[Release] Agent - an all-games on any platform text hooker, use this for machine translations!

[adamstan]

I think I finally got it fully working on 64bit PPSSPP. The crash I experienced previously when trying to use signature was because the signature was too vague, and it actually hooked to the wrong address - I only noticed it when I paused emulator before attaching. When I tried more specific signature - it worked.

I did that on "test 2" version.

[0xDC00]

@adamstan How about test1, please check it too.

[adamstan]

39 minutes ago, 0xDC00 said:

@adamstan How about test1, please check it too.

Okay, I'll test that too. Also I think now I'll focus on making it work with '&' - Sora no Mukou de Sakimasu you ni -  on rpc3 emulator. I found out that this game uses utf8 encoding with 00 as terminator, I found an address when one of the lines is stored, and have a bunch of instructions that access it. Among those, one seems to be the instruction that reads the line one character at time, since it executes the same number of times as there are characters in the line. But when I try to hook to any of those instructions, I either get no reaction or a crash (the endless loop of "onEnter <address>" messages) When I get back home I'll post a screenshot with those instructions - maybe you'll have a better idea which one would be good to try.

But it's also possible that rpcs3 works in a weird way, just like PS3...

Edited July 27, 2021 by adamstan

[adamstan]

Okay, as far as PPSSPP64 is concerned - "test2" version works properly for me:

When using the same signature on "test 1" version, it hooked to the correct address but it crashed when I pressed the button to advance the game.

 

And here are search results for rpcs3 and the abovementioned '&' - Sora no Mukou de Sakimasu you ni -

But when trying to hook onto one of those instructions I get either no reaction or crash.

Edited July 27, 2021 by adamstan

[0xDC00]

@adamstan

Latest: https://github.com/0xDC00/agent/archive/refs/heads/master.zip
 

Try this script.
Note: create a new .js file inside the scripts folder and load it instead of ExecutionWatch.

globalThis._FIXED_DCODE_ = '$utf-8,|0|00,|rbx+[rbp+0x38]|4883EC28837D14000F85????????48????48????B0FEFFFF48??????????B0FEFFFF48????204C????????????4C????1848????????????48??????????78FFFFFF48';
// func 0x46328

var previousString = '';
globalThis.filters = function(s) {
    if (s === previousString) {
        console.log('>');
        return null;
    }
    previousString = s;
    return s.replace(/\n+/g, ' ');
}

require('_ExecutionWatch.js');

Source: https://docs.google.com/spreadsheets/d/14k5TBc2cAed8Fcx2fb5schlPh6Ah24dmW3dJpxvNAbc/edit#gid=0&range=E18

Edited August 2, 2021 by 0xDC00
fix image link

[adamstan]

7 hours ago, 0xDC00 said:

@adamstan

Latest: https://github.com/0xDC00/agent/archive/refs/heads/master.zip
 

Try this script.
Note: create a new .js file inside the scripts folder and load it instead of ExecutionWatch.

globalThis._FIXED_DCODE_ = '$utf-8,|0|00,|rbx+[rbp+0x38]|4883EC28837D14000F85????????48????48????B0FEFFFF48??????????B0FEFFFF48????204C????????????4C????1848????????????48??????????78FFFFFF48';
// func 0x46328

var previousString = '';
globalThis.filters = function(s) {
    if (s === previousString) {
        console.log('>');
        return null;
    }
    previousString = s;
    return s.replace(/\n+/g, ' ');
}

require('_ExecutionWatch.js');

Source: https://docs.google.com/spreadsheets/d/14k5TBc2cAed8Fcx2fb5schlPh6Ah24dmW3dJpxvNAbc/edit#gid=0&range=E18

I was full of hope, but sadly, it's again perhaps the problem of different compilations on different systems - on mine, I get "No Result!" message when trying your custom script with d-code. Could you describe how did you arrive at this code? Then maybe I'll be able to work out the one for my system.

[0xDC00]

@adamstan function 0x46328
 https://docs.google.com/document/d/1Gc42hY8MYmZJMtgrzG6PqWT6X0BBou-P2HbWnRD2jbE/

[adamstan]

1 hour ago, 0xDC00 said:

@adamstan function 0x46328
 https://docs.google.com/document/d/1Gc42hY8MYmZJMtgrzG6PqWT6X0BBou-P2HbWnRD2jbE/

Thanks! I found correct signature for my system, and now it works

(Screenshot got weirdly streched, but that doesn't matter )

If I may ask - how did you know that it should be function 0x46328 ?

[adamstan]

@0xDC00

I have one feature request (or perhaps a bugfix?) for ExecutionWatch script.

It's related to the problem I encoutered when playing Princess Evangile. This game uses UTF16 encoding with 0000 word used as terminator. And here's the problem - it seems that when searching for that pattern, it stops when it finds two consecutive 00 bytes regardless of their position, which leads to the situation from picture below:

Since this game has also latin letters and various formatting codes incorporated into the script, it is possible to encounter characters with codes that start or end with 00. And sometimes it leads to the premature clipping of the text, like in this example - there was 3E00 004E sequence - and those two middle bytes got misinterpreted as terminator pattern. I think the solution would be to modify the script so it would consider specified encoding and pattern length when looking for the terminator. So in this example, since it's utf16 and 2-bytes long terminator pattern, I think it should always read the data as 16bit words, not as a series of single bytes. That way we should be able to avoid this problem.

(BTW If I'm not mistaken, 004E code is 一 character, since I got the same problem with the word 一名, which is also always preceded by formatting code or special character ending in 00.)

 

Edited August 18, 2021 by adamstan

[0xDC00]

@adamstan 
Please test latest build: https://github.com/0xDC00/agent/archive/refs/heads/master.zip

[adamstan]

14 minutes ago, 0xDC00 said:

@adamstan 
Please test latest build: https://github.com/0xDC00/agent/archive/refs/heads/master.zip

Now it works correctly, and doesn't cut off at those points Thanks

[adamstan]

57 minutes ago, 0xDC00 said:

@adamstan 
Please test latest build: https://github.com/0xDC00/agent/archive/refs/heads/master.zip

I have one more question. What is exact syntax for replace function? For example, I'd like to filter out ruby text/furigana. In case of this game, it means removing "<R*/" (with * here acting as a wildcard) and ">". What arguments should I provide to replace function?

[0xDC00]

@adamstan Custom script.

globalThis._FIXED_DCODE_ = '$utf-16,|50|0000,46??????????45??????F3??????????F3??????????41'; // <-- YOUR CODE HERE

//globalThis.decode = function(buffer) {
//    return _decode(buffer);
//}

globalThis.filters = function(s) {
    return s
        .replace(/(<R[^\/]+.)([^>]+)(.)/g, '$2') // trim ruby
        .replace(/<[A-Z]+>/g, '')                // trim tag
        //.replace(/\n+/g, ' ')                    // single line
        ;
}

require('_ExecutionWatch.js');

regex101

Edited August 21, 2021 by 0xDC00
fix regex single line

[adamstan]

9 minutes ago, 0xDC00 said:

@adamstan Custom script.

globalThis._FIXED_DCODE_ = '$utf-16,|50|0000,46??????????45??????F3??????????F3??????????41'; // <-- YOUR CODE HERE

//globalThis.decode = function(buffer) {
//    return _decode(buffer);
//}

globalThis.filters = function(s) {
    return s
        .replace(/(<R[^\/]+.)([^>]+)(.)/g, '$2') // trim ruby
        .replace(/<[A-Z]+>/g, '')                // trim tag
        //.replace(/\n+/, ' ')                   // single line
        ;
}

require('_ExecutionWatch.js');

regex101

Thanks, works like a charm And thanks for the RegEx explanation/example

[adamstan]

Hello, @0xDC00

Could you perhaps help me with another PSP title? This time I'm trying to work out the D-CODE for PSP version of DameKoi

I found the signature that almost works:

44 0FB6 4C 03 00  - movzx r9d,byte ptr [rbx+rax+00]

44 0FB6 4C 03 00     
E9 19000000          
3D 00000100          
72 07                
3D 00400100          
72 E7                
8D 00                

$shift_jis,00,440FB64C0300E9190000003D0000010072073D0040010072E78D00

But it has one little problem - it keeps truncating the first byte of the captured string, which leads to garbled up first character of the text:

How can I fix that?

 

[adamstan]

Okay, I found the solution, using 'expressions' field of the DCODE. Since original function pointed to rax+rbx, I just changed it to rax+rbx-1, to shift captured area one byte 'left'. Now it works as it should

So here's working D-CODE for PSP version of DameKoi:

$shift_jis,1|50|00,|rax+rbx-1|440FB64C03??E919????????????????72073D0040010072E78D00

(It has to be attached after starting the game)

 

Edited September 26, 2021 by adamstan

[adamstan]

@0xDC00, I have another question.

When I loaded my abovementioned DameKoi script into latest agent version (0.1.1) it seems to work (console shows "onEnter" message and memory contents of the address it landed on), but it doesn't copy text to clipboard.

So my questions are - how do you create custom scripts for this new version, and how do you search for hooks for it?

I found no information on github or discord.

 

Update:

My old script for Princess Evangile works. The only difference between the two is that PE uses utf16 encoding, while Damekoi uses shift-jis, and it seems that when I load it into new version of agent, it doesn't recognize the encoding.

Edited December 13, 2021 by adamstan

[0xDC00]

@adamstan sorry for my delay
> So my questions are - how do you create custom scripts for this new version, and how do you search for hooks for it?
> I found no information on github or discord.

https://discord.com/channels/867944111557201980/867944111557201983/941316823675187290
I will add a new script for DameKoi soon... (get it via script updater)

https://discord.com/channels/867944111557201980/940962653533265950

 

> Damekoi uses shift-jis, and it seems that when I load it into new version of agent, it doesn't recognize the encoding.

Looklike TextDecoder (engine iconv lite - js) is unstable.

Script request?, please create an issue here: https://github.com/0xDC00/scripts

Edited February 11, 2022 by 0xDC00