Data extraction thread

[CryingWestern]

The reason it wasn't working was because your extractor didn't extract them properly.

[Deep Blue]

which tool should i use if not crass? I tried with arc_conv and it is working fine now, i will need to figure out how to pack them again but it's progress 

Edited November 5, 2015 by Deep Blue

[CryingWestern]

I used AE for extracting it out of the archive.

[Guest]

Btw, sorry for the main posts being down. The reason is simply that the last forum update has messed it up completely, The spoiler tags don't work, the editing is broken, many images aren't uploaded anymore, and other problems.

Until all of these problems are fixed, I won't put it back up on the forum. Aside from the pictures, most of the problems are no related to me though.

My appologies

[Deep Blue]

So I'm going mad with this problem...how the hell do I repack the txt into .mjo and then into .arc I tried everything and the files are always 0kb I already tried so many combinations  >_>

any place where i can find info about this engine specially the text scripting and how to use the tools to pack and unpack the files? 

EDIT: Using AE i managed to pack the .mjo into .arc but i still cant convert the txt into mjo

Edited November 6, 2015 by Deep Blue

[Scorp]

So I'm going mad with this problem...how the hell do I repack the txt into .mjo and then into .arc I tried everything and the files are always 0kb I already tried so many combinations  >_>

any place where i can find info about this engine specially the text scripting and how to use the tools to pack and unpack the files? 

EDIT: Using AE i managed to pack the .mjo into .arc but i still cant convert the txt into mjo

All you need is arc_conv from w8m. Version from that github is fine: https://github.com/GAEndGears/majiro-tools/blob/master/arc_conv.7z?raw=true

To unpack .arc (I would assume you use update.arc) type in console: arc_conv update.arc
Folder update.arc~ would be created. Copy one of the files from this folder to the folder above (copy nard00.mjo for ex).

To convert file to text format type (I use nard00.mjo as example): text_conv majiro nard00.mjo > nard00.txt

After you're done with .txt changes, to pack back to .mjo type: text_conv majiro nard00.mjo nard00_.mjo < nard00.txt. This will create file nard00_.mjo in same folder (also initial nard00.mjo file should be present! do not delete it).

After that copy that file nard00_.mjo to update.arc~ folder, delete old nard00.mjo and rename it to nard00.mjo (e.g. replace old nard00.mjo with newly created) and pack it back to update.arc using this: arc_conv.exe --pack majiro update.arc~ update.arc 2 (notice 2 - that's majiro format version, in your update.arc it was V2, so that's why I put 2 here).

You should get newly created update.arc.

[CryingWestern]

If that doesn't work, i've used my extractor for the texts. Your choice to use them.

[Scorp]

If that doesn't work, i've used my extractor for the texts. Your choice to use them.

Texts were already present on TLWiki, he needed a way to put them back.

[CryingWestern]

If he's using the scripts that were done by Nagato, only his tools will repack it.

Edited November 6, 2015 by CryingWestern

[Deep Blue]

I finally did it, so my mistake was on not letting the original mjo file in the same folder when I tried to convert the txt into mjo again so you need to do this

text_conv majiro originalfile.mjo yournewfile.mjo > youredited.txt 

what a pain in the ass, then I just use AE to compress them, also I need to edit both files to make the things i edited  to work correctly, when i say both files I mean

nar3_00.mjo which contain some lines of the scrip and then the main file nar3.mjo, i guess they did this because all the stories are divided into chapters and you can select them individual but it also checks the main scrip (?) anyway if you dont do this it will generate a chaos....this engine is a mess, the way it handle the lines, for example how you  add the furigana

lets say I want to write 私達 and add furigana then i need to write it like this

1わたし
2私
3たち
4達
what a piece of garbage...

also everytime you go to the next line the program jumps to the next line to, even NScripter was much more robust and simple to work with... I'm not sure i want to spend time with this at all, also when I try to bypass the main files using the update file I need to add the number of the line i want to edit...is just a nightmare to work with.

I tried and it works with AE Majiro Arc ver 1.000, with 3 and 2 it doesnt even start it just tell me (I guess) that is a dif majiro ver

Thanks both of you for helping me with this.

EDIT: for anyone how wants to unpack the files in the future, use arc_conv because AE sometimes it breaks the files so when you try to convert the mjo to txt it will give you a 0k file but it doesnt happen with arc_conv

Edited November 7, 2015 by Deep Blue

[Nanashi3]

Anyone know bytecode script decompiler for Regista games? The ones here: http://regista.co.jp/pro.htm (apart from older ones like Ever17)

 

Script usually stored in sn.bin. So for example Cross+Channel on PSP have that type of script. I believe even some recent games like Phantom have it (upgraded obv). It is compressed with usual lzss from 89, with no modifications, so you can easily decompress it.

 

Obv header have the offsets for script parts, but what about script itself? I ask this because text is mixed with script bytecode (as usual), and script have jumps forward/backward, counted in bytes, so seems without full list of byte commands (not necessarily real meaning, just correct list of parameters, obv) would be not possible to translate any of these.

Hey there Scorp,

if you only need the script for extraction, I released a set of python 3 scripts for dumping sn.bin archives (tested on a handful of psp/ps3/x360 titles).

https://github.com/mchubby/yetireg_tools

 

I won't be around to help much, so if you need proper tools, you'd rather recruit someone to hack this.

 

Greets and out,
Nanashi3

[Scorp]

Anyone know bytecode script decompiler for Regista games? The ones here: http://regista.co.jp/pro.htm (apart from older ones like Ever17)

 

Script usually stored in sn.bin. So for example Cross+Channel on PSP have that type of script. I believe even some recent games like Phantom have it (upgraded obv). It is compressed with usual lzss from 89, with no modifications, so you can easily decompress it.

 

Obv header have the offsets for script parts, but what about script itself? I ask this because text is mixed with script bytecode (as usual), and script have jumps forward/backward, counted in bytes, so seems without full list of byte commands (not necessarily real meaning, just correct list of parameters, obv) would be not possible to translate any of these.

Hey there Scorp,

if you only need the script for extraction, I released a set of python 3 scripts for dumping sn.bin archives (tested on a handful of psp/ps3/x360 titles).

https://github.com/mchubby/yetireg_tools

 

I won't be around to help much, so if you need proper tools, you'd rather recruit someone to hack this.

 

Greets and out,
Nanashi3

Well, I unpacked and extract the portions myself, that's trivial thing, I believe you can just unpack using Okamura own LZSS.C code from 1989.

That's a pity you did not digged much into what these opcodes meant, but thx  Will try to take a look later, as I believe there would lot of compatibility issues with different titles.

Btw, I do not see any jump/call bytecode command elaborated in your extjis_psp-cc.py. Did you really missed them? As if yes you would never be able to recompile script back if you edited something.

[K450]

Hi,

I was planning on making edits to the files eyecatch_0705 to eyecatch_spring in hoshimemo to include the year as well as the date,

and even used the exact font (via test drive, no way am I spending $500 on one font), correct format (24BMP with alpha) and everything, and discovered crass can't repack them into bins. 

Unfortunately, repacking tools rest on the EH project, so I'm left with nothing to work with, unless by some miracle I can work out/find a method for doing so, or someone points me in the right direction.

[CryingWestern]

You can try these tools though not sure if it will work (you'll need to link a path through cmd to use these as well I believe).

I believe AE has a converter and packer. Haven't used it yet though. So whether it works or not, i don't have a clue.

[Nanashi3]

Well, I unpacked and extract the portions myself, that's trivial thing, I believe you can just unpack using Okamura own LZSS.C code from 1989.

That's a pity you did not digged much into what these opcodes meant, but thx  Will try to take a look later, as I believe there would lot of compatibility issues with different titles.

Btw, I do not see any jump/call bytecode command elaborated in your extjis_psp-cc.py. Did you really missed them? As if yes you would never be able to recompile script back if you edited something.

Yes indeed, the script merely dumps text and has no way to reconstruct. It is left as an exercise to the reader

Unfortunately, I'm afraid my MIPS proficiency has rusted. delay slot etc. */me forgot it all*

If you need a few pointers for PSP C+C, I gathered a few annotations when I browsed the eboot. Hopefully PPSSPP improved its debugging capabilities.

Data symbols

ptr_curSN  091D1700
g_nScripts 09A25CE0
ptrOpcode_ 09A41A70   << Instruction Pointer, should be useful for locating jumps

Function symbols

Lzss_decompress 08850F34
opcode_sub2b 088567D8
opcode_sub4b 08856854
OP00_0b  088568B0
OP01_var 088568E4
OP10     08857040 

And the function table for opcodes seems to be .data:0888F164

Sorry I can't be of much help XO

 

[K450]

You can try these tools though not sure if it will work (you'll need to link a path through cmd to use these as well I believe).

I believe AE has a converter and packer. Haven't used it yet though. So whether it works or not, i don't have a clue.

I tried running bin_archiver through cmd, imported the module, and ran "bin_archiver -c graph graph.bin" (usage instructions were the format "bin-archiver.py folder output.bin")

The folder was in the same directory as the script, and the same name as given in the above command. I got:

File "<stdin>", line 1
   bin_archiver -c graph graph.bin

   bin_archiver -c graph graph.bin

                                              ^

SyntaxError: invalid syntax

No dice

And is AE the same as Akerou? If so, he's planning to add a repacker, though I don't know when.

Edited November 13, 2015 by K450

[CryingWestern]

Are you using Python3?

[Corsad]

Sorry, I've tried Extractdata and Crass to extract the bgm.dat file for Ore no Ragnarock game. However, after it finished the process, the Extractdata showed a plain page and the Crass didn't output any directory.

Can someone teach me how to extract the file?

Thanks

[K450]

Are you using Python3?

This is what I used

[CryingWestern]

The one you are using is python2, you can get python3 here.

[K450]

Alright, uninstalled all other versions and installed the recommended (3.0) release. This was the result:

[binaryfail]

Alright, uninstalled all other versions and installed the recommended (3.0) release. This was the result:

 

You're not invoking the tool correctly. Check here: http://i.imgur.com/NUU6Anv.png

METHOD #1 : Usually, you should run a script directly from the command prompt (and not python's shell)

METHOD #2 : If you really wanted to run the pack function from the python shell, that's how it should be done. Make sure bin_archiver has the .py file extension, otherwise it won't import.

METHOD #3 : Don't repack a new bin. For the FVP engine, you can make a new folder within the game directory. Give it the same name as the original bin (excluding the .bin extension) and put your modified files in that folder.

[K450]

Ah, thanks for helping!

I appreciate the help. 

[K450]

Success!

What I did was use akerou's fvp toolset, compiled with qt, and decoded the original graph.bin. Then, I converted those .bmps I made to png (Photoshop's automation saves the day!) and set fvp to convert them back.

Curiously, when these new files were copied to the replacement graph folder as per Method #3, they defaulted to x0 y0. However, when adding them to another copy of the graph folder (converted using bin_archiver.py),

and after appending their correct numbers (ex. 0882_eyecatch_0705), I repacked this folder, again using bin_archiver. I copied this new bin over to my game directory, and lo and behold, the replacement files had their original placement!

Now, if there's a way to decode/encode the pos.dat file itself to allow editing the image coords to fix their alignment...

[binaryfail]

Success!

What I did was use akerou's fvp toolset, compiled with qt, and decoded the original graph.bin. Then, I converted those .bmps I made to png (Photoshop's automation saves the day!) and set fvp to convert them back.

Curiously, when these new files were copied to the replacement graph folder as per Method #3, they defaulted to x0 y0. However, when adding them to another copy of the graph folder (converted using bin_archiver.py),

and after appending their correct numbers (ex. 0882_eyecatch_0705), I repacked this folder, again using bin_archiver. I copied this new bin over to my game directory, and lo and behold, the replacement files had their original placement!

Now, if there's a way to decode/encode the pos.dat file itself to allow editing the image coords to fix their alignment...

Akerou's tool compresses the data in pos.dat using zlib -- a very odd choice.
But you can decompress it, edit the data with a hex editor, and recompress pos.dat again.

If you haven't tried it yet, see if nvsg_converter (from the same toolset as bin_archiver) does what you need.
That tool requires you to specify the X/Y positions on the command line, rather than stashing it in a file.

Running the images with method #3 shouldn't act any differently from repacking the bin file.

Here's a random image I made. It's using method #3 and nvsg_converter. Coordinates are set to the lower-left corner.