Data extraction thread

[sena_kaito]

On 2/25/2019 at 2:25 PM, Ulysses said:

E_AYA07_A.dat & E_AYA07_B.dat are WMV video files, open directly via PotPlayer.

C2_EA.PSP is PSB file, however it is either incomplete or encrypted or obfuscated (there seems to be many 0xFF inserted), it's not possible to be loaded in my FreeMote.

If you believe C2_EA.json is JSON file, it could be XORed, too.

Solved with latest GARro
Thank you.

[onii]

SiglusEngine issue with g00 files: how do I create "No Data" layers?

I tried using dummy transparent images but it causes a glitch, it may be something to do with the file output format as well but I can't find a documentation for G00PackMax and vaconv-cpp doesn't seem to support multi-layer PSD files.

Now, I have almost completely finished editing the interface and half the scene file, that is the only obstacle I couldn't manage to overcome so far... I made this thread here with more details if you need.

Edited March 5, 2019 by onii
reduced to relevant info

[thrae]

On 24. 5. 2014 at 10:34 AM, maefdomn said:

YU-NO

Download Yuno_arctools.zip and Yuno_gp8tools.zipnot the latest version.

Is there any good soul that still has this "Yuno_arctools" and could reupload it somewhere? The link is not working and I'm not able to google it anywhere else. I'm trying to extract music.arc from this game to turn down BGM volume a bit (because it drowns out voice overs) with no luck for the last two days and I would much appreciate any help.

Edited March 6, 2019 by thrae

[Nanashi3]

12 minutes ago, thrae said:

Is there any good soul that still has this "Yuno_arctools" and could reupload it somewhere? The link is not working and I'm not able to google it anywhere else. I'm trying to extract music.arc from this game to turn down BGM volume a bit (because it drowns out voice overs) with no luck for the last two days and I would much appreciate any help.

You may download the v0.4 here: https://web.archive.org/web/20171201163843/https://tlwiki.org/index.php?title=File:Yuno_arctools.zip

[thrae]

On 6. 3. 2019 at 11:49 PM, Nanashi3 said:

You may download the v0.4 here: https://web.archive.org/web/20171201163843/https://tlwiki.org/index.php?title=File:Yuno_arctools.zip

Thanks mate, you saved me! It didn't cross my mind to try the other versions when I previously found out the archive link to the newest one isn't working. Considering how much hours I've spent googling and trying of different arc tools, this is really emabarassing. Anyway, I just finished editing the files and the game is so much better with background music actually being in the background! Once again thank you!

Edited March 7, 2019 by thrae

[Eco-Friendly]

I'm trying to unpack squeez cgpack by using arc_unpacker , but only thing i get is 0_nopics.gr or something like that. Can anyone help me ?

[Nikdo]

Hi, it's me again. This time I'm trying to unpack Hentai Ouji to Warawanai Neko visual novel (PSP, 2013), again looking for character images.

I have found it for some reason in the INSDIR directory, inside the RES.DAT (it has 47 50 44 41 / GPDA signature). The only program able to extract it was the old good arc_conv-master, so here I've got the script_charactor.dat file, again with GPDA signature.

But there goes the problem.

If you open the script_charactor.dat file in a hex editor, you can see it contains the desired *.gim files, which can be opened with GimConv or Gim Viewer:

But unpacking it again with arc_conv-master only creates a "damaged" files. It has no extension and if you open it with a hex editor, you can see it's got a GPDA signature again and the content is clearly messed up (a lot of zeros after every few lines).

Isn't there a better way to extract those files? Anything I've found on the internet was about emulators... :/

EDIT:

OK, I have found something called "tenoritool" (http://rsk.twilight.free.fr/ps2/home/tenoritool-1.0alpha.7z), which seems to go well with other PSP games, where you need to extract res.dat (like Oreimo or Toradora).

However, when I've tried it for this game, it's showed some kind of error written in my native language (something about "spaces in the string"), so I guess it is more like cmd error than anything, but... What the heck?

EDIT:

OK, everything I said before was wrong.

To use the tenoritool, you must be just very direct with the command, so this one actually worked:

tenoritool.exe --list="" -v -x -d extract res.dat

And with that, I was able to extract the script_charactor.dat file. And from there I've finally got the desired *.gim files.

But those files wasn't the usual gims. It was just gzip files, that contained... Guess what? The dat file that I've got from the arc_conv-master at the beginning!

But I know the solution now.

Using tenoritool on it again creates 22 dat files. One of them has clearly the biggest size, so all you have to do is to change its extension to *.gim and you can open it/convert it to an image.

Wow, this was harsh. Sorry for bothering you when I was able to solve it myself the same day I wrote it here, but maybe someone else will find these instructions helpful.

 

On 13. 3. 2019 at 11:59 PM, Eco-Friendly said:

I'm trying to unpack squeez cgpack by using arc_unpacker , but only thing i get is 0_nopics.gr or something like that. Can anyone help me ?

Try GARbro. It isn't omnipotent, but it's the best user-friendly unpacker I know so far...

Edited March 17, 2019 by Nikdo

[Eco-Friendly]

15 hours ago, Nikdo said:

Try GARbro. It isn't omnipotent, but it's the best user-friendly unpacker I know so far..

Thanks , but it didn't helped . Saying it can't be opened as archive 

[Nikdo]

15 hours ago, Eco-Friendly said:

Thanks , but it didn't helped . Saying it can't be opened as archive 

OK, I'm gonna look at it.

Can you send here another pack of the same game, that isn't that large?

[Nanashi3]

4 hours ago, Nikdo said:

OK, I'm gonna look at it.

Can you send here another pack of the same game, that isn't that large?

Try https://github.com/xyzz/taiga-aisaka

Here is a precompiled copy with Java 8: https://mega.co.nz/#!7s90TQwD!JeRYuxxB62f9bt19RU_-mwYqX0eWgHwNAn0Uu4L8YTE

Example usage:

java -jar Gpda.jar res.dat

Compilation howto:

javac Gpda.java
jar cfe Gpda.jar Gpda *.class

Slightly modified to extract regardless of whether ".dat" is lowercase or not.

[Eco-Friendly]

7 hours ago, Nikdo said:

OK, I'm gonna look at it.

Can you send here another pack of the same game, that isn't that large?

I opened it with GARbro , it just worked for a third time !
Thank you for response !!))

[Veshurik]

Oh, I think, extracting files from PSP, PSVita, PS3/PS4 eroge is terrible... Even harden then for PC... If you are totally noob.

[Nikdo]

19 hours ago, Nanashi3 said:

Try https://github.com/xyzz/taiga-aisaka

Here is a precompiled copy with Java 8: https://mega.co.nz/#!7s90TQwD!JeRYuxxB62f9bt19RU_-mwYqX0eWgHwNAn0Uu4L8YTE

Example usage:

java -jar Gpda.jar res.dat

Compilation howto:

javac Gpda.java
jar cfe Gpda.jar Gpda *.class

Slightly modified to extract regardless of whether ".dat" is lowercase or not.

Well, thank you.

This program seems to work similary to tenoritool, but it also has the same "bug" when it comes to Hentai Ouji to Warawanai Neko.

While it normaly extracts res.dat and inner script_charactor.dat (which contains the base for characters images), the faces are stored inside script_charactor_face.dat.

Both tenoritool and Gpda extract the latter as "no data".

In tenoritool case, it contains the names and filesizes, but there is nothing inside (just zeros).

In Gpda case, it contains only name. The file sizes are always 0 B.

I have upload the file on a cloud, so you can download it here: https://dfiles.eu/files/cociazoa9

Just looking inside it with a hex editor, you can clearly tell that it does have the data there, both programs just fail to extract them.

Do you have any idea why?

[Nanashi3]

2 hours ago, Nikdo said:

Well, thank you.

This program seems to work similary to tenoritool, but it also has the same "bug" when it comes to Hentai Ouji to Warawanai Neko.

While it normaly extracts res.dat and inner script_charactor.dat (which contains the base for characters images), the faces are stored inside script_charactor_face.dat.

Both tenoritool and Gpda extract the latter as "no data".

In tenoritool case, it contains the names and filesizes, but there is nothing inside (just zeros).

In Gpda case, it contains only name. The file sizes are always 0 B.

I have upload the file on a cloud, so you can download it here: https://dfiles.eu/files/cociazoa9

Just looking inside it with a hex editor, you can clearly tell that it does have the data there, both programs just fail to extract them.

Do you have any idea why?

Ok so it seems there was an unhandled case in the file parsing.

I have compiled a new one for you @ https://github.com/mchubby/taiga-aisaka/releases (source changes at https://github.com/mchubby/taiga-aisaka/commit/14e066)

The resulting files could be further processed by the SDK's GimConv (if you need to mod & convert back) and Gim Viewer http://hsreina.shadosoft-tm.com/display/post/56

 

[Nikdo]

On 19. 3. 2019 at 8:29 PM, Nanashi3 said:

Ok so it seems there was an unhandled case in the file parsing.

I have compiled a new one for you @ https://github.com/mchubby/taiga-aisaka/releases (source changes at https://github.com/mchubby/taiga-aisaka/commit/14e066)

The resulting files could be further processed by the SDK's GimConv (if you need to mod & convert back) and Gim Viewer http://hsreina.shadosoft-tm.com/display/post/56

 

Haha, and there goes all my expectations.

Thank you so much for creating this version for me, but the only information I've got from this is, that the script_charactor.dat file nor any other of those now accessible files actually contain what I'm looking for.

You see, inside the script_charactor.dat file you can find images like this:

So I have logically expected, that the script_charactor_face.dat file will contain what is missing, but there are just "teasers" like this:

I'm starting to hate this game, really.

There is even someting like emote, but you just can't use it to finish the face, because it creates completely different images.

I know that the desired part may be assembled in just one image that I just happened to miss, but... There is actually still a file that wasn't extracted correctly.

It's name is "adummy.dat" (35,938 kB) and even now it contains only zeros.

Unfortunately, it's path is res.dat\adummy.dat so I have to share the whole resource now: https://dfiles.eu/files/hwpn85k6r

If it's not there, then I'm just lost.

Edited March 21, 2019 by Nikdo

[Nanashi3]

18 hours ago, Nikdo said:

Haha, and there goes all my expectations.

Thank you so much for creating this version for me, but the only information I've got from this is, that the script_charactor.dat file nor any other of those now accessible files actually contain what I'm looking for.

You see, inside the script_charactor.dat file you can find images like this:

Well the tools are not perfect but after extracting the RES.Dat using the java tool

cmd

cd RES\script_charactor.dat

ren *.gim *.gim.gz

 

Extract those gz files using 7-zip for example:

=> AZU_1B.dat, etc. to YOKO_3L.dat are created.
These are GPDA-headered files too but the java tool fails with a heap memory error.

Tenoritool can split those but with the wrong extensions

- the first files are textual configs [AZU_1B.0001.dat and AZU_1B.0002.dat]

- The largest file, like the one you posted,  is a GIM image [AZU_1B.0003.dat]

- the rest are eyemasks as GIM images [AZU_1B.0004.dat to AZU_1B.0022.dat]

 

 

[Nikdo]

4 hours ago, Nanashi3 said:

Well the tools are not perfect but after extracting the RES.Dat using the java tool

cmd

cd RES\script_charactor.dat

ren *.gim *.gim.gz

 

Extract those gz files using 7-zip for example:

=> AZU_1B.dat, etc. to YOKO_3L.dat are created.
These are GPDA-headered files too but the java tool fails with a heap memory error.

Tenoritool can split those but with the wrong extensions

- the first files are textual configs [AZU_1B.0001.dat and AZU_1B.0002.dat]

- The largest file, like the one you posted,  is a GIM image [AZU_1B.0003.dat]

- the rest are eyemasks as GIM images [AZU_1B.0004.dat to AZU_1B.0022.dat]

 

 

OMG you are kidding me.

So I had it in my palm the whole time, but I just toss it away as a trash... I'm such an idiot. Thank you, really.

But still, for the education sake, why that one file isn't extracted?

Is it because it's too large and some kind of space-limit overflow?

I have created some data-handling programs myself (mostly just wav or bmp though), so I have some basic idea what are those extractors doing, but... I don't know Java or C#, so those source codes tells nothing to me...

[pmrowla]

I've been working on a tool for extracting and patching LiveMaker/LiveNovel games and figured I should post it here. It started out as a fork of tinfoil's irl which I guess was kickstarted and then abandoned a while back?

Current status is:

• LiveMaker assets can be extracted from .exe or .dat files
• LiveMaker's binary .lsb format for their "chart" scripting can be dumped as text or xml
• LiveNovel scripts can be extracted from LiveMaker's binary .lsb format
• (Translated) LiveNovel scripts can be compiled and inserted back into a .lsb file
• Modified .lsb's can be patched back into a livemaker game and run with (cp932 encoded) english text, but currently the application will lock up when you try to exit.

My current code is here: https://github.com/pmrowla/pylivemaker/tree/wip (you have to use the wip branch link since I'm not pushing anything into master until I figure out the crash on exit thing)

The code itself is mostly fully commented but I haven't published any of the actual docs yet, so there's a github issue w/a summary of what works right now and what doesn't: https://github.com/pmrowla/pylivemaker/issues/1

 

Honestly idk if there's even any livemaker games worth patching but I was bored and felt like reversing something so I picked an obscure engine that afaik hasn't been dug into very much before. I think GARbro supports viewing .gal CG's from LiveMaker archives, but that doesn't cover the text extraction and patching side of things.

[Haiyami]

I'm trying to extract the scenario files from the md scr.med file from saiminjutsu4. I tried garbro but i can only extract the contents from the root file. I can't extract any of the scenario files so I can change coding such as translation and sprite image generation.  Anyone have anything useful to help out?

https://imgur.com/a/nx3j75p

 

 

Edited March 31, 2019 by Haiyami

[pmrowla]

On 3/29/2019 at 10:31 AM, pmrowla said:

I've been working on a tool for extracting and patching LiveMaker/LiveNovel games and figured I should post it here. It started out as a fork of tinfoil's irl which I guess was kickstarted and then abandoned a while back?

This thing is mostly done now, everything at https://github.com/pmrowla/pylivemaker works and there's some actual documentation now. It's still more of a proof-of-concept thing than an actual tool someone could use to make a full translation patch, but I probably won't get around to making it particularly user friendly until/unless someone actually wants to make a full patch for something.

If you are interested in RE I wrote up a short post about it here: https://pmrowla.com/blog/reverse-engineering-livemaker/

[Azukee]

Thought somebody might be interested in this; me and a friend of mine are currently working on an all-purpose archive unpacker (mostly for visual novel archives)

The link can be found at https://github.com/Azukee/Macaron currently in development is the Criware unpacker (unpacker works, conversion to custom files atm not) and Kirikiri unpacker (Header reading works atm; can be found in the Kirkiri branch)

You can find the currently supported archives in the readme file of the repo, and if you have any archive requests you can request them in the issues page with the "archive request" tag

[Zakamutt]

It's possible this exists somewhere else but I couldn't find it: something that can convert the BGI / Ethornell engine's sysgrp UI images to something you can work with. (target: senmomo). My searches brought me to xupefei's BGIKit, which had no binary releases, so I had to compile it myself (ok visual studio makes this easy actually). I'm not sure exactly what platforms these binaries will work on bar windows 64-bit, but at least you don't have to compile them yourself: https://drive.google.com/file/d/1cy1SnLSQZ_rCW8H6AsD9SZDxZrUd7qtZ/view?usp=sharing

Only the sysgrp tool is tested and only from game format to .bmp, have fun. To use that, call the program from the command line (cmd.exe or powershell) with the path to the file to be converted  to bmp / from bmp as the sole argument, or more concretely:

SysgrpConverter.exe filenamehere

 

[Koioniaku]

everytime i try to extract sound from kikokugai, it says its encrypted... anyone know what i should do? its .npa

[Haiyami]

So i've able to extract the scenario files from saiminjustu 4. But of course those files don't have any file type extension on them. When i try to open any of the scenario files in notepad++ I get a garbled mess.

 

any help I'm trying to change the picture references for uncenosred pictures as well as translating. Thank you.

[ichigop3p]

22 hours ago, Haiyami said:

So i've able to extract the scenario files from saiminjustu 4. But of course those files don't have any file type extension on them. When i try to open any of the scenario files in notepad++ I get a garbled mess.

 

any help I'm trying to change the picture references for uncenosred pictures as well as translating. Thank you.

Change the unicode from ansi to any japanese lang like JIS. . .If does not work maybe the extraction failed