Data extraction thread

[LewdRidereros]

HAo there, I'm trying to translate this VN, https://vndb.org/v4803,  well more like I want to translate it since I dont know how to extract the data. The game uses some kind of .dat files or something like that and its a real pain in the ass to get it working. I tried using arc_unpacker and a lot of other extractors to no avail. Arc_extractor at least pointed that maybe it was an NScripter engine but I used Filextractor, nsaout and nsdec and it was fruitless as well. Im actually about to give up at this point. 

[MantasB]

I see. How about Narcissu Steam version then?
https://github.com/3licie/majiro-tools/issues/1

[sena_kaito]

Hii~

Is it GARbro already dead?
Any tools like GARbro?

[GuyInDogSuit]

18 hours ago, sena_kaito said:

Hii~

Is it GARbro already dead?
Any tools like GARbro?

 

Wondering the same thing. Seems like it hasn't been updated in over a year.

And it's sad because I can't figure out how to extract files from the game "White Wings ホワイトウィングス." Apparently it uses the KiriKiri engine, but I don't know what encryption it used.

Edited July 20, 2020 by GuyInDogSuit

[Yoshino Reiji]

Hi, just want to ask here, is there anyone knows how to extract CG from the game Kishin Houkou Demonbane? It is the ported PC version from the PS2, which came out last year. Because it used NPK engine so i don't know how to extract it, even my friend so far tried all methods but none of them work either. Much appreciated if anyone can help.

https://vndb.org/r61822

[marcus-beta]

1 hour ago, Yoshino Reiji said:

Hi, just want to ask here, is there anyone knows how to extract CG from the game Kishin Houkou Demonbane? It is the ported PC version from the PS2, which came out last year. Because it used NPK engine so i don't know how to extract it, even my friend so far tried all methods but none of them work either. Much appreciated if anyone can help.

https://vndb.org/r61822

Try this, you need found the game key before, this have a tool to that too.
then put the game key using the command line arguments
https://github.com/marcussacana/MwareStuff
That said, maybe don't works, the VNDB said the engine isn't the mware...

Edited July 27, 2020 by marcus-beta

[Yoshino Reiji]

1 hour ago, marcus-beta said:

Try this, you need found the game key before, this have a tool to that too.
then put the game key using the command line arguments
https://github.com/marcussacana/MwareStuff
That said, maybe don't works, the VNDB said the engine isn't the mware...

Thanks, we will try.

Oh and just asking, do you know how to extract pak files? This is from the game Kishin Hishou Demonbane. My friend used quickbms to extract, he got a 400 MB data file but the images can't load, it just like this.

Hope you can help too.

[marcus-beta]

Generic extension, not sure.

[Craztik]

Anyone have the Eushully unpacker? All links around here are dead, can you fix them please? Thank you. ~

(Also the first post misspelled Eushully as Enshully please fix for future people)

[waterflame]

On 8/2/2020 at 3:02 PM, Craztik said:

Anyone have the Eushully unpacker? All links around here are dead, can you fix them please? Thank you. ~

(Also the first post misspelled Eushully as Enshully please fix for future people)

https://mega.nz/file/yp5DzZxC#esCxDMUhESP9wuD5awpeFwOHWg2n6MrAstIW4dTQ_98 Is this what you're looking for?

[darksshades]

On 8/16/2020 at 1:16 AM, Plastic Memories said:

My problem has been posted in thread above ,if anyone can help with .AGF image repack tool or anything relates to data repack /convert that would be awesome !

 

Its been a while since I used it so I'm not sure If I remember it right.
But I think you could use vfstool. It can convert the agf for you.

The only problem I remember is that on some games the image you get will be flipped vertically. So you had to flip it back when repacking again.
After you edit the images and repack them back into .agf you can just use the bbtools to repack them back normally.

I'll post the download for vfstool here are the old link(on hongfire) does not exist anymore. Please reupload it somewhere as this is a temporary link.
http://www.mediafire.com/file/s6ju6dn6a94sjrf/vfstools.zip/file

If you need any help PM me and we can text over discord.

PS: Not sure if you had this problem but there were a few different versions of bbtools going around. If the tool crashes when repacking the interface(text) scripts, them you probably have the wrong version. (Not sure if there are different versions now but I have 2 different versions here)
PS2: If I remember correctly there were a few games that could recognize the image even if there were not agf when you repack it back. You had to save them as BMP and remove the extension when repacking it back. But not sure if it was a older or newer game now and it might had been an exception.

Edited August 20, 2020 by darksshades

[banos]

New to all this, but Onee-chan no Yuuwaku uses KiriKiri but instead of an .xp3 file, the .ks scripts were in data.bin. I'm not quite sure how to repackage them. Making a new bin file resulted in an error. 

[GuyInDogSuit]

I know this is a long shot, seeing as how morkt hasn't been active for months (since March 30, apparently), but IxSHE Tell (English) just came out on Steam and I can't seem to extract any image files from it successfully. I mean, they DO extract, but then they're not open-able. The archives are in the root.pfs format (Artemis Engine?) and while GARbro recognizes them as archives and allows me to look into them and extract them, the images themselves come out as if they're still.... encrypted. Every other file format works fine, though. For some odd reason, I can only successfully view or extract backgrounds. Audio (ogg) and such plays just fine, too.

Here's a sample of one of the images that doesn't open once extracted:

https://user-images.githubusercontent.com/13460691/91682612-fa4e7b80-eb06-11ea-877c-239e127bdd34.png

Edited August 31, 2020 by GuyInDogSuit

[GuyInDogSuit]

On 8/30/2020 at 9:26 PM, GuyInDogSuit said:

I know this is a long shot, seeing as how morkt hasn't been active for months (since March 30, apparently), but IxSHE Tell (English) just came out on Steam and I can't seem to extract any image files from it successfully. I mean, they DO extract, but then they're not open-able. The archives are in the root.pfs format (Artemis Engine?) and while GARbro recognizes them as archives and allows me to look into them and extract them, the images themselves come out as if they're still.... encrypted. Every other file format works fine, though. For some odd reason, I can only successfully view or extract backgrounds. Audio (ogg) and such plays just fine, too.

Here's a sample of one of the images that doesn't open once extracted:

https://user-images.githubusercontent.com/13460691/91682612-fa4e7b80-eb06-11ea-877c-239e127bdd34.png

 

Bump?

[crymaydevil]

hello, anybody knows any software which can merge composition-ed character sprites like this?

it's from psb files unpacked (from psbpack.dat of ANIM games) using GARbro.

 

here

[GuyInDogSuit]

On 8/30/2020 at 9:26 PM, GuyInDogSuit said:

I know this is a long shot, seeing as how morkt hasn't been active for months (since March 30, apparently), but IxSHE Tell (English) just came out on Steam and I can't seem to extract any image files from it successfully. I mean, they DO extract, but then they're not open-able. The archives are in the root.pfs format (Artemis Engine?) and while GARbro recognizes them as archives and allows me to look into them and extract them, the images themselves come out as if they're still.... encrypted. Every other file format works fine, though. For some odd reason, I can only successfully view or extract backgrounds. Audio (ogg) and such plays just fine, too.

Here's a sample of one of the images that doesn't open once extracted:

https://user-images.githubusercontent.com/13460691/91682612-fa4e7b80-eb06-11ea-877c-239e127bdd34.png

 

Bump?

[GuiraGueen]

N/A

Edited September 19, 2020 by GuiraGueen
nevermind

[GuiraGueen]

Has anybody figured out how to rip sprites from Koikishi Purely Kiss (恋騎士Purely☆Kiss) yet?

https://vndb.org/v7260/

I've tried literally every xp3 decoding software but they either can't extract them or can extract them but extracts them as encoded .tlg files which are impossible to convert.

So far I'm tried:

CRASS (with the advanced options)

xp3viewer

xp3 tools

GarBro

Arc_Unpacker

krkrextract

All to no avail. Does anybody have any idea how I can decode these files? They game isn't listed in the menu of krkr, GarBro or Arc_Unpacker.

[ligigi]

On 2/5/2019 at 3:44 AM, sena_kaito said:

Using this tool,
http://dnrv2.cyriaca.net/makuta/projects/hello/hate2.exe

How to use,
Open CMD on hate2 folder
Then type:
hate2 "input file" "output folder" (without quotes)
hate2 X:\img.dat X:\img (example)
And voila!

Nb: maybe you need turn off your antivirus, in some case, they treat as fake detection.

Good luck!

 

the link is no longer available. can you help me?

[AstralSword]

On 10/9/2020 at 3:49 PM, ligigi said:

 

the link is no longer available. can you help me?

Is this still related to Sorcery Jokers? I looked back to the original post, but I can't really check if this is what you want to extract data from.

[ShiroChikara]

Hello my friend, i cannot go on the page of Tlwiki, is down the server¿ can you help witch that¿

[loam]

I heard that some were having trouble extracting .dat files that come with Aokana and AokanaEXTRA1, despite it apparently being based on another known format from what I've heard.

Here's a tool that allows you to both unpack .dat files to folders and folders back to .dat files that will work with the game if you decide to modify its contents. It supports both the PC and Switch formats. Early versions of this tool were used to make the Aokana +18 patch for the switch, which consisted in simply unpacking the PC +18 files and pack them into the Switch's version format.

Binary: http://www.mediafire.com/file/0zgkf38g3dswoy6/AokanaDatTool.zip/file

Source code (C#, .NET Core): http://www.mediafire.com/file/51oelx9qq91gh6s/AokanaDatToolSrc.zip/file

Edited November 10, 2020 by loam

[iGSfYMkbtk]

On 11/8/2020 at 6:49 AM, loam said:

I heard that some were having trouble extracting .dat files that come with Aokana and AokanaEXTRA1, despite it apparently being based on another known format from what I've heard.

Here's a tool that allows you to both unpack .dat files to folders and folders back to .dat files that will work with the game if you decide to modify its contents. It supports both the PC and Switch formats. Early versions of this tool were used to make the Aokana +18 patch for the switch, which consisted in simply unpacking the PC +18 files and pack them into the Switch's version format.

Binary: http://www.mediafire.com/file/6ssm4q8m2kb1qsi/AokanaDatTool.zip/file

Source code (C#, .NET Core): http://www.mediafire.com/file/wq6cn4j6zfrr74t/AokanaDatToolSrc.zip/file

Hello!
I tested the program and found a problem about "repack" part of this tool:
The output *.dat file seems not be accepted at all or loaded correctly by Windows-Steam version of both Aokana and Extra 1!
The CGs (or even maybe sounds and scripts too?) inside repacked part of the game shows up as gray screen,
Even without modifying anything and then just straight up pack them back immediately!

Proven by unpack all of the 12 unpackable *.dat files in "AokanaEXTRA1_Data" from Windows-Steam Extra 1,
then pack them back with this tool, replace the original *.dats,
then the game won't even start the startup sequence after launch the game...(whole screen just went black.)
Commandline used: "AokanaDatTool.exe" --pack --input "unpacked_root_folder" --output "repacked.dat"

I also tested to unpack the repacked *.dat from this tool back to a different folder,
then compares with the unpacked file hashes from the original *.dat, and nothings different...
Then i tested another known working script from GitHub:
https://github.com/hktkqj/Aokana-Extractor
Use that to unpack the original games *.dat to another folder again,
compares with the result from this tool but again... All of the files hashes are the same...
So the "unpack" part of this tool is ok.

(BTW, the unpacking speed of this tool is significantly faster than that script from GitHub mentioned above!
Great job man! Sorry for my bad english...)

Edited November 9, 2020 by iGSfYMkbtk

[loam]

14 hours ago, iGSfYMkbtk said:

Hello!
I tested the program and found a problem about "repack" part of this tool:
The output *.dat file seems not be accepted at all or loaded correctly by Windows-Steam version of both Aokana and Extra 1!
The CGs (or even maybe sounds and scripts too?) inside repacked part of the game shows up as gray screen,
Even without modifying anything and then just straight up pack them back immediately!

Proven by unpack all of the 12 unpackable *.dat files in "AokanaEXTRA1_Data" from Windows-Steam Extra 1,
then pack them back with this tool, replace the original *.dats,
then the game won't even start the startup sequence after launch the game...(whole screen just went black.)
Commandline used: "AokanaDatTool.exe" --pack --input "unpacked_root_folder" --output "repacked.dat"

I also tested to unpack the repacked *.dat from this tool back to a different folder,
then compares with the unpacked file hashes from the original *.dat, and nothings different...
Then i tested another known working script from GitHub:
https://github.com/hktkqj/Aokana-Extractor
Use that to unpack the original games *.dat to another folder again,
compares with the result from this tool but again... All of the files hashes are the same...
So the "unpack" part of this tool is ok.

(BTW, the unpacking speed of this tool is significantly faster than that script from GitHub mentioned above!
Great job man! Sorry for my bad english...)

Hey, thanks for reporting this. The bug was caused because the paths were being written with a trailing \ into the .dat file when packing them, because my code is removing the base path without considering that and just removing what you give it in the --input. If you use --input "unpacked_root_folder\" with the broken version, you will notice it will work properly. I used this tool like that while making the Switch +18 patch. This is clearly not the intended behavior, so I've fixed it:

Updated download links:

Binary: http://www.mediafire.com/file/0zgkf38g3dswoy6/AokanaDatTool.zip/file

Source: http://www.mediafire.com/file/51oelx9qq91gh6s/AokanaDatToolSrc.zip/file

Thanks for letting me know!

PS: I didn't knew a python script existed out there already, but it's clear there wasn't an attempt to understand what the code actually did. I tried to make the code as clear as possible with meaningful variable/function names so I hope somebody finds that useful.

Edited November 10, 2020 by loam

[GTX1080Ti]

Hi everyone. I'm trying to extract the opening/ending "image sequence" or raw file from the games, Yosuga no Sora, Haruka na Sora, Natsu no Ame.

It seems the opening is stored in the opening.mei file (the file name kinda gives it away). By using GARbro I am able to see a stream of images inside, however its not able to extract it in png format. Is there any tool which can deal with this? A quick google led me to believe this is Entis/ERI format, but literally no pointers to how I can get something out of it.