mnakamura

Ok, could you please name the hex editor that could show me at least zlib, AES, 3DES substreams (preferably whole lot more, like RLE, LZ/LZW/LZSS, etc)?

98% is just a translation. I've just described my way. If you're able to do more than me just by applying your brain — congrats, you're probably smarter than me. I'm not really comfortable with doing lots of repetitive hex arithmetics in my head or applying zlib decompression / XOR / bit shifts / rotations / etc just by looking at the hex dump.

Hi, fellow VN explorers, I've just published an article on basic reverse engineering of unknown file formats of visual novels. It requires little technical knowledge (actually, everything except for the optional latest part can be done without programming) and it tackles the basic process of discovering information about simple container format and exploring its structure to extract files from a game archive. Hope it would be useful Feel free to ask any questions.

You see, my focus is not Mikan engine, but the .story format it uses. Engines come and go, but the solid interchange format is here to stay. Like, Gutenberg project collects ebooks in plain text format since 1971. How many platforms it outlived? I guess half a dozen already. Today it's HTML engine, tomorrow someone might want to do a Unity-based engine (that will still run off the same .story JSON), or a native Android/iOS engine, or whatever. I'm not really a programmer, I'm more like a reverse engineer, so I'd be happy if someone more proficient in JavaScript took over the engine itself (and actually it's happening right now). Work on customizable UIs proceeds, actually, Noratoto port already has an option to use UI that looks pretty similar to original.

Well, here you go again - another title named just of the the blue - and you've already tried and checked it To be honest, I've really just wanted to take a look at it without purchasing a real PSP while the series were on air, but as the series concluded (IMHO, in a very boring way), I kind of lost interest to even trying that.

Of course, that comes first. I still have a solid backlog of things I've already promised, so probably I won't even touch Doddler's engine in a month or so. You're really modest Still, you seem to have at least touched virtually everything I've tried so far And still I haven't tackled basically anything somewhat exotic (like, for PS games I've only barely touched 2 titles: Gran Turismo and Subete ga F ni naru). From what I've seen there, it's all pretty complex - i.e. custom sound formats, custom image formats, weird multi-layered textures + embedded 3D models for even the simplest stuff like overlaying the eyes over face sprite, etc, etc.

You know, in my day life, I'm a team leader in an infosec division of a large Japanese corporation. I do know a thing or two about time management, risk evaluation, delivering the results, fulfilling the promises and meeting the deadlines. If we're choosing the tool for a new, not-so-well-known area of work, it's only natural to spend some time to do preliminary research. It could take a week or two, but ultimately you would be choosing a tool X because it "performs better", "has larger userbase (thus we can just hire them and save on the training)", "has best SLA", "can be super-quick for prototyping purposes", etc, etc, and *not* because "I only know X and screw everything else". I've already been through multiple situation when this time spent on research paid off well. Anyway, I guess it's kind of pointless and off-the-topic to discuss who's more "mature" or whatever. The best thing I've got from this conversation is probably a new challenge for me: to try doing "Really? Really!" port for Mikan

Wow, what else can I say? Thanks! I feel really lucky that I've met you. I admire your skills, hands down. You always seem to have answers for everything I just keep wondering if there's anything you haven't tried reversing yet?

I'm not sure that Doddler aims to "recreate 100% same experience". A quote from the 2014 article: "In the original, a shattering sound played and the screen dropped away piece by piece, but I found it very difficult to pull off that effect in Unity as I would need to do pixel-by-pixel manipulation of a texture (something that’s really slow to do). So instead, I came up with my own effect. I used a unity asset store script “Ultimate Fracturing and Destruction” to create an object that I could apply a snapshot of the screen to, and then used the built in Unity physics to handle it falling away." Unity is somewhat high-level and obviously has its pros and cons. Actually, it's pretty similar to HTML + WebGL in that aspect (save for the fact that Unity obvious has *tons* of ready-made effects / assets / scripts / etc already implemented). It would be hard to achieve some effects, so substitutes some naturally. Some of them might actually look better, but it's still not "100% same". I haven't yet reverse-engineered any of the Doddler's ports to check out whatever's inside, but it doesn't look like that he's "keeping the original script", as you mentioned. 2015 article mentions that he actually does some compilation into a bytecode and conversions of the original script.

Probably you took much more effort that me, seeking all these emulation / substitution tools for mouse, custom keyboards, etc. Using Windows tablet to read while commuting was my dream like a year or two ago. I've also tested ~60-70 VNs on Windows tablet (however, that was Windows 8 ) and my experiences were not that good. Unfortunately, I don't remember exact titles (I've tried lots of low-profile ones I got for cheap at garbage bin sales), but about 5-6 out of 60-70 used bizzare "right mouse click to continue", "left mouse click to step back" scheme, which required you to do right clicks. It is possible with touch, but takes like forever to go on in the manner. Quite a few older games don't actually work in Windows 8. I've managed to run them by installing Windows 98 and XP in VMware, but that (a) heats up the CPU and eats battery like crazy, (b) again, quite bothersome to use touch UI on VMware emulated ancient OSes which have zero knowledge of touch interface at all. Probably your mouse pointer software would have helped here, though.

First of all, I'd like to apologize if I sounded rude anywhere or offended someone. I didn't mean to. I'm sorry, but (a) you're citing May, 2015 article, but I'm citing June, 2014 article, (b) context or outcomes do not actually matter here. If a person chooses a tool blindly, just because he/she knows a tool - that *is* immature. If it turns out that he/she was right afterall - that's great, but that's luck, not a well-weighted decision. Quoting the article: "So that’s how I did it, the whole port done in C#/Unity, over the course of about 4-5 months." Please don't attribute me things that I haven't said and make me look like I'm an evil person. I haven't ever said anything bad about MangaGamer, much less calling them "a bunch of people who has no idea what they are doing" or anything like that. I fully respect them and recognize their role in current upbringing of VNs in the West. They really do their best and I'm in no way competing with their business. All the words about "X is performing better than Y" without measurable proofs carry little weight. You're completely right about different aims: they do business (and consequently do huge PR work), I do not. They earn money (thus they're professionals), I don't (thus I'm not). There are quite a few other aspects beside technical merits of Unity or not. For example, chances are that using Unity makes it much easier to pass all the required tight certifications for proprietary gaming platforms such as PS / XBox / PSVita / Wii / whatever. Or it's easier / cheaper to hire additional help (thus scaling the team) for Unity vs other competitive solutions. You're also completely right about the sizes of the games and amount of testing. Actually, it makes an interesting challenge for me: to take the same Really? Really! that is actually the object of 2014 article, port it to Mikan and measure the time. That's probably the only way to get hard results.

Unfortunately, no tablet would run *all* VNs. There are quite a few older VNs that are meant to be keyboard-driven (and they're full screen and they don't really work well with virtual on-screen keyboard), or, for example, use some intricate thing like right mouse click to continue (which *is* possible with touch-to-mouse emulation, but that's a world of pain to play). OTOH, some of more modern VNs require a relatively powerful videocard with recent shaders & stuff (i.e. they fail to run on integrated Intel chip). There is no silver bullet.

My bad, how could I forgot about that Thanks! I've read that article by Doddler, and, honestly, I don't think that he's all that almighty. I'm sorry, but rationale like "The main reason is, I don’t know Ren’Py. I do know Unity though." sounds like a pretty immature attitude. Of course, things are somewhat easier for them, given that they (probably) have access to original VN sources, but still, their current technologies make them spend like 4-5 months per port. For comparison, I typically one usually in less than a week (KFHD, for example, took me literally 1 evening) and that includes all the reversing stuff. And, yeah, IMHO, Unity's "online" versions have *a long* way to go...

I've missed that, thanks! I've imported first chapters they've completed translating and looking through their tools now. Actually, I'd love to contact these guys and show them my demo. Any ideas on how to do that? xtenext doesn't seem to have e-mail published on Github.

I doubt that it is the case here: probably the team that started the translation in 2011 just faded out years ago, it's not about official translation announcement. In fact, I'm curious if Mangagamer or other official translators would be interested in making use of porting to HTML engine? It could be beneficial to the business, as it dramatically hastens deployment and combats piracy, as it's less trivial to copy a website than to copy a local file(s). Neko* is actually a pretty poor engine from the technical POV, compared to Ren'Py or krkr2. Any image effects support (blur, colormapping, etc) is basically nonexistent, lots of incompatibilities, traditional problems with script operators written in Japanese (yes, they literally use [立ち絵ロード] command to load a sprite), etc, etc. The main pro of the engine is that it *is* a high-level engine.

Just got yet another port request basically running. This time it's Kyonyuu Fantasy - running NekoPack / NekoSDK / NEKOSDK_ADVSCRIPT2 engine. Obligatory screenshot: Most basic things like backgrounds / sprites / texts / voices / background music work. Partial English translation is "made by Zyxas. Special thanks to xtenext." This particular VN is actually quite a number for me, because it's the first one that I'm doing completely blind: I couldn't make it run on any of my Windows desktops, so I just dug in and tried to guess whatever's going on. As always, feel free to request demos. As far as I understand, Mangagamer's official translation is going on to happen somewhere in 2016, so nobody's interested in translating it into English anymore?

Yeah, you're right, and I'm sorry for that. Actually, I've kind of cheated and just used some external program to do unpacking, let me check which one was that. I think it was either Crass or arc_unpacker. Probably the latter. UPDATE: Actually it was arc-reader by Alexander Roper. I'll try to bring up some docs up to date in the days to come. Hopefully it will help understanding what's going on with all that porting stuff.

You know, Mikan is different Prior to inventing .story format, I've seen through various open / semi-open options available (like tjs format, VNDS format, ren'py format) - they are either too primitive to convey modern games (with animations, layers and stuff) properly, or they are too generic (i.e. they are actually a proper Turing-complete programming language - and thus programmers are tempted to invent yet-another-engine-inside-engine with them. That's actually an architectural error: they fail to properly differentiate "presentation" vs "business logic" vs "actual story data". Obfuscation and stuff like that is, unfortunately, blatant ignorance either by programmers or (more frequently) by publishers who still think that publishing a offline DVD in 2016 with some "copy / extraction protection" will stop their game from being pirated and/or reversed.

I just wonder why people keep writing such stupid compilers... And actually why people keep reinventing the wheel and write so many engines in the first place...

BGI is pretty weird too, especially modern BGI with their subroutines. They have like 5 or 6 ways to just change a background, for heaven's sake... And I'm not talking about any fancy transitions, or anything like that. Another fun fact: their compiler always compiles negative literal numbers like -42 as "push 42; push -1; mul". Care to show a bit of N7/E17 arithmetics disassembly?

Wow, ok, https://tlwiki.org/. It seems that they have a whole fanscinating world of web servers configs to learn before them, I guess.

Yeah, you're hit the point with structure. The idea is to have one repo per game (port_*) and one repo per engine (engine_*). *2story is obsolete and they will be joined into engine_* repos. Ruby is not obligatory by any means, of course. As for scripts in bash... Well, do you know of any alternatives? This whole VN reverse engineering tool business is pretty weird, if you'd ask me. Take a look at tlwiki, for example: it's a very diverse collection of scripts in various languages (most of them are just uploaded as is, so no VCS, no change history, no nothing) and binary tools written in hardcore languages like C or C++ (the sources are frequently missing, sometimes they're GUI only). It's nightmare with licenses and copyrights - nobody seems to care. And, the ugliest thing - all these tools are meant to be ran manually. Almost no automation or anything like that exists. The closest possible stuff I've seen is VNDS converters - they're written in Java (and include sources) and they're huge UI applications. Just checked in, Never7Converter is ~400KB of sources, ~12K LOC. In contrast, my noratoto port actually is 287 LOC for the port + 608 LOC for the engine tools < 1K LOC totally. I guess we can try to cater for Windows users later by doing some sort of "converter shell" that would actually bring shell scripts / Ruby / Python interpreters to Windows box and call them in a pretty GUI. Or, screw that, 99.9% of Windows users would just download a ready-made port from torrents anyway. Yeah, thanks for the heads up! I'll put that into Mikan engine README soon.

In actual news, I've finished porting demo of Aiyoku no Eustia today - demo quality (lacks some fancy animations), but playable altogether. It uses newer BGI engine. Obligatory screenshot: Ports and engine-specific tools are published at: https://github.com/mnakamura1337/engine_bgi - engine-specific tools https://github.com/mnakamura1337/port_eustia - Eustia-specific scripts, metadata, etc (basically, they call "engine_bgi" stuff with proper arguments, that's all) More documentation will probably follow. If anyone wants to take a look without doing all the conversion with scripts on your box - PM me, as usual.

As I said, the problem is that he ignores my mails. I don't know if there's a transmission problem, he's busy, or he doesn't want to reply.

Actually, I've tried to contact both ViLE team and Zus multiple times — and I haven't got any answer. Zero activity on ViLE since 2012, lots of unanswered questions on ViLE blogs / comments, and several other people reporting failure to contact suggest that BaSF is no longer active in the community. By the way, someone still pays for vilevn.org domain - and that is probably "Vegard Fiksdal", as per whois info. Zus seems to be alive and kicking (at least he answers questions in a dialog box on tss.asenheim.org), but seems to ignore me. Hey, @Scorp, if you have a solid method to contact him, could you relay a message from me? Or at least confirm that he does not want to have any discussion with me?