Cannot convert given narrow string to wide string

[Kisegami]

I am translating Kami-sama no You na Kimi e. It's using Kirikiri 2 with .ks is the script.

After changing the encoder of .ks file from SJIS to UCS-2 LE, i translate some of the line to vietnamese and then save to test

I use XP3Pack to make a patch, when i start the game, it showed error: "Cannot convert given narrow string to wide string"

Is it the font issue, or something?

[Zakamutt]

Are you sure it's using BOM (byte-order mark)? The real encoding you want is UTF-16 LE with BOM, which is very close to UCS-2 LE but not quite methinks. Yes even if the badly translated sourceforge docs say the opposite. Other than that the other tip I've seen is to convert every file to UTF-16 LE BOMc including all the systme script files.

[Darbury]

Was about to say the same thing. I've run into this problem as well, and saving as UTF-16 LE with BOM always fixed it.

How to tell if you did it right? Crack open your UTF-16 LE .ks file in a hex editor. It should start with "FF FE".

Edited March 1, 2021 by Darbury

[Kisegami]

1 minute ago, Darbury said:

Was about to say the same thing. I've run into this problem as well, and saving as UTF-16 LE with BOM always fixed it.

How to tell if you did it right? Crack open your UTF-16 LE .ks file in a hex editor. It should start with "FF FE".

 

5 minutes ago, Zakamutt said:

Are you sure it's using BOM (byte-order mark)? The real encoding you want is UTF-16 LE with BOM, which is very close to UCS-2 LE but not quite methinks. Yes even if the badly translated sourceforge docs say the opposite. Other than that the other tip I've seen is to convert every file to UTF-16 LE BOMc including all the systme script files.

Any suggest about a tool that can convert all of it to UFC-16 LE? Notepad++ cannot do that @@

[Darbury]

Visual Studio Code and Atom are both fine cross-platform options.

Down in the lower right of their editing windows, they'll show you the current encoding of a file and let you re-encode it before saving.

[Kisegami]

6 minutes ago, Darbury said:

Visual Studio Code and Atom are both fine cross-platform options.

Down in the lower right of their editing windows, they'll show you the current encoding of a file and let you re-encode it before saving.

I have opened the file that encoded with Notepad++ (USC-2 LE) in VSC. But VSC show that it's already in UTF-16 LE. Do i need to convert all of the files (ks and tjs) to that encoding? Even when i haven't translated anything of them

[Zakamutt]

remember that you need to have BOM

[Kisegami]

6 minutes ago, Zakamutt said:

remember that you need to have BOM

How to get UTF-16 LE with BOM. Both VSC and Atom doesn't have that

Edited March 1, 2021 by Kisegami

[Kisegami]

When i try to convert the .ks file from Shift JIS to UTF-16 LE (VSC don't show with BOM or not). It's appear a bunch of code. Is it normal, or have i broke the file?

 

[Darbury]

Welcome to the imperfect world of wrestling with file encodings. Actually, I'm sure it makes perfect sense to all the CS majors out there, but...

Looking back on my project notes, seems like my workflow involved copy/pasting the original Shift-JIS source into a new TextEdit doc (yup, I'm on Mac) then re-saving it out from there as a UTF-16 LE file. Apparently TextEdit did the best job of keeping the BOM where it needed to be.

Anyway, this is the part where you get to experiment and find a workflow that works best for you. Or bribe one of those fancy CS majors into explaining the "sensible" way of doing it...

[Zakamutt]

Fwiw I found this old-ish python 3 script on tlwiki for converting encodings, but it didn't have a way to specify writing the BOM, so I hacked it in. It seems to work? Have fun with the command line tool experience and all that, and remember it will overwrite stuff without mercy if input dir = output dir.

As for how I've done it personally, I basically always used some form of Kotlin program to reinsert text into scripts since it was my comfort language at the time I worked on shinobi harisenbo (which ended up being SJIS still because I was MISLED TO BELIEVE THAT THE BOM WAS WRONG BY THE DAMN MISTRANSLATED DOCUMENTATION ahem that was annoying anyway) and ichigo & kyuugo. It's not particularly hard to output to any reasonable encoding you want in kotlin / java, but the BOM did turn out to be annoying and require me to first write the magic number and then start writing the actual file.