Decoding QFG SAV Import files

[Charles]

Binary: http://blazingstix.com/apps/qfgimporter
Source: https://github.com/Blazingstix/QFGImporter

Hi All, I've started a thread in the QuestForMoreGlory forums about this, but at Collector's suggestion, I'm posting here too, since he has the space to allow attachments... that will make this a lot easier.

So, long story short, I'm trying to fully decode the QFG character SAV file starting with QFG3, and I could use your help.

As a little history (verbatim from my post on QuestForMoreGlory: http://www.questformoreglory.com/forums ... topic=3392)
Way way back when, I wrote the QFG Importer '95... which Collector has graciously been hosting on his SierraHelp.com pages for quite a while. I just recently released the source-code on github (https://github.com/Blazingstix/QFGImporter95), too. I'm pleased that many of you have found it useful over the years, but that program doesn't help me now because the hidden secret of my program from all those years ago is that it never created any files from scratch... it just regurgitated SAV files the games themselves had spit out.

I'm sure many of you have seen the awesome work by Adam Plocher, in the last couple of years, of actually decoding QFG1 and QFG2 (http://www.bitcollectors.com/Project/QfgCharacterEditor). He reverse engineered the field format for the first two games in the series, and I've expanded on his original work by mapping out neary every bit and byte of the QFG1 and QFG2 import files.

I think this is really cool stuff, but not nearly as impressive as Adam's work in actually decoding them in the first place. You can look for yourself and see some amazing hidden secrets of what actually got stored in the import files... things that maybe only the AGDI's had seen before: https://github.com/Blazingstix/QFGImpor ... r/QFG1.txt
https://github.com/Blazingstix/QFGImpor ... r/QFG2.txt.

The format of QFG3 and QFG4 is subtly different from QFG1 and QFG2, but I think I have a large portion of it figured out (I can show more details in a follow-up post). I can clearly identify *where* the skills/attributes are (strength, climbing, stealth, etc.), but I cannot correctly identify their values. And here's where I can use your help... I need as many QFG3 SAV files as I can get, in order to compare them against one another, and help see patterns. It would be most helpful if I could get both the .sav file *and* a screenshot of the stats (and inventory and magic) at the end of QFG3, but if you only have the sav file, that is excellent too.

I can also greatly use QFG1 .sav files because there is one bit of the QFG1 format I haven't identified... literally one bit (i.e. 1/8 of a byte)... the 7th bit of the 5th byte.

Anyway, I cannot guarantee success, nor even expediency... I'm only able to work on this in the little spare time I have, but I could very much use all the help I can get.

Thank you for listening.

[Tawmis]

I'm probably going to state the obvious here...

But couldn't you -

1. Make a new character, and make Strength a specific number (say like 20). Save the game. (So it has no real inventory, ruling out additional placements).
2. Make a new character, put it all in Intelligence, make it that same number. Save the game.

And rinse and repeat - and that way you can see where that same number/value (I know it won't say "20") - within the save game appears in different areas as values?

Once you have figured out the stats - then work on inventory items, etc? That way you're starting with a narrow field, and expanding from there.

[Collector]

I'll try to play through the last bit of QfG1 to generate a SAV within the next few days.

[Charles]

Hi Tawmis, that's actually a really good question. In truth, that's exactly what I did do for much of decoding QFG1 and QFG2, but it's not as simple or quick as it sounds.

Fortunately for me, somebody else had already done the heavy-lifting of decrypting QFG1 and QFG2, so I just had to poke and prod by trial and error to find out what all the remaining bits of the code meant. There is one flag still undefined for QFG1, and that is not related to anything you *have* but is instead something you have *done*. That part is the time-consuming process. I'm hoping to narrow down what it could be by having a large sample-set. I suspect it is related to a magic user (perhaps beating Erasmus at Mage's maze), so with enough data, I can potentially see that only characters with magic have this flag checked... then I can do my own pin-point testing against anything magic-user related and see if that sets the flag.

Secondly, this is the .sav export/import save's I'm talking about, which can only be done once you finish the game... now, I'm not asking anybody to play through the game for me, but rather I'm hoping many of you have previously exported .sav files (as I do) that you can send me... those alone would help tremendously (but if anybody also has the corresponding saved game near the end, so they can tell me the stats and inventory and magic, etc, I will not turn those away).

As with any reverse-engineering or code-breaking, more data is better, and I've already gone through the previously saved games I have. QFG1 and QFG2 were easy to jump ahead to the end-game/exporting by way of debug codes, but for QFG3 I would have to play through the entire game. Not an impossible task, but if others already have saved files and exported characters from that point that they're willing to share, it would be much simpler.

Now, with QFG3 there is the much harder task of not just identifying which bits mean what, but also decoding/decrypting the data.

Again, I do not want anybody going through a whole game from start to finish, but rather I ask you to share any end-game saves and characters you already have.

To give you a quick idea of the process, here is one of my QFG3 Thief exports:

Code: Select all

 051 050 1 1 44b 22f 44b 22f 44b 22f 44b 44b 22f 44b 22f 44b 22f 22f 44b 22f2d392b152b182b182b182b182b182b182b182b182b182b182b182b182b182b182b182b182b182b182b232b222a222a222b1d2b24294a41 e d62 d2d d18 f c d 6

Now, assuming this is two bytes per value, I can break it down into groups accordingly:

Code: Select all

0051 0050 0101 044B 022F 044B 022F 044B 022F 044B 044B 022F 044B 022F 044B 022F 022F 044B 022F 2D39 2B15 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B18 2B23 2B22 2A22 2A22 2B1D 2B24 294A 410E 0D62 0D2D 0D18 0F0C 0D06

And applying a similar decoding as with QFG1/2 (using 0x0053 as the initial cypher) I get

Code: Select all

0002 0001 0151 054A 0664 0664 0664 0664 0664 0664 0000 0664 0664 0664 0664 0664 0000 0664 0664 2F16 062C 000D 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 003B 0001 0100 0000 013F 0039 026E 6844 4C6C 004F 0035 0214 020A

I can already draw several inferences from this data... to start with, the first word is 0002 (0000 is Fighter, 0001 is Wizard, 0002 is Thief, and 0003 is Paladin), which means Thief. Words 5 -> 19 are the skills... I can see that word 10 (which is Parry) has a value of 0, as does word 17, Magic. The rest of the skills are 300, which I know from outside knowledge (i.e. loading into QFG4). So how does 0664 == 300 ? Is it a problem with my decoding algorithm, or some other factor I haven't identified yet?

All of this leads me to think I need more data, and honestly, I'm kinda hoping that posting this in a public forum will also lead to a different way of thinking, and hopefully a bit of needed inspiration.

[Collector]

Have you considered contacting Adam Plocher for any ideas he may have? There are a couple of members here that might be able to give some insights, if they see this thread.

As to contributors, it is a good excuse to replay the game .

[Qbix]

Oh that sounds like fun!
I decoded a few save formats (non-sierra games) and it can be very challenging, but fun.

Please post your decoded data fragments.


Could you do the decoding with 0x51 instead of 0x53 ?

[ghost_rider]

Hi Charles,

Here are the two .SAV files with corresponding stats screenshots. If you have any questions about them just let me know.

Cheers

[gumby]

Charles, can you give us a couple more examples, besides 0664 = 300 points? If you could provide values across the 300 point spectrum it would be helpful. Say (approximately) 1, 50 , 100, 150, 200, 250, 299? The more the better, if they are handy. Closely grouped values would be helpful too (eg. 50, 51, 52)

[Charles]

ghost_rider:
Thank you very much for those two files. I'm finding them very helpful already.

Collector:
I posted a comment to a post on his blog, back in November/December when I started on QFG3, and he was very encouraging, but I got the distinct impression that he has a few other things on his plate at the moment. That said, I would love to have his input, especially how he cracked the encryption on QFG1/2.

qbix:
I can try with a 0x51 but I'm doubtful that will give an improvement... the encryption for QFG1/2 is basically EncryptedByte(N) = Byte(N) XOR Byte(N-1), and you use 0x53 in place of Byte(-1). This still works in QFG3 (if I use 16-bit Shorts instead of 8-bit Bytes)... sorta... it sure gives me the first values I expect to see... and it's giving me the 0 values I expect to see... maybe it's because I'm using big-endian bytes, and perhaps they should be little-endian... even then, I'm not getting the values I expect...

gumby:
The trouble is there has to be something wrong with my decryption assumptions in the first place, because it's not always 0664 = 300... in that one file it is, but in the Wizard saved file that ghost_rider donated 0630 = 300, 021e = 264, 0230 = 214... I suspect there's something wrong with how I'm adding two bytes together, but I can't fathom what...

Hopefully I'll be able to set aside some time later this week to work on it... it's like putting together a jig-saw puzzle, or working on a Rubik's cube... it's fun, and challenging... but it can be super-frustrating before it's done.

[Charles]

Sorry for the double-post, but I've made a breakthrough...

In QFG1/2 they used a really weird way of storing the currency... they used two bytes, but not the full two bytes. Instead of a standard way of storing two bytes, where AAAA = BB CC (where BB is the top 8-bits and CC is the bottom 8-bits) they made BB = value / 100, and CC = the remainder.

It turns out they did the exact same thing for QFG3 (and I assume QFG4)... after encrypting the values using the same EncryptedShort(N) = Short(N) XOR Short(N-1) they would split the Short up into EncryptedShort(N) / 100 and the remainder.

The problem is, this allows the possibility of an overflow condition (most notably with the Experience value), and that causes some QFG3 files (and many QFG4 files) to have longer data-lengths than they should.

I'm now able to decrypt any QFG3 files has a data-length of 208 characters (excluding character name) — I'll work on compensating for the overflows, but my first task is identifying the remaining unknown Shorts, like inventory, etc.

Regardless, I'm confident this is now enough information to be able to generate fully-compatible QFG3 character exports.

ghost_rider:
Can you confirm the skill values for your Wizard character's magic?

• 163 Open
• 32 Detect Magic
• 44 Trigger
• 24 Dazzle
• 73 Zap
• 28 Calm
• 259 Flame Dart
• 55 Fetch
• 180 Force Bolt
• 17 Levitation
• 89 Reversal
• 7 Juggling Lights
• 24 Lightning Ball
• 68 Summon Staff

And tell me how much currency you had (46 royals?), and which inventory items you had with qty 1, 3, 15, 17? (There should be at least a couple with qty 1).

[Collector]

Here is a QfG1VGA sav

[Charles]

That's an excellent saved character, Collector. It has the unidentified flag checked (What I'm calling Flag 7).
I can see that you had 120 Gold (including the reward at the end of the game), 1 Dagger, 11 Healing Potions, 9 Magic Potions, 12 Vigor Potions, 1 Undead Unguent.
You also had the Broadsword, Chainmail Armor, Lock Pick, Thief's Tool Kit, and turned Baba Yaga into a frog to become the Hero of Spielburg.

Can you recall any other notable events you did along the game? I believe Flag7 is something you did, rather than something you had.

I was hoping it was something related to defeating Erasmus in Mage's Maze, but you don't have the Dazzle spell, and your Magic spells are low enough that it looks like you only even used Open once or twice.

I've previously checked if Freeing or Killing the Baronet flagged it, or Splitting the Antwerp, and it wasn't those ... did you defeat Toro?

[Collector]

I am sure that I did just about everything but the Mage's Maze and robbing the old lady. I am sure that I freed the Baronet. I did the Dryad task. The only cheat I did was the mushroom from the fairy ring, but that is not modifying the game, just exploiting a small bug. I don't remember all as it came from a save game that I made at the end from a game that I played years ago.

[Charles]

Okay, so I've pretty much finished writing a complete QFG Export Editor, for QFG1-4 (QFG5's export file is an entirely different beast, and one that I don't have any interest in examining at the moment).
You can download it from: http://blazingstix.com/qfg/QFGImporter.zip

You can open, edit, and create from scratch export character files that are identical to those exported from QFG1-4.
There is even a feature to detect if your file is one of the difficult to open files, and fix the problem.

The program is functionally complete, but there are still a couple things I'd like to do to it over time:

• some GUI changes, maybe add more graphics (character class, inventory items)
• fill in the gaps in knowledge (i.e. QFG1 Flag7, QFG3/4 Unknown Item 1-3, all of QFG3/4's flags)

But for the time being, I'm kinda burnt out on this program.

If anybody tries it and finds any bugs, or has any suggestions, I'm listening. Enjoy!

[Collector]

I'll try to find some time to take a look. Would there be any point with QfG5? Unless it could be re-imported back in itself or one of the earlier games, what would ever do with it?