I invested quite some time into figuring out, how to decode the files containing the different camera views for all the scenes in Ecstatica. Well, I finally got somewhere, after some unsuccessfull attempts. At least those attempts made me learn a lot new things about Huffman codes, the LZW algorithm and RL enconding. The amazing game mod wiki helped much again.
The phenotypic facts
I rather quickly found out that:
- The scene files are much to small to contain a raw pixel representation.
- Changing a byte somewhere in the file sometimes affected a single pixel in the game, sometimes a sequence of pixels
- All changes had only local impact, just sometimes the decoding algorithm in Ecstatica seemd to crash and omitted all following pixels. Amazingly this didn’t make Ecstatica crash. Congratulations to the programmers for this stability!
Could it be LZW?
The wikipedia article about the LZW algorithm is quite insightful. After reading it and reconsidering the facts, I came to the conclusion, that it its propably not LZW. Why? As noted above, changes in the pixel values have only local impact. This is also true for the first few pixels. But in my opinion changing values in the first stage of a compressed LZW data stream should have global impact on the decompressed stream. For example:
Could it be RLE?
Well, could it be RLE? I learend a lot about the algorithm from the wikipedia article about the PCX format and the respective modding wiki article. What made me doubt that we have RLE here was, that in my favourite HEX editor, I could see no connection between the most-significant bits and the impact of a value change.
So what is it?
It is some kind of RLE. A special form of RLE: T’here are color bytes and there are RL bytes, like in the standard RLE. The RL bytes are structured in the following way:
- The 6 most-significant bits are the run-length, let’s call the value L
- The 2 least-significant bits are flags.
The flags have the following meaning
|bit #0||bit #1||meaning|
|0||0||L pixels. Each pixel’s color is determined using the previous pixel’s color and a 4bit value giving the relative change of color|
|0||1||L pixels of the same color|
|1||0||L pixels whose colors are represented absolutely using 1 byte each. If none is set, there are following L pixels in which each pixel|
The RLE data starts at position 10 (decimal) in the file.
Is that all?
No, absolutely not. What I was able to decode was merely a part of the scene files. There’s a lot more in there. Some candidates
- z-Values. In order to correctly let walls occlude characters, there has to be a Z map or something like that.
- Camera properties. To be able to render the characters correctly, the camera properties (focal length, etc.) have to be known.