3D Lemmings file formats

[Pooty]

Hello, you. /waves

So, I've been busy for the last week or so. I've been trying to understand the technical stuff that makes 3D Lemmings work. I've made a good amount of progress, but I'm going to have to cut it short for now as I must focus on my university studies for the next few weeks.

I'll dump what I've learned so far in this thread. I've even made a level that you can try out, which I've posted in the level design forum. The level was planned using Minecraft because imagining a puzzle in 3D was a lot harder than I expected. Also, the level was made solely through hex editing, so I needed something to reference as I was going along.

Anyway, here's what I know so far. These aren't particularly tidy notes since these were made on the go, but I'm hoping that if you squint your eyes, you can make some sense of them.

[Edit 2019] This post has been updated with notes compiled by me and namida. More detailed notes have been put up in the L3DEdit repository, and may be more up to date than what's listed here.

===Technical overload begins here===

LEVEL.###
This file is in the LEVELS folder, and is where almost all of the level data is stored. Normally, this file is compressed using the RNC compression algorithm version 1. It's a common enough algorithm, so any tool that can decompress RNC1 files will work here. Also, the file will load whether it's compressed or not. The uncompressed file size is around 65Kb, which is not a lot these days.

Level Information

Code Select Expand

Integers are stored in little endian.

0000      Byte                 Timer (Seconds)
0001      Byte                 Timer (Minutes)
000A-001B Struct (2 members)   Skill ID? (Byte), Number of this skill (Byte)
0022-???? Struct (3 members)   LandVertexX (Byte), LandVertexY (Byte), ??? (Byte)
00E2      Int16                Release Rate
00E4      Int16                No. of Lemmings
00E6-00E7 Int16                To Be Saved
00E8      Byte                 Texture Set
00E9      Byte                 Land Graphic
00F0-010F String               Level name (String requires null termination)
0110      Byte                 Non-interactive object set
0111      Byte                 Sign Set
0112      Byte                 Sea Graphic
0113      Byte                 Animated non-interactive object set
0114      Byte                 Interactive object type (e.g. traps)
0115      Byte                 Sky Graphic
0117      Byte                 Walls Set
011D-013C String               Comment (Not used in game, often used to store level author)
0143      Byte                 Theme index (see "Music & Wallpaper" section)
0151      Byte                 Music index
0153-0156 Array (4 elements)   Level border definition (kills lemmings that leave the border). Order: -X, -Z, +X, +Z
0159      Byte                 Preview Camera Pivot Y
015A      Byte                 Preview Camera Pivot Z
015B      Byte                 Preview Camera Pivot X
015C      Byte                 Kill ceiling (level border definition for +Y)
015D-017C Struct (5 members)   Initial camera positions (see "Camera Positions" section)
0200-7DFF                      Level Block Data
8200-FDFF                      Objects Data

Refer to the attached image to see how some of these relate to the objects seen in the game.

You'll notice there are gaps in there. There are plenty of things I couldn't figure out. For example, I don't know why the water moves the way it does. In level 2, the water is animated. I couldn't work out how this could be done.

Camera Positions (015D-017C)
There are four cameras. Data for each camera is stored in 8 bytes of data as follows.

Code Select Expand

Integers are stored in little endian

+00   Signed Int16  Camera Y Position
+02   Signed Int16  Camera Z Position
+04   Signed Int16  Camera X Position
+06   Byte          Camera rotation. 4 values.
+07   Byte          Unknown


Level Block Data (0200-7DFF)
Each level block data element is stored as a single 16-bit integer in little endian.

Code Select Expand


[MSB] XXXXXXYYYYZZWWWW [LSB]

X   6 bits   Block graphic ID (refer to BLK file)
Y   4 bits   Block type
Z   2 bits   Rotation. The block will rotate counterclockwise (90 degrees * Z)
W   4 bits   Block segments (blocks are split into 4 vertical segments from top to bottom, and all true bits must be consecutive)


The order in which the blocks are stored are along the Z axis (blue), Y axis (green), and then the X axis (red). The level size is stored in memory as 32 blocks long, 32 blocks wide, and 16 blocks tall. However, there are glitches in the game that prevent the full field from being used.


Objects Data (8200-FDFF)
The objects data also uses 2 bytes to store data, and the order in which the game objects are stored are exactly the same as the level blocks. The definition of the two bytes, however, are different. Editing the second byte has done nothing as far as I've seen, so the following information relates only to the first byte.

Code Select Expand


00     Draw nothing
01-1F  Static Object (from OBJ.xxx)
20-3F  Signs (from SIGNS.xxx, first 6 bits = type, last 2 bits = rotation)
40-4F  Unknown
50     Animated Object (from ANIMOBJ.xxx)
51     Animated Object using four large Static Object frames (scrolling through frames 05-08)
60-67  Interactive Object (from TRAP.xxx)
70-C7  Wall (7#-C# represents the direction)
D0-E7  Half Wall (D#-E# represents the direction)
F0-F7  glitchy wall


Music & Wallpaper
Memory address 0143 is where the music and the wallpaper data is stored per level. The wallpaper refers to the image that you see before you start the level. If you've played the PlayStation and Saturn versions of the game, this image will also appear after you finish the level (win or lose).

Code Select Expand


01  Castle
02  Egypt
03  Space
04  Sweets
05  Golf
06  Computer
07  Army
08  Maze
09  Circus
0A  Lemgo (coloured blocks)

The alternative songs for each theme can be chosen at the memory address 0151. From what I've seen, values from 00-03 are accepted. Other values can result in some strange glitches.

BLK Files
These files are also stored in the LEVELS folder, and are linked to the level with the same file extension, e.g. LEVEL.000 is linked with BLK.000. This file represents a dictionary of blocks that can be placed in the level. There are 64 block definitions per level.

Some block indexes have special rules.

Code Select Expand


Block 0     Entrance. This requires the top two segments to be solid and the bottom two segments to be empty. There is an effective limit of 4 entrances per level.
Block 1     Exit. The lemmings interact with the +Z face unless the block is rotated.
Block 2     Splitter block.
Blocks 3-5  Unknown
Block 6     One-way block. It does not automatically put in any one-way block graphics, and they must be implemented using the Face Data below.
Blocks 7-8  Unknown


In official levels, normal block data begins from index 09.

Each block consists of 22 bytes.

Code Select Expand


00-01  Unknown
02     Block flags
03     Unknown
04-06  Data for +Z face
07-09  Data for -Z face
0A-0C  Data for +X face
0D-0F  Data for -X face
10-12  Data for +Y face
13-15  Data for -Y face


Block Flags (offset +02)

Code Select Expand


01   Double-sided rendering. The inside of the block will be rendered with the same textures as the outside of it (example: bridge in level 10).
02   Steel. The block is indestructible.
04   Liquid. Lemmings will drown if standing on this block.
08   Unknown
10   Non-solid for lemmings. The camera interacts with this block, but the lemmings pass through it.
20   Unknown
40   Slippery. Lemmings will slide on the top of this block.
80   Non-solid for camera. The lemmings interact with this block, but the camera passes through it.


Face Data (Offset +04-15)
These represent the graphics that each face of the block will use.

Code Select Expand


+00   Texture Index
+01   Texture Modifier Flags
+02   Shading


The graphics used by each face is stored in the level's texture file. The offset for each graphic is set to the value of the Texture Index multiplied by 4096 (0x1000). However, if any of the flags below are set, this can change the behavior in a currently unknown way.

Texture Modifier Flags

Code Select Expand


01-10   Unknown
20      Face will animate between 4 consecutive face textures.
40      Face will animate using special rules (see below).
80      Face will treat colour 0 as transparent.


The special rules of 40 depend on the value given for the Texture Index. Any indexes not listed below are unknown.

Code Select Expand


00      Four-frame animation from frame 12-15
02      Eight-frame animation from frame 00-07
06      Seems identical to 02


Shading
This byte reduces the brightness of the face texture.

Palette Data
The in-game palette is stored in a file called GFX/LM3D.PAL. The file consists of 256 RGB triplets. The colour values range between 00-3F.

Other files in the GFX folder (*.000, *.001, etc.)

Other files in the folder are stored as a single channel image. By linking these files to the palette, you'll get the graphics as they're seen in-game. The width of each type of graphics file are as follows:

Code Select Expand


AnimObject  64
Bgrd        320 (it's the "3D Lemmings" background that you can see in the main menu)
Land        128
Obj         64
Sea         64 (not confirmed)
Signs       64
Sky         1024
Texture     64
Traps       64
Walls       64


The contents of each file is described below.

AnimObject

Code Select Expand


000   Egyptian Torches
001   Flag bearing the Clockwork Games logo
002   Flashing lightposts
003   Rings of fire
004   Lemming flag
005   Another lemming flag (different from 004)
006   Chequered flag
007   Candy cane torches
008   Neon exit sign
009   Exit flag
011   Pillar torches
013   Juggling lemmings
022   Rocket flame
023   Duplicate of 001
090   "Block Here"
091   "Turn Here"
092   "Bomb Here"
093   "Build Here"
094   Duplicate of 093
095   "Bash Here"
096   "Mine Here"
097   "Dig Here"
098   "Climb Here"
099   "Float Here"


Land

Code Select Expand


000   Dirt / sand
001   Grass
002   Black diagonal tiles
003   Red and yellow triangle tiles
004   Lemgo Red
007   Chocolate
008   Circuit board
009   Grass 2
015   Grass 3
017   Chocolate 2
018   Dirt / sand 2
028   Blue
038   Solid blue


Obj

Code Select Expand


000   Pyramid
001   Castle
002   Maze
003   Circus
004   Lemgo
005   Maze
006   Army
007   Candy
008   Maze 2
009   Golf
010   Pyramid 2
015   Maze 3
016   Army 2 (Snowy)
018   Computer
038   Pacman Ghosts
095   Warning Signs


Overlay
This file stores the cracks that you see when a lemming is bashing, mining or digging through terrain. It also masks the terrain debris that appear during these processes so that they don't appear so blocky. These are both stored as 1 bit-per-pixel images.

Code Select Expand


0x0000 - 0x1FFFF   Terrain cracks
0x2000 - 0x2FFFF   Debris masks


Sea

Code Select Expand


000   Quicksand
001   Dark blue water
002   Normal blue water
003   Green slime
004   Lemgo water - blue
005   Lemgo water - red
006   Icy water
007   Liquid chocolate


Signs

Code Select Expand


000   Pyramid Splitter, Credit Signs ("One Giant Step", "Oh No", "Lemmings 3D") - front only
001   Castle
002   Credit Signs ("Programming By" x2, "Graphics By", "Raytracing By") - front and back
003   Credit Signs ("Produced By", "Eclipse", "Sony", "Playstation") - front and back
004   "Paradise This Way" replaces "Keep Off Grass", Credit Signs ("Hi Ho", Waterski, "Oh No" (dup), Juggle) - front and back
005   Maze
006   Army
007   Candy
008   Duplicate of 04
009   Castle 2, also has a LemLab-looking splitter entrance?
011   Castle 3, mostly identical to 09 but "Keep Off The Grass" isn't replaced and has Castle splitter instead of LemLab-y
095   Maze 2


Sky

Code Select Expand


000   Pyramids
001   Castles
002   Space
003   Trees
004   Lemgo constructs
005   Stone wall with arches
006   Ruins? (usually used with Army theme in official levels)
007   Plain sky
008   Space Red
009   Golf course
012   Space, bigger stars
016   Snowy trees
017   Chocolate mountains
018   Space (blue)
027   Pinkish sky


Texture

Code Select Expand


000   Pyramid
001   Castle (???)
002   Spaceship
003   Circus
004   Lemgo
005   Maze
006   Army
007   Confectionary
008   Computer
009   Golf
010   Pyramid 2
011   Castle 2
012   Spaceship 2 (/ Lemlab?)
013   Circus 2
015   Maze 2
016   Army 2 (Snowy)
018   Computer 2
019   Combination of Pyramid and Golf?
021   Castle 3 (???)
022   Space 3 (Dot to Dot)
028   Computer 3 (Breakout?)
036   Army 3 (Corkscrew Digger)
038   Computer 4 (Lemmtris?)
046   Golf 2 (Land Ahoy)
048   Computer 5 (Lemmtris?)
095   Maze 3


Traps

Code Select Expand


000   Flame blower
001   Squasher
002   Bear trap
003   Trampoline
004   Spring
005   Teleporter
006   Grabby slime trap
007   Laser trap
008   Rope slide


Walls

Code Select Expand


000   Pyramid
001   Castle
002   Space
005   Maze
007   Pyramid 2 (seems identical to 1)
008   Computer
011   Castle 2 (???)
018   Digits


BombNumb.gfx
This file contains the graphics for the bomber counter, the camera number label, the '?' when a lemming runs out of builder bricks, and the highlight lemming cursor. It is arranged in this format:
12
34
5?
↓

Cogs.gfx
The cogs appear when the game is loading. I think the image width is 88 pixels, but I'm not certain. Also, I'm not sure where the palette data for this comes from.

Lemmings.fnt
This 32 pixel-wide file is where the animated lemmings used in the password screen comes from.

Minilemm.gfx
This file is also 32 pixels wide, but this file represents the lemming icons at the bottom of the game screen.

[GuyPerfect]

This is awesome! I love the technical expertise we have in this community.

[Prob Lem]

I'm not a super-ultra-technical type, but this makes for fantastic reading nonetheless. http://www.lemmingsforums.com/Smileys/lemmings/thumbsup.gif" alt=":thumbsup:" title="Thumbs Up" class="smiley" />

I'm still hoping to obtain rips of the sprites for the lemmings themselves in, say, PNG format. Is that any closer, now?

[ccexplore]

I'm still hoping to obtain rips of the sprites for the lemmings themselves in, say, PNG format. Is that any closer, now?

Well, between the info on compression, palette data and "other files in the gfx folder", at least some graphics and textures can definitely be extracted.  Not having played the game though, I'd think for the lemmings, you'd need the 3D models as well?  If so then that information is currently not documented it seems.

I've even made a level that you can try out, which I've posted in the level design forum. The level was planned using Minecraft because imagining a puzzle in 3D was a lot harder than I expected. Also, the level was made solely through hex editing, so I needed something to reference as I was going along.

Minecraft actually sounds like a very good idea.  Like 3D Lemmings, I haven't actually played Minecraft, but based on what little I know about it, sounds like it might be plausible to make use of Minecraft as a poor-man's level editor so you don't have to write one from scratch, as long as someone implements a way to convert from Minecraft representation to the Lemmings 3D game files' representations?  (Granted, I don't know how much information there is out there on Minecraft's own file formats.)

[Prob Lem]

Not having played the game though, I'd think for the lemmings, you'd need the 3D models as well?

In Lemmings 3D, the lemmings don't have 3D models - just sprites, shown from half a gazillion* angles, that are scaled as needed. http://www.lemmingsforums.com/Smileys/lemmings/tongue.gif" alt=":P" title="Tongue" class="smiley" />

*Totally scientific, I swear!

I'm guessing that they're contained in the files lemm.mhc and lemm128.mhc (the latter, I assume, being the high-resolution lemmings), from the directory lemm.

[Pooty]

I'm guessing that they're contained in the files lemm.mhc and lemm128.mhc (the latter, I assume, being the high-resolution lemmings), from the directory lemm.

Excellent guess. I think you're right. http://www.lemmingsforums.com/Smileys/lemmings/thumbsup.gif" alt=":thumbsup:" title="Thumbs Up" class="smiley" />

The demo and winterland versions of the game contains a surprising amount of uncompressed data. lemm128.mhc is not available on those versions, but lemm.mhc is.

I opened it up in Paint Shop Pro as a raw image of 26 pixels wide, and...
http://i144.photobucket.com/albums/r188/Pootyechidna/LEMMMHC.png" alt="" class="bbc_img" />

Almost there, but I can't quite get it right. Maybe the rest of the data in the file is important somehow?

Edit: I've just looked at the contents of OBJ.038.
http://i144.photobucket.com/albums/r188/Pootyechidna/OBJ038.png" alt="" class="bbc_img" />

[Prob Lem]

Whoah! Could there have been some sort of Pac-Man knock-off theme at some point? http://www.lemmingsforums.com/Smileys/lemmings/shocked.gif" alt=":o" title="Shocked" class="smiley" /> (EDIT: I mean, the game's "Lemgo" set was originally called "Lego", with no apparent regard for trademarks, wasn't it? The music for those stages even retains that name... Or perhaps the Pac-Man style stuff is just from some sort of test?)

Also, excellent work with the lemming sprites, so far! http://www.lemmingsforums.com/Smileys/lemmings/laugh.gif" alt=":D" title="Laugh" class="smiley" />

[ccexplore]

I opened it up in Paint Shop Pro as a raw image of 26 pixels wide, and...
<snip>
Almost there, but I can't quite get it right. Maybe the rest of the data in the file is important somehow?

I don't have the tools at the moment to do this myself, but try opening it with a different width and see what happens.  Even if it doesn't divide evenly into the total number of pixels (it's always possible some of the bytes are filler or otherwise unused by the game).

I'd start with 24 because it's divisible by 8.  If there's a "correct" width it's likely to be lower than the current 26, given the leftward skew of the graphics right now as you go from top to bottom.

[Pooty]

I don't have the tools at the moment to do this myself, but try opening it with a different width and see what happens.  Even if it doesn't divide evenly into the total number of pixels (it's always possible some of the bytes are filler or otherwise unused by the game).

I'd start with 24 because it's divisible by 8.  If there's a "correct" width it's likely to be lower than the current 26, given the leftward skew of the graphics right now as you go from top to bottom.

I've tried other values, but for some reason, 26 was arguably the closest to looking correct.

I've had another look at it, and I've realised that there is a second layer of compression. Just before the lemming gets drawn for the first time, there is a repeated pattern of 40 00 40 00 ... (in hexadecimal). The 40 is the offset for the current line in the sprite (in this case, 64 pixels), and the 00 is the number of pixels at the end that are set to the transparent value 0. Since the sprite is 64 pixels wide, this essentially means to draw a blank line.

The first non-blank line reads:

Code: [Select]

1D 1C 6E 6F 6F 72 6E 73 6F
This means: begin drawing the sprite at 0x1D pixels across (29 pixels), and leave 0x1C pixels at the end (28 pixels). Add these two values together, and take the result away from 0x40. The result is 7, so there are 7 pixels that are not blank. The next 7 bytes are the palette values that fill this gap.

I got started with that idea, and it definitely seems to work. Unfortunately, I made a mistake just after the shoulders, so I present to you half a lemming. http://www.lemmingsforums.com/Smileys/lemmings/tongue.gif" alt=":P" title="Tongue" class="smiley" />

http://i144.photobucket.com/albums/r188/Pootyechidna/LemmingProgress-1.png" alt="" class="bbc_img" />

[GuyPerfect]

Attach that file (or a zipped copy) and I'll take a whack at it. Cracking data formats is my http://mlp.wikia.com/wiki/Cutie_mark" class="bbc_link" target="_blank">cutie mark. (-:

[Pooty]

Here you go. I've also attached the palette file.

[GuyPerfect]

Thanks, though I'm a step ahead of you. I forgot I had the disc image on my hard drive still because I missed it when backing everything up to DVD archives. (-:

Your earlier work was a good start. I've figured out the remainder of the file. The format is as follows:

Code: [Select]

File
===============================================================================
Manifest    Entry[568]    Cell manifest (0x08E0 bytes)
CellData    Byte[]        Cell data
===============================================================================

Entry
===============================================================================
Animation    Byte      The animation this cell belongs to
Flags        Byte      For 128px cells, 0x80 means "last cell in animation"
Offset       UInt16    The location in CellData of the Cell structure
===============================================================================

  Offset is stored as a 16-bit value to save space. To resolve the actual
  address, first add 0x08E0 (for the manifest), then add Animation * 0x4000 to
  resolve the actual offset of the cell in the file.

Cell
===============================================================================
LineOffsets    UInt16[]    Offsets relative to the Cell structure per Scanline
LineData       Byte[]      Scanline data
===============================================================================

  The LineOffsets are relative to the beginning of the Cell structure, not the
  beginning of LineData.

  The number of elements in LineOffsets depends on the file being processed.
  For LEMM.MHC, it's 64, and for LEMM128.MHC, it's 128.

Scanline
===============================================================================
Lead      Byte      Number of transparent pixels on the left side of the line
Trail     Byte      Number of transparent pixels on the right side of the line
Pixels    Byte[]    Width - Lead - Trail bytes of pixel data for the line
===============================================================================

  Pixel values represent palette indexes.
Attached to this post are ZIP files containing every sprite from both the 64 and 128 pixel files. For the rest of you, some sample rips:

http://i7.photobucket.com/albums/y255/bgng/0008E0_00_00_zpscec8936e.png" alt="" class="bbc_img" /> http://i7.photobucket.com/albums/y255/bgng/0C2F7D_30_00_zpsb6724b12.png" alt="" class="bbc_img" /> http://i7.photobucket.com/albums/y255/bgng/162AE5_58_00_zpsf2dffca0.png" alt="" class="bbc_img" /> http://i7.photobucket.com/albums/y255/bgng/26EEFC_9B_00_zpsfce2e547.png" alt="" class="bbc_img" />

[Pooty]

Bloody hell, that was fast. Nice one. http://www.lemmingsforums.com/Smileys/lemmings/thumbsup.gif" alt=":thumbsup:" title="Thumbs Up" class="smiley" />

[Prob Lem]

AIEE!! Awesome! Thanks very much! http://www.lemmingsforums.com/Smileys/lemmings/thumbsup.gif" alt=":thumbsup:" title="Thumbs Up" class="smiley" />

This also seems to confirm a suspicion I had about the "Rasputin" cheat - it looks from this like it really does have unique sprites for assassinated lemmings (it was a little tricky to tell in-game due to the scaling), since there are indeed separate sprites for stunned-on-landing lemmings in here. http://www.lemmingsforums.com/Smileys/lemmings/shocked.gif" alt=":o" title="Shocked" class="smiley" />

EDIT: I should clarify, those thanks are directed at both of you. http://www.lemmingsforums.com/Smileys/lemmings/laugh.gif" alt=":D" title="Laugh" class="smiley" /> I'm going to get some of these printed up as standees, I think, hehe...

[GuyPerfect]

I'm going to get some of these printed up as standees, I think, hehe...

*Imagines a desk littered with assassinated Lemmings*

On an unrelated note, Rasputin was always the name of my Bulbasaur. I am not joking.