Hacking Lemmings Revolution

[GuyPerfect]

After much experimentation, it has become apparent that I won't be able to inject custom code into existing DirectX interfaces: there's just too much funky stuff going on behind the scenes and lots of things break when I try to mess with them. At this point it's really a matter of using them as they are, or not at all.

So what I'm looking at is similar to what I mentioned in the first post: I'll have to implement IDirectDraw entirely, where all of MY methods just bounce through to the actual DirectX methods except when I need to hook into something. That won't be hard, but it WILL require a complete restart of the project. (-:

[GuyPerfect]

Wheee! I tried a fully phony approach and it worked! I'm just allocating my own fake DirectX interfaces in memory and linking them with actual interfaces spawned by DirectX proper, then using a simple implementation to bridge the game. The bonus here is that if I want to hook into any particular method, I've got the function body right there for adding code to.

Another bonus is that I stuck debug output in every method, so I can see all of them as they're called by Lemmings Revolution:

Code: [Select]

DirectDrawEnumerate()
DirectDrawCreate()
  Created IDirectDraw 0x0284E860
IDirectDraw::QueryInterface(0x0284E860, IID_IDirectDraw4)
  Created IDirectDraw4 0x006B99D8
IDirectDraw::Release(0x0284E860)
  Object deleted
IDirectDraw4::SetCooperativeLevel(0x006B99D8)
IDirectDraw4::QueryInterface(0x006B99D8, IID_IDirect3D3)
  Created IDirect3D3 0x0284E8D8
IDirect3D3::EnumDevices(0x0284E8D8)
IDirectDraw4::GetCaps(0x006B99D8)
IDirectDraw4::EnumDisplayModes(0x006B99D8)
IDirectDraw4::GetDisplayMode(0x006B99D8)
IDirect3D3::Release(0x0284E8D8)
  1 reference remains
IDirectDraw4::Release(0x006B99D8)
  Object deleted
DirectDrawCreate()
  Created IDirectDraw 0x006B8FE0
IDirectDraw::QueryInterface(0x006B8FE0, IID_IDirectDraw4)
  Created IDirectDraw4 0x006B9D08
IDirectDraw::Release(0x006B8FE0)
  Object deleted
IDirectDraw4::GetAvailableVidMem(0x006B9D08)
IDirectDraw4::GetAvailableVidMem(0x006B9D08)
IDirectDraw4::SetCooperativeLevel(0x006B9D08)
IDirectDraw4::SetDisplayMode(0x006B9D08, 640, 480, 16)
IDirectDraw4::QueryInterface(0x006B9D08, IID_IDirect3D3)
  Created IDirect3D3 0x0284E880
IDirect3D3::FindDevice(0x0284E880)
IDirect3D3::EnumZBufferFormats(0x0284E880)
IDirect3D3::CreateDevice(0x0284E880)
IDirect3D3::CreateViewport(0x0284E880)

(Main menu here)

IDirectDraw4::GetAvailableVidMem(0x006B9D08)
IDirect3D3::EvictManagedTextures(0x0284E880)
IDirect3D3::Release(0x0284E880)
  2 references remain
IDirect3D3::Release(0x0284E880)
  1 reference remains
IDirectDraw4::SetCooperativeLevel(0x006B9D08)
IDirectDraw4::Release(0x006B9D08)
  Object deleted
IDirectDraw4::CreateSurface() is excluded from this output because there are lots of them. It returns a http://msdn.microsoft.com/en-us/library/windows/desktop/gg426178(v=vs.85).aspx" class="bbc_link" target="_blank">DirectDrawSurface4 (even though this link is for version 7) interface, which as you can see has a lot of members. I'm not THAT bored right now, so I'll get to implementing that particular bridge later, but I do have IDirectDraw, IDirectDraw4 and IDirect3D3 implemented for peering into.

Somewhere just beneath the IDirectDrawSurface4 and/or IDirect3D3 interfaces is the parameter responsible for causing the colors to show up incorrectly. With any luck, it will just be a simple EXE edit to pass the right value, rather than requiring this DLL thing to be present.

[GuyPerfect]

I've ruled out IDirectDrawSurface4 as the source of the problem. This leaves IDirect3D3 as the culprit; specifically its CreateDevice() method which results in an http://source.winehq.org/source/include/d3d.h#L1128" class="bbc_link" target="_blank">IDirect3DDevice3. I can't provide an official link to that, though, because the internet seems to have no recollection of it ever existing. This so-called "device" object is the rendering context--the interface that communicates with the video back-end. It supports viewport selections, target device contexts, transformation matrices and--oh goody!--drawing primitives. Consider the images in the first post, showing the clock icon in the bottom corner of the screen. The way those colors are screwed up, I seriously doubt it's a full-color texture. It looks paletted, and the palette itself is going wonky and eventually AWOL. Not even Wine in all its accuracy will display those icons correctly.

Thing is, though, how exactly does IDirect3DDevice3 make textures? Version 8 introduced a CreateTexture() method, but that's nowhere to be seen here in version 3. My guess is that it's using QueryInterface() with something like IID_IDirect3DTexture, but I won't know that for sure until I implement the interface and see what happens.

[GuyPerfect]

I was able to get IDirect3DDevice3 implemented in my DLL without issue. It functions properly, and it was not in fact creating textures via its QueryInterface() method. The only other place that could come from, I figured, was the IDirectDrawSurface4 objects. Subsequently, I was able to verify that textures are indeed coming from IDirectDrawSurface4::QueryInterface() passing IID_IDirect3DTexture2 as the interface ID. This is exactly the information I've been wanting to know about, because it provides a solution to the original problem.

The implication here is that we can intercept texture data as it comes from Lemmings Revolution, swapping them out with different data of our own design that function properly on the display. If we construct true-color textures and inject those in place of the paletted ones, it won't matter worth beans what the states of the internal palette objects look like. I'm familiar with the game having been playing it recently, and the following need to be addressed:

• The Lemming, Balloon and Clock icons on the right-hand side of the main game screen should be replaced with true-color bitmaps.
• The on-screen text and selected-skill hilight border need to show up as white.
• The text in the menus needs to change color. Sometimes it's red, sometimes it's black. It's the same character map texture either way.

Those first two should be doable with a straight-up texture replacement. The last one will take some clever manipulation of the game engine, but it won't be difficult. I've already got IDirect3DDevice3 ready to bend over backwards for me, so we'll see what needs to be done.

Links of interest, mainly for self reference:

• http://msdn.microsoft.com/en-us/library/windows/desktop/gg426136(v=vs.85).aspx" class="bbc_link" target="_blank">IDirectDraw4::CreateSurface()
• http://msdn.microsoft.com/en-us/library/windows/desktop/gg426208(v=vs.85).aspx" class="bbc_link" target="_blank">IDirectDrawSurface4::Lock()
• http://msdn.microsoft.com/en-us/library/windows/desktop/gg426114(v=vs.85).aspx" class="bbc_link" target="_blank">DDSURFACEDESC2

[GuyPerfect]

Wow, this keeps getting stranger. Take a look at this debug output:

Code: [Select]

IDirectDraw4::CreateSurface(0x009CA0C8)
  16x16 @ 0x00000000
  Created IDirectDrawSurface4 0x009C99C0
IDirectDrawSurface4::QueryInterface(0x009C99C0, IID_IDirect3DTexture2)
IDirectDrawSurface4::GetSurfaceDesc(0x009C99C0)
IDirectDraw4::CreateSurface(0x009CA0C8)
  16x16 @ 0x00000000
  Created IDirectDrawSurface4 0x009C9A90
IDirectDrawSurface4::GetDC(0x009C9A90)
IDirectDrawSurface4::ReleaseDC(0x009C9A90)
IDirectDrawSurface4::Blt(0x009C99C0)
IDirectDrawSurface4::Release(0x009C9A90)
  Object deleted
IDirectDrawSurface4::Lock(0x009C99C0)
IDirectDrawSurface4::Unlock(0x009C99C0)
First off, pixel data is NOT sent to the surface object when it's created. So much for the easy way.

Secondly, it creates a second surface object (using the descriptor from the first), which copies its contents into the first then deletes itself? Weird, but so far so good...

Then... Wait, what goes on in between that GetDC() and ReleaseDC()? A debugger confirms there's a http://msdn.microsoft.com/en-us/library/dd183370(v=vs.85).aspx" class="bbc_link" target="_blank">BitBlt() in there, which is not an unusual way to initialize the contents of a surface object. Okay, fine. Nothing strange there.

But then there's also a Lock() and Unlock() right after that? Lock() is used to map the video memory into the process's address spaceOk, but... it doesn't need to do that to initialize the pixel data! Does it need to read it for whatever reason? This doesn't make sense.

Okay, so blitting is a one-way operation. Since it's being used, we know bitmap data is being transferred from process memory to video memory in between GetDC() and ReleaseDC(). Whatever that Lock() and Unlock() are used for, who knows. Hopefully we won't have to worry about it.

Since GDI is being used to manage texture data rather than DirectDraw directly, I'll have to redirect KERNEL32.DLL as well (since apparently the EXE is importing BitBlt() through that somehow, even though it's located in GDI32.DLL). Hopefully I can use the device context handles from that to extract the bitmaps it's writing into texture memory. And assuming there isn't even more weirdness going on, hopefully supplementing those with true-color bitmaps will solve the first part of the compatibility issue.

[mobius]

The implication here is that we can intercept texture data as it comes from Lemmings Revolution, swapping them out with different data of our own design that function properly on the display. If we construct true-color textures and inject those in place of the paletted ones, it won't matter worth beans what the states of the internal palette objects look like. I'm familiar with the game having been playing it recently, and the following need to be addressed:

• The Lemming, Balloon and Clock icons on the right-hand side of the main game screen should be replaced with true-color bitmaps.
• The on-screen text and selected-skill hilight border need to show up as white.
• The text in the menus needs to change color. Sometimes it's red, sometimes it's black. It's the same character map texture either way.

Those first two should be doable with a straight-up texture replacement. The last one will take some clever manipulation of the game engine, but it won't be difficult. I've already got IDirect3DDevice3 ready to bend over backwards for me, so we'll see what needs to be done.

don't forget; the bomber timer over a lemming doesn't show correctly. This is particularly strange since it doesn't show up as black but instead doesn't show up at all.  (however you can still see the "blank image: sometimes.
Also some objects in the background image are funny; for example the lolly pops and sheep in the "river background" are black
also, *sigh* I forgot another few items; the tips of switches should be red, they are discolored in the same way.
-The fish, in water tanks, they were orange or goldfish; same thing.

I apologize if what I say you already talked about but I don't understand the programming jargon.
It seems that most of the faulty objects are all objects that move or are dynamic. the letters, numbers, switches. Does the programming group the objects in this way (or some way)?  That might help locate the problem

[GuyPerfect]

I'm onto something. It's not DirectX after all: it's Lemmings. I was able to intercept textures on the way in from the application and got these:

http://i7.photobucket.com/albums/y255/bgng/NotQuite_Source.png" alt="" class="bbc_img" />

No alpha channel is present in that image, but it's the correct, full-color, non-paletted icons for the status figures on the screen.

Interestingly, Windows 7 didn't dump these. I guess there was an error in the DC I was reading from. I'll look at it more closely later on. What we know is that, for whatever reason, Windows 7 probably doesn't get these pixels whatsoever.

After those are applied to DirectDraw, they turn into this:

http://i7.photobucket.com/albums/y255/bgng/NotQuite_Before.png" alt="" class="bbc_img" />

The subsystem stretched all three out so they were 32 pixels wide. Iiiiiinteresting...

Remember how I said you can populate a texture with GetDC -> BitBlt -> ReleaseDC? Well, that's where these images are coming from. In fact, I was dumping them from the DC right before and after ReleaseDC() was called.

Remember also how I said there was an additional Lock() -> Unlock() right after? Well, take a look at what that did:

http://i7.photobucket.com/albums/y255/bgng/NotQuite_After.png" alt="" class="bbc_img" />

Uh-huh. Look familiar?

http://i7.photobucket.com/albums/y255/bgng/LookFamiliar.png" alt="" class="bbc_img" />

Lock() is used to give the program access to the texture memory as though it was ordinary RAM. What goes on between that and Unlock() happens in the program itself, reading from and writing to texture memory.

Lemmings Revolution itself is screwing up the colors. And can you blame it? Rather that feed colors directly into DirectX, it first blits into a GDI DC, then subsequently accesses the memory through Lock(). That is not--I repeat--that is NOT the way to initialize a texture. Apparently at some point DirectX decided that textures should be square or something (I really don't know what's going on with that one) and Lemmings was expecting everything to work exactly as it did in Windows 98.

That's the bad news. The good news is that, since it's not an issue with DirectX whatsoever, once all of the textures are loaded correctly, every last piece of the color problem will sort itself out.

[GuyPerfect]

I was able to isolate the function responsible for loading pixel data into IDirectDrawSurface4 objects. It constructs a DC that's used as the source in BitBlt(), and all of the textures from that exported just fine, even in Windows 7. It seems the DirectX destination DC isn't accepting some of them. I'm still not sure what's going on between Lock() and Unlock(), but it looks like it has something to do with the alpha channel. In fact, all textures that have built-in alpha channels (as opposed to using a color key for transparency) appear to be missing all their color.

[GuyPerfect]

Alright! Just as I was hoping, simply not calling that function at all results in pure, un-textured polygons. And what's more, they're the right colors! In fact, even the numbers that show up for Exploders are back, albeit white squares at the moment. Everything's functioning properly without bogus texture data involved!

All I need to do is make a replacement for that function and we should be good to go.

Here we have the level results screen, with very red lettering.
http://i7.photobucket.com/albums/y255/bgng/ItIsRed.png" alt="" class="bbc_img" />

Likewise, the shadows behind the text on the level select screen are properly red as well.
http://i7.photobucket.com/albums/y255/bgng/ItIsNotBlue.jpg" alt="" class="bbc_img" />

EDIT:
Personal note in case of catastrophe: The function in question is in process address range 00472DB9-0047393B

[GuyPerfect]

I'm up to my eyeballs in assembly, but progress is being made. I opted to just pick apart the function instruction-for-instruction and generate an equivalent to the source code that eventually became Lemmings Revolution.exe. There's a lot of code to go through (it's one of THOSE functions), so there's still a few hours of this ahead of me, and don't be surprised if I do other things between now and the time I finish. (-:

Once an analog to the source code is produced, I'll test that out to be sure my work yields the same results as the function in the game. After that, it will be a simple matter to make changes such that the textures are once again loaded correctly. It'll be the first time in ten years anyone's seen that Lemming icon with the colors it's supposed to have. (-:

[GuyPerfect]

I'm close. Very, very close. We now have proper alpha channels:

http://i7.photobucket.com/albums/y255/bgng/GoodIcon_Lemming.png" alt="" class="bbc_img" /> http://i7.photobucket.com/albums/y255/bgng/GoodIcon_Balloon.png" alt="" class="bbc_img" /> http://i7.photobucket.com/albums/y255/bgng/GoodIcon_Clock.png" alt="" class="bbc_img" />

What Lemmings does is load a BMP image into a GDI bitmap object, then passes that to a function to convert it into a Direct3D texture object. This is accomplished with BitBlt(). However, all of the textures are passed as either paletted or 24-bit RGB, which means no transparency except in the case of a color key. What it does instead is pass in an alpha buffer when appropriate, then writes that to the texture after BitBlt() by using Lock() and Unlock(). Or at least, that's the idea. It doesn't do that correctly, though. I haven't looked totally into it since I now know how textures are supposed to be processed, but I'm pretty sure it's incorrect handling of the RGB channel bit masks that's causing the colors to screw up.

Right now, I appear to be missing a little detail that either tells the game to accept the textures I'm generating or tells Direct3D to just-plain-work-dammit, but as soon as I figure out what it is that turns my byte arrays into on-screen pixels, we're in business.

EDIT:
Got it! Direct3D wasn't liking my custom channel masks. Now all I gotta do is give it the texture data. (-:<

[GuyPerfect]

Woo! Got it working! However, due to the way I'm processing all textures as 32-bit RGBA, any of the paletted textures that handled transparency with a key index... aren't transparent. However, I have a solution for that which should make the entire game just the way it was on launch day!

Colors are back, Exploder numbers are back, right-side-of-screen icons are back... Today is a good day.

http://i7.photobucket.com/albums/y255/bgng/Almost_Levels.jpg" alt="" class="bbc_img" />

http://i7.photobucket.com/albums/y255/bgng/Almost_InGame.jpg" alt="" class="bbc_img" />

[ccexplore]

Wow, awesome progress! http://www.lemmingsforums.com/Smileys/lemmings/thumbsup.gif" alt=":thumbsup:" title="Thumbs Up" class="smiley" /> http://www.lemmingsforums.com/Smileys/lemmings/cool.gif" alt="8)" title="Cool" class="smiley" />

Only thing I want to add is, when you're all done, it would be good to include information on what patching is done on your EXE, sorta like the xvi32 screenshot you made for the DirectSound-related patch.  Because there seems to be multiple versions of the game out there with various other fixes, and no one has been able to sort out which such version is the most updated.  So it may be best to eventually have a patch program that tries to find and patch the relevant bytes even if their exact location in file may be slightly different between versions of the EXE.

[GuyPerfect]

There's a patch on Download.com, so it's built off of that. If you get that patch to run and work, applying MY patch should work as well.

Which, by the way, is done now. Go check it out. (-:

http://www.lemmingsforums.com/index.php?topic=620.0

[ccexplore]

Okay, I'll give that a try and do diff between the two to work out the exact changes you made, and then I should be able to make it in the future so that it's possible to apply your changes to "any" version of the game.

The reason that is of interest is because there seems to be quite a few versions out there it seems.  For example, I alone have two copies of the game, one published by "Take 2 Interactive", another by "Take 2 Interactive Europe".  The former's CD actually comes with both an unpatched and patched version of EXE (which are not diff-identical), and the EXE from the latter is also different from the first two.  (Granted, the differences may turn out to be superficial, I certainly haven't bothered to analyze in any manner.)  This is not to mention what other people may own, given what we find out onhttp://www.lemmingsforums.com/index.php?topic=585.msg12473#msg12473" class="bbc_link" target="_blank">this thread.  Also, patched/unpatched versions (at least what I own) have known http://www.lemmingsforums.com/index.php?topic=585.msg12490#msg12490" class="bbc_link" target="_blank">subtle differences in game mechanics.  All that confusion makes it reasonable to want to be able to apply the sound/graphical fix to multiple versions of the EXE, rather than having to stick with the version on CNET downloads.

But for now I think I'll just wait until you've finished tweaking your changes first.  Nice work once again.