[BUG][PLAYER] Zooming In Fails to Preserve Mouse-On-Land [FIXED]

[Simon]

NL 12.12.5

The land are the physics pixels, i.e., the pixels where Lemmings physics apply. Zooming into the map will draw the land bigger, and we will see less of the land.

The canvas is the bitmap where we paint the land, then paint the lemmings, then paint the mouse cursor, ..., and finally paint the entire bitmap to the screen. After zooming in, we'll draw a land pixel as a square across several canvas pixels. The mouse position on the canvas stays the same throughout zooming in, that is good.

The mouse position on land will ideally remain the same before and after zooming in. When we're in small zoom levels and the land doesn't fill all of the canvas, preserving mouse-on-land won't always be possible. NL wants to fill the canvas with as much land as possible, and that's good.

The bug is now:

• Zoom in until the land fills all of the canvas. Don't zoom in further yet, i.e., keep available at least one deeper zoom level for later.
  
• Scroll to a position near the boundary of the land. Ideally, you're not at, but only near the boundary.
  
• Put the mouse cursor anywhere except perfectly in the middle of the canvas.
  
• Look at where the mouse points on the land.
  
• Zoom in.
  
• Look again at where the mouse points on the land. Now, it's a different pixel than where the mouse pointed on the land during step 4, before you zoomed in during step 5.
  

Expected instead: In step 6, mouse-on-land agrees with step 4's mouse-on-land.

Surprisingly, the bug doesn't happen when you're near the center of the map. There, NL computes and preserves mouse-on-land correctly when zooming in.

There are singular cases where the zooming preserves mouse-on-land even after back-and-forth zooming-in-and-out. Easiest example: Scroll to the perfect middle of the land. Put the mouse in the perfect middle of the canvas. Zoom in and out.

I haven't tested zooming out. That should ideally use the same code to preserve mouse-on-land, and fixing the bug for zooming in should ideally also apply to zooming out.

-- Simon

[Simon]

More research: Zooming-in never zooms onto the cursor unless the cursor is exactly in the middle of the canvas. Let's ignore this special case. (Reason for ignoring this: In general, any two linear maps agree at one point.)

Zooming-in seems to look at where the mouse is on the canvas, threating the position as a relative location of a point (mouse) on a rectangle (canvas), finding the corresponding relative location on the land (which doesn't even have the same aspect ratio), and zooming onto that point of the land.

As a result, when you don't move the mouse and repeatedly zoom-in, zoom-out, zoom-in, zoom-out, ..., you'll end up in one of the four corners of the level, depending on the quadrant of the canvas where your mouse was.

-- Simon

[Simon]

The zooming behavior is so aggravating that, several times now, I stopped playing NL before I wanted. And I don't understand how everybody else apparently doesn't care and happily plays on. Either everybody else is so thick not to notice any problem, or, more likely, subconciously avoids the nasty zooming even for precision work and just crawls into the screen really close.

Rule: Zooming must preserve mouse-on-land* and mouse-on-canvas.

This rule should apply even at the cost of not shifting all the land into view, i.e., the canvas should allow some land to remain out of view even when there is still void visible. Both Lix and NL fail this extended requirement. I should improve it in Lix.

Anyway, NL fails the rule even when the canvas is filled with land, and no void is visible. Please fix NL to follow the rule at least for when the canvas is filled entirely with land both before and after the zooming operation.

(*) There is a fun special case for the rule when the mouse points into the void, not onto the land. We should probably clamp the mouse-on-void coordinates then to get mouse-on-land coordinates, and preserve the clamped coordinates. This is hard to describe, and it's not necessary to understand yet because I'm mainly concerned with NL's zooming behavior when the entire canvas is filled with land, i.e., when no void is visible at all.



Edit 2023-05-06: I filed the extended requirement ("This rule should apply even at the cost of not shifting all alnd into view") for Lix: Zoom: Preserve Mouse-on-Land even when zoomed-out; don't screen-center (github #460)

-- Simon

[WillLem]

Yes, I see what you mean. The cursor jumps all over the place when zooming in. Ideally, it should be the "central" point when zooming. The same is true for the Editor, if I recall. More than happy to help debug this one, give me a shout if you're looking at it.

[Dullstar]

I don't understand how everybody else apparently doesn't care and happily plays on. Either everybody else is so thick not to notice any problem, or, more likely, subconciously avoids the nasty zooming even for precision work and just crawls into the screen really close.

Honestly that could be it. I rarely use the zoom function, but weirdly, in Lix, I *will* use the zoom function way more often. Although in fairness Lix levels have a wider variety of starting zooms which could just make it feel like a more natural part of the engine. But it's also completely undeniable that the zoom function in Lix is much smoother than its NeoLemmix counterpart, and generally works exactly as I would expect for the most part until you zoom out too much relative to the size of the level (and idk, maybe that's the least bad solution at that point).

[Dullstar]

Another NeoLemmix zoom quirk, but this one might actually be intended: the panel will be located as close to the center of the screen as it will fit when the level's vertical size fits in its entirety, until you zoom in enough to force the panel all the way to the bottom. Thus, zooming in and out can cause the panel to move around.

Lix leaves the panel in place and anchors the bottom of the level to the panel. Personally, I would have expected a similar behavior as well. Still, it's probably worth asking around to see what behavior people expect.

[WillLem]

Lix leaves the panel in place and anchors the bottom of the level to the panel. Personally, I would have expected a similar behavior as well. Still, it's probably worth asking around to see what behavior people expect.

It's possible they're part of the same problem, the more I think about it. Maybe anchoring the panel and/or level would actually fix it...?

Something to keep in mind is that we'd have to then decide whether to anchor the panel to the bottom of the screen or some arbitrary spot in the lower-middle. If the bottom, levels would be crunched to the bottom of the screen which would seem very wierd in Fullscreen at 1x zoom (even in Lix, this can appear a bit strange on smaller levels). The fact that panel & level move dynamically together as they do seems intended to me, given the much lower resolution of most levels in NeoLemmix.

[namida]

I can confirm that the panel moving with the game area is intentional. The other possibilities I considered were either anchoring everything to the bottom of the screen, or keeping the panel at the bottom while centering the level area; neither really felt right to me.

Re: the zoom matter, I recall several attempts to address this kind of issue in the past (although that might have been the editor), I'll have to take another look. I haven't yet. If no one's really been complaining about it being broken, I suspect no one will complain about fixing it either, but it'll go through an RC release either way.

[Simon]

There is merit to the floating panel: It keeps the mousing range smaller.

Anchoring the panel to the bottom felt intuitively right, but I never thought about floating panel in Lix. Panel sits at bottom because it has always sat at the bottom. And I make the mousing range small by rendering the land directly over the panel.

Fitts's law says: Buttons at the bottom edge are easier to click because the mouse can't overshoot. But that holds only in Lix. NL doesn't trap the mouse. Wow, NL traps the mouse within the narrow corridor that has the buttons at the bottom. Never realized it. Nice!

I think I prefer the panel to stay in one place. But this preference might merely be from habit. Surprisingly, I'm not annoyed when the NL panel floats around.

If you anchor the panel, then whether to anchor the map to the panel depends on how thoroughly you want to preserve mouse-on-land. In Lix, forcing the rendered land to touch the panel felt right to me only because I minimize the amount of void on the canvas first, and only preserve mouse-on-land as a second aim. If I go all the way in Lix with preserving mouse-on-land even when the land doesn't fill the canvas, I'll have to allow the map to float around instead of anchoring the map to the panel.

-- Simon

[WillLem]

NL doesn't trap the mouse. Wow, NL traps the mouse within the narrow corridor that has the buttons at the bottom. Never realized it. Nice!

There's also a hotkey to untrap it, which I'd suggest as an option for Lix. I'll post a topic for it though rather than discussing that here.

[WillLem]

This bug is also present in the Editor: zooming doesn't seem to anchor to any particular thing, be it scenery or cursor position.

I'd suggest that whatever fix we find for the Player should also be applied to the Editor.

[Simon]

Code Select Expand


procedure TGameWindow.ChangeZoom(aNewZoom: Integer; NoRedraw: Boolean = false);
var
  OSHorz, OSVert: Single;
  DoZoomOnCursor: Boolean;

  procedure SetCursorToCenter();
  var
    MousePos, ImgCenter: TPoint;
    ImgTopLeft, ImgBottomRight: TPoint;
  begin
    // Clip the Mouse position to the Image rectangle
    MousePos := Mouse.CursorPos;
    ImgTopLeft := Img.ClientToScreen(Point(0, 0));
    ImgBottomRight := Img.ClientToScreen(Point(Img.Width, Img.Height));
    MousePos.X := Max(Min(Mouse.CursorPos.X, ImgBottomRight.X), ImgTopLeft.X);
    MousePos.Y := Max(Min(Mouse.CursorPos.Y, ImgBottomRight.Y), ImgTopLeft.Y);
    // Get center of the image on the screen
    ImgCenter := Point(Trunc((ImgTopLeft.X + ImgBottomRight.X) / 2), Trunc((ImgTopLeft.Y + ImgBottomRight.Y) / 2));
    // Move the image location
    Img.OffsetHorz := Img.OffsetHorz - (MousePos.X - ImgCenter.X);
    Img.OffsetVert := Img.OffsetVert - (MousePos.Y - ImgCenter.Y);
  end;

  procedure ResetCenterToCursor();
  var
    MousePos, ImgCenter: TPoint;
    ImgTopLeft, ImgBottomRight: TPoint;
  begin
    // Clip the Mouse position to the Image rectangle
    MousePos := Mouse.CursorPos;
    ImgTopLeft := Img.ClientToScreen(Point(0, 0));
    ImgBottomRight := Img.ClientToScreen(Point(Img.Width, Img.Height));
    MousePos.X := Max(Min(Mouse.CursorPos.X, ImgBottomRight.X), ImgTopLeft.X);
    MousePos.Y := Max(Min(Mouse.CursorPos.Y, ImgBottomRight.Y), ImgTopLeft.Y);
    // Get center of the image on the screen
    ImgCenter := Point(Trunc((ImgTopLeft.X + ImgBottomRight.X) / 2), Trunc((ImgTopLeft.Y + ImgBottomRight.Y) / 2));
    // Move the image location
    Img.OffsetHorz := Img.OffsetHorz + (MousePos.X - ImgCenter.X);
    Img.OffsetVert := Img.OffsetVert + (MousePos.Y - ImgCenter.Y);
  end;

begin
  aNewZoom := Max(Min(fMaxZoom, aNewZoom), 1);
  if (aNewZoom = fInternalZoom) and not NoRedraw then
    Exit;

  DoZoomOnCursor := (aNewZoom > fInternalZoom);
  Img.BeginUpdate;
  SkillPanel.Image.BeginUpdate;
  try
    // If scrolling in, move the image to center on the cursor position.
    // We will ensure that this is a valid position later on.
    if DoZoomOnCursor then SetCursorToCenter;

    // Switch to top left coordinates, not the center of the image.
    OSHorz := Img.OffsetHorz - (Img.Width / 2);
    OSVert := Img.OffsetVert - (Img.Height / 2);
    OSHorz := (OSHorz * aNewZoom) / fInternalZoom;
    OSVert := (OSVert * aNewZoom) / fInternalZoom;

    Img.Scale := aNewZoom;

    fInternalZoom := aNewZoom;

    // Change the Img size and update everything accordingly.
    ApplyResize(true);

    //// If scrolling in, we wish to keep the pixel below the cursor constant.
    //// Therefore we have to move the current center back to the cursor position
    if DoZoomOnCursor then ResetCenterToCursor;

    // Move back to center coordinates.
    OSHorz := OSHorz + (Img.Width / 2);
    OSVert := OSVert + (Img.Height / 2);
    // Ensure that the offset doesn't move part of the visible area outside of the level area.
    Img.OffsetHorz := Min(Max(OSHorz, MinScroll), MaxScroll);
    Img.OffsetVert := Min(Max(OSVert, MinVScroll), MaxVScroll);

    SetRedraw(rdRedraw);
    CheckResetCursor(true);
  finally
    Img.EndUpdate;
    SkillPanel.Image.EndUpdate;
  end;
end;


WillLem found this code section and PM'd me. I wrote back; my replay is below the horizontal line here. The answer is from end of March 2023.



Hi,

I need more information about those names and will have to look at the full source repo. I won't have time this week [March 2023] for that. But if you have time, I have some ideas for promising attacks.

For a start, let's clarify:

What is "image":
The entire land, before rendering a section of it?
The section of the land that we are going to render?
The target bitmap onto which we draw during rendering?

What is "cursor position":
On the screen?
On the land? (I.e., it's what I call mouse-on-land? Have we really already recompute backwards from screen coordinates to land coordinates?)

// If scrolling in, move the image to center on the cursor position.
[...]
// Get center of the image on the screen
ImgCenter := Point(Trunc((ImgTopLeft.X + ImgBottomRight.X) / 2), Trunc((ImgTopLeft.Y +

Center? On first skimming, this looks like a dubious approach. Remember what we want to do: Preserve mouse-on-land before and after zooming. To do that, there is no obvious need to compute the center of anything.

... unless that's NL's preferred way of storing scrolling positions, because we need some reference coordinate for the scrolling position. Typical choices are indeed the center of the visible section of the land, or the top-left corner of the visible land. Does NL store the scrolling position via center of visible area or top-left of visible area? If it's center, fine. If it's top-left, we should be suspicious of computing the center in your shown source.

//// If scrolling in, we wish to keep the pixel below the cursor constant.
//// Therefore we have to move the current center back to the cursor position

I assume "pixel below the cursor" means the land pixel.

Then I don't see why (move the current center back to the cursor position) accomplishes (keep the land pixel below the cursor constant).

Idea to investigate: What does the code assume to draw that conclusion? The distance on the land between mouse-on-land to center-of-rendering-area-on-land doesn't stay the same before and after zooming. Explicitly:

A = mouse on land before zooming
B = (center of rendering area) on land before zooming
C = mouse on land after zooming
D = (center of rendering area) on land after zooming

Then your qouted code looks like it wrongly assumes A − B = C − D.

From NL user experience, the code behaves worse the deeper we zoom in. Maybe (before zooming) in the wrong assumptions is really (at 1x zoom), even though I don't see how that would work in your quoted source.

Draw some examples on paper.

-- Simon

[Simon]

Trying to play contest levels, and of course the zoom bug rears its ugly head and kills the mood. Has namida or WillLem worked on this since? Or has feedback on the research?

I'm beginning to think that I should offer money for a fix. The problem is: For $200, I can buy a simple Windows laptop, install the build tools, and fix the bug myself. But anything less than $200 isn't going to convince namida to take time off his day job.

Although then the second-most annoying bug will be that the game exits on all lemmings lost. I thought we had that nailed years ago. And the third bug might be slippery RMB scrolling, which looked hard to fix without library calls to freeze the mouse on screen. Ramble ramble grumpy grumpy. Grumpy Simon!

-- Simon

[WillLem]

Trying to play contest levels, and of course the zoom bug rears its ugly head and kills the mood. Has namida or WillLem worked on this since? Or has feedback on the research?

If you want to get on Mumble, I can screenshare my laptop and we can look at it together. I'm pretty fluent with the codebase now so I should be able to find things pretty quickly. If you know of a screensharing host that allows remote access as well, I'm happy for you to take control when necessary.

Although then the second-most annoying bug will be that the game exits on all lemmings lost

SLX has fixed this one

[Simon]

Happy to have a session if it helps you dig into the bug.

Mumble: I can make Saturday, 09:00 UTC. (This is in the morning, i.e., it's not 21:00 UTC.) Or I can make Sunday, 09:00 UTC.

I don't know about good screen sharing tools. My spontaneous idea was: On lixgame, I'll run a private RMTP server (like a twitch backend), e.g. Owncast, and you stream there with OBS (or similar screencasting software). The problem is that this would be a public livestreaming setup, optimized for chat, multiple viewers, audio etc., and not 1-to-1 screensharing.

-- Simon