Replay Insert Mode for Lix (like NeoLemmix's "blue R")

[Simon]

Hi,

this thread assumes that you've kept checked the option Keep replay after ◀▮ at all times. If you uncheck it permanently or temporarily, also look at topic: Rewind as Undo: Still Preserve a Loaded Replay.

Rewind is browsing. You keep "Keep Replay After ◀▮" checked at all times. This is the default. You treat the replay functionality like a book, and use framestepping to browse through this book. Only when you're really sure, you cut content by clicking air or by assigning. You accept that Lix occasionally surprises you with unwanted assignments when you forget to cut the replay.

Let's try to improve this style even more. The behavior of Lix 0.10.3 is:

• While you're viewing a replay (a loaded one, or your current attempt after rewinding), Lix will assign by itself the remaining future assignments from that replay.
  
• Click air (really, click anywhere in the game map where no lix is) to cut the replay. Cutting discards all future assignments from the replay. Because you're now at the end of the replay, the floating R disappears.
  
• Or assign to a lix. This first cuts the replay as if you clicked air, then adds your now assignment to the replay. It also advances physics once, therefore the assignment is already in the past. Again, you're now at the end of the replay, and the floating R disappears.
  
• The tweaker (filmstrip icon) allows you to discard assignments (past or future), or to move assignments by single physics updates. Tweaking preserves the remaining future assignments, and this is deliberate -- it's really a tool to edit history and see what future results from that single change only. To discard all future assignments, click air as usual, or discard all future assignments manually in the tweaker.
  

Design hole: In Lix 0.10.3, there is no way to insert new assignments into the replay without cutting the replay.

I'd like to offer replay insert mode. Let's design it. Insert might become a user option (that you toggle from the options menu only, and never during play), it might become a mode during play, or it might even become a default.

In my recent livestream, I've experimented with a hacked Lix version that inserted all assignments instead of cutting the replay before appending the assignment. This was very useful when many lix had to work together in a tough singleplayer puzzle. But it was annoying on ccx's 100% Built By Lixes: Here, many builder assignments go to a single lix, and each assignment desynchs future assignments to that lix. I got surprised and annoyed from Lix constantly replaying future assignments because I failed to cut them. As a result, I'm considering:

Idea (*): Click air to cut all future. Assign to discard only that lix's future.

Example:
In phyu 100, lix #0 builds.
In phyu 200, lix #1 mines.
   Phyu 300 is now.
In phyu 400, lix #0 digs.
In phyu 500, lix #1 climbs.
In phyu 600, we nuke.

(Phyu means physics update, a.k.a. physics frame.)

In Lix 0.10.3, when you click air here, Lix cuts the two future assignments and the nuke. Likewise, when you assign to either lix #0, lix #1, or an entirely different lix, Lix cuts the two future assignments and the nuke, then inserts your new assignment for phyu 301. Even with idea (*), when we click air here, Lix cuts the two future assignments and the nuke.

The difference of idea (*) is when we assign. If we assign to lix #0, we discard the assignment in phyu 400 and the nuke, then add an assignment to lix #0 at phyu 301. If, instead, we assign to lix #1, we discard the assignment in phyu 500 and the nuke, then add an assignment to lix #1 at pyhu 301. If, instead, we assign to lix #29, we discard only the nuke.

Is (*) sensible? Or do you see a different way to improve Lix in this direction?

Should we offer (*) as an option, in addition to offering how 0.10.3 always cuts all future actions on assignment? Should it be a user option in the options menu, or should it be a button during play? Either way, should (*) be the default or should 0.10.3's cutting be the default?

-- Simon

[mobius]

Ill replya more in depth later. My initial reaction is I like NL's function; two modes: insert mode and non-mode and being able to switch it on or off during play. While insert mode is super useful it has very specific uses for myself anyway, im more often not using it so it would get annoying

[namida]

I personally would not want this to be always-on. While I do make use of this mode at times in NL, the majority of the time I would have it turned off - it's essentially an "only when I specifically want it" feature for me.

Not sure how I feel about cutting assignments only to that particular lix.

[Dullstar]

Many Lix users probably also use NL, so its behavior and defaults may be a good starting point since presumably users will expect it to behave similarly. That said, we also shouldn't chain ourselves to that: if we think we can do better, we should go for it.



I think if you want to default to on, it's *extremely* important to have an easy way to cancel the replay, and I'm not sure clicking the air is the best solution here, as it seems like a very easy misclick. I think a hotkey would be better: still possible to mispress, but I would suspect it to be less likely than accidentally clicking the air next to a Lix.



I also think that there's some promise to cutting some assignments, though I think we can do better. Replay insert mode as implemented in NL is very prone to causing assignment desyncs, as NL has no way to detect that this happened. In many cases when you're using replay insert mode, the assignments are independent enough that they don't interfere, but if there is any amount of interaction between the inserted assignment and lemmings queued to perform assignments in the future, then while sometimes it'll still work out by chance, most of the time those future assignments just become nonsense. I think a simple solution to this in Lix would be to design it such that the replay additionally stores the position and direction of the Lix: if the Lix is not where we expect it to be when it's time to make the assignment, the assignment has become desynced, and should be cut. I think this generalizes better than simply cutting all future assignments to a particular Lix: some might cause few if any problems (e.g. assigning a climber or floater to a Lix that is busy doing multiple queued tasks with no climbing/floating in between), while others (anything that alters the terrain, really) can easily affect the queued assignments of other Lix.

Minor quality of life edge cases

We could easily be consistent and just cut any desynced assignment and I wouldn't expect it to cause any problems but there may be a few cases that could be convenient to the user to fudge a little:

- What if the Lix is within a certain, potentially configurable margin of error from its intended assignment location, e.g. only one or two pixels off and still facing the correct direction?
- What if the Lix originally queued to perform the skill is in the wrong position, but there is a different Lix in that location that the assignment could be reasssigned to?
- If we allow automatic reassignments, should permanent skill loadouts be considered, e.g. should we assign a floater to a walker if the floater was originally assigned to a climber? What if we were assigning a builder instead of a floater?

IIRC, the replay system is also closely related to the networking code, specifically, distributing assignments to other players, so this may also help detect desyncs (which I don't think would generally come up except maaaaaaybe with some latency edge cases that ultimately end up getting resolved successfully, but there have been desync bugs in the past, e.g. when there was a hatch assignment bug, though I'm not sure if any of them survived long enough to actually affect a planned multiplayer session). Of course, this doesn't help FIX the desync but it could be helpful diagnostic information (and possibly maybe for automated testing?).

[geoo]

Ill replya more in depth later. My initial reaction is I like NL's function; two modes: insert mode and non-mode and being able to switch it on or off during play. While insert mode is super useful it has very specific uses for myself anyway, im more often not using it so it would get annoying

I personally would not want this to be always-on. While I do make use of this mode at times in NL, the majority of the time I would have it turned off - it's essentially an "only when I specifically want it" feature for me. [...]

I think the question to ask here is: why would it be annoying if it was always on? It annoyed Simon when it was still assigning to the Lix whose task you just overwrote, but if the assignments to that Lix were cut, or if all desynched assignments were cut (as Dullstar proposed, something that had come to my mind as well), would there still be instances where it'd be annoying? I think it's hard to know without trying it for a while.
If in the majority of cases this is the behavior you desire, and in the few other cases there's an easy mechanism to cancel the replay, then this is probably good behavior.
Note that I do not see any use case where you actually want to keep the future assignments for the lix whose assignment you just overwrote, because they will be desynched anyway. The one exception is assigning permanent skills, however, you only need to assign them just before they actually take effect, and in that case a desyc of future assignments happens as well. Are there any use cases where you want to keep desynched assignments? If not, Dullstar's proposal is arguably even cleaner, and just cancelling the assignments of the lix you're overwriting is merely a heuristic that is easier to implement (and thus a sensible first step).

It seems cumbersome to me to have it something you turn on and off during play, especially if mentally you need to keep in mind whether it's on or off at a given time. (Tracking a state always takes mental capacity, and to me tracking whether the replay is still going or not is already complex enough, and I don't think I ever consciously look for the visual indicator to check for the state).
Having two different sequences of input actions corresponding to the two seems like it would be less confusing (because it's state-less), and the proposal (simply assign to insert, assign and click somewhere else to cancel) seems to be as simple as it gets and consistent with previous behavior for cancelling replays (and this behaviour will still be needed in the future, e.g. to cancel when backward frame stepping or loading from a savestate).

Also note that if you want to change an assignment to a lix/lemming to a later point in time, this behaviour is already required, as you have to click first to cancel the assignment.
This ties into the discussion in the other topic: https://www.lemmingsforums.net/index.php?topic=6120.0 In the context of option (1), the proposed behavior in this topic is simply a practical implementation of insert mode that is consistent with everything else. In the context of (2), whenever you go back to a previous point in time (rewind, load state, restart), future actions are cancelled, and the whole point of this topic is moot as remembering replays (and therefore insert mode) wouldn't exist.

I think if you want to default to on, it's *extremely* important to have an easy way to cancel the replay, and I'm not sure clicking the air is the best solution here, as it seems like a very easy misclick. I think a hotkey would be better: still possible to mispress, but I would suspect it to be less likely than accidentally clicking the air next to a Lix.

I disagree here. I really wouldn't like to use another hotkey for this, personally, as keyboard estate is really precious and I'm already out of keys within easy reach with all the other hotkeys. Clicking into air has been in place for ages, and I personally haven't had any issues with misclicking. (Admittedly, when you assign a skill it cancels as well currently, so the cursor position didn't matter and now it would, but I'm having a hard time seeing this a big issue. Given the many hotkeys already in place, I think fatfingering the one in question is more likely to me than misclicking, and that's from a user who's very clumsy with a mouse and prefers keyboard wherever possible.)

[Dullstar]

I will say that cancelling assignments corresponding to a specific Lix is a weak heuristic, as while it *does* cover a case where desyncs are likely, many skills are capable of desyncing *other* Lix (i.e. basically anything that alters pathing) and those desyncs are a big part of why replay insert mode can be annoying at times.

[Simon]

My main concern is: When improve a default here, we're breaking user patterns with all of L3, L3D, WinLemm, NL, and prior Lix. The margin of improvement must be big enough to convince most players on first sight. The hope is that idea (*), click air to cut all future and assign to cut that lix's future, is unintrusive enough. I'm not so sure now, and I believe we need to add some modes (in a pinch) or options (I'd like those more) regardless of what eventually becomes default.

Viewed from the other side: Let's not expect (*) to be overly intrusive. Sure, the closest analog is NL's blue R, a mode that inserts everything. By design, (*) is already less annoying than inserting everything. namida's reaction is wise: It's too early to tell how annoying it will be.

Dullstar nails an excellent point: The experience hinges on how the game treats replayed assignments that aren't 100 % applicable. In Lix 0.10, assignments are optionally forced left or right, have a lix ID, a skill, and a physics update. An assignment becomes illegal if the lix's direction mismatches, or if the ID'd lix isn't able to accept that skill in that exact frame. Position and other lix are ignored. There is no feedback, e.g., in the tweaker, about which past assignments were legal and which were skipped for illegality. My hunch is that adding more requirements to these assignments will be a physics change, unless I jump through more hoops to not write them into replay files or send them over the network.

I think geoo is correct in that cutting the assigned lix's future is a good first step, and it's easy to implement. I'd really like to ship insert mode (possibly hidden behind options) in a few weeks, to get everybody to try it. And we can still improve failed/relocated assignments later.

It's conceivable to add hotkeys and checkmark options: A hotkey for cancelling replays, no binding by default. Two checkmarks for what happens when you click the air: Advance physics by 1 frame, cut replay, both (current default), or neither. Hotkey real estate is scarce at 14 skills, potentially 15 or 16 in a few years, and it's customary to have advance-physics and cut-replay bound to the air click. The air click UI is also improvable, e.g., see github #400: Scissors mouse cursor while replaying.

In the options menu, the game hotkey page is getting crowded, and it's conceivable to open a second page for all non-hotkey options. Or maybe keep all non-replay options on the first page, and move the replay options to the second page.

-- Simon

[Simon]

Lix 0.10.3 cuts all future assignments when you assign. This is already annoying in a different way: Any disjoint union, or any two separated lix, will constantly cut each other's assignments. You'll likely want some form of inserting as default on multi-hatch maps. E.g., assignments might cut all future assingments of lix from that same hatch. I still believe (*) is better, but cut-future-of-same-hatch sounds viable enough, too, that I can't easily argue against it.

Extra requirements for assignments (putting more values into the struct): Those I'd rather not add in the next few weeks. But let's keep them in mind to make the inserting even better in the future.

The biggest worry remains that I roll out (*) and everybody gets upset over losing assignment-cuts-global-future. That was a reasonable behavior after all and it annoyed mainly on multi-hatch maps.

-- Simon

[geoo]

I will say that cancelling assignments corresponding to a specific Lix is a weak heuristic, as while it *does* cover a case where desyncs are likely, many skills are capable of desyncing *other* Lix (i.e. basically anything that alters pathing) and those desyncs are a big part of why replay insert mode can be annoying at times.

Certainly doesn't cover all cases, but in practice might serve the purpose reasonably well, espcially if you alter somewhat recent assignments and not assignments at the beginning of the replay. Always depends on the use case, and should certainly be better than nothing.

In the future, the same considerations will also need to be applied to the replay tweaker, i.e. desyncs caused by adjusting the frame of an assignment.

E.g., assignments might cut all future assignments of lix from that same hatch. I still believe (*) is better, but cut-future-of-same-hatch sounds viable enough, too, that I can't easily argue against it.

For single-hatch levels, that's basically having no insert mode. Usually, as a single-hatch level progresses, some worker lix are separated and operate essentially independently (and independent of the crowd), and so there are good use cases for insert mode.

The biggest worry remains that I roll out (*) and everybody gets upset over losing assignment-cuts-global-future. That was a reasonable behavior after all and it annoyed mainly on multi-hatch maps.

For a prototype, having an option in the options menu to enable/disable sounds like a good start, it allows people to experiment while being able to disable it if they get fed up. At the same time, you don't have to worry about in-level UI yet.

[Simon]

Attached is Windows 64-bit executable that inserts all assignments, but cuts future assignments to same lix and future nukes. Warning: It's experimental, I haven't tested it much.

On Linux: Build branch repinsert Build the normal master branch from my unstable Lix repository.

There is no new option yet. You can't toggle this insert mode; as long as you're replaying, you'll insert, cutting only that same lix's future. The next step for me will be creating these options.

-- Simon

[Dullstar]

Dullstar nails an excellent point: The experience hinges on how the game treats replayed assignments that aren't 100 % applicable. In Lix 0.10, assignments are optionally forced left or right, have a lix ID, a skill, and a physics update. An assignment becomes illegal if the lix's direction mismatches, or if the ID'd lix isn't able to accept that skill in that exact frame. Position and other lix are ignored.

Having a check for direction is already a good start, I think. Position is probably sufficient additional information to ship the feature as on-by-default, I think (I'd want to test it more, but I also suspect it could be enough to make an option unnecessary). Considering other Lix could be convenient but I could also reasonably see it being annoying, and honestly it's not important. In general, though, I'd probably rather it drop too many assignments (within reason) than not enough assignments.

Cutting future assignments as a heuristic is probably fine, but I don't think it's sufficient. Including other Lix from the same hatch feels a bit hacky -- it might be more convenient to program than improving the detection of desynced assignments, and at the very least it does reduce the probability of random junk assignments, but it reduces the usefulness of the mode and doesn't 100% eliminate them. And while it makes perfect sense on a technical level why NL keeps the junk assignments in its replay insert mode, I don't think it's good UX.

[Ramon]

My personal preference for this would be to have a hotkey such that holding it while clicking a Lix (similarly to force direction) would "insert" a skill rather than cut off the rest of the replay. I do however agree that the hotkey assignments are getting quite crowded and I have no proper suggestions on a clean solution.
Just curious though, are the "Control" and "Alt" keys not used for any ingame (level play mode) functions at all? I can maybe see randomly hitting Alt+F4 being a reason.

[mobius]

My personal preference for this would be to have a hotkey such that holding it while clicking a Lix (similarly to force direction) would "insert" a skill rather than cut off the rest of the replay. I do however agree that the hotkey assignments are getting quite crowded and I have no proper suggestions on a clean solution.
Just curious though, are the "Control" and "Alt" keys not used for any ingame (level play mode) functions at all? I can maybe see randomly hitting Alt+F4 being a reason.

Honestly I really like this idea; sounds very reasonable to me. This is the most often way in which I use insert mode in NL: inserting only a few skills, not major changes.

[Simon]

Option in 0.10.4: Lix 0.10.4 will have a single new checkmark option for insert mode. By default, this is off, and you get the old behavior: New assignments first cut the replay, then add the assignment. If you switch it on, it'll be permanently on during play. You'll get (*): All your assignments during replay first cut the same lix's assignments, then add your assignment.

This is mainly to ship 0.10.4 within a week, and to reconcile all experimental work with the stable version. Most likely, I'll change the feature over the months/years.



UI findings from streams: During my livestreams in January 2023, I had (*) active at all times. I got to be the front-line guinea pig of a potential UI revamp.

Insert mode has grown on me, I'd really like to keep playing with (*), and I'd like to improve it even more. There will probably be new hotkeys. There will be options about what those keys do.

Suddenly, air misclicks have become a problem. Now, I'd rather cut the replay with a more explicit action, not merely by clicking the air. Reason: I'm often deep inside the replay, and I'm solving most multitasking levels completely out of their normal timeline. I insert many skills, and each assignment is a click with the deliberate choice to keep other lix' futures.

Dullstar, you've foreseen correctly that, with insert mode, we would probably want a more explicit way to cut replays. I doff my hat.

But if you don't insert habitually, then airclicks are so very very nice to cut replays, and they're deeply rooted in tradition since L3, L3D, NL, earlier Lix, ... I'd like to continue to offer airclick as a way to cancel the replays, and that's why we need options.

Furthermore, at times, I would have liked to cut a single lix's future without assigning a new skill. There is no good way to do that. You'll have to guess the lix's ID, then erase all her future assignments in the replay tweaker one-by-one. I'm beginning to want a future-cutting skill.

Here's a UI idea. Everybody, feel free to shred it to pieces.

• In the options menu, you choose between insert-by-default or cut-replay-by-default.
  
• With either choice, the skillbar gets a new button for the other, non-default way.
  
• The button reacts to a user-configurable hotkey.
  
• If it's an insert button (because you selected cut-replay-by-default), it can either be hold-hotkey-to-use (I assume that's better) or toggle to insert mode by tapping the hotkey.
  
• Airclicks cut the replay only in cut-replay-by-default. In insert-by-default, airclicks do nothing. Probably airclicks also do nothing when you're cut-replay-by-default and activate the insert button.
  
• If it's a cut-replay button, it might become like a skill button: You can either select a regular skill or the cut-replay functionality, not both at the same time. I'm imagining the button to show scissors.
  
• When you "assign" scissors, you cut that lix's future.
  
• When you airclick with scissors, you cut the global future.
  

I'd still like to encourage everybody to try insert-by-default, but if you really don't want, you don't have to throw away tradition or your muscle memory.

Feedback in the UI can also become better regardless of insert-by-default or cut-replay-by-default. For example, when you tweak a lot, you'll want to see a hovered lix's ID printed somewhere.

-- Simon

[Simon]

I still play with always-insert. I misclicked air again, this time during Think Inside the Box. Rage! Had to redo most of the replay, luckily I still had it in my mind.

If I design the scissors button, it might get a third usage (item #2 in the following list) that would have been helpful in Think Inside the Box:

• Scissor a lix to cut only that lix's future.
  
• Scissor a hatch to cut the future of all lix from that hatch.
  
• Scissor air to cut the global future. Maybe it should take a double click, as with the nuke, to cut global future with scissors. But single click is probably fine, given that we already selected scissors explicitly.
  

Remember: This ingame button is still pure theory. I'd offer the scissors button only when you selected insert-by-default in the options menu. If you selected cut-by-default, you'd instead get an insert button.

So much to do!

And so much Lix and also Lemmings these days. What a wonderful time.

-- Simon