[DISC.][PLAYER] High-res support

[namida]

I'd like to look at implementing support for a high resolution mode in NeoLemmix. If not much else comes up, I might even make this a target feature to have for V12.8.X.

First - let's cover a few things that I'm not interested in arguing about, but will be "this is how it is - deal with it":
- There will be no separate high-res physics. Physics will remain unchanged. High-res mode will be a purely graphical matter.
- Custom styles will not be obliged to provide high-res graphics. It's purely optional.
- The feature will be intended for use for low vs high resolution, not for alternate skins. Technical and/or rejection-of-style-submission measures may be taken to enforce this.

WillLem has been working on some high-res lemming sprites that we can use. For the official styles, we can rip high-res graphics from an existing version that has them, such as WinLemm or Mac Lemmings.

Some things that I feel should be discussed:
- Should graphic sets remain required to provide low-res copies of graphics, or should it be acceptable to provide only high-res ones (and NL downscales these for players using low-res mode)? Or perhaps, should it be required for terrain (due to terrain directly impacting physics) but optional for everything else? (My thought: Downscaling is fine, as long as it's a custom-implemented algorithm and not reliant on the graphic library's downscaling algorithm.)
- If a user is playing in high-res mode and encounters a level that some / all components of don't have high-res versions, should NL fall back to low-res, or try to upscale the low-res pieces? (My thought: Upscale.)
- If a user is playing in high-res mode and specifically custom lemming sprites that are in use don't have high-res versions, what's preferable as a fallback: upscaling of the equivalent low-res sprites, or use of the default high-res sprites? (My thought: Again, upscale.)
- Is it important for the editor to support high-res mode? This would be a lot of work to implement. (My thought: No. Editor is fine to remain low-res.)



Todos:
- Bug with backgrounds

[IchoTolot]

I would say:

- Graphic sets are required to either provide high or low res physics (or both). I think there could be sets that would work best in a higher res.

- If a user plays in a high res mode and plays a set that doesn't have high res graphics, NL should fall back to low res graphics. I think upscaling (or downscaling) could go wrong and create inaccurate or quite weird graphics. The same goes for the lemming sprite point. If I play in low res and I encounter a high-res only set I think it would be better if NL keep the graphics for this set high res. I could be proven otherwise here though as I am quite sceptical of the accuracy of scaling algorithms.

- I don't think we need a high res editor, especially when it's a ton of work.

[GigaLem]

While High rez stuff is nice, I don't really deem it practical, I doubt many people will use it or be able to pretty up their graphic sets for high rez mode to match or anything. I appreciate what efforts willLem can go through but I don't really see this being necessary.

[namida]

- Graphic sets are required to either provide high or low res physics (or both). I think there could be sets that would work best in a higher res.

I'm not sure what you mean by this - "graphic sets must provide at least one resolution", that's pretty much implied; or "we should choose one resolution that all styles are required to have, but the other is optional", if we go this route then of course low-res would be the required one as that's what physics are based on and what all styles already have.

If we don't have upscaling / downscaling and use a "you can provide either resolution", this means low-res-only and high-res-only styles cannot be mixed together. It would also more or less mean that providing a high-res version of any given style is on an all-or-nothing basis.

Most of all - if physics remain low-res-only (and they will be, this is not up for debate), we cannot have high-res-only styles without downscaling. Thus, if we don't have downscaling, all styles are either "low-res only" or "both resolutions".

[IchoTolot]

Yeah you are right here. Maybe it's simply better to say low-res is mandatory to provide for sets and high-res is optional.

[mobius]

I'd like to look at implementing support for a high resolution mode in NeoLemmix. If not much else comes up, I might even make this a target feature to have for V12.8.X.

First - let's cover a few things that I'm not interested in arguing about, but will be "this is how it is - deal with it":
- There will be no separate high-res physics. Physics will remain unchanged. High-res mode will be a purely graphical matter.
- Custom styles will not be obliged to provide high-res graphics. It's purely optional.
- The feature will be intended for use for low vs high resolution, not for alternate skins. Technical and/or rejection-of-style-submission measures may be taken to enforce this.

WillLem has been working on some high-res lemming sprites that we can use. For the official styles, we can rip high-res graphics from an existing version that has them, such as WinLemm or Mac Lemmings.

Some things that I feel should be discussed:
- Should graphic sets remain required to provide low-res copies of graphics, or should it be acceptable to provide only high-res ones (and NL downscales these for players using low-res mode)? Or perhaps, should it be required for terrain (due to terrain directly impacting physics) but optional for everything else? (My thought: Downscaling is fine, as long as it's a custom-implemented algorithm and not reliant on the graphic library's downscaling algorithm.)
- If a user is playing in high-res mode and encounters a level that some / all components of don't have high-res versions, should NL fall back to low-res, or try to upscale the low-res pieces? (My thought: Upscale.)
- If a user is playing in high-res mode and specifically custom lemming sprites that are in use don't have high-res versions, what's preferable as a fallback: upscaling of the equivalent low-res sprites, or use of the default high-res sprites? (My thought: Again, upscale.)
- Is it important for the editor to support high-res mode? This would be a lot of work to implement. (My thought: No. Editor is fine to remain low-res.)

I completely agree with the no-argument points (not that it matters since there's no argument )

Having said that I must honestly say I vote this be a low priority. I'd much rather see other things like skills or other new game features be implemented or bugs fixed before seeing this. Adding this would add a lot of new complications and subsequent issues that need their own resolving. As your questions sort of indicate. And we already have a long list of issues that need resolving atm.
-Having high and low res graphics in the same game feels really weird to me.
-Seeing low res graphics in editor then see them change in the player also seems weird. I'd like the editor to represent the levels as accurately as possible.

If it is implemented; would it be able to support the Lemmini graphics? Which would be very nice since then at the very least all the original sets would easily have high res replacements.

[ccexplore]

If it is implemented; would it be able to support the Lemmini graphics? Which would be very nice since then at the very least all the original sets would easily have high res replacements.

Yeah, in fact I'd almost go as far as to say that the Lemmini/WinLemm (and maybe Mac Lemmings) graphics may likely be the only use of high resolution in NeoLemmix for quite some time, unless we also explore upscaling low-res graphics (though I'm skeptical on the efficacy of upscaling).  Your point about priority certainly reflects past feedback as well--hi-res graphics hasn't really come up too much until recently.  On the other hand, because the person who really wanted it was able to put in some work graphics-wise, we're probably in a situation right now where this otherwise lower-priority feature request may actually be relatively a little closer to ready compared to things like new skills.

I do wonder if it's too restrictive though to always require low-res version of graphics.

[Dullstar]

I definitely think that requiring low-res graphics is too restricting. I could see an interesting case, for example, for high-res re-imagining of specific sets that wouldn't necessarily be 1:1 replacements for their low-res counterparts, which level designers could use if they wanted.

[namida]

I definitely think that requiring low-res graphics is too restricting. I could see an interesting case, for example, for high-res re-imagining of specific sets that wouldn't necessarily be 1:1 replacements for their low-res counterparts, which level designers could use if they wanted.

This is specifically something that I don't want such a feature to be used for. That should be done via having a completely separate style, especially if it isn't a 1:1 replacement physics-wise.

I'm fine with not requiring low-res, but it does mean we'd need to accept downscaling of the high-res for players who are playing in low-res. Perhaps an option could be implemented to allow a style designer to choose from several algorithms to use for upscaling / downscaling (as applicable), which might somewhat mitigate the concerns around poor quality resampling. Any scaling would be done at load-time, not run-time, so it shouldn't affect performance too much.

[Dullstar]

Would a restart be required to toggle between high-res and low-res? I'd probably use low-res myself for the original sets, but I wouldn't mind falling back on high-res for sets where low-res isn't available - honestly, I think that would be preferable to me compared to downscaling.

And to clarify, the non-1:1 re-imagining would be a separate style, which is why I thought of it as an example of when you might have high-res graphics, but no low-res graphics to accompany them.

[namida]

Would a restart be required to toggle between high-res and low-res?

I'm currently thinking that you would not be able to change mid-level, but you could change without restarting NeoLemmix itself. Of course, I can't say for sure until I actually work on implementing it, but I think what I've described here should be acheivable. Changing on-the-fly while in-game probably isn't; the only way I could see that being feasible is if NL always rendered both resolutions and the option simply chose which one to display, which would likely have an extreme performance impact for minimal gain. (EDIT: Alternatively, NL keeps a history of every modification to the visual terrain map, and replays these upon resolution switching. While this wouldn't have the performance hit of the previous suggestion, it would instead be very code-intensive, again just for the gain of "you can switch mid-level" which I don't see as a huge deal.)

And to clarify, the non-1:1 re-imagining would be a separate style, which is why I thought of it as an example of when you might have high-res graphics, but no low-res graphics to accompany them.

Ah, gotcha.

[namida]

Some things that I feel should be discussed:

- If a user is playing in high-res mode and encounters a level that some / all components of don't have high-res versions, should NL fall back to low-res, or try to upscale the low-res pieces? (My thought: Upscale.)

I do get the concerns some people have expressed about upscaling; but the more I think about it, the more I realise "upscale" is the only sane answer here. "Revert to low-res mode" just leads to too many situations where one piece or one graphic might end up requiring the entire game to revert to low-res mode, and that seems silly when upscaling can be done.

I do feel it would be beneficial here, to allow each style to individually specify an upscaling algorithm to use. Pixel art styles would benefit most from a different upscale algorithm than full-color art, for starters. We don't want to be overkill on this though - we want maybe 4 or 5 choices at most, probably including Nearest Neighbour as one of them (which ultimately looks no different from "use low-res, but zoom in further", except that it can be mixed with higher-res graphics instead of making the entire game fall back), not hundreds. Maybe even just three - nearest neighbour, a pixel art upscaler, and a full-color art upscaler.

- If a user is playing in high-res mode and specifically custom lemming sprites that are in use don't have high-res versions, what's preferable as a fallback: upscaling of the equivalent low-res sprites, or use of the default high-res sprites? (My thought: Again, upscale.)

I stay with "upscale" here mostly due to the concept of "don't make a special rule where one isn't needed". Like any other content, "upscale the low-res content" is the behaviour.

- Is it important for the editor to support high-res mode? This would be a lot of work to implement. (My thought: No. Editor is fine to remain low-res.)

No one seems to be concerned with this, so - at least for now (I can reconsider this later if there's demand) - editor will be low-res only.

- Should graphic sets remain required to provide low-res copies of graphics, or should it be acceptable to provide only high-res ones (and NL downscales these for players using low-res mode)? Or perhaps, should it be required for terrain (due to terrain directly impacting physics) but optional for everything else? (My thought: Downscaling is fine, as long as it's a custom-implemented algorithm and not reliant on the graphic library's downscaling algorithm.)

For the sake of physics simplicity, and in particular taking into account that it's a lot trickier to find a globally-acceptable downscaling algorithm than an upscaling one when individual pixels matter, I think part of the answer here going forward is: All styles will be required to provide low-res copies of every terrain piece. They can (but do not have to) additionally provide a high-res one. For everything else, pixel-perfect accuracy isn't critical, so downscaling might be more acceptable - I'll give some more thought to this (and of course, feel free to give your input).

Now, I've put this one last because I wanted to write about the editor first. Because that has to be taken into account here - if the editor remains low-res, then it's easiest to require all styles to provide fully low-res copies. Otherwise, the editor also needs to be able to downscale the high-res graphics for any case where only those are present. This does make me lean towards "all styles must provide a low-res version".

[WillLem]

Thanks for the shout-out regarding the sprites; I had fun making these, it'd be great to see them in use!

I am obviously in 100% support of this happening, I think that NeoLemmix is a very capable Lemmings engine and offers some exciting new features which enhance and re-imagine the classic game of Lemmings. When I switch from SuperLemmini to NeoLemmix there's always the feeling that whilst the engine itself is more capable, the graphics just...aren't as pleasing, which is a shame because some users have created excellent custom content.

However, as I've mentioned in numerous other posts, there are things which make NeoLemmix way better (continued support, framestepping, in-game skill previews, in-game clear physics, the list goes on...) hence the reason I'd fully encourage a visual uplift.

To address your questions, and speaking as someone who knows very little absolutely nothing about developing and maintaining software, but who knows a lot about art, music, enjoying a good game of Lemmings and making things look pretty:

- Should graphic sets remain required to provide low-res copies of graphics, or should it be acceptable to provide only high-res ones (and NL downscales these for players using low-res mode)? Or perhaps, should it be required for terrain (due to terrain directly impacting physics) but optional for everything else? (My thought: Downscaling is fine, as long as it's a custom-implemented algorithm and not reliant on the graphic library's downscaling algorithm.)

I would suggest that this is down to the user/creator - if they want their content to be played by as many people as possible, they should provide both high and low-res graphics. Not only would this be simpler from an engine point of view (I can imagine), but it would give the creator full control over how their content will look to the end-user.

- If a user is playing in high-res mode and encounters a level that some / all components of don't have high-res versions, should NL fall back to low-res, or try to upscale the low-res pieces? (My thought: Upscale.)

I agree with IchoTolot that upscaling might look a bit odd... I think it might be simpler to make it so that if a user chooses high-res mode, they only have access to content that has high-res graphics created for it (and vice versa). This would encourage creators to provide custom graphics in both resolutions in order to reach maximum people.

- If a user is playing in high-res mode and specifically custom lemming sprites that are in use don't have high-res versions, what's preferable as a fallback: upscaling of the equivalent low-res sprites, or use of the default high-res sprites? (My thought: Again, upscale.)

Again, I would keep each mode exclusive so that content from one mode won't work in the other unless there is an equivalent graphic.

In addition to all of the above points, I would say that the priority should always be to keep things as simple as possible for everyone concerned. If it would be easier to simply upscale/downscale graphics in either scenario, then - of course, go with this option!

- Is it important for the editor to support high-res mode? This would be a lot of work to implement. (My thought: No. Editor is fine to remain low-res.)

Would this mean that the Editor couldn't be used to create hi-res content? Or would it work similar to creating a .lvl level in 1.43 and then saving it as a .ini - the Editor automatically chooses the correct equivalent graphics for the resolution? If the latter, this is absolutely fine as a way of getting levels made.

Just out of interest - if NeoLemmix Editors in the past have been more than capable of handling multiple formats and resolutions, why the shift towards restricting the format? I can imagine it's to keep things simpler...

Let me know if there's anything I can do to help with any of this. I'm currently working on:

- Hi-res Xmas sprites (these are nearly done, I'll send a pack over soon)
- Hi-res NeoLemmix-exclusive elements (buttons, pickup discs, splitters... if anyone has a list of these that'd be helpful in compiling them)
- Hi-res panel/buttons (these will have the chunky cartooniness of the Windows ones but the more crisp, familiar style of the DOS/Amiga ones - see my current low-res custom panel for an idea of what I'm aiming for. I'll create "clicked" versions of these as well in case you want the option to have tactile buttons). It would be good to back-and-forth these with someone to get a final version that everyone's happy with.

All best,

-WillLem

[namida]

Just out of interest - if NeoLemmix Editors in the past have been more than capable of handling multiple formats and resolutions, why the shift towards restricting the format? I can imagine it's to keep things simpler...

Two factors here. Firstly, the current generation editor that works with NL V12.X.X, was an entirely new editor written from scratch - it's not even in the same programming language as the old editor. So, it wasn't so much removed, but more never implemented in the new editor in the first place.

That aside, supporting more formats makes the code more complicated. At the very minimum, code is needed to load and save the different formats. The modern editor doesn't even have this - it can only load and save in modern NeoLemmix format (while you can tell it to load a level in a few other formats, what actually happens here is that it gets the engine - which for now does have that support, though it's load-only - to perform the conversion). On top of that, either the user has to manually keep in mind what features do and don't work with the engine they're using, or else the editor must keep track of what one's being used and enable / disable options as appropriate. Then, if there's a feature that another engine supports but NL doesn't, that's an extra feature to implement support for.

Would this mean that the Editor couldn't be used to create hi-res content?

The editor would work exclusively in low-res, and the engine would display the content in either low-res or high-res depending on user preference. The high-res mode will be a purely visual change; physics remain low-res, and thus, low-res versions of the styles still need to exist (but high-res versions are optional).



In terms of content - no rush on that. My main concern for V12.8 is getting the engine support there; the graphics to go with it can be introduced over time as part of styles updates or future NL updates.

[Dullstar]

I agree with IchoTolot that upscaling might look a bit odd... I think it might be simpler to make it so that if a user chooses high-res mode, they only have access to content that has high-res graphics created for it (and vice versa). This would encourage creators to provide custom graphics in both resolutions in order to reach maximum people.

That sounds like a bit of a content maintenance and user experience nightmare, to be honest. Users who select the high-res mode might be confused as to why some levels disappear when they select it, while content creators would have to either give up styles that lack high-res versions or pester people to change their graphics settings in order to play their levels. There's few enough original styles as it is anyway, as the vast majority of custom styles are game rips. Forcing people to provide two sets of graphics in order to avoid having their style cause issues with the player will exacerbate this problem as well as create bloated style downloads: high-res graphics necessarily significantly increase the size of the style, since now there's two copies of everything, but styles created in low-res may simply be upscaled to prevent the game from hiding levels from high-res users, resulting in a style that is at minimum twice (plus probably extra because the second copy has 4x the pixels) as big as it needs to be if the game could simply upscale the graphics itself.