[Added][Discussion][NXP Player] Handling of pack specific tilesets

[IchoTolot]

There is one big problem I see with the upcomming new formats and it's the storage of the pack-specific tilesets which are not included by standard in NL.

I am always very concerned about maintainability of packs and strife for a big database of functional packs from all the different authors. Sadly some packs are lacking updates to keep them up-to-date and running on the new NL versions which makes me a little bit sad inside seeing good packs decent into incompability.

As I thought deeply about the new formats I came to a point which can result in chaos: Each pack still has to come with the non-standard tilesets it is using and as these cannot be stored hidden in an nxp anymore, they go directly into the tilesets folder and here is my point --> I highly doubt every pack creator keeps the tilesets bundled with his pack up-to-date!
This can result in total chaos when a tileset creator releases a critical update. Some users use the new version and the pack isn't working anymore. Other users overwrite the new with the pack version and now every other pack has a chance of not working.

So what to do?

I've come up with 2 solutions and a proposal which can help reducing the problem in general:

[Solution 1]

This is not my preferred solution and you will see why.

Letting packs have (again) internal tilesets which have priority over the tileset database within the pack. This ensures the pack is working all the time while still not influencing the other content.
Still we want to go away from hidden files and binary blobs and that's why this is far from optimal.

[Solution 2]

Each pack can have a modification (mod) folder where it stores modded tilesets in the new format. This is similar to SuperLemmini which also has a mod folder for tileset modifications.
Here each pack can (!= must) store it's non-standard tilesets and the author does not need to update the pack when one of the tilesets there gets an update. It's still preferable to update the mods as well, but no different tileset versions will overwrite themselves, causing possible broken content.
This could lead to having tons of versions of the same tilesets and that's why I highly advertise to keep the mods up-to-date as well, resulting only in safety duplicates.
Also now minor modifications to existent tilesets will be possible without overwriting the original maybe made by a different author.
Furthermore trash tilesets don't clutter the already big tileset folder -- like Sega2.dat I use in Reunion. They can now be stored the the tileset's mod folder.

This would be my preferred solution.

[Helpful proposal]

Expanding the by standard included tileset list of NL.

If more tilesets are included by standard, fewer tilesets need to be included with downloading a new pack. Of course these should be tilesets which are relatively stable and high quality ---> not having many major content breaking overhauls anymore.

Why not include the following to the standard list:

-- Gronkling's tilesets (Menace,Beast II,Cyber,Slime and minimal)
-- My Dune tileset, Lemmini conversions of Pieuw's castle and Prince Jamie's City tileset.
-- zanzindorf's glacier and sKiwi tilesets
-- I don't know how many of them are currently stable: GigaLem's freedom planet tilesets.
-- Arty's lego tileset

Raymanni's tilesets are currently still in the development phase, but after that they are also a hot candidate.
I 100%missed some potential candidates, but simply write in here!

With all these included by standard the ammount of nessesary tileset additions to packs will drop dramatically!

This will also lead to the reduction of the safety duplicates if [Solution 2] is chosen.



What do you think about the problem and how should it be adressed after you?  Do you also want a bigger standard NL tileset database, or which tilesets did I miss here?

I will also soon update the topic on the upcomming changes to the L2 tilesets, but don't panic no content breaking stuff there. Only removal of some no-trigger objects and the adding of a few tiles, which won't be problematic with the new formats.
 

[Nepster]

1) In your post, you always assume that non-standard graphic styles always come with the packs themselves. I don't think this will work at all. Instead all up-to-date versions of the graphic styles should be hosted on namida's homepage and anyone missing them should download the latest versions from there. I think the only exceptions to this should be pieces like VGASpecs and similar.
Of course this demands some more forethought from style designers, as they will have to keep their styles (almost) backwards-compatible.
Whether some or all of them are considered "standard" styles that already contained in the NeoLemmix player's zip file, is another question. I would suggest to keep the initial download file rather lean (perhaps even without namida's styles) and let the user download other styles whenever needed.

2) I think we should have a style "special" collecting all the VGASpecs and similar. Even now we have "orig_special", "lpiii_special", "lpiv_special" and "lpo2_special", and if we keep adding special styles for almost each pack, we will soon have more "special" styles than usual ones. Even just my updates to the old Lemmix packs would create a dozen more special styles.
So I think it makes sense to bundle the special pieces a bit, with a (semi-inforced) naming rule to avoid naming collisions.

3) IchoTolot, you talk about a style "Sega2.dat". I was not aware that there are packs out there that use modded versions of existing styles. How many of such modded styles do you use, resp. know about? How much difference is there between your Sega style and the original one?
Do you really need a modded version of this style, or could you use the official Sega style (but perhaps with a different background color so that the solid black pixels become visible)?
I would very much prefer not to use "modded" versions of graphic styles at all. So if this would only be used for a few levels in one or two packs, then perhaps we should look for another solution.

[IchoTolot]

1)
Hosting all avialable tilsets on the NL page, with each pack giving a list about which ones you need, is also a very good solution. Style designers only need to announce when sth critical will be changed, but I still would advice to stay as backwards-compatible as possible.

2)
This is an excellent proposal!
Otherwise the VGASPEC count would explode and clutter the whole tileset folder.

3)
I am using "Sega2.dat" and "Christmas.dat" in Reunion. These tilesets are currently hidden in the nxp and are the Lemmini equivalent conversions of "Sega.dat" and "Xmas.dat" as these have a completly different ordering of the tiles. Converting tilesets was a lot easier than rebuilding the levels from scratch.
A conversion table could be used to convert the levels as Sega2 and Christmas are a subpartitions of the NL versions with just minor obtical differences (steel + arrows), but that means additional work + I have no idea about the conversion table level conversion progress and creating a tool for it. Replacing of arrows would be needed, but other than that I think the tilesets haven't any major physics changes (they even lack some tiles + objects).

I know that you are not the type to add modifications to tilesets, but other people might want to maybe add a custom object or sth.

EDIT: Still I want to propose to increase the list of tilesets build in by standard in NL as this makes everything easier.

[Nepster]

3)[...] A conversion table could be used to convert the levels as Sega2 and Christmas are a subpartitions of the NL versions with just minor obtical differences (steel + arrows), but that means additional work + I have no idea about the conversion table level conversion progress and creating a tool for it.

Have a look at the files in "data/translation". There you have the translation tables as text files. So the only thing you would have to do is to find the "sega.nxtt" file, copy it and rename it to "sega2.nxtt". Then open it and change the numbers after the lines starting with "INDEX", so that the actual images in the style folder match up with the indexes of the pieces in your special style. Then have this "modded" sega translation file present when you run the NXPConverter to turn the nxp file of Lemmings Reunion into the new file-format pack.
I would suspect that modding the "sega.nxtt" once purely for converting your pack is less work in the long run than maintaining a modded style over the next years.

EDIT: Still I want to propose to increase the list of tilesets build in by standard in NL as this makes everything easier.

The player will not have a single built-in tileset. All styles come in the form of single images in the "styles" folder.

EDIT: Corrected slight inaccuracies.

[Simon]

I find this highly interesting. Excellent design thread, good popcorn thread. Thanks to Icho for making the topic!

This could lead to having tons of versions of the same tilesets and that's why I highly advertise to keep the mods up-to-date as well, resulting only in safety duplicates.

While excellent for hacking, this breaks down when distributing mods.

Two authors modify the same tile T to TA and TB which are incompatible with each other, then each author ships a pack that relies on TA resp. TB. How does a player deal with this? Copy files around by hand whenever player runs a level from the other pack?

Expanding the by standard included tileset list of NL.

Good. Or auto-download the tiles' blessed version from a centralized website lazily. From a dependency-management standpoint, both solutions are almost the same.

but other people might want to maybe add a custom object or sth.

Add to tileset and bump the non-breaking version.

Or add as loose tile. If the system demands each tile be in a tileset, make a tileset with one tile.

Converting tilesets was a lot easier than rebuilding the levels from scratch.

Don't duplicate, alias instead. NL will support this.

-- Simon

[IchoTolot]

I was talking to Nepster in IRC and I came to the conclusion I will indeed use the Translationtables method to get rid of these 2 otherwise useless tilesets and free Reunion from an old dirty workaround.
I will wait until the exp version becomes "save to convert" though and not just a pure test version. Then I can combine this with updating my other tilesets with all the changes I have planned (the no-effect objects stuff etc...) and the conversions of my packs.
When building a new pack with .lvl level files I assume the new FlexiToolkit will convert them into the new level format as well? Just asking even though the new version should be backwards compatible.

Yes, I see the disadvantages of the modding idea now and this can indeed lead into chaos as well.

I think on what we all can agree is that expanding the standard library with roughly stable and high quality tilesets is a good idea, resulting in fewer manual downloading of tilesets for packs. Combined with a central database of all tilesets on the NL homepage where every pack creator can point at saying you need A,B,C and D is a very good solution to my problem. Authors just need to keep track on critical tileset updates and by demand recheck their levels --> pointing out of critical updates is even more important!

[Nepster]

Yes, I see the disadvantages of the modding idea now and this can indeed lead into chaos as well.

Well, to be honest, I had more or less the same discussion with namida via PM about a week ago, with me suggesting an idea similar to your modding idea. And it took me two answers by namida to be convinced that the disadvantages are indeed huge.

Converting tilesets was a lot easier than rebuilding the levels from scratch.

Don't duplicate, alias instead. NL will support this.

Not "will", but "already does". The translation tables are the tool designed to do the aliasing.