[SUG][NeoLemmix Level Packs] Difficulty Ratings Scale

[WillLem]

What really matters is having some kind of system to help players find content at the level they want to play at.

Yes, exactly! +1 for this.

i rate all mine as easy and icho said some of my levels arent as easy as i think so id have no idea how to rate them lol

Your levels are definitely not "easy", I'd say they're intermediate and up. In the above suggested scale, I'd say yours are usually around the "4" mark, with the occasional one in "3" territory when you're feeling generous (or haven't backroute-proofed the level!)

My tendency seems to be to underestimate difficulty.

I think most creators do, to be honest. It definitely seems to be a recurrent theme/issue in these conversations.

maybe testers could offer suggested ratings?

This is a great idea. It would help to encourage use of a universal ratings system as well, since creators, testers and players would need to be aware of the general definitions.

[WillLem]

OK, so... just to pad these definitions out a bit based on discussion so far:

1 - Anything that's a tutorial level, any-way-you-want-it, far more skills than needed, and/or with very obvious solutions.
2 - Levels that require a bit more thought, but still have multiple possible solutions and plenty of available skills.
3 - Levels with fewer possible solutions and requiring more specific tricks/methods, but perhaps with some leniency in the skillset and/or relatively clear indications of what's meant to be done if the skillset is limited. Design features such as tight time limits, red herrings, limited skillsets and difficult save requirements may start to appear at this level of difficulty, but will usually be there mainly to enhance an otherwise straightforward level.
4 - Levels requiring one very specific and relatively obscure solution, with few if any clues for the player and usually only providing the necessary skills for that solution. Levels with this rating will usually not provide alternative solutions, and will be designed specifically to prevent anything but the intended solution. Design features will be more unforgiving, and expect a high degree of skill from the player.
5 - Levels requiring one very specific solution involving very advanced (or even brand new) tricks and methods not widely known to the community, and only providing the necessary skills for that solution. These may also be particularly complicated levels requiring a lot of time and effort to solve.
6 - Indicates levels with an extreme degree of complexity, with the expectation that only the highest-skilled players will be able to solve the level.

What does everyone think? Should there be more levels of difficulty? Are the definitions too loose/not loose enough?

And, importantly - would this system be usable?

[Strato Incendus]

Okay, based on those definitions, I get the impression that I might finally be able to approach level 6 territory with Lemmings Open Air... ^^ But considering there is one level from Pit Lems that kaywhyn is currently stuck on, maybe even that pack has a few select "outliers".

I'm not sure I will ever make any difficulty 1 or 2 levels again. Unless I'm making tutorial levels - which I actually quite enjoy. But the problem is, about halfway through designing those levels, I usually also want to find some way to still make them interesting to more experienced players, as well... which at best results in a tight skill limit and/or saving everyone, and at worst includes some simple but not-quite-obvious skill combination that will definitely come as a surprise for complete newcomers.

But yes, complete any-way-you-want-it levels, I think that's what original DMA Lemmings is for. The Fun rank does an excellent job at making X-of-everything levels... well... fun! While a hastily-made X-of-everything level will usually result in an ONML Tame level. You youself made "Tame Gone Wild" for a reason!

So I wish we would include fewer of those open-ended levels in our custom packs, and just put more emphasis on pointing complete newbies to original Lemmings first (to be played in NeoLemmix, of course ), for learning all the basic skills. This would free up much more design space for custom levels, instead of every custom pack having a first rank of 20 to 30 no-brainer levels.

[Proxima]

But yes, complete any-way-you-want-it levels, I think that's what original DMA Lemmings is for. The Fun rank does an excellent job at making X-of-everything levels... well... fun! While a hastily-made X-of-everything level will usually result in an ONML Tame level. You youself made "Tame Gone Wild" for a reason!

So I wish we would include fewer of those open-ended levels in our custom packs, and just put more emphasis on pointing complete newbies to original Lemmings first (to be played in NeoLemmix, of course ), for learning all the basic skills. This would free up much more design space for custom levels, instead of every custom pack having a first rank of 20 to 30 no-brainer levels.

I am starting to get seriously annoyed by this. You do not have the right to prescribe to other authors what difficulty level they should aim at. Just enjoy content at your level, and accept that some content isn't for you. (If you want to complain that we don't have a good system to help players find content, I would be totally on your side, but that complaint should not be directed at pack authors.)

Of course I agree that "hastily made" levels tend to be bad; that's true at any difficulty. But looking at the content that's been produced recently, I can see a lot of high-quality easier levels, such as Sammings and the early ranks of Lemminas. If you are still of the opinion that most usermade easy levels are more Tame-like than Fun-like, then -- and I mean this seriously -- maybe you should list the packs you've played / looked at, so that I can see which ones you haven't looked at and make recommendations? It would also be an interesting study, to help us look at how designing tendencies have changed over time.

As for "every custom pack having a first rank of 20 to 30 no-brainer levels", that hasn't been true for years -- the later Lemmings Plus packs start with a first rank that's mostly easier linchpin puzzles, and packs such as Nessy's, Nepster's and IchoTolot's likewise jump straight into medium or hard puzzles. In fact, I can't think of a large multi-difficulty pack that does start with a "Tame" rank, unless I look all the way back to Lemmings Plus I, and that is now seven years old.

[Crane]

To add to this, my pack "Holiday Lemmings '16" (which I still need to convert to the latest NeoLemmix version, because some levels are now impossible due to the deadly ceiling)... it only has 30 levels and the difficulty climbs quickly, but the first three levels start with X of everything to get you into the groove - however, each has replayability in the form of a talisman.

Level 1 - 20 of everything... easy enough starter level with some slightly tricky terrain to navigate.  Comes off as a classic 20-of-everything level. Has a talisman that challenges you to complete it without any builders.
Level 2 - 10 of everything... starts with a deadly drop (and more than 10 lemmings).  Completing it in under 1 minute nets a talisman.
Level 3 - 5 of everything... skills are more limited.  Has two talismans: complete with 100% (difficult because there's a steel floor under the trapdoor, so stopping the lemmings from going to the left requires 3 Builders), and complete with no Bashers (requires the use of bombers and careful skill conservation).

Level 4 - 3 of everything... doesn't have a talisman, and barely counts as an X of everything level, but you have to use what you have so 3 lemmings can help each other reach the exit through a large steel obstruction.

----

To mention another pack... I helped convert some levels in Revenge of the Lemmings to the Picnic rating which is largely Fun and Tame-like levels.  A couple of levels that stand out is "Hunting Season" and "Singularity" because the intended solution in their harder versions is disguised in some way.  The former, "Wish you had them?", has no Builders and it's not obvious how you get to the exit without them.  For the latter, "Duality", the limited skills mean you can't just brute-force your way to the top with Builders, and uses, in my opinion, an absolutely beautiful sacrifice as part of the intended solution.

And this is just me... when I played the Fun levels and realised they repeated in later difficulties (as well as the first 13 Tricky levels), I started to wonder with anticipation when the next repeat will appear!  In fact, the only non-special Fun level that doesn't repeat is level 3: "Tailor-made for Blockers", probably because it was actually tailor-made!  I tried to follow the same philosophy with the Picnic levels in Revenge of the Lemmings with, at the time, only "Picnic 29" not repeating (although a harder version was made for it later).

Finally, with all these easyish levels in Picnic, you can just skip straight to Hootenanny if you don't want to play them.

----

As mentioned before, one problem with difficulty is that, as a level designer, it's hard to accurately measure the difficulty of one's own level.  I thought the difficulty of my level "Dropship 12" was on par with high Mayhem or Havoc, given it uses a variant of a famous Mayhem technique, but feedback from others suggest it is one of the hardest levels I have ever made.  Granted, such overestimation or underestimation of difficulty is generally not too extreme... only one or two levels out.

Though not with Lemmings, I have used a similar system for rating difficulty in the form of stars.  Usually rated from 1 to 5 stars (although 0 can easily be added to this), and "Flashing 5-star" for something insanely difficult.  I've noticed a lot of packs rate the highest difficulty as "Extreme".

[Strato Incendus]

@Proxima: Fair enough; I just happen to currently be playing Lemmings Plus I again, so it's very prominent in my memory. NepsterLems also starts with these open-ended levels. I have indeed played a good chunk of Lemminas; haven't looked at Sammings yet, though, so thanks for the suggestion!

And that's also what my post was - a suggestion, a wish, and a description of what I prefer. Not a prescription.

[mantha16]

personally I deliberately try to design easier packs as i feel theres easier pack deficit (in terms of new content) but I dont even create what I consider tutorial levels at least not on purpose.  I assume people that download original content that aren't meant to replicate original lemmings games have some level of knowledge so don't need tutorial levels.  Plus I don't consider difficulty rankings so.  I'm interested in theme hence packs like Sammings and Sammings2.  I'm interested in style hence All The Styles and the yet to be resurrected TVLems and 6 Nations Lems.

I kinda get a bit annoyed by bitching about other peoples packs I mean sure offer constructive criticism but if someone doesn't like a pack then they don't have to play it. Besides which builderfests, x of everything levels are great for challenges and for replaying, like if I replay one of those levels I try for a different solution everytime.

I do think difficulty ratings would be really useful for people I just dont think that authors are the best judge of difficulty since they know the solutions.  Maybe players could offer their views and after a majority vote the pack moved to a sub-thread for whatever difficulties are agreed on. 

Obviously opinions will vary

[WillLem]

This discussion kinda petered out, but I've had a number of comments on the advertised difficulty of my packs recently (and changed Lemminas to "Medium" as a result), so I think it's worth reviving this topic.

It is difficult to know exactly where to place your pack on the difficulty scale if there is no difficulty scale. All it would take is for us to come up with an agreed list of criteria which designers can refer to, make it official (i.e. give it its own sticky topic), and suggest that designers use it wherever possible.

Of course, some designers will ignore it, or not know about it, and still others may get it slightly wrong and make an incorrect guess as to the difficulty of their pack. But, if the ratings and their criteria are sufficiently detailed and clearly explained, then this is less likely to happen. And those occasional packs which miss the mark will likely get picked up on anyway.

So far, I've made a few suggestions which received some interest but didn't really take off. And, I get it, maybe I'm not the person to make such a scale, but as long as custom content is regularly being created, and to encourage new users to find packs at their own ability level, I do think it needs to get done.

Some initial suggestions:

The scale should be viewed from the point of view of a complete beginner, so "Easy" would mean can't-miss, any-way-you-like, open-endedness, etc. Most of us would find Mayhem "Easy" nowadays, but we grew up playing Lemmings; the Mayhem levels are still "Hard" to a beginner, even in NeoLemmix.

The difficulty rating given to a pack should indicate its maximum difficulty, ideally. Whilst level designers may wish to include "Fun-style" levels early on in their packs, that doesn't necessarily need to be indicated in the pack's given rating.

[Gronkling]

Custom packs aren't generally played by complete beginners, if it's intended for complete beginners then "Difficulty: Beginner" like the introduction pack is fine. Most people who are going to be playing custom packs will have beaten the original games I imagine. Stuff that's about the same level as the original games would be "Easy", more standard custom levels "Medium", things that require an in depth knowledge of the game "Hard", and stuff that even the best solvers on the forum find really difficult "Very hard".

[WillLem]

Custom packs aren't generally played by complete beginners

Whilst that might be true (but is impossible to know for certain), difficulty ratings shouldn't preclude any level of ability.

Most people who are going to be playing custom packs will have beaten the original games I imagine.

I'm aware of several people on the forums who haven't beaten the original games fully yet. Also, again - your statement, whilst not entirely unfounded, is impossible to know for sure and therefore shouldn't form the basis for a standard, forum-wide difficulty rating.

The only thing we can be certain about is that there are people who have never played Lemmings before, and there are people who are very experienced at playing Lemmings. The only way to have all difficult ratings covered is to cover everything from Beginner and up.

[Proxima]

The only way to have all difficulty ratings covered is to cover everything from Beginner and up.

That's certainly true; but your original conclusion that difficulty ratings should be assigned from the point of view of a beginner simply doesn't follow. To a beginner, Fun 1 to 20-ish are "easy", late Fun and most of Tricky is "medium", Taxing and Mayhem are "hard", and everything from mid-tier DoveLems upwards is an indistinguishable porridge of "impossible". Such ratings are of no use to non-beginners wanting to know how hard a pack really is.

[WillLem]

I'm absolutely happy for this not to be done in the way that I've suggested, I'm just providing a starting point. The key message here is that something needs to be done about difficulty ratings.

I suppose the problem is that no two people are going to regard any particular pack as being the same level of difficulty, which is why I suggested viewing every pack from the POV of a beginner. OK, most people on the forums are not beginners - I get that - but we all have been, and I remember really struggling with Oh No! More Lemmings because of its advanced level of difficulty, which is way beyond that of original Lemmings.

There comes a point when levels' difficulty stops being quantifiable in degrees, as well. The levels just start to seem outright impossible. I'd say that here is where it's not really worth having a scale, because it no longer becomes useful. Here, we just need the rating to say "beginners, casual players and those of around average ability will likely find this pack basically impossible, whereas those of way above average and advanced ability will find it a suitable challenge".

Therefore, my suggested scale would be:

Beginner / Easy
Novice / Medium
Expert / Hard
Master / Advanced
Genius / Extreme
Legend / Super-Extreme*

*The latter of these is really an "extra" rating to denote that even highly skilled players will find the pack an extreme challenge could be useful. Beyond that, though, it just gets into silly territory where difficulty ratings start to become meaningless anyway.

It's worth noting that, whilst a lot of custom content is harder than the original games, once solved it's just as "Easy". That's why, again, I'd suggest using a beginner's POV to design whatever scale we use.

[IchoTolot]

We already have a scale that is widely used: Easy - Medium - Hard - Extreme

So I don't think there is action needed in that regard. Maybe mods should be able to hand out/change ratings as sometimes creators misjudge their packs, but other than that the current system seems fine to me. It's just that some packs could need a re-evaluation.

I don't agree that viewing every pack from the POV of a beginner is a good thing here as most stuff would then simply fall in Hard or Extreme. The original games aren't really a good scale anymore on top of that.

[Proxima]

OK, so... what do we do, then?

Firstly, we relax and have a cup of tea. I'm on your side here even if we disagree about the details, and I don't like being shouted at

Beginners really only need to know whether content is beginner-appropriate or not, so the first two ratings should be "Beginner" (suitable for complete beginners) and "Easy" (accessible to complete beginners and will begin to stretch them).

But intermediate/advanced players also need to have rankings that help them find content at a level most suitable for them. And for such players, the gulf between late DoveLems and late United is much wider than the gap separating late DoveLems from "Just dig", so lumping them into the same rank doesn't help anyone.

On a quick side note:

Lemmings Destination is a perfect example of a pack which describes its early ranks as "Easy" when they're far more comparable to the later puzzles in ONML, which itself I would probably regard as "Advanced". We can say ONML is not Advanced now, because we've all beaten it, but I still remember struggling as early on as the Crazy rank.

Lemmings Destination is a one-off example of a pack whose difficulty is clearly mislabelled. Everyone agrees on this. But I think it's the only example of a really clear mislabel, not a symptom of a wider problem.

And yes, ONML is not "advanced". "We struggled on it once" just isn't a good metric. I struggled on Fun 24 the first time. Lemmings is a puzzle game, and ideally all levels should be a bit of a struggle at first.

I was going to continue by making my own proposal for a difficulty scale, but then Icho posted, so I'll frame my proposal as a reply to Icho instead.

[Gronkling]

How I judge it:
Beginner - For new players who may still have trouble with some tricky/taxing levels
Easy - About the same level as the original games
Medium - Quite a few hard puzzles, definitely a step up from the original games, but doesn't require an in depth knowledge of how everything works. This is the standard level for most custom packs so makes sense to be in the middle.
Hard - Puzzles are very difficult, and generally an in depth knowledge is expected
Very Hard\Extreme - For if even the most advanced solvers in the forum find it difficult to solve

These ratings just seem pretty much intuitive and use the most common words for difficulty levels.