Points system for completing levels

[covox]

Hey guys (if anyone's still around),

If there was a theoretical points system, which acted as a function of number of lemmings saved and time remaining, what would (in your opinion) be the best balance.

Code Select Expand

def getScore(self):
        if [all lemmings need to be saved]:
            return 5000+10*[time remaining in seconds]
        else:
            return 1000 + int(4000*([number of lemmings rescued] - [number of lemmings needed to be saved])/([number of lemmings in total] - [number of lemmings needed to be saved]))+10*[time remaining in seconds]

(flagfall of 1000 points, + up to 4000 points for saving additional lemmings, + 10 points per second left on the clock)

That is the current unpolished mechanism that sitting in my clone (which will be released once I finish my yearly academic spadework). The aim is to give a reasonable score that fully takes into account the difficulties involved with completing that level. This could be used in cumulative totals, average scores, etc., for each difficulty/game set, as well as providing a single number that's easier to gloat about  :winktounge:

Of course, my current "saving additional lems" bonus is biased against levels which force you to sacrifice a couple and reflect this in the level requirements. And I'm worried about the time modifier, as it is linear rather than proportional (thus racking up huge advantages for levels with surplus time). And inserting another variable for the current difficulty level would be good.

So, I ask the question, roughly how valuable is saving lemmings over saving time?

(and no, this isn't going to completely replace percentage/time highscores)

[STT]

Hmmm... clever, but what about those levels where you can only save, say 6/10 max?

[Shvegait]

I'm going to throw in another variable without answering your question, covox, but what about unused skills? If you plan on doing a scoring system, certainly economy of skills should come into play.

[Proxima]

Hmmm... clever, but what about those levels where you can only save, say 6/10 max?

I don't see this as being a huge problem; it just means you can't get points for saving more than the requirement on those levels.

It's just a matter of finding the right balance.... levels can have up to 9 minutes, which is 540 seconds, but it should *never* be possible to finish a level with more than about 5 minutes (300 seconds) to spare (except in very rare situations like optional multi-tasking -- 10 Tricky); if it is, the designer has allowed too much time. So 10 points per extra second seems reasonable enough.

I'd make the basic score for completing a level depend on the rating category; perhaps 1000 points for completing a Fun level and an increment of 500 or even 1000 for each category above that? That may sound like a lot, but it does compensate for the fact that earlier levels tend to have more spare time and lower save targets.

Instead of a separate formula for 100% needed levels, I'd allow some points for number (or percentage) of lemmings saved, but more points for saving more than the minimum. That way, you don't create a needless difference between "100% needed" and "all but one needed, 100% impossible anyway".

A simple mechanism would be (25 x percentage of lemmings needed) + (50 x percentage of lemmings saved above requirement), so you'd automatically score 2500 on a 100% needed level but could get up to 5000 by saving more than the minimum. Perhaps saving 100% when not required could have a fixed score of 4000 and you get this or the score from the above formula, whichever is greater.

Forget unused skills -- the highest number of skills on a standard level is 320 but a user-defined level could have up to 792, so even 10 points per skill would unfairly swamp out the lemmings saved and time bonuses, while any fewer would make the scores from best play on a level with fewer skills available not worth having.

[Shvegait]

Forget unused skills -- the highest number of skills on a standard level is 320 but a user-defined level could have up to 792, so even 10 points per skill would unfairly swamp out the lemmings saved and time bonuses, while any fewer would make the scores from best play on a level with fewer skills available not worth having.

Why are you assuming that the score must be such a simple function? What if you got a bonus for the *percentage* of skills that you have leftover. Then you have no such problems for high numbers of skills, since you can set a hard cap. Make it very low, sure.

The point is, if two people save the same number of lemmings, and have the same time remaining, the person who used fewer skills should get a better score!

--------------------

The problem with any such scoring system is that it only makes sense in the context of a single level. In my mind, the best theoretical scoring system would be where each level had an equal "perfect" score (most saved, minimum time, minimum number of skills used), and you *deduct* points from that score for every imperfection.

Of course, this has several implementation problems. For one, who determines the most possible saved, the minimum time possible, and the minimum number of skills used? Surely you might think you have the best solution without actually having it. Two options:
1) A dynamic retro-active scoring system could solve this issue. If you save more than the max thought to be possible, then the max is set to the new max. Etc. for the other possibilities. [This would work best with an online hall of fame.]
2) Just add a small bonus formula, so you can get higher than the "perfect" score. (I would not really recommend using the negative version of the deduction formula, because for instance in a lose 2 level where lose 2 is thought to be the max possible, saving 100% would result in an explosion (or divide by zero, or something of that nature) of that part of the score.) [This would work best with an isolated single player game.]

Another problem, precedence for scoring would have to be set up properly. What is "more perfect", having more time left over, or using fewer skills? Is saving one second better than saving 10 builders? And even within skills, is saving 2 climbers better than saving 1 builder? For the most clarity, the precedence should be clearly defined, so that for instance, saving 1 second IS better than saving 10 builders, OR that saving 1 builder IS better than saving 2 minutes. And probably each skill should be treated equally for simplicity.

So probably deduct the most points for a lemming lost, less points for a skill used (can certainly use a percentage based system here), and single points for each extra second used (or use a percentage based system, but then you are liable to run into ties when scores differ only by 1 second in long levels).

The system probably sounds a bit complicated, but it has the advantage that the maximum score the player is striving for is clearly defined, and that all levels would carry the same weight. This value could be multiplied by the difficulty rating, so that Fun levels are worth at most 5000, Tricky 10000, Taxing 15000, Mayhem 20000... Or whatever (and if you use the bonus system for exceeding expectations, of course you can go over those maximums slightly). Then when you see that you got 18240 score on a Mayhem level, you know that you can do better... and if you get 14997 on a Taxing level, you probably used the intended solution, just slightly slower. (And very high scores above the "perfect" score would instantly flag a backroute!)

[Proxima]

Why are you assuming that the score must be such a simple function? What if you got a bonus for the *percentage* of skills that you have leftover. Then you have no such problems for high numbers of skills, since you can set a hard cap. Make it very low, sure.

Actually, I did consider a percentage bonus system and decided it would run into the same problem. However, I suppose it could *just* work if you make it 10 x percentage of skills unused for a maximum of 1000; then saving one skill on a level with, say, 20 skills in total is worth the same as saving five seconds on the clock. That still seems to have the problem of overvaluing unused skills on levels with a large number of surplus skills while undervaluing them on levels with very few, but it's up to you whether you think even that's better than not counting unused skills at all.

[covox]

Phwoar, there have been some brilliant ideas mentioned. Base score, remaining skills, hmmmmmm...

Code Select Expand

def getScore(self):
    baseScore = 1000 + [difficulty, 0-3]*500

    if [all lemmings need to be saved]:
        rescueScore = 4000
    else:
        rescueScore = 4000*([number of lemmings rescued] - [number of lemmings needed to be saved])/([number of lemmings in total] - [number of lemmings needed to be saved])
   
    skillScore = 0
    for skill in [every possible skill]:
        if [the skill was empty to begin with]:
            skillScore += 300
        else:
            skillScore += 300*[amount of skill remaining]/[initial amount of skill]

    timeScore = 10*[time remaining in seconds]

    return (baseScore + rescueScore + skillScore + timeScore)

Here's a possible way it could pan out. So far, the bonus hierachy is:
Extra lemmings > Extra skills > Extra time

I like the idea of a glass ceiling depending on difficulty, however it'd be impossible to hit the maximum (if you throw time and skill use considerations in the mix). An online ladder would be nice however that's slightly ambitious at this stage, never mind how delightfully easy it would be to hack.

Instead of a separate formula for 100% needed levels, I'd allow some points for number (or percentage) of lemmings saved, but more points for saving more than the minimum. That way, you don't create a needless difference between "100% needed" and "all but one needed, 100% impossible anyway".

The "if" fork in the first formula was to prevent a nasty divide-by-zero error (it determines the bonus using the number of extra lemmings saved. which of course is zero when you're obligated to save 100%  :winktounge:)

[Mindless]

An online ladder would be nice however that's slightly ambitious at this stage, never mind how delightfully easy it would be to hack.

If you wanted to prevent hackability, you would have the client (game) record the user's solution to the level, then this would be submitted (along with the score, if you want to assume good faith) and the server would verify that the solution's score by replaying the recording.

Of course, this would mean a lot of work for you, so I don't expect this to ever happen.

[covox]

If you wanted to prevent hackability, you would have the client (game) record the user's solution to the level, then this would be submitted (along with the score, if you want to assume good faith) and the server would verify that the solution's score by replaying the recording.

That's actually not a bad idea. Emulating is maybe a bit over the top; replaying every single recording might be a bit taxing on my poor server, and the code would have to get by using nothing Pygame-related at all. However there's nothing stopping an audit of the demo (i.e. see if the correct number of lemmings get released, submit the X position of the entrance and exits, see what skills get used, regenerate the score again etc).

Of course, it's not bulletproof by any means, and if the user base ever gets big enough there will be one smartypants, willing to dig through miles of obfuscated Python source code, just to see their name next to a 1 with eight 0s trailing behind. Still, if the demos are put on display, it'll become quite obvious who's been telling porkies (perhaps through a hot-or-not voting system)

Of course, this would mean a lot of work for you, so I don't expect this to ever happen.

You underestimate how childishly easy Python CGI scripts are to write. Perhaps for the third release milestone

[LemSteven]

Hi everyone, I'm back! :smiley:

I really like this scoring idea.̆ Thanks, Covox, for coming up with it!̆ However, I agree with Shvegait's comment that that the systems that have been suggested so far are more complicated than they need to be.̆ I've created a much more simple system, using some of the ideas mentioned in this thread:

If the minimum save percentage is not reached, the level score is automatically zero.

Otherwise,
Lemmings Saved Bonus = 200 points for each percent saved
Time Bonus = 1 point per second left
Unused Skill Bonus = 10 points per skill
Rating Bonus = 0 for Fun, 500 for Tricky, 1,000 for Taxing, and 1500 for Mayhem

I think this system creates a good balance between the bonus categories.̆ Each percentage point is worth the same as twenty skills, and each skill is worth the same as ten seconds of time.̆ The only major flaw I see with this system is that there is no "perfect" score, but with new records still being set and broken on occasion, that is not all bad in my opinion.̆ Feel free to disagree with me, though.

Here are some examples of my scoring system in use:

Fun 1: 100% saved with 260 seconds to spare and 9 diggers remaining
Level Score = 200*100 + 260 + 9*10 + 0 = 20350 points

Tricky 15: 70% saved with 275 seconds to spare and 2 bombers remaining
Level Score = 200*70 + 275 + 2*10 + 500 = 14795 points

Tricky 21: 100% saved with 250 seconds to spare, using 15 of the 528 skills given
Level Score = 200*100 + 250 + 513*10 + 500 = 25880 points

Mayhem 3:̆ 100% saved with 5 seconds to spare, using 6 of the 8 skills given
Level Score = 200*100 + 5 + 2*10 + 1500 = 21525 points

Mayhem 20:̆ 98% saved with 200 seconds to spare, using all of the given skills
Level Score = 0 points (the minimum save percentage is 100%)

[Proxima]

Each percentage point is worth the same as twenty skills, and each skill is worth the same as ten seconds of time.

I still say that radically overvalues unused skills -- especially on levels like your example of Tricky 21 where there are hundreds of skills you have no reason to use. Why not give one point per unused skill?

Come to think of it, I think you have the points for lemmings saved way too high as well -- you can see from your examples that it makes the category bonus negligible. Why not 25 or 50 as I suggested?

[geoo]

One point of interest is whether you also want to compare scores from a set of levels (i.e. "Who has the best score in the range of e.g.  Mayhem 1-10") or only between single levels (i.e. "Who has the best score on e.g. Taxing 18"). I suppose the former would be even harder to achieve, the latter might be not that bad.

Just some thought how I'd do it, it'd be an approach for the latter:
First, some 'level' score: lv = [rating value] - constant_l*(p - q) + 2*constant + 1;
p and q being either 1 or 0; p being 1 if the level allows to lose more than 4 lemmings; q being 1 if the amount of all skills (except for the builder) is equal and higher than 9. I consider p and q effects which apply mostly to less difficult levels. 'constant_l' being a constant devaluate/revalue the impact of p and q on the level score.
[rating value] is 0 - 3 from Fun to Mayhem, 0 for Tame, 2 - 5 from Crazy to Havoc.
Therefore the 'level' score is always at least 1, and at maximum depending on the constant, e.g. for constant = 1 it is 8 for Havoc with none of p and q true.

'time' score: time_scr = factor_t*([time left]/[time given])

'lemmings save' score: save_scr = factor_s*(lv + constant_s_lv)*(1 - ([lemmings lost]+constant_s)/([maximum amount of lemmings that may be lost]+constant_s))
constant_s_lv devaluates the effect of the differences between the 'level' scores; constant_s devaluates the score if all lemmings have been saved and not all have been required.
I added the 'level' score here because it's usually more difficult to save more lemmings on more difficult levels.

'skills left' score: skills_scr = factor_sk*(lv + constant_sk_lv)*(1 - .125*sum(([skills of that kind used] + constant_sk)/([skills of that kind given] + constant_sk)))
constant_sk devaluates the effect of using none of the skill of a kind.
Level score added because it's usually more difficult to save skills in more difficult levels.

The overall score is the sum of these scores.
constant_s_lv should be smaller than constant_sk_lv, because that way the amount of lemmings saved gets a more quickly increasing impact on the overall score with an increasing difficulty level than the amount of skills left. For the time, I left the level score out, which is equal to using infinite for constant_time_lv.
That way, the amount of lemmings saved in easier levels has (relatively) much less impact on the score than in harder levels, the amount of skills left (relatively) not as much less impact, and the time has (relatively) equal impact always.

Setting some proper values for the factors and constants the system would approximately work the way I want, which is not necessarily the way you want it to work.
So, these are just my (quite complicated) thoughts. :wink:

[Fleech]

This sounds like a good idea, so I'll give my thoughts for what it's worth. There's some complicated ideas here, but I agree with LemSteven and Shvegait that a more simple formula will do just fine.

This is all based on the idea that the number of lemmings saved is the ultimate decider on how well you've done, with skills used and time used the 2nd and 3rd comparisons respectively. If you make time/skills used too important, you'll end up with a situation where you can earn more points saving less lemmings than saving as many as possible, which surely shouldn't be allowed to happen?

Base Score - IMO having a base score based on difficulty is needless. We all know the levels aren't ordered correctly anyway, and saving more than required is much harder on some early levels than it is on later ones (Cascade anyone?). Can you justify giving All Or Nothing the same base score as Save Me?! It doesn't really matter whether it's used or not, but it's unnecessary since everyone will be given it anyway, making the extra points completely worthless when comparing scores.

Lemmings - it makes sense to use the save percentage rather than the amount saved here, which everyone seems to have agreed on already. The fact that some levels aren't 100%able is irrelevant - the more you save the more points you get, simple. I don't see why it should matter if there's variation in the max possible scores for each level, since trying to judge if you're better at one level than another is fairly pointless anyway.

Skills - the huge amount of excess skills in some levels can be ignored by removing points for those used rather than adding points for those left. Using the percentage of those used (as suggested) is okay, but it still means the unnecessary skills are earning you points, which they shouldn't. This way, you're only judged on what you actually use.

Time - again, simply removing points for time used is the only way I can see that makes sense. There are plenty of levels that give a ridiculous amount of unneeded time, and this extra time should not be earning you points.

My suggestion then;

([Save %] * 100) - (10 * [Total Skills Used]) - ([Time Taken In Secs])

This is a ratio of 100:10:1 for Saved:Skills:Time.

I've not really tested these figures, but the general idea should be fine. I realise this makes the time taken a fairly negligible number, but it doesn't matter. It's not the score that's important, it's the score compared to other scores that is. Saving 3 seconds might not make much difference to the actual score, but the time is only considered as a final decider when everything else is equal anyway.

Just think of it as say, Minutes:Seconds:Tenths. One tenth in a 10 minute race seems negligible, but it makes all the difference in a photo finish :tongue:

[covox]

LemSteven: Perhaps dumping the base score isn't too bad. It does seem disproportionate giving out points for lemmings which have to be saved though; it makes most scores padded out to the 16000 mark. Would it be possible to just use the fraction extra?

geoo89: Oh my word. :mikehuh2: Isn't seven symbolic constants a little bit over the top for a score generator?   

SgB: I agree with the "less is better" philosophy when calculating score; if there are too many extra tweaks then the end result isn't very meaningful. However, it would appear that the formula you proposed would generate mostly negative scores

[Shvegait]

SgB: I agree with the "less is better" philosophy when calculating score; if there are too many extra tweaks then the end result isn't very meaningful. However, it would appear that the formula you proposed would generate mostly negative scores

How's that? Assume you save only 50%, then your base score is 5000. If you take a whopping 10 minutes, your remaining score is 5000 - 10 * 60 = 4400. You'd have to use > 440 skills to get a negative score! Maybe you didn't follow the formula carefully, it seems quite reasonable.