Polls for possible changes to the contests

[IchoTolot]

As stated in the recent Contest Topic I want to do 2 polls regarding the rules of the contests:

Poll 1.) Change the voting round duration from 3 days normally and 4 days for finals to 2 days normally and 3 days for finals.

Poll 2.) Base the distribution of levels not on contest rules anymore and instead just pseudo-randomly distribute all levels evenly onto a reasonable number of groups.

I will leave each poll open for 3 days and make a decision based on the results.

[IchoTolot]

The first poll is up!

[Simon]

Rename the contest from
Lemmings Forums Level Contest
to
Level Design Contest.

Reason: Other contests are on Lemmings Forums, too, are about levels, too, and are contests, too. Example: The contest where we minimize skills on a given level. Thus Lemmings Forums Level Contest is not precise.

I don't mind the author/rules/versioning in the filename, but please include the level name. The level name is the handle, not the author or the rule. When I have a loose replay called Nepster_r1v3, from what contest is that?

-- Simon

[IchoTolot]

Rename the contest from
Lemmings Forums Level Contest
to
Level Design Contest.

Reason: Other contests are on Lemmings Forums, too, are about levels, too, and are contests, too. Example: The contest where we minimize skills on a given level. Thus Lemmings Forums Level Contest is not precise.

Good idea!

I agree that Level Design Contest is more precise.

I don't mind the author/rules/versioning in the filename, but please include the level name. The level name is the handle, not the author or the rule. When I have a loose replay called Nepster_r1v3, from what contest is that?

How about: Author_Levelname_RuleVersion?

[Simon]

How about: Author_Levelname_RuleVersion?

Good!

-- Simon

[IchoTolot]

The poll resulted in the following:

3 days for regular votes and 4 days for finals:
    4 (40%)
2 days for regular votes and 3 days for finals:
    6 (60%)

So I will shorten the voting period to 2 days for regular votes and 3 days for finals.

[IchoTolot]

The next poll regarding the voting system is up!

[Proxima]

But for a big change I want 2 things:

1.) A majority for an overhaul determined by a poll.

2.) After that: A concrete plan to what the new system should be. As I don't know how which desired attributes a new system should have and what exact flaws shall be corrected - here the persons wanting a change shall help me out. I want concrete ideas!

Okay, sorry I haven't gotten round to this up to now, I've been very busy with my course, and getting up early every day has meant that whenever I've not been busy, I've been too tired to really do anything

I'll start with some thoughts, and then move on to concrete proposals.

The biggest problem with the current system is far too many voting rounds. The most recent contest vote required eleven rounds to pick out a winner from twelve levels. That requires a lot of unnecessary effort from both the organiser and voters; and it means voters are not participating equally, unless they visit the forum every few days and have enough time to look at which levels are up for voting and make a vote. That makes the results far less meaningful, since the winner is heavily dependent on the luck of who was able to participate. And this will be more of a problem with the current proposal to shorten the time for each round.

So, how can we reduce the number of rounds without moving to a completely different voting system?

Firstly, no tiebreakers before the semi-final round. For example, in the last contest, the Rule 2 vote was intended to pick three of five levels. Two clearly made it through; another two tied for the third slot. What's the harm in letting both of those two go through? They had less support than the two clear winners, so they're not going to beat them in the final either.

Secondly, reduce the survival rate. In the last contest, the survival rates from each rule ranged from 50% to 66%. Do we really expect that a level that placed third of five has any chance in the final? And if it somehow did win the final, would that be a fair result, when the first poll shows it was liked less than two other levels? I propose that survival rates should be roughly equal to current attrition rates, i.e. rules with 2, 3, 4, 5, 6 levels should go down to 1, 1, 2, 2, 2 survivors.

Thirdly, have at most three rounds in the mixed group. My second proposal would reduce the mixed group to at most 6 levels, so it shouldn't need more than three rounds in any case. (I'm tempted to propose reducing it to two, but it's nice to know the third-place winner and to have a dramatic showdown between only two finalists.)

I think that with these changes (instead of reducing the length of each round, not in addition), the voting process would be a lot smoother and more enjoyable, and voting in each round would feel a lot more meaningful.

[IchoTolot]

I considered your proposals and have made up another possible plan based on that. In the end I 100% cannot satisfy everyone though. As I think even the first poll showed that the forum is somewhat split about the process in general and in the end I will have to say: THIS is how we do it. Otherwise it is an endless discussion.

Ok, but now to the plan I have:

- We pseudo-randomly distribute all levels evenly onto a reasonable number of groups (LOTY like). This will ensure the group size or the rule doesn't matter anymore as this was brought up as a critique point.

- We bring down the groups to 1/2 levels in one round each. If there is a minor tie (like 2 or 3 levels) all the levels advance. Bigger ties will still need a tiebreaker (like 4 level tie for the 2nd place or so), as this will result in a very over sized mixed group. This should be rare case in general though.

- No tiebreakers before the determination of the final 3 levels - only if there is a major tie which will result in 5+ levels advancing from a group there will be an emergency tiebreaker. So only when we are voting for the final 3 and 2 levels tiebreakers can happen normally.

- The mixed group will ensure the last 3 levels with tiebreakers (to determine 1st, 2nd and 3rd place) and there will be at maximum just 1 round between the start of the mixed group and the semi-final with 3 remaining levels.

- The voting duration will stay at 3/4 days.


This way I will cover the following critiques:

- The group size and the rule doesn't matter anymore

- Fewer voting rounds

- Reducing of tiebreakers

- Reducing of the survival rate

- Only 3 major mixed group votes

- Enough time to vote

Please share your opinion on this plan!

[Simon]

Hats off to Icho for being open (to change election system) after the poll (about duration of voting in existing election system that has many votings) even if it brazenly overrides the poll result. This is sensible: The main problem is that the elections take too long overall, and there has been incentive to vote (for shortest poll time) merely to accelerate the overall process.

Fewer rounds with long voting times is good. Reducing survival rate is good.

Tiebreakers should be rare. If survival rate is low, you can let tying levels pass instead of tiebreak round, in a pinch. That's good.

If the winning level has to beat levels of other design-rules, then it's unnecessary to group by design-rule in the first place. Yes, make random groups across all entries; in a pinch, prefer fewer groups with more levels each. Then vote within each bucket, then merge the survivors from all groups.

If a contest has < 10 entries, you don't even need groups; everything can go into a single poll. The only reason to make groups seems to be cognitive ease? It's easier to pick a favorite 3 from 8 than it is to to pick a favorite 5 from 23.

-- Simon

[IchoTolot]

Hats off to Icho for being open (to change election system) after the poll (about duration of voting in existing election system that has many votings) even if it brazenly overrides the poll result. This is sensible: The main problem is that the elections take too long overall, and there has been incentive to vote (for shortest poll time) merely to accelerate the overall process.

I initially thought the 2 main problems were a too long voting period and the group size inequality and the 2 polls here were the easiest way to address the problem.

I did not consider the solutions and points from Proxima before his post and just thought that this would be indeed better, having a still longer time to vote, but fewer rounds. I am more in favor of this proposal now.

Both polls, the 1st one and the current one, are very close and not one-sided. So I feel not very bad for deviating from the result.

Also I still want to ensure that everybody has indeed a chance to vote.

If a contest has < 10 entries, you don't even need groups; everything can go into a single poll. The only reason to make groups seems to be cognitive ease? It's easier to pick a favorite 3 from 8 than it is to to pick a favorite 5 from 23.

I would not draw the line at 10 though. I still think a group of 8 is a bit too big. My line would rather be at roughly 6. For 10 I would make 2 groups of 5. I am still in favor of smaller groups.



As a result from this I am more and more in favor for changing the system up to the plan I proposed earlier:

- We pseudo-randomly distribute all levels evenly onto a reasonable number of groups (LOTY like). This will ensure the group size or the rule doesn't matter anymore as this was brought up as a critique point.

- We bring down the groups to 1/2 levels in one round each. If there is a minor tie (like 2 or 3 levels) all the levels advance. Bigger ties will still need a tiebreaker (like 4 level tie for the 2nd place or so), as this will result in a very over sized mixed group. This should be rare case in general though.

- No tiebreakers before the determination of the final 3 levels - only if there is a major tie which will result in 5+ levels advancing from a group there will be an emergency tiebreaker. So only when we are voting for the final 3 and 2 levels tiebreakers can happen normally.

- The mixed group will ensure the last 3 levels with tiebreakers (to determine 1st, 2nd and 3rd place) and there will be at maximum just 1 round between the start of the mixed group and the semi-final with 3 remaining levels.

- The voting duration will stay at 3/4 days.

I already got 2 voices from the discord chat that they have no objections against this and if no flood of angry posts arrives saying that this is terrible, I think I will try this out. We can still make adjustments if needed.

[Proxima]

The new plan sounds very good to me. There's only one thing I don't like about it -- I would rather keep the first round being grouped by rule so that we establish the winner of each rule as well as the overall winner of the contest -- especially for rules like "Make a Tame level" where all entries have no chance of winning overall. But if people want to get rid of rule-grouping, I'm okay with that, and it's still a big improvement on the current system.

[Simon]

Whatever cuts number of rounds is good. We can try those ideas.

For the record/future discussion, I believe we should prune much, much faster per round than (10 go into two groups of 5 with 2 or 3 surviving). I would run 10 entries like the French elect their president, with exactly two rounds: First round has all entries, best two advance to second round.

I admit that it's taste, some want a spectacle, and some want sub-winners among the levels for the same design rule, it's all good. Also the organizer should, to an extent, just do what makes themselves feel best.

-- Simon

[namida]

The new plan sounds very good to me. There's only one thing I don't like about it -- I would rather keep the first round being grouped by rule so that we establish the winner of each rule as well as the overall winner of the contest -- especially for rules like "Make a Tame level" where all entries have no chance of winning overall. But if people want to get rid of rule-grouping, I'm okay with that, and it's still a big improvement on the current system.

I agree with this. Some rules are likely to lead to more successful (I wouldn't necesserially say "better") levels than others, so I think it's worth running them separately.

[IchoTolot]

The new plan sounds very good to me. There's only one thing I don't like about it -- I would rather keep the first round being grouped by rule so that we establish the winner of each rule as well as the overall winner of the contest -- especially for rules like "Make a Tame level" where all entries have no chance of winning overall. But if people want to get rid of rule-grouping, I'm okay with that, and it's still a big improvement on the current system.

I agree with this. Some rules are likely to lead to more successful (I wouldn't necesserially say "better") levels than others, so I think it's worth running them separately.

I think this is the main point to discuss still.

If some rules lead to better levels, then those from "worse" rules will drop out anyway during the contest. They will get into the mixed group sooner or later. I still think the equal group size with equal surviving rates is a higher goal to achive than delaying the inevitable mixing.