Giving feedback on custom packs

[WillLem]

I think there needs to be a protocol regarding giving feedback on custom packs.

Whilst feedback is useful, welcome, and encouraged, when unchecked it can also be quite harsh and put people off wanting to create custom content. I'm sure that there are many people who have thought about making a pack, but have seen the nature of some of the feedback that gets posted and have been put off.

Similarly, I am very sure that there are people who have removed content due to particularly negative feedback following an intense period of working on that content, leaving them feeling like their hard work and time was all for nothing.

As a basic suggestion, let's say you play a pack and there are several things you don't like about it. Before taking to the keyboard to type your frustrations straight away, take a moment to see if there's anything you do like about it, and then be sure to include that in the comments.

Ideally, each bit of negative feedback given should be balanced out with a piece of positive feedback. Praise is important: it keeps people motivated to try harder and put the extra effort in to improve their content. Simply listing a pack's drawbacks is only going to make the creator feel like a failure. Giving feedback is a powerful thing, and with great power comes great responsibility: use it wisely.

Suggestions to creators

ALWAYS get your pack playtested by at least 2 people, and be prepared to take a bit of harsh criticism initially - this will help you to shape your pack into its final product.

Then, when you release your finished pack, add a poll at the top. Something simple like:

What did you think of my pack?

1 It's excellent!
2 It's good, but some things could be improved
3 It's not really my cup of tea
4 I haven't played it yet

(Or even just a star-rating system from 1-5).

That way, you can get a good idea of what people who've downloaded your pack think, and it could help to balance out any negative feedback you get initially (unless of course everyone votes 3, in which case it's definitely worth having another look at it anyway!)

A poll would also encourage those who aren't predisposed to typing paragraph after paragraph of detailed feedback to leave something for you, for better or worse. That way, if a lot of people vote 1 or 2, but someone leaves a blazingly heated comment about how awful your pack is, at least you can see not everyone feels that way, and it'll be that much easier to take the criticism on board constructively, even if it wasn't necessarily presented to you that way.

[mantha16]

I would also add that when commenting on aesthetics it would be helpful if people remembered that styles, custom terrain backgrounds etc is an art form and as such is entirely subjective.  Just because you dont like how something looks doesn't mean its bad or needs to be changed just that its not to your taste and feedback should reflect that.

[kaywhyn]

Whenever I give feedback, I tend to focus on the positive things rather than the negative. That's why it's better if one focuses on giving the constructive feedback type instead of the harsh criticism type, the latter which tends to be hurtful. I'm one of those people who rather praise people instead of admonish them for doing things incorrectly or not so well. This is something I feel like the whole world needs to do more: Focusing on praising people rather than be blind by prejudice/discrimination. Thus, I always find at least one good thing to say about something/someone daily.   

[Dullstar]

As a community, I'm noticing that while we've had a lot of discussions about what makes a level good or bad, we don't actually seem to have this information gathered up into one convenient spot new designers can look to for advice. Instead, it's spread through a bunch of pack reviews and old threads on the level design board. We could probably save a lot of pain for both creators and with a do's and don't's pinned thread, because at least half of new designers seem to get hit with some variant of "I shouldn't need to use Clear Physics Mode to play your levels" at least once. CPM is kind of a blessing and a curse: it's a powerful tool for cutting through deceptive (intentionally or otherwise) designs, and I find it quite useful for checking the exact details of trigger areas and checking objects I want to know more about (i.e. is this thing a trap like I think it is?), but when I have to use it to solve a level, it leaves me feeling kind of annoyed, and I think some designers just either don't understand why this is annoying, or don't understand why their designs cause players to reach for the CPM button in the first place. You could say it's because of aesthetics, but the problem is, once I reach for CPM, the aesthetics go away, because CPM takes all those visuals away.

Lots of designers do this, so I'm not trying to call out any specific individual here.

[mantha16]

if anyone can't be constructive in their feedback and not understand that some things are subjective and things don't always have to take a laissez faire attitude.

Surely the idea is to be supportive not dismissive

[ccexplore]

I agree it's a fair point especially for those who are new to creating levels, that the feedback should probably be coached in more encouraging tones.  It certainly helps to end on a more positive note if nothing else.

At the same time though, we need to be mindful not to also discourage people from either giving any feedback, or to make them feel like they can't be honest with their feedback.  Balance is important but from what's said so far, it feels like everyone has very different ideas of where that balance actually lies.

The poll is probably a good idea, it's certainly pretty efficient in soliciting quick feedback, as it often takes more effort to write any detail feedback whether good or bad, a poll by comparison is much less work for everyone involved.  Though even with a poll, early results especially may still not always be as positive as one might like, and worse, it's going to outside of any single person's control.  And you also don't get to learn precisely what is working and not working for people--for example if it's "not my cup of tea", which aspects specifically aren't?  The poll won't tell you.

Speaking of art, consider that many visitors strolling through a modern art museum, some are bound to make fun of one or more works they encountered.  This is the flipside to "art is subjective".  Usually the artists aren't too fazed by this though (and in some cases for some works, may even expect this).  I think just as there are ways to improve the dialog on the feedback giver's end, there are also ways on the feedback taker's end to reel in unbalanced feedback.  If you're struggling to find the positives in some feedback, challenge the feedback giver to clarify whether there's anything they do enjoy about the levels, things that they'd like to continue to see.  I'd also encourage everyone to not assume the worst in people or take things too personally.  Language is tricky, written language even more so, and being overly negative can often be unintentional.  Give people benefit of the doubt and talking things out I believe can ultimately help clear things up and reach a better understanding for everyone involved.  And learn to become more comfortable with different views.  In the end even in worst case, it's just one person's feedback no matter what.  Someone may never like fish just because, it doesn't mean fish is bad in general or that you should stop cooking fish for everyone else either.

[mantha16]

I don't think that the onus should be on the receiver to challenge feedback.

Its called constructive criticism for a reason.

I honestly don't care what people think of my aesthetics because that is subjective and i like chaos a lot of people like order thats fine.  It's ok to not like the same thing thats cool.

For me its like if you don't like certain things you don't have to see them.  Like I don't like jump scares in movies, I don't find horror movies enjoyable but I don't demand they should cut the gore or make them less scary I just don't watch them. 

I like the idea of their being a generally accepted list of things to bear in mind when creating levels/packs.  Having a you MUST do this or you MUST NOT do that is more problematic to me.  If you have an open medium that allows creative freedom then it should be as open as possible.

like regarding CPM if you said "please bear in mind if the way to play the level requires CPM use then some players may not enjoy it" thats fine if you said "you HAVE to be able to play a level without using CPM" I find that a little bit over the top

[Simon]

I assume this sparked the discussion: Dullstar's feedback to Telelems

This frankness is exactly what I prefer to hear from others about my work: Direct focus on problems. I wouldn't want replies to me pampered merely to comply to feedback rules. I don't see it as rude either. Especially the sprite problem isn't even a problem of the pack entirely, it's also an issue in the engine.

Can't advise how to treat whoever doesn't appreciate frank discussion. It's delicate. We all would like the hobby to grow, super-thick skin shouldn't be required.

Community expectations about level design need exposure, that's right. They're always under debate anyway, whether they're more exposed or not.

-- Simon

[mantha16]

again just thats cos its what you prefer doesn't mean I am wrong for feeling differently.  and I'm sorry but you don't know how thick my skin is.  Feedback should be constructive thats not being delicate thats just what the purpose of feedback should be.

the phrase is CONSTRUCTIVE criticism  not just criticism and there was nothing constructive or helpful about what Dullstar said.

I didn't agree with a lot of what WillLem said either but I'm not annoyed about that because he offered constructive feedback.

[Simon]

Everything what Dullstar wrote was actionable, thus constructive. Feel free to ask if it's not clear how to fix the issues. Feel free to disagree that something is an issue.

Steel/earth distinction is perhaps the least actionable; it's not necessarily clear which exact tiles the problem are. In a pinch, you can ask more people to test.

-- Simon

[mantha16]

that assumes I agree everything he said is an issue and no it wasnt constructive but tbh im over this discussion and I not longer wish to be part of it so

peace out

[IchoTolot]

As long as there are no direct isults in it, I think feedback must be honest and directly pointing at problems.

Just writing: "This is great!" over and over only gets you so far. No, it must be pointed out what works and doesn't work. So that you can see which elements are worthwhile keeping and what should be considered to adjust.

Back when I released my first pack (Lemmings Reunion), Nepster gave me a ton of feedback. Some of it was harsh, but to the point. It helped me to figure out the bad parts and I adjusted a lot of things because of it. He also wrote what worked well and that gave me hints on which things should be focused on more.
Removal of timers, unnessesary precision, making some traps more visible and reducing of redundant elements (too many builders) where his most common criticisms back then. I learned my lessons and my content got better because of it.

Feedback must reflect the true feelings of the player, otherwise it is nearly worthless and can be even misleading. When reading it the creator does not need to adjust everything pointed out, it's up to them what lessons are to be learned and it's ok to disagree with some of the comments.

In my opinion, the most important lesson here is:
People need to learn to free themselves from feelings while reading feedback and focus more on logic. Feelings can often get in the way of thinking clearly and wrong impulsive decisions are being made because of it. Take a deep breath, take a step back and analyse in peace what information can be exacted from your feedback.

[mantha16]

IT WASNT CONSTRUCTIVE

[kaywhyn]

As others have already said, it's just one person's opinion. Thus, don't take these things personally. Instead, these should be looked at as ways to improve. Perhaps it's best to look at them as suggestions instead. Of course, as the level pack author, you don't have to follow them. You can choose to follow all, some, or even none of them. It's also ok if your first pack didn't turn out as well as you wanted it to. Many of the members here had first packs that were ok but they would be a lot better if this and that changed, and these authors have improved tremendously since with every subsequent pack they have released. This is all a learning process. I can't say the same for myself, as I have never made one, and I likely don't intend to, as I am no good when it comes to making levels and am much better at solving.

Also, keep in mind that it's impossible to please everyone. Just look at the various topics in the discussions/suggestions board. Some issues were very easy to resolve, while others had a very lengthy discussion before any action/changes were made. The best you can do, in most cases, is tailor your pack to some, but not all. Of course, don't disregard your own preferences as well. In the same way that art is subjective, there will always be levels that some people like but others won't. Yes, there are some levels from L1 and ONML that were generally liked or hated by most, but there are some who thought otherwise. This is very true of the difficulty of levels. I have struggled on levels that some found easy or vice versa, where there are hard levels that most have struggled with but I didn't find so bad.

[mantha16]

the only thing i have to say is it wasnt my first pack but the feedback i got from sammings I did put into effect where I agreed with it.  I am largely really pleased with the aesthetics and IF I do decide to make changes and re-publish people should not expect a lot of aesthetic changes other than trying to better define backgrounds v terrain and steel v non steel. 

And once I understand the technical issues in changing the recolourings for athletes I will change that but only that (and correcting the actual errors) 100% I will not be redesigning the sprites because I love them