Also, whenever you create a rule or interlocking system of rules that involves numbers, subject it to an 'expected value' calculation to see if it functions the way you intend. As an example of what goes wrong when you fail to do this, I submit the sad story of the combat system in Decipher's Lord of the Rings roleplaying game. While in many respects a beautiful and thoughtfully made game, the combat system looked sort of familiar on casual inspection but was actually a disaster. The problem was that the expected value of damage per turn (chance of doing damage x average damage per successful attack) divided into the typical hit points of garden variety foes meant that it took something like 20 turns of combat to kill an orc. It was sort of like a D&D-based fantasy heartbreaker where damage and to-hit chances were taken from OD&D and typical hit points from 4E. The thing looked sort of o.k. if you hadn't played it, but was totally unplayable. Fan boys argued that it was fine by pointing out that if you did maximum damage every turn you could kill someone in 2-3 turns or something. Which is complete horse shit because the odds of that happening are 1 in thousands. You can save yourself a lot of heart ache by just working through in your head the most likely, 'mean' outcome of various sorts of rolls, and thus the survival chances and times to complete combats or other actions.

First you need to run it a few times so people know how the game works.

Then you need to turn over the rules and leave, and find out later what they did to your baby.

Quote from: Gronan of Simmerya;953873First you need to run it a few times so people know how the game works.

Then you need to turn over the rules and leave, and find out later what they did to your baby.

And be emotionless and clinical.  When you return to find your baby has been skinned, dissassembled, the skeleton looking now something like a hunchback Skaven with random parts haphazardly sewn back on, don't scream and reach for the shotgun, just go "Huh, interesting.  Why did you shove his left eye up his ass"?

Quote from: Larsdangly;953869Also, whenever you create a rule or interlocking system of rules that involves numbers, subject it to an 'expected value' calculation to see if it functions the way you intend. As an example of what goes wrong when you fail to do this, I submit the sad story of the combat system in Decipher's Lord of the Rings roleplaying game. While in many respects a beautiful and thoughtfully made game, the combat system looked sort of familiar on casual inspection but was actually a disaster. The problem was that the expected value of damage per turn (chance of doing damage x average damage per successful attack) divided into the typical hit points of garden variety foes meant that it took something like 20 turns of combat to kill an orc. It was sort of like a D&D-based fantasy heartbreaker where damage and to-hit chances were taken from OD&D and typical hit points from 4E. The thing looked sort of o.k. if you hadn't played it, but was totally unplayable. Fan boys argued that it was fine by pointing out that if you did maximum damage every turn you could kill someone in 2-3 turns or something. Which is complete horse shit because the odds of that happening are 1 in thousands. You can save yourself a lot of heart ache by just working through in your head the most likely, 'mean' outcome of various sorts of rolls, and thus the survival chances and times to complete combats or other actions.

I write programs to test this doing thousands of iterations for each level of damage, hit points, and defense. Which reminds me, I need to run that for my latest setup.

Quote from: Larsdangly;953869Also, whenever you create a rule or interlocking system of rules that involves numbers, subject it to an 'expected value' calculation to see if it functions the way you intend. As an example of what goes wrong when you fail to do this, I submit the sad story of the combat system in Decipher's Lord of the Rings roleplaying game. While in many respects a beautiful and thoughtfully made game, the combat system looked sort of familiar on casual inspection but was actually a disaster. The problem was that the expected value of damage per turn (chance of doing damage x average damage per successful attack) divided into the typical hit points of garden variety foes meant that it took something like 20 turns of combat to kill an orc. It was sort of like a D&D-based fantasy heartbreaker where damage and to-hit chances were taken from OD&D and typical hit points from 4E. The thing looked sort of o.k. if you hadn't played it, but was totally unplayable. Fan boys argued that it was fine by pointing out that if you did maximum damage every turn you could kill someone in 2-3 turns or something. Which is complete horse shit because the odds of that happening are 1 in thousands. You can save yourself a lot of heart ache by just working through in your head the most likely, 'mean' outcome of various sorts of rolls, and thus the survival chances and times to complete combats or other actions.

I have some detailed spreadsheets for that sort of thing.

What stumps me more is non-combat stuff, because it's less based "how likely is this to happen" and more based on "how can failure be interesting."

Quote from: Ashakyre;953595I do this in music very easily. The only thing that's tricky is... first the other person has to know what you're trying to achieve, right? I always say I can't give you advice if I don't know what you're aspiration is. So... someone could say "combat is too hard" and I might say "you didn't even try to run away."

But what I just said easily becomes an excuse to not take criticism. Very easily. So what I've learned form music is... listen to the problem but take the other person's idea for a solution very skeptically... but try everything you can within your concept to address the problem they brought up.... If that still doesn't work either your concept is self-defeating or the other person isn't your audience.

As far as the other person knowing what you're trying to achieve, you should make it very clear to your players that you are engaging in a Playtest.  That you need them to point stuff out to you, and that they should understand it might not work like a regular RPG campaign; possibly including the rules changing as you go along.

As for the rest, yes, you do have to consider the nature of the criticism, and whether any suggested fixes are really going to be fixes. But I've seen way more people err on the side of not listening to good criticism than listening to bad criticism. Usually in a decent sized playtest group, you can judge by what the majority of players think, as to whether a problem is really a problem (and a solution is really a good solution).