Forum > Pen and Paper Roleplaying Games (RPGs) Discussion

'B-but that didn't come up in the playtest'

<< < (3/4) > >>

Steven Mitchell:
Yeah, good testing is hard and expensive.  The GM has a brain and should use it, because the game can't cover everything, even if somehow magically designed, developed, and tested perfectly. 

That's not an excuse to skip testing entirely or use bad practices (thus wasting whatever testing time you have) or otherwise do multiple idiotic things (like make multiple rules so bad that a thoughtful editorial pass should catch them long before testing). 

Tester feedback is probably the hardest part of testing.  It's laughable, almost hysterically bad at coming up with solutions to problem, and often completely off the mark on even the diagnosis of the root problem.  But when the feedback is negative, it's nearly always one of two things: 

1. The tester wants the thing to do something it wasn't designed to do.  Which means you need to discover and correct why the tester got the wrong impression.
2. There is a real problem, but it bears no resemblance to what the tester thinks it is.  Which means you need to dig out the real problem, correct, and retest. 

In general*, people are really bad at understanding why something doesn't work, but really good at recognizing what isn't working for them.

* There are, of course, numerous exceptions to this in any specific field, but even people good at testing and analysis are surprisingly likely to revert to the general norm outside their area of expertise.  That is to say that good testing is a combination of understanding the thing being tested and being good at testing processes.

Bedrockbrendan:

--- Quote from: Steven Mitchell on October 15, 2021, 08:41:51 AM ---

Tester feedback is probably the hardest part of testing.  It's laughable, almost hysterically bad at coming up with solutions to problem, and often completely off the mark on even the diagnosis of the root problem.  But when the feedback is negative, it's nearly always one of two things: 

1. The tester wants the thing to do something it wasn't designed to do.  Which means you need to discover and correct why the tester got the wrong impression.
2. There is a real problem, but it bears no resemblance to what the tester thinks it is.  Which means you need to dig out the real problem, correct, and retest. 




--- End quote ---

This is pretty important: you can mess up a game responding to feedback, as much as you can ignoring it. It takes a while to figure out how to sift through feedback and figure out what feedback is relevant and what is going to make the game worse. Some of my biggest mistakes designing games have been responding to well reasoned feedback that just didn't fit what I was trying to do with the game. Usually the feedback itself is perfectly fine, and well intentioned. It just isn't accounting for big picture stuff, and when you are in the weeds of play testing, you can easily get disoriented. Now I have a much better idea of how to keep things focused.

There is also the added problem that full campaigns and individual playtests are different things. So some things that seem like problems during playtest, turn out not to be in the context of a long campaign, and some things that don't seem like problems in short playtest scenarios, become problems in larger campaigns. And larger campaigns are the harder thing to test because those take time and each long campaign only gives you a narrow glimpse into what other groups are going to experience playing your game.

Another issue worth talking about is time for development. Everyone is different, but I shifted to a minimum of two years for most books (excluding things that are more minor or small scale). The one I am working on now, has been in development since 2017.

Chris24601:
One of my solutions to getting better feedback from my testers was to deliberately test alternatives; my previous example where I tested both static modifers and an advantage-like mechanic was just one example where I did that.

A similar thing that can help is to actually reach out to specific playtesters when you notice something. That was how I figured out the Halfling issue. I noted in the feedback from blind tests involving char-gen that no one was making Halflings. So I started asking some of my testers and the pattern of answers indicated the “too disturbing” aspect.

And I’ll definitely agree you need blind tests if you’re going to do any at all. I found several issues in my char-gen system where people dove right past the section on attributes and ultimately had to ask me how they were determined because they couldn’t find them.

Another was that originally I covered backgrounds before classes, but the classes often offered some extra options to the boon selections backgrounds offered and people running straight through the book for character creativity had already picked their boons before they even looked at what extras the class offered.

So even though your background is functionally more important to defining most PCs than class is (backgrounds are all your non-combat options, classes cover only combat) I had to put classes first in the layout because most blind testers would note the options that way and then also check those when they later reached the background section.

That’s not something I would have caught if I’d actively involved myself in those char-gen tests (I was AT the session so I could observe and if someone was stuck I could answer questions… but just the need to ask me a question was an important datapoint).

Hakdov:
Steve Jackson wrote long ago in his book on game design on the value of playtesting, especially blind playtesting.  And you can tell that he actually practiced what he preached from the very tight designs of his early games like Ogre and Melee. 

Shawn Driscoll:

--- Quote from: Spinachcat on October 14, 2021, 04:00:14 AM ---As to "where's the quality?", consider yourself a gold miner. You gotta hack through a lot of not-gold to hopefully find the hidden gold. And once you find that "golden game", play the damn thing with lots of friends and sing its praises for the rest of us.

--- End quote ---
I recently discovered a fairly new game designer that has been producing game after game each week or two, it seems, for the last couple of months. I plan to do videos once I finish my reviewing.

It's been years since I bought an RPG, I think, other than RPGPundit's books.

Navigation

[0] Message Index

[#] Next page

[*] Previous page

Go to full version