This is a site for discussing roleplaying games. Have fun doing so, but there is one major rule: do not discuss political issues that aren't directly and uniquely related to the subject of the thread and about gaming. While this site is dedicated to free speech, the following will not be tolerated: devolving a thread into unrelated political discussion, sockpuppeting (using multiple and/or bogus accounts), disrupting topics without contributing to them, and posting images that could get someone fired in the workplace (an external link is OK, but clearly mark it as Not Safe For Work, or NSFW). If you receive a warning, please take it seriously and either move on to another topic or steer the discussion back to its original RPG-related theme.

Author Topic: What do you expect out of play testing?  (Read 936 times)

Vic99

  • Old Guy That Still Games
  • Sr. Member
  • ****
  • Posts: 302
    • Wicked Cool Games FB page
What do you expect out of play testing?
« on: June 28, 2022, 12:27:44 PM »
I'm a one man operation writing a rules light fantasy d20 system.  After a year, writing is basically done, but still needs some formatting and such.  I've been running a game for 6 months using these rules (they've evolved some) and a few one shots with other groups and have gotten great feedback from my groups.

Three of my buddies will be using the system soon, one is outside the game group.  I hope to give it to a few other acquaintances and just say go to see if they can get it without my help.  I'm not giving them a scenario.

Although a lot of this is informal, I also want to standardize play testing some so I have good comparable data.

1.  What would you recommend that I ask for set up to gather good data? - not interested in another giant project . . .

2. If you were a consumer and used your precious time used a system for a game like this, what would like to have seen in play testing to say, this is pretty solid I'll give it a try?

Thanks.

Jaeger

  • That someone better.
  • Hero Member
  • *****
  • Posts: 1536
Re: What do you expect out of play testing?
« Reply #1 on: June 28, 2022, 06:28:10 PM »
My 2cents...

1: The game is to be played 100% RAW by the playtesting groups for extended periods. No fixes, no house rules.

This needs to be done so that you can get reliable mechanical feedback. If they have played several sessions, and feel something needs a fix - then they should be able to articulate exactly why it was "broken" to them.

You should have a 'feedback' schedule; every x number of session played, the group reports in.

2: Absolutely give them a scenario.

Grab something sandboxy that they can play for a while; like Keep on The Borderlands. Re-stat the NPC's/Monsters for your system, and let fly.

You want a 'control' scenario where the NPC/Monsters are a known quantity. Yes the different groups/GM's will do different things. But the idea is to establish as much play commonalities as possible with your new system between the various playtest groups.

The scenario should also be run RAW: One guy running your test scenario but passing out magic items left and right will give very different feedback to one running it straight up.

3: You should have a handout explaining exactly how you intend the game to be run. All your "test GM's" need to be on the same page.

Same with the players. i.e. You want players who will actually engage with the system. "Playtesting" with groups filled with players that constantly ask "What do I roll again?" is counter productive.

All the players need to be 'on board' with playing the game and test scenario straight-up.

You want to establish as many points of commonality as possible between the playtest groups.


As much as possible you want an objective assessment of your game: Does system X produce the intended result at the table during actual play? If not -why not?
"The envious are not satisfied with equality; they secretly yearn for superiority and revenge."

Opaopajr

  • SeƱor Wences
  • Hero Member
  • *****
  • Posts: 7768
Re: What do you expect out of play testing?
« Reply #2 on: June 28, 2022, 08:43:05 PM »
Yes, yes, yes to everything Jaeger brought up. It may not seem glamorous, maybe tedious even, but common baselines and moderate stress testing from pro-active people who have buy-in at the outset is so helpful. "No plan survives contact with the enemy," as it were, but you really want to avoid casual orthogonal playing and uncritical supportiveness. That means Rules-As-Written and constrained scopes (hence the value of Dungeon Crawls).

If it can survive the heat of that kitchen, and still be fun *to want to play* again, then it can branch out into the looser elements of play and thrive.
Just make your fuckin' guy and roll the dice, you pricks. Focus on what's interesting, not what gives you the biggest randomly generated virtual penis.  -- J Arcane
 
You know, people keep comparing non-TSR D&D to deck-building in Magic: the Gathering. But maybe it's more like Katamari Damacy. You keep sticking shit on your characters until they are big enough to be a star.
-- talysman

Steven Mitchell

  • Hero Member
  • *****
  • S
  • Posts: 3770
Re: What do you expect out of play testing?
« Reply #3 on: June 28, 2022, 11:13:32 PM »
I will put one caveat on what Jaeger said:  You might not be ready for that degree of playtesting.  If your three buddies already have some experience with the system (e.g. playing while you run it), then when they run it, that's not a blind playtest.  On the other hand, a blind playtest is an incredible amount of work, that you don't want to engage prematurely. 

So you might think about doing a playtest first with your 3 buddy groups that is kind of a dry run for later, blind playtest.  Yeah, have an adventure.  Try to stay somewhat removed (e.g. don't be a player in the game).  If they are not bothered by it, maybe observe play--without saying anything during the sessions about the game. This breaks several blind playtest rules.  On the other hand, let's say you do that kind of testing and you find things that are clearly not right yet.  It's a useful test.  The more of the semi-obvious things you can uncover and fix before starting the real playtest, the more efficient your real playtest will be. 

And let's face it, game companies don't even do blind playtest correctly, with a lot more resources.  With a 1-man shop, you want to start that kind of playtesting only when you know you are ready.

Jaeger

  • That someone better.
  • Hero Member
  • *****
  • Posts: 1536
Re: What do you expect out of play testing?
« Reply #4 on: June 29, 2022, 12:14:34 AM »
...
And let's face it, game companies don't even do blind playtest correctly, with a lot more resources.  With a 1-man shop, you want to start that kind of playtesting only when you know you are ready.

What's even crazier is that the big companies can have people lining up by the thousands to blind playtest for free.

Yet WotC consistently rolled out every edition from their 'playtesting' with known issues...
"The envious are not satisfied with equality; they secretly yearn for superiority and revenge."

bromides

  • Full Member
  • ***
  • Posts: 123
Re: What do you expect out of play testing?
« Reply #5 on: June 29, 2022, 12:20:19 AM »
A lot of these big houses have "free" playtesters (or more like those dudes who pay in to kickstart, like you're paying them for the privilege? Idiots)... except they aren't actual playtests. The playtesters are giant fans... or the playtests are done to create the illusion of playtesting, and they never actually read the playtest feedback, so the input from the (few) non-sycophant playtesters doesn't ever actually get into the game... or the sycophants drown out the more professional playtesters, so that on the preponderance of evidence in the testing results, the sycophant WOO WOOs who say everything is good tends to be what the designers hear.

I swear for the big houses, it's just a bunch of WOO WOOs, really. And that's for everything with big houses getting fanboi feedback. WOOOOOOO!

HappyDaze

  • Hero Member
  • *****
  • H
  • Posts: 5337
Re: What do you expect out of play testing?
« Reply #6 on: June 29, 2022, 12:44:55 AM »
A lot of these big houses have "free" playtesters (or more like those dudes who pay in to kickstart, like you're paying them for the privilege? Idiots)... except they aren't actual playtests. The playtesters are giant fans... or the playtests are done to create the illusion of playtesting, and they never actually read the playtest feedback, so the input from the (few) non-sycophant playtesters doesn't ever actually get into the game... or the sycophants drown out the more professional playtesters, so that on the preponderance of evidence in the testing results, the sycophant WOO WOOs who say everything is good tends to be what the designers hear.

I swear for the big houses, it's just a bunch of WOO WOOs, really. And that's for everything with big houses getting fanboi feedback. WOOOOOOO!
Can confirm this happened with FFG.

Slipshot762

  • Sr. Member
  • ****
  • S
  • Posts: 478
Re: What do you expect out of play testing?
« Reply #7 on: June 30, 2022, 04:53:48 AM »
for me its the oddball interactions of arcane mechanics producing unintuitive results as the normwhich i seek to ID in a playtest; for example last playtest we did for some mass combat rules a player turned his battalions (rectangular cards) sideways to double the number of battalions in melee contact...an unforeseen possibility and one for which no sort of standing rule or guide had yet been written.

finarvyn

  • Hero Member
  • *****
  • Posts: 1646
Re: What do you expect out of play testing?
« Reply #8 on: June 30, 2022, 07:22:03 AM »
All that Jaegar mentioned in his post.

For me, the purpose of playtest is to see what happens when someone not connected to the author opens the book and tries to run the game. Person opens book, reads rules, tries to play. Playtesters should get an adventure to run, probably, but not a lot of additional information that wouldn't be included in the rulebook somewhere.

If there are glitches in the rules, are confusing things written in the rules, or whatever -- those are the things that the playtest is supposed to find.
Marv / Finarvyn
Kingmaker of Amber
I'm pretty much responsible for the S&W WB rules.
Amber Diceless Player since 1993
OD&D Player since 1975

Omega

  • Hero Member
  • *****
  • O
  • Posts: 17093
Re: What do you expect out of play testing?
« Reply #9 on: July 01, 2022, 02:42:41 PM »
...
And let's face it, game companies don't even do blind playtest correctly, with a lot more resources.  With a 1-man shop, you want to start that kind of playtesting only when you know you are ready.

What's even crazier is that the big companies can have people lining up by the thousands to blind playtest for free.

Yet WotC consistently rolled out every edition from their 'playtesting' with known issues...

That is because WOTC was adding and changing stuff even as 5e was getting ready to go to print. And some of 5es problems early on were printing gaffes rather than outright rules problems. The rest seem to be "working as intended" by contrary designers.

Omega

  • Hero Member
  • *****
  • O
  • Posts: 17093
Re: What do you expect out of play testing?
« Reply #10 on: July 01, 2022, 02:54:54 PM »
Quote
1.  What would you recommend that I ask for set up to gather good data? - not interested in another giant project . . .

2. If you were a consumer and used your precious time used a system for a game like this, what would like to have seen in play testing to say, this is pretty solid I'll give it a try?

Ive been playtester for several games now from board to RPG to video and even one LARP.

1: That can be as simple as asking playtesters to fill out some questionnaires along the way. Or just send in feedback as they find things right or wrong. Pretty common over on BGG.

2: That may be an impossible question as everyone has their own ideas on what is enough and what is not.
For me d20m Gamma World was a miserable failure on so many damn levels its not even remotely funny. The system is broken. The art is broken, the designers are barely that. But it still has one of the best DMGs of any RPG I've seen in a long long time. How the hell they dropped the ball so thoroughly on everything else is baffling. Well till you realize who was designing it.

Theres also the problem of false feedback during playtesting. I've seen this several times where a good design gets killed or the project stonewalled because one or more of the playtesters are trying to turn the game into effectively another game. Very often to the detriment of the game. My best advice here is that you have to step back and assess if the feedback is really feedback. This is where it can get really hard and really discouraging.