This image spawned the following exchange over on Google+. (Given that it’s challenging to get people to a place where they can even begin to start discussing rpg design, I thought it was worth reposting this here.)
Jeffro: This is how the vast majority of people view rpg products. When they pick up a new one, they assume that this will be their overall experience. I don’t think that’s a problem, really, but the tone of most rpg rule sets come off as if the author is completely oblivious to the fact that this is the case. If I’m reading someone’s game book, I’d like to see some indication that they’ve not only played the game themselves, but that they’ve paid attention to other people that have as well. The less actual play is involved in the development process, the more people will have to do without thier genius in favor of whatever actually works.
Lewis Pulsipher: In other words, playtest the game. But RPGs have tended more and more to be about stories, so the game becomes secondary, hence (in the mind of the author) needing less (or no) testing? In great contrast to the “big” RPGs that get lots of playtesting before publication.
Tim Jensen: Wait, we might have to start naming names here.
Most of the small press RPGs I play these days (various Powered by the Apocalypse games, Fate, Microscope/Kingdom, Hope Inhumanity, Dog Eat Dog, Swords Without Master, Itras By) had open playtesting both at conventions and online, sometimes for years before being published.
The “big” RPGs I play or have played recently (Numenera, Night’s Black Agents, Rogue Trader, 7th Sea, and the World of Darkness games) all suffer from an obvious lack of playtesting. I’ve read interviews with game designers who haven’t played their own games, or who never follow the rules when asked to run them at conventions. Entire game lines have been written by unpaid interns with negligible RPG experience, given a single pass by a line developer/editor, and sent out. I have been in multiple GenCon seminars where RPG developers have said that their games “don’t need playtesting” because they know how to write games so well.
D&D4 skimped on playtesting after level 15, and D&D5 has some striking character balance issues despite their famous playtesting program. Granted, part of the charm of these, Pathfinder and Savage Worlds is finding the exploitable bits to build the most optimized character, but I’m not sure if that’s what the designers intended.
There is also an anti-design, nostalgia movement in the hobby which is concerned with emulating the play experiences with the earliest editions of D&D. DCC has been pretty good with innovation in this space, but the rest haven’t added anything to the conversation that wasn’t said by 1981.
Playtesting is hard. It takes forever even when those who are doing it don’t flake out. And there is an older segment of the RPG designer community that either doesn’t understand or care about game design enough when it conflicts with their livelihood.
Lewis Pulsipher: Different definition of “big” – I was thinking D&D and Pathfinder (and I don’t actually know how much PF is/was tested, but since it’s strongly derived from D&D3 there was already lots of playtesting, practically speaking). The ones you name aren’t big to me, more middle tier.
Yes, playtesting can be hard – or at least, time-consuming.
Note: you can follow Dr. Pulsipher at his blog, on Google+, and on Twitter.