I feel kinda guilty for talking about either science, politics, or video games lately. I just haven't had a lot of truly interesting religious thoughts. At least, not interesting enough to share here.
In any event, over at Twenty Sided, Shamus asked what made a great RPG. I gave my answer there, but I thought I'd share my thoughts and ask for yours.
Part of the trouble everyone had in answering this question was that there are two very different conceptions of an RPG. Western RPGs tend to let you pick a role and then set you loose to play in the sandbox. Japanese RPGs tend to give you a role and then let you play it out. The characteristic that connects them both is that you typically don't engage in a lot of fast, frenetic gameplay (although that is allowable). The games aren't based on fast fingers and great keyboard skills alone, but the time and progress you're willing to put in. You'll have a character who will grow more powerful over time, with in-game statistics that evolve as the game advances, and sometimes options for acquiring better equipment as well.
So, what makes a great RPG to me? The same thing that makes any game great: It's fun to play.
That being said, I should address some rather specific issues.
A lot of people in Shamus's comments harped about freedom and story. They wanted a story that was strong and compelling while being free to make some/any decision in the world they've been planted in. I think both are overrated. Story is necessary to a degree, yes, but
not every game requires a story that is novel or ultra-compelling. The Paper Mario series, for example, was ridiculously fun, yet had a story that most would characterize as shallow. Freedom comes out either way. The Elder Scrolls games were a very open-ended, while any Final Fantasy game will put you on a very specific path ("railroading"). Both are great fun, though.
Three issues that I see as necessary for a great RPG: Advancement, Variety, and Pay-off.
If you're playing a game where the core mechanic revolves around statistics, it should be both reasonable and worthwhile to raise those stats. It shouldn't take 200+ hours to become powerful, but you shouldn't spend a long time playing just to earn increases that are nearly irrelevant. Players like to think that if they start off knowing the magical equivalent of a cigarette lighter, they'll eventually be able to detonate nuclear warheads from their fingertips.
Variety is necessary in any game, but RPGs can be won or lost on this issue. If all the enemies are just palette swaps as you progress ("Oh look, now the soldiers are wearing green instead of blue! Scary!"), there's very little sense of advancement. If you're stuck in some city for half the game, you're going to start feeling wanderlust and boredom. And this ties directly into advancement, too. Every rookie Jedi plans on taking out the dark lord of the Sith, every 1st level adventurer wants to go toe to toe with a dragon. That kind of variety gives a game a meatiness that a good story just can't account for. After all, if the story is great but the game play isn't any fun, why not just turn it into a novel?
Finally, I should note the pay-off. If you play the game for 40+ hours, you ought to be a monster-crushing legend. You ought to be rewarded for sticking it through for so long. An epic boss battle? A tear-jerking wrap-up to the story? Whatever your prize is, the pay-off should be satisfying. Too many games offer up 30 seconds of video as a denouement and expect players to be happy with it. Such a decision can make the difference between an okay game and a legendary game.
If you've stuck with me for this long, good. I just have a few more notes to make, as the original question also asked what mistakes can make a good RPG bad.
My first example is "The Bad-Guy-Bait-and-Switch." I'm sure you'll recognize this. You've spent the game chasing after the Bad Guy, just waiting to get your hands on him and dispense some sharp, stabby justice. Suddenly, you're told he's not the real Bad Guy. Maybe he was being controlled by the Big Bad Guy. Maybe he was just a flunky for the Big Bad Guy. Maybe you just misunderstood his actions for the last 20 hours and didn't realize he was trying to fight against the Big Bad Guy. Either way, your emotional attachment to this villain is now gone, and you're left to fight some Big Bad Guy for reasons that are fuzzy at best.
My next example is the deus ex machina. This involves somebody rescuing you from defeat, rather than allowing you to achieve victory alone. Perhaps the BBG is just too powerful for you, and some guy finishes the job when all seems lost, or weakens him enough for you to strike the final blow. Perhaps your dead parents/friends/lover comes to your side and saves you from the BBG's devastating attack. Either way, you are not a hero, just an "almost-victim." You're forced to watch the end of the game as a spectator, not a participant.
Almost as bad is the touchy-feely story. Your characters bicker and whine a lot and ultimately get in touch with some very sensitive issues and feelings. This can be touching if done right, but almost always ends up trying your patience. When I want to listen to whiny emo kids, I'll just go on MySpace, m'kay?
Finally, I should mention a non-story related faux pas, especially since I didn't feel like ruining any plot devices by naming names. While the game mechanics should be satisfying, an overly-complicated system can make the game too intimidating and ultimately drive away all but the hard-core players. As Final Fantasy games evolved, this started to become a problem. In VI, it was "espers," which allowed every character to learn every spell, ultimately leaving the characters with only superficial differences. In VII, it was a goofy system of "materia" which granted abilities in complicated mixing and matching schemes. In VIII, you had "guardian forces" which bonded to the characters, as well as magic that was siphoned off of enemies and turned into stat boosts. Ten was the worst, with its gargantuan grid of spheres offering skills and spells and stat boosts to whoever would dare probe its vastness. If none of that made any sense to you, then you're already aware of the problem I'm describing. If you have to study to understand the system, fun gets lost somewhere in there.
Well, thanks for sticking around to the end of this lengthy, rambling post. Maybe in the future I should blog about what makes a great blog post. Not that I'd know anything about that.