Reviewing MMOs: Is the Last Word the Final Word?

Dan Stahl of Cryptic Games, developer of Star Trek Online raised a very interesting point earlier this week on MMO reviews. His opinion, verbatim, basically went:

“In my opinion, the whole game rating business doesn’t necessarily do a great justice to MMOs. MMOs are designed to grow over time and get better with every major release. It might be better if sites like Metacritic could find a way to rate MMOs by releases instead of just the initial day one. There are plenty of MMOs that have made huge strides since days one and some that have even gotten worse. Until then, we will continue to offer the game for free and ask for people to try it out and decide for themselves.”

Game reviews depend on a multitude of factors. Say it’s a game where knowing and experiencing everything within the game is tantamount to a useful review. Oftentimes, if the game is terrible, even for a good portion of the gameplay, it’s not hard to pan it even if the ending is gold (or unusual, as was the case with Inversion).

But MMOs are different – they are constantly evolving beasts, receiving fixes and additions like no other genre. A game like Mass Effect 3 may receive the periodic DLC update, but The Secret World needs to be putting out more new episodes and quests each month. It’s the very nature of the genre – if players have finished everything, and have nothing to do but level grind, why continue playing?

And then there are the patches and fixes, that might have made a previously unbearable aspect of the game quite fun. Some gamers might right off these little fixes as insignificant. But ask any fighting game fan who’s moved from one sequel to the next about balance issues. For ordinary people or reviewers, it may seem minsicule but for the gamers themselves, it could be worlds apart from what they played before.

How does one go about keeping track of every single MMO, be it RPG or shooter, with every new update and quest that comes about? Because our role isn’t to iron out every single possible scenario and then conclude if it’s just right. That’s a QA tester’s job. A reviewer should know if it’s fun or not, and if an update improves something, a quick run through of the game should be made to see just how different the experience is. But again, in order to notice the change, one has to be constantly playing these games, observing the nuances firsthand. Not an easy task when you consider all the other games that demand a reviewer’s attention.

However, in light of Stahl’s views, let’s look at PC games in the late 90’s to early 2000’s, which began demanding more patches than usual. Were those games, some buggy and unplayable, others featuring minor nuisances, judged differently? Was there a “scope for improvement” scale when evaluating a game? How much fun the game would be if all the errors were rectified? Understanding the core gameplay, and identifying what makes it fun is vital. So even if there’s less bullshit in the way of enjoying it, the game automatically goes up in value.

Then again, many games were looked down on at release for needing to be patched – it was even cited as a lazy route by developers to be able to ship on time, while working on fixes later. Why can’t they just delay the title and ensure it’s perfect, rather than shipping a buggy game and patching it later? Some developers even went ahead and did that, but such a tactic is unthinkable in today’s high-pressure, AAA environment. Maybe it’s the reason most blockbuster come to consoles first – a fixed architecture to optimise for reduces dev time significantly, not to mention the number of users is significantly higher for all consoles combined compared to PC gamers. But that time is being cut short, thanks to yearly sequels. Either way, don’t expect to see Call of Duty or Assassin’s Creed delayed for any other platform but PC. In fact, when the first Black Ops was released simultaneously for PS3, PC and XBox 360, it was notorious for being almost unplayable on the former two.

Coming back to MMOs, should they be given special treatment? It’s definitely a worthy avenue to explore, even in a limited capacity, but shouldn’t be interpreted as a clean slate for devs. If the core gameplay wasn’t fun the first time around, no amount of polish or additional content is going to fix that.