RSS icon Home icon

    Posted on August 5th, 2009 keving 10 comments


    How well is this game gonna do saleswise, anyway? The more optimistic among Japanese game sellers were hoping for five million in that country alone. Game journo Kiyoshi Shin has a lot to say about that, starting with Famitsu’s perfect review of the game:

    Weekly Famitsu’s cross-reviews receive the closest attention from members of Japan’s game industry. The total score from the four reviewers is seen to have a certain level of impact among retailers and users, so everyone has to be conscious of it. But is this cross-review system fulfilling its purpose? Developers and users have split opinions about this.

    When it comes to power, game media is going to lose out to game companies every time. Japan’s game companies have an aversion to getting scores applied to their releases, and the media is obligated to consider that in their actions — if a publisher refuses to give an outlet advance information, then that’s it. I had the editor-in-chief of one publication [presumably an online one -k] tell me once that “adding scores is simply a difficult proposition for us as a business.”

    Meanwhile, American game media is filled with score-based reviews. There’s even a “metascore” that calculates the average score across multiple media outlets. It doesn’t end with professional reviews, either: Functionality that lets users submit their own reviews (with scores) is standard-issue. Whether pros or gamers, the scores get compared and contrasted with each other, and that in turn puts pressure on media to make their reviews fair and appropriate. As a result, you often see cases where major releases with enormous advertising budgets behind them are faced with low scores. Meanwhile, games with high-scoring reviews are usually backed up by users and have a tendency to be long-selling hits; poor games receiving high scores are a rarity.

    It says something about the Japanese press, maybe, that Shin seems amazed at the concept that a game’s score should have nothing to do with its advertising.

    DQIX received a perfect 40/40 from Famitsu, but users have been using the game’s Amazon page as a soapbox to vent to the heavens about the RPG. The result: DQIX’s average score on there is only 2.72 out of 5. This has led to online arguments across Japan’s forums, with many accusing trolls of fanning the flames and posting reviews without even playing the game. The thing is, this isn’t only occurring on Amazon — discount shopping site also displays an average score of 2.72/5 (coincidentally), and amateur-run review database Nintendo DS mk2 has it at 57/100, or around 2.85/5.

    So why are these scores on disparate sites agreeing with each other — and disagreeing with Famitsu?

    In his book The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies, Cal Tech professor Scott E. Page uses his research into group knowledge to build the foundation for the theory that “diversity beats ability.” The theory: A randomly-picked group of people is more capable of solving problems effectively than a solitary talented person. There are some caveats: The problem has to be divided into a reasonable size and strictly defined, and the group’s members should be as diverse as possible in age, gender, and background, in order to increase the number of parameters at work.

    Page writes that professional groups have a tendency to all think the same way, no matter what. Meanwhile, a diverse group working under many parameters provide a wealth of different viewpoints. When these viewpoints are put together, they create an accurate font of group knowledge. The rise of the Internet has provided us with an environment that makes forming these groups simple.

    Comparing the user reviews on American game sites with actual sales figures, I get the impression that we’re seeing this theory at work. It isn’t strange at all that Amazon and the other two sites are giving nearly the same results. If you assembled another group with similarly diverse parameters, their reviews would very likely average in the upper 2’s as well — and the way the userbase as a whole feels about the game would very likely be close to that number.

    While I don’t like Shin’s conclusion (mainly because it assumies that game media is worthless for anything besides straight-on reviews, some conventional wisdom I’ve spent years trying to overturn), the fact is that Japanese gamers’ opinions have never had the direct impact on sales that they have in America and Europe these days. Now, Amazon and other sites have given them a soapbox. Japan’s publishers should be watching — it’ll be interesting to see if DQIX’s long-term sales are affected by these users at all.