The Modern Review Curve

Warning: This article was written by a cynic, and the jaded edges shown here might just prove how ignorance and bliss are often better than being well informed. Or maybe I'm just an idiot, either or let's jump in.

Let me start by saying the only true way to fail as a critic is to give a game a good score without fully explaining why.
— SuperBunnyHop-Metal Gear Solid 5: Dissociative Disorder

 

And the only way to truly fail the public is to observe a problem, and refuse to correct it.

 

Note: The examples given here are not meant to inflame or defame any one website or company. This issue persists throughout all levels of our industry, and unless deliberately looked upon, can easily be overlooked. These are cherry picked examples shown to further identify the ongoing problem, so that decent and honest people can better correct the industry to prevent such actions from occurring. Indeed yours truly has himself taken part in the problem and drank the preverbal kool-aid.

Over the past few years I've began noticing a trend when it comes time to push a game to release. The reviews start rolling out, and consumers start googling the earliest scores available. Companies start rolling out press events, setting embargos, and purchasing lavish ads. Critics begin plowing through the game as fast as humanly possible, marking off each tick mark on their general spreadsheet that tells the sign of a "good game", and the consumer starts searching for the review that best fits the narrative they want to hear. Yes it's a wonderful time where all the critical thought and careful planning tends to get pushed aside in the chaos. And this is often how companies want the behavior to work. At the end of the day they need to make money, and curving reviews to up the score or prevent the drop in pre-orders is a number one priority. It's gone from a moderately sporadic process of trying different tactics to see what works, to a very refined and deliberate system whose implementation has gone almost completely unnoticed. Fair warning, this is not a quick read. In fact it's extremely daunting just how much needs to be discussed here, and a good portion will probably be overlooked due to the vast amount of wording I already need to use just to describe some of what I feel are the basics.

A wise man named William Shakespeare once said, “Brevity is the soul of wit”...which is probably why I’m not funny.
— Alex Bernhardt

The time it takes to fully explore a games systems and mechanics, chug through the story(if there is one) and allow for the diminishing returns to take full effect is not really in the time table of your average reviewer. The modern view that the press and publisher holds is that a game's reviews are best right around the launch of the title. Meaning from about 2-3 weeks before release to about 1 week after. As far as they're concerned, a review 2 or 3 weeks after launch is dead in the water. And to their credit, are not entirely off on that front. All the data points to it.

The process goes something like this:  Around the time of release a typical consumer will start googling reviews for the game they're interested in, and click on the nearest website they trust the most. They'll usually read the first few sentences, if that, then scroll to find the score. With little to no effort they'll repeat this process on several websites, and then start making logical deductions. The risk-reward parts of the brain start firing off, and if the product is overall well received, make the purchase. If the scores are varied or overall tell the tale of a bad game...well...

Note: The previous example applies only to uncertain buyers and general populace, not fans. No offense, but a fan will usually buy dookie so long as the dookie has the brand name they've been anticipating.

Anyways, the point of me explaining this is to state that generally a person will look at five or six reviews, and make their mind up based off of the few scores they've cataloged. After that there's no point in reading anymore reviews, waiting for another person to give their opinion, or even trying unconventional means to test the game themselves(more on that later). Again, bringing this all back in, this IS what happens. We have the data to back it up. Smarter people than I have studied it, and written their conclusions. A review IS dead after a few weeks. And believe it or not, these things still drive sales. Not as much as good old fashioned hype, but still a determining factor nonetheless.

This creates a feedback loop, where the reviewer will plow through as much of the game as possible as fast as possible, jotting down what they do and don't like about the game, not even fully exploring it in order to get a review online. It's further exacerbated by the tighter and tighter time limits publishers keep inflicting on the industry through tiny embargo windows and press events. It's no longer about having a well documented and informed thought on the game, it's about the initial impressions you've had with the game based on the last twenty or so hours. In meetings the phrase, "You don't need to finish the game to have a good impression on it" is tossed around. However that's just simply not true. Games are not movies, you can't walk into one, play it, and assume that's going to be the same every time. The whole difference between games and other media is the dynamics in delivery. Movies and TV are passive experiences. Once you watch that episode, it will be the same no matter what. A games story may be like this, but the gameplay itself is interactive. That means every time you fire it up, the potential for a completely different series of events is there. And it's the rate at which the possible satisfying returns of this interactivity diminish that acts as an internal quality assessment we have for a game. That's not to say it's THE determining factor, but it's a big one.

Take Metal Gear Solid 5 for example. Here's a game that is seemingly perfect for a good portion of the game. Indeed if you just sped through it you'd have no indication that it was at all flawed. In the review done by gamespot the writer states:

When it comes to storytelling, there has never been a Metal Gear game that’s so consistent in tone, daring in subject matter, and so captivating in presentation. The Phantom Pain may be a contender for one of the best action games ever made, but is undoubtedly the best Metal Gear game there is.
— Peter Brown, GameSpot

Which is to say, I’m sorry, not really true. Playing the game delves deeper into a world that the creator, Kojima, never even really wanted to make in the first place. The care and polish placed on the mechanics of the gameplay are by far an amazing technical feat where every action feels as though there’s some input to it. However, story being the most captivating? Presentation being the most well thought out and structured of the series? I’m sorry but no. Yes, this industry is very subjective, and yes each person will walk away with their own opinion, but that doesn’t mean there’s no such thing as objective analysis in this field. A quick dive into chapter two of the game shows a clear lack of finish, with a culmination of random unseen tasks you need to complete in order to get a bait and switch ending that was already a fan theory nearly two years before the game released. That’s not to say it isn’t good. The bits and pieces of the story we do get are pretty interesting in and of themselves, but presentation? This story is all over the place, and not in the good way like MGS4, but rather in an incomplete fashion where every conversation and cutscene feels like its own episodic show set in the same universe. Nothing here really ever ties together, and missions are reused for god's sake. The entire game was clearly unfinished, and Kojima and his team were forced to polish...and I mean come on  blah blah blah blah

And as he jammed on his keyboard forsaking Peter Brown for two sentences he used to finish his review, Alex’s head imploded into the black hole of nerd rage that had been slowly building mass in the corner of his mind while writing about something he totally got derailed on.
— Morgan Freeman, Narrator

Peter Brown is not really to blame here. Most professional scores are roughly the same, and compared to user scores show the lack of time that was allocated to play such a Jovian game. Peter and friends reviewed much of this at a press event where travel and accommodations were supplied by Konami. Average Joes like you and me bought and played it after release; dumping countless hours into this to only find the turd lying under the shiny glistening golden foil. And no, I’m not saying the game is bad, we’ll talk Metal Gear another time. I’m saying the large disconnect between consumer and reviewer was not a lack of principles(mostly), but rather the lack of time needed for diminishing returns to start its advance. Playing the game for one or two days is not going to give you that effect. Your brain isn’t able to process boredom in such a short time frame when it’s being overloaded with content. And to be fair, it is fun…initially. The wear and tear comes from extended hours of use.

Rainbow 6 Siege Snap Shot-Ubisoft

The same but reverse can be said for a game like Rainbow Six Siege. The game was given poor reviews, or at least what the industry considers poor, back when it first came out. However, anyone who’s spent a decent time playing Siege with friends knows it’s far from an 'average' game. It seems to completely dispel with the idea of diminishing returns, and accomplishes this by ridding itself of all the baggage that had built up over the series’ extended lifetime. No story or real backdrop was needed. Just fun gameplay with the simple premise of terrorists vs. good guys. And by doing this, was allowed to focus on all the intricate and detailed ways you could go about fighting in a 3D, destructible environment. If you never had the time to really sit down and understand the core intricate design happening, you’d never think twice about calling this generic and bland. Sure it needed work, and that’s on the developers, but the core structure and idea was there. Again not to say it's perfect, but it is great. And yes, there are issues, one becoming more and more prevalent is the DLC surrounding the title. A worrisome issue that I need to cover in the next month or two. But at the end of the day it is something my friends and I can still enjoy...when I'm not too busy procrastinating and keeping myself enveloped in work.

The crunch to review games has become so outrageous that I feel as though half of the games with complex structures and mechanics that descend through the pipeline are not even finished, or are rushed through so quickly little time is given for critical thought. As referenced by one of previous entries. And, again, that’s the problem surrounding our industry and their critics, TIME. And it's here where the mainstream audience comes into play. By being sucked up into the hype and anticipation of a game, we’re often forced to not only review a game as fast as possible in order to get the hits, but we often have to curve it to avoid backlash from the community and publishers. No longer can I say something mean about a game, and downright belittle it for having short comings. Instead I need to gently lay my opinion on a pillow, and fool myself into thinking what I’m playing is currently “The Best”. It's this love triangle between editor, publisher, and consumer that keeps most critics from being brutally honest. The idea of you scratch my back I'll scratch yours that's plagued the professional side for years, is what makes the curve feel all too dirty and sleazy. Ask anyone in the industry, a score less than 8.5 can spell death for a game. People just don’t have enough money or energy to invest in anything less these days. And that’s fine. As a consumer you shouldn’t be forced to play sub-par products. However, the lack of patience also means a huge bug has been left in the system which allows publishers to exploit. It has generated a system where a publisher/developer will use rush tactics to fool the audience into early buys and curved grades.

This is the part where Alex says his writing sucked, don’t worry it still does.
— Voice of God

And it did, and so do most of the reviews on written sites. They’re often bland, dry, lack any motivation, and feel contrived rather than natural. Part of that being a particular person may be assigned to review a game type they don’t usually enjoy, but also because there are such asinine systems and standards each website adheres to. When reading a review it can often feel like you’re reading an extended ad for the game through the eyes of someone having played it for ten minutes, then trying to describe it in as few sentences as possible. Length is always an issue when it comes to writing, and trying to write an epic about the game is not really feasible when the average attention span of a reader is nothing more than a few minutes.  Time, again is the enemy here. This time, it's how well you can captivate someone for as long as possible. And given the current state of reviews, is something that must be condensed into four or five paragraphs. A good chunk of this has been brought on by the age of infotainment. Where a video can be more captivating, entertaining, and as a result often more informative than the written counterparts. Why spend twenty minutes reading a review, when you can get the same effect in about five while watching gameplay and listening to commentary demonstrating the game in a manner much better than any amount of writing ever could?

There's a clear and definite reason why people are turning more and more to YouTube personalities like Dunky and SuperBunnyHop, aside from the obvious infotainment aspects, that make this a concerning issue for print media. The majority of these people are generally doing there own thing, in their own way, with no corporate or editorial oversight (USUALLY) By cultivating a unique persona they are able to tap into the frustratingly bored market, and bring fresh perspective to an otherwise bought out industry. The only downside is that they may not be entirely generating well informed opinions in the eyes of their viewers, but rather those who simply regurgitate what the show told them to say. See, watching a video or listening to a podcast is a very passive experience. You're being told what to think by the narrator. However, in writing you are forced to think in order to comprehend what's going on. It forces you to exercise editorial authority on your own. And I'm not trying to say that these shows are bad in any way, god no! I love them and enjoy the editorials and often humorous reviews the personalities generate. However, it's something that should be only supplementary to our consumption of this type of media. And the reason it's becoming a larger aspect of the market is due to a failure in our ability to write captivating material.

The thing we need to acknowledge is first and foremost that we are consumer advocates. Our job is to be the voice of the general public and represent their needs and interests towards a system of corporations that may not necessarily care, or may be actively trying to manipulate. Most of us have failed in that right. When our reviews don't mention the outrage at having DLC be necessary to enjoy a game, or when we fail to call a company out in order to keep our relations with them the same, we fail our audience. There can be no tangible middle ground here. Sometimes a company will do amazing things, and other times that same company will do extremely shady shit. It's our job to be the pall bearers, and use editorial authority to take a stance on these events. I cannot stress this enough, but if print is to survive in gaming we must become less and less industry oriented, and more on the side of consumers. In other words, think as if the only game you'll be able to afford this year, is the one you're playing right now.

Switching gears before this rant goes off the rails(As if it already hasn't) there is one aspect to the industry that's disappeared which used to act as a gateway buffer between critic and consumer, demo's. At the end of the day being able to play a vertical slice of the game is better than any video or article could ever hope to achieve. Publishers have long since known this, and many of them have long decided that it's too risky. They've trashed that model in favor of gameplay obfuscation.

If you don’t trust the critics, and the bit we’ve shown you looks amazing...then I don’t know maybe you’ll just have to buy and play the game yourself. Just saying, we ain’t got no demo for ya, so ya might just wanna see where it goes. If you don’t like it well...eh here have a look at our no refund policy. I don’t know, you tell me.
— Some Exec, Some Board Meeting-Some Big Wig Publisher

Publishers do everything to generate hype and anticipation for a product, using flashy models and epic excerpts from their games to create a high in the buyer, which generates an impulsive mentality. They use test models and data collected from surveys and playtime to force developers to adhere to positive returning values which may or may not inadvertently pervert the designers original ideas. All in the name of averting risk.  At the same time publishers employ various tactics and methods prior to release to keep that hype train going and curve the presses view. Including things like not releasing demos to the public; holding exclusive press review events where the industry has about a day or two of boot camp to play and generate opinions of the game; drafting embargos that push a reviews green light to just next to the games release; and paying off youtube personalities to play their game, where the expectation is lack of thought and the soft passive aggressive term, “Don’t bite the hand that feeds”. Seriously, don't bite the hand that feeds? I'm sorry, when was it ok for us to disregard objectivism in favor of monetarism? I get it we all need to make a paycheck, but that paycheck shouldn't come at the cost of alienating your own audience.

Critics are so pressed for time during this phase they have to resort to press templates and brief recollections of their encounter with the game. Because of the varied circumstances in which they came into the contact with the product there may also be some “Wining and Dining” altering the review. They must also meet fan approval and opinion subconsciously, and keep relations with publishers civil if not cool, leading to a softened review that may be under the influence of a curved grade even before the game has been touched. At the same time, they are forced to manufacture an opinion that in most cases lacks critical thinking into the games structure and mechanics with the added flavor of time sensitivity further antagonizing the already warped view on a game that may not be as good or bad as initially thought. Most reviews now happen in a vacuum where a low budget title with a ten buck price tag will be equated with a AAA having mass marketing and a forty million dollar backing, and diminishing returns are almost completely over looked so long as enough gloss is applied over the product. Something publishers and developers know, and have no problem exploiting.

Consumers are often tied to two rafts. Either fans or undecided and 'casual' buyers. The unfortunate turnout to this is that almost all critical thought is lost, and in turn developers must adhere to the standards set by the public to get funding for their creative vision. Setting a tone of mediocrity across the board which is only randomly interrupted and adjusted through indie titles doing their own thing, and developers with godlike status wading into deeper waters often at the risk of potential career suicide. All in the name of getting an arbitrary score and ensuring the masses will continue to gorge themselves on this stuff. For anyone doubting these kinds of things happen, take a stroll down memory lane to the time of Killzone 2.

It’s system we have all created through our lack of creative thinking, our inability for patience, our willingness to be manipulated by spectacle rather than substance, a sheer disdain for anything we don’t agree with, and a total lack of regard for how manipulated a review might have been so long as it agrees with us. It’s a beast we all created, and a beast that holds this industry together. There is no evil in this but the one we allow to happen. I'm just another fanboy jamming on his keyboard about how inept I am at coping with the way things are. This isn’t a pressing issue in everyday life, hell not even in video games. These are pieces of entertainment, coded onto our screens and designed for the sole purpose of living another reality, if only for a few hours.

The changes we'd need to make are not that hard to comprehend. Things like: refusing to pre-order a game until a free to play demo of the product is offered; Writing in a less boring and mundane fashion, and using editorial authority to invoke a stance on some of the practices in this industry; demanding that publishers adhere to our needs, and that any product we receive will need due process before going to print; telling our bosses higher up that we're going to take our time writing the review; not letting ourselves be blinded by the glitz and glamour of press events; raising the price of video games themselves to make up for the necessity to push DLC; ditching the pay to win model; demanding of ourselves the necessity to be first in line to talk about something, and the list goes on and on.

Yet, it wasn't until my fifth or sixth re-edit of this article(When I was getting preachy with all of the solutions above) that I came to the conclusion that these aren't truly solutions to the problem. They are 'A' solution to the problem, but not one that is feasible or likely to happen. As a physics major I love to solve problems, and the limiting parameters here make for an amazing doozy of a problem to solve. It's one I want to explore further, but the obvious ones listed above are only possible if the majority of the industry, including you the consumer, agrees to go along with it. Something I don't see happening any time soon(There goes that cynicism again) It's not all bad, though. There are plenty of publications that have come out and spoken against some of the things going on in our world. Even bigger named sites like IGN have taken stances on this issue, like when they decided not to attend the CoD: WWII event in order to review the game on their own terms. The problems I'm trying to point out are not actively being ignored by everyone, but the key here is that a majority must engage, not a select few.

All of these simple, but unlikely, changes act as solutions to a problem that mainstream audiences may not spend too much time dwelling on. In fact I’ve only started to see more people complaining about the gap between critic and user review in the past two or three years, despite this being a problem for decades. The only conclusion I’m left with, is that the practices used to curve the grade are only now becoming more prevalent. Perhaps the most degrading bit about this whole thing is how small of a problem it is. In the industry alone there are more pressing issues and concerns. It's humiliating how much time I've taken from both you and me to talk about something so trivial. However, it is a problem that is big enough to make it onto our radar. We know something is off, but we just can't place it. It's also appears to be a problem that branches into more concerning issues that our community faces. Things involving how the sixty dollar price tag is chocking the process all the way up to companies using DLC to work the tax code in their favor. In Physics we know to start solving huge problems by tackling the little ones. We know that a large problem is not usually one huge issue per-say, but rather a ton of smaller individual problems comprising the whole. 

I'd like to spend the next year or so exploring these issues to see where it takes us, but that's going to take time. Here is something easy and intangible. There's loads of factual evidence to debunk and support what I'm saying. And while I believe what I am saying rings a bell of truth, I'm mostly talking out of my ass. However, these more pressing issues need solid evidence and clear concise points to be relevant. Again, time is our enemy. Time to gather resources, and time to connect the dots to see how all of these issues are seemingly interconnected. This time next year, I hope we can revisit this post, and reflect upon all that we've come to learn.

I’ve enjoyed my time with you, I hope you’ve enjoyed mine.
— Professor Francis Graham
No author bio. End of line.