•  
  • RicardoT
  • Level: 30 (53%) 
  • Rank: Wicked Sick!
  • Member since: May 3, 2002
  • Last online: 02/05/10 6:56 pm PT
  • My Emblems:
    • Staff
    • Greatest Game Hero Bracket Submitter
    • Greatest Game Hero Preliminary Voter
    • First espisode of TOTS!
    • Tokyo Game Show 2007
    • +1 Orator of Distinction
    • Staff
    • +1 Orator of Distinction
    • Tokyo Game Show 2007
    • First espisode of TOTS!
    • Greatest Game Hero Preliminary Voter
    • Greatest Game Hero Bracket Submitter
     
     
    Close

My Recent Ratings

Title Score
Panzer Dragoon Orta 10
Osu! Tatakae! Ouendan 10
30Apr 08

The scoop on the GTA IV score

Right, so let's talk about that GTAIV score. For those who haven't seen or heard, here's the deal: there was a bug in our publishing system on Monday that caused the incorrect score for GTAIV, a 9.5, to appear for on some pages for a while. This happened hours before we posted the actual score with the review, a 10. This has all led to a lot of talk and speculation so I'm going to set things straight, short answer: it's not true. Longer answer requires me to run you through stuff which I'll be doing now.

Our review process is like so: Once our reviewer has finished the game, he writes his review, submits it to copyedit, and then goes back over the text to look over the changes copyedit made before submitting it to production who then produce the review. Once the review is produced as a Web page on our QA servers, a link goes out to the reviews team for the peer review process. During that time the review score is not final. I'll say that again for dramatic emphasis: During that time the review score is not final.

In the case of GTAIV, that non-final review had a score of 9.5 attached to it--a score which the peer review process is designed to fine-tune and then set in stone. As has been standard GameSpot operating procedure for years, the last step of the peer review process is a discussion about review content and score. The content discussion entails making sure that relevant gameplay features and whatnot are mentioned to ensure the review is accurate as possible--and that said score matches said review.

As Justin noted in his blog, there were enough people in the reviews group who felt GTAIV deserved a 10 that we all holed up in a room to settle the matter....as the review was in the QA process on our staging servers with a 9.5 score attached to it. Ironically, one of the staff poked their head in the room during the at-times heated discussion and mentioned that it looked like a bug on the staging site had caused the QA score to go live for a split second. The importance of this event didn't sink in at first, since were so embroiled in our talks. Why were we so focused? Because we take matters of review score seriously. I'll reiterate what Justin said in his blog: We don't hand out 10s lightly--we've given out just four in GameSpot's entire history.

I've hit up our tech folks to find out what the deal was. After some back and forth, they got me answers because I wanted to post about this and include some info. So here's the tech nitty gritty for those that care:

* a 9.5 score, which as I noted above was not final, was the original score placed on the GTA4 page and became the page that was cached while testing and reviewing the site for quality assurance.

* Due to a bug in our publishing system that has since been corrected, a cached page containing the score and review blurb, but not the review itself, appeared on the Production version of the site.

* The length of time this was up is not precisely known but logs show that changes were made from 4:45PM PST - 8:20PM PST, with the actual review going live at 8:20PM PST.

And there you have it. A publishing snafu + some tremendously awful timing = needless drama. I'm well aware there's a bunch of folks out there happy to add this to the conspiracy theory tapestry that has been woven about GameSpot. Adding to said conspiracy was a blog post from a moderator saying the 9.5 score was deliberately put up as a "red herring" to test reader reaction. The same moderator has since corrected himself, pointing out that in 2003, an accidental score of 0.0 was posted for one of the highest-review games of the year--Star Wars: Knights of the Old Republic (9.1).

I'm disappointed that, end of the day, one of the best-written and best produced text and video reviews is being cynically written off by some. None of the naysayers know the effort that went into making sure our review of probably the biggest--and so far, the best--game of the year was as bulletproof as possible. The personal attacks on Justin are unnecessary. Apparently people don't realize that Justin's been at GameSpot for 9 years--starting with the original GameSpot UK--and has reviewed all sorts of games in that time.

But so it goes. Bugs and assorted glitches are a fact of life on the web just as misspellings and assorted printing issues can be with print magazines. We take our reviews seriously here and we don't post them until they're ready. This is why GTAIV went up when we wanted it to with the score we wanted it to have. For those that gave us the benefit of the doubt and that didn't jump to conclusions, you have my thanks. To the others? ...

  • Posted Apr 30, 2008 4:33 pm PT
  • Category: Editorial
  • 49 Comments

49 Comments

  • JordanElek

    Posted Apr 30, 2008 4:57 pm PT

    I always wondered why we sometimes saw review scores (but no actual review) a few hours before the review text goes live. I thought it was just a preview, something to get people excited for the review.

    And now we know.. the rest of the story. Thanks Ricardo.

  • shaunmc GameSpot staff member

    Posted Apr 30, 2008 4:58 pm PT

    For those unfamiliar web publishing jargon, when an article is in QA, that means it's been placed on a PREVIEW version of GameSpot. This QA environment is only available to those in the GameSpot offices. It's so editors can see how an article will look before it goes on the live site.

    So basically, QA = rough draft.

  • StanBlocker

    Posted Apr 30, 2008 5:09 pm PT

    So that whole QA thing makes it clear now. I was wondering why sometimes I would get an update in my RSS and the link never worked. Just having to wait an hour or so and it worked though. Personally I think all the GTA players knew they would get it regardless of the score. Anyway its just .5.

  • snake670670670

    Posted Apr 30, 2008 5:35 pm PT

    Ok i trust your integrity and am actually delighted you gave the game a 10 rather then a hazy 9.5

    but, i do have 1 question. How does tha game go from a 9.5 to a 10. Who can decide this. Is this decided by people who have read the review (but may not have necceseraly played tha game yet) and feel that the text reads more like a 10 or do the people who are involved in this decision have to have played the game. How does that work?

  • MAILER_DAEMON Site moderator

    Posted Apr 30, 2008 6:52 pm PT

    Like I've said in and out of the mod boards, I'm sorry that my comments added to this mess, as I misinterpreted a joke from the live chat.

  • shaunmc GameSpot staff member

    Posted Apr 30, 2008 7:12 pm PT

    snake670670670: Justin Calvert, the reviewer, made the decision based on input from other editors.

  • Glazkrak

    Posted Apr 30, 2008 7:17 pm PT

    Thanks for clarifying this situation for us all.

  • UnlivedPhalanx

    Posted Apr 30, 2008 8:40 pm PT

    even a 9.5 is really really good, but a 10 from Gamespot is unheard of in recent times. I don't see what people are complaining about either way.

  • mjcowley Site moderator

    Posted Apr 30, 2008 9:06 pm PT

    The level of detail that Gamespot goes through for its reviews is quite impressive. With your in depth view of the process, a quality review from Justin and an explanation of the score change, I certainly hope all are now appeased.

  • Forlorn79

    Posted Apr 30, 2008 10:04 pm PT

    My ONLY question is... did Big Rigs get so much analysis? Only a handful of games got the highest score of 10, but only 1 game has received the lowest score of 1!

    In all seriousness, the only problem should be that the score changed, not that it got a 10 instead of 9.5, but you've explained why it was changed, and why the 9.5 was not official, pretty well. The only thing I would add is how often review scores change before going live. I can only assume it happens semi-regularly. If this tweaking of the score happens very rarely, however, I think most people would want to know why. I got the game at the midnight launch (GTA IV, not Big Rigs) and am sure I will love it!

  • room234

    Posted Apr 30, 2008 10:48 pm PT

    The sad thing is, in the wake of all the drama GS has seen this year with Jeff, Alex, Ryan, and so forth leaving... this is the nonsense you're going to have to deal with.

    I don't doubt that this is exactly what happened... but you have to admit that GS hasn't had the most "scandal-free" reputation lately. So any little thing that goes wrong, or any major thing that GS does (like hand out a 10) is going to get drug through the mud because there are lots of people out there that just don't trust you folks anymore.

    If I were you, and clearly I'm not, I'd find the CNET execs that botched the Gerstmann-gate thing, show them things like this ridiculous little drama, and make them understand how badly that donkey show is still hurting the site.

  • Kevin-V GameSpot staff member

    Posted Apr 30, 2008 11:16 pm PT

    Forlorn, it's worth noting that at least since I've worked at GS, I've worked under both scoring systems, and scores have often been tweaked to make sure they match the text and accurately reflect our impression of a game before being published. That's what peer review is about. The process is no different now than it has been for years.

  • SirSpudly

    Posted May 1, 2008 12:52 am PT

    It feels almost odd...When Gerstmann offered Zelda:OOT a review score of 10, he was the one who had to convince his boss that things like talking, better texturing, and true sound would not have made the game any greater. And the argument rang true.
    Yet today it was the voice of many telling one man that he was too critical of flaws, that it was he who over-evaluated their impact on the gameplay and fun. A true indicator of what business is today in group-led problem solving. Many people, one research, one solution.
    Perhaps people should learn why a 10 is a ten and learn why either way still produced the same result. A game that cannot be reasonably improved at the time it came out.

  • Lunar52

    Posted May 1, 2008 1:34 am PT

    Interesting they way GameSpot makes the reviews.

  • Light_Bahamut

    Posted May 1, 2008 1:44 am PT

    The fact of the matter is, this is needless rhetoric over a game that under the old review system would probably not have received a 10. It seems to me, from reading between the lines of this snafu, that Justin, under the old review system, would have awarded the game a 9.6-9.7. Apparently he originally chose to round down to a 9.5, but the rest of the review staff felt he should round up, due to the content of the text and whatever else goes into the peer editing process (which is a crucial aspect of any review, and I'm sure many games' final scores have been tweaked over the years. My only question is, if this kind of extensive peer editing takes place, then how the hell did Frank get away with a 6.8? Haha)

    Ultimately, it doesn't much matter, and this little ordeal simply highlights the pettiness of people on the internet who invest themselves entirely too much in spewing negativity. I'm not asking for everyone to sing kumbaya here, but damn people, give Gamespot a break would ya?

  • dannyodwyer

    Posted May 1, 2008 2:18 am PT

    I didnt even notice this lol. interesting how you guys come to the final decision though.

    needless drama indeed. Im sure the infoil hat croud loved this episode

  • markwlin

    Posted May 1, 2008 2:20 am PT

    "Testing fanboy reactions" has nothing to do with the "conspiracy" that I've been hearing about -- namely, Gamespot cashing a big, fat check from Rockstar. Why wasn't this mentioned anywhere within this blog entry? Perhaps, it's true, after all? I must have seen at least 10x the accusations about this as there were about "fanboy reactions."

  • Caddy06_88 Site moderator

    Posted May 1, 2008 3:14 am PT

    Great explanation Ricardo! markwlin: So are you suggesting that Rockstar provided every single reviewer who gave the game a 10, there are A LOT of them, with a "big, fat check". The game is fantastic, reviewers from every part of the industry think so, I don't see how giving a fantastic game a great review results in a pay check.

  • Japots

    Posted May 1, 2008 6:07 am PT

    So whats up with the reply from a moderator saying the review was a "red herring" ?

    "The same moderator has since corrected himself, pointing out that in 2003, an accidental score of 0.0 was posted for one of the highest-review games of the year--Star Wars: Knights of the Old Republic (9.1)."

    Heh, ever heard of 1984?

  • edubuccaneer

    Posted May 1, 2008 6:33 am PT

    GTA is one of those games that people don't need to read a review for. Whoever was waiting for it, already got it even before GS's review was up. With such a comprehensive review that confirmed what they wanted, they had to cling on into something as silly as a .5 increment. With the new system it's either a 9.5 or a 10! Bah, I still think scores should be taken down altogether. It's the review that matters, not a number after it. After Justin's hard work to make a comprehensive review for a highly anticipated game like GTA IV, it's not even close to fair having something so small as a score get so much attention.

  • Kevin-V GameSpot staff member

    Posted May 1, 2008 8:19 am PT

    SirSpudly, your take does not reflect how the process worked. If Justin did not feel the game deserved a 10, there would not be a 10 on the review. I have submitted several reviews without having settled on a final score, knowing that peer feedback would ultimately be a valuable tool in helping me make that final call.

    Bahamut, seeing things in terms of the old system isn't an accurate analogy. We don't give a score in the old system and then "round." There is no such thing as "rounding." The old system is gone; there is nothing to "round."

    markwlin, your suggestion is ridiculous. Insane, fictional conspiracy theories have no intellectual merit and don't even closely resemble reality.

  • DiGiTaL_SiNs

    Posted May 1, 2008 9:23 am PT

    Although I certainly can see that GTA IV is a superb game, I'm not going to say that GTA IV deserves the 10/10 score because I did not play the game thoroughly yet. And even if I do play it to completion, It's going to be really hard to prove whether it deserves a 10/10 or a 9.5 which is merely a 0.5 lower score (which still makes it a superb game).

    But what concerns me (and probably everyone else who was surprised to see the 10/10 score) is that when you see a 10/10; you think "wow, that game must be perfect", eventhough in the review and the video review Justin said -and you guys approved- that it's not perfect. Even if you guys keep saying that 10/10 is not perfect it just means its "prime', I find that very confusing.

    If you really don't want to consider 10/10 perfect, then maybe you should have went with the more lenient 5 star scoring system (or the letter grading system). 5-Stars to me doesn't translate to perfect as strongly as 10/10 does. Besides, I bet that in the 5-star rating system, the whole disagreement you guys had about a 0.5 score difference woudn't have been an issue, because you would've all agreed that it deserves a 5-star rating anyways.

  • Lionheart08

    Posted May 1, 2008 10:39 am PT

    I swear, people are quick to out and make conspiracy theories about GTAIV getting a 10.0. I was shocked myself, but I figured there was bug or error when the 9.5 was first shown.

  • Smoov_B

    Posted May 1, 2008 3:00 pm PT

    Well thanks for clearing that up. Interesting to hear about the process. Despite the apparent score change, from what I saw around here people were terribly impressed with the high quality and objective stance of the review itself. Justin did a great job keeping it unbiased and staying focused on the games merits.

  • dannyodwyer

    Posted May 1, 2008 3:42 pm PT

    markwlin:
    On second thoughts, im with you buddy. Clearly Ricardo and the rest of Gamespot have been infected with the same mind-mongering alien bacteria i read about on snopes. If you look at your net connections open ports you can actually see them transmitting gorgon-waves to your keyboard in an effort to infect you also. Close the doors, cut off the water supply and whatever you do dont buy GTAIV, no matter how much of a totally awesome, universally critically acclaimed game it is. THATS JUST WHAT THEY WANT YOU TO DO!!!

  • jayelcey

    Posted May 1, 2008 7:06 pm PT

    "The same moderator has since corrected himself, pointing out that in 2003, an accidental score of 0.0 was posted for one of the highest-review games of the year--Star Wars: Knights of the Old Republic (9.1)."

    So, based on the explanation given here regarding GTA IV, that means the original score by the reviewer of KotOR was 0.0 until the other staff convinced him to raise it? Also, what are the chances that the other staff saw that nearly all other sites were giving the game a perfect score and were influenced by that since Gamespot was slower than many of them about getting the game reviewed? Would hate to draw the ire of millions of gamers, eh Gamespot, that would think you are out of line and unreliable? Hmmmmmmm...

  • vaejas

    Posted May 1, 2008 8:43 pm PT

    Wow, I missed it. This is the most drama since Jeff's 7.7 LocoRoco review was pulled off GameSpot UK. See what I did right there? Totally made something out of nothing. It's easy!

    Thanks for the post Ricardo, it feels ridiculous that you had to explain anything about this. Keep doing what you're doing.

  • PSdual_wielder

    Posted May 1, 2008 10:09 pm PT

    Seeing a hyped game getting the second highest possible score on a scale was apparently not enough for some people. This would normally never happen, but its just funny seeing how frustrated some people get seeing a game get 9.5/10.

  • shaktiIIIgta

    Posted May 1, 2008 10:59 pm PT

    for snake670670670: got no knowledge of press jargon nor im part of Gamespot, just read this: http://www.gamespot.com/misc/reviewguidelines.html and... the stuff that speaks by itself to give a reason why the they review like this (for me GS gives great reviews) is this one: '...at its time of release, a game earning this score could not have been improved upon in any meaningful way.'

  • dutch777

    Posted May 1, 2008 11:35 pm PT

    RicardoT, thanks for your explanation. just ignore the nutters that question Gspot. what the hell do they know. i think you Rock!!!!

  • noelveiga

    Posted May 2, 2008 6:50 am PT

    Here's what I don't get from the controversy about GTAIV's scores, here or elsewhere: Nobody's saying the game is bad.

    Really, look it up. Some people claimed Halo was boring, or Gears of War repetitive when similar compaints appeared, but GTA? Nobody, anywhere, is claiming GTA is a bad game, or worse than any other game. As far as the people complaining know, 9.5 is no worse for the game than 10.

    So, I guess my point is: who the heck cares?

    Please, do get rid of numerical scores. It's an aberration of game criticism, music, book or movie critics don't bother with such stupidity. Take notice.

  • bondoghini

    Posted May 2, 2008 11:30 am PT

    I have to say I agree with stpaul unfortunately, Gamespot has let all of us gamers down, thanks guys

  • Rando1017

    Posted May 2, 2008 4:05 pm PT

    I can understand this explaination, but can someone please clarify how a game mostly about driving gets a perfect score when the driving system is horrid at best, and broken at worst. Seriously, you can't take a corner and see where you're going. The E brake doesn't work and you just slide into stuff. Anyone??

  • cbutkus

    Posted May 2, 2008 6:21 pm PT

    Sorry but this explanation just doesn't cut it. Its pretty sad how far this site has fallen.

  • DiGiTaL_SiNs

    Posted May 5, 2008 3:19 am PT

    BTW; if you guys keep saying that 10/10 doesn't mean it's perfect, it just means it's "prime" and that GTA is not perfect, how come this headline is up on the site?

    "ON CNET: Grand Theft Auto IV is PERFECT "

    I can put up a screenshot if you want. And "PERFECT" is in all caps, I'm not just capitalizing it for dramatic effect or anything.

  • red_x2004

    Posted May 5, 2008 12:21 pm PT

    When you take a game with the scale of GTA IV you definitely have a 10 game at your hands ,i mean whatever the flaws are in this game they are nothing to what it has to offer, every single detail was taken care of in GTA IV, and frankly, it has never been done so will before.

  • illkillyou

    Posted May 6, 2008 9:59 am PT

    No offense, but you're being awfully condescending to your readers in this blog.

  • illkillyou

    Posted May 6, 2008 10:06 am PT

    Jeez, just read Kevin V.'s comments too, and holy crap... HARSH! Just how much are you guys trying to alienate your readers here? You guys really need a chief editor (maybe you do) and that cheif editor really needs to monitor staff blogs and comments so you guys don't say something stupid and offensive to readers... like, you know, you just did .

  • trecipticon

    Posted May 6, 2008 5:16 pm PT

    there is nothing condescending to the readers. I found nothing offensive in that blog post or the comments. lighten up a little,

  • DiGiTaL_SiNs

    Posted May 7, 2008 2:58 pm PT

    And here's another one:

    http://www.gamespot.com/news/6190520.html?action=convert&om_clk=latestnews&tag=latestnews;title;0

    "...which is just the fifth game in GameSpot's history to score a perfect 10"

    So you guys really consider 10 to be perfect then? What the h* is going on? You guys keep saying that the game is not perfect and the 10 doesn't mean it is, and then all over the website you post that it is PERFECT!

  • DiGiTaL_SiNs

    Posted May 8, 2008 9:53 am PT

    After contacting the Gamespot news team, they admitted that it was just a mistake, and corrected the news headline.

    It now reads:
    ""...which is just the fifth game in GameSpot's history to score a 10"

    Finally! Things were starting to get way too confusing (even for the GS news team it seems).

    Thanks to news editor Brendan Sinclair for replying to my emails (quickly I might add) and fixing the issue.

  • Skull-Fire

    Posted May 16, 2008 9:54 am PT

    You're saying you're surprised that people jumped to conclusion? You shouldn't be after what happened last year. People were still leaving this year because of all that.

  • ChiefFreeman

    Posted May 17, 2008 3:04 pm PT

    thanks for being so open and honest. I think you're doing a great job, and I haven't given up on the site.

  • metalgear-solid

    Posted May 23, 2008 10:14 pm PT

    Anyone have a picture of the 9.5 score? I'd really like to see it, even just for fun.

  • Yongying

    Posted Jun 1, 2008 5:08 pm PT

    Should have left the score a 9.5..even if it was a mistake!

  • naju890_963

    Posted Sep 16, 2008 5:11 am PT

    Yeah there are so many glitches round GS i cant coun them

  • dennishachey

    Posted May 18, 2009 2:12 pm PT

    nice pick of E3 of GOD OF WAR 3 i cant wate to

  • joedog05

    Posted Jul 27, 2009 5:55 pm PT

    OMG, the gta 4 score went from 10 to 9.5! - for some people, for a small amount of time. But still omg, like that's such a huge issue, makes you think that you just take game scores for granted. Any day another score could drop 0.5 and the editor-in-chief will have to write a short book on what happend.

My Recent Ratings

Title Score
Panzer Dragoon Orta 10
Osu! Tatakae! Ouendan 10
advertisement
Click Here