Are oranges the solution to the QA quandary?

Published on October 6, 2015

I'm kicking off with a thinly veiled, zesty rant about the approach to QA (Quality Assurance) in the games industry.

It’s an issue that has been gaining attention over the past year, but is still extremely relevant today. Firstly, and obscurely, we’ll open with this question:

Why did the orange stop halfway up the hill?

Well… the games industry is seeing a rapidly boiling media surge against rushed out product, resulting in tarnished reputation, followed by grovelling verbal and financial apologies. It’s now so mainstream that even the BBC have been posting about it – since last year. And things appear to be the same a year on – in the last month, issues plaguing Tony Hawk’s Pro Skater 5 after its launch have led it to be labelled ‘a dumpster fire on wheels’. Activision has acknowledged the bugs and is reported to be working to fix them.

Is this down to poor quality assurance? Most of the time, the answer is no. Sure, there may be the odd dosser sleeping at their post, but the real problems are deadlines, financial targets and unexpected re-scope, to name but a few… So, what gets squeezed? QA, of course. And, if we imagine the development process is an orange, all the juice, that lovely QA juice, is lost!

You can chuck sixty freshly picked (or underripe) QA resources at it in the last week, but all that tends to happen is six hundred valid – but obscure – rushed issues are flagged for Dev to trawl through and, ultimately, ignore. All the little issues get bumped aside or labelled ‘as designed’ and brushed under the carpet. There’s no time to filter all the bits from the juice!

So, what’s the problem?

The problem is that the community of gamers have become unwitting members of QA post-release. They are not brushing issues under the carpet – instead, they are putting compilations of all those missed pips that got squeezed out of the orange on Youtube and into the media. They are aggregating a massive ugly ball of all the horrible gubb you didn’t have time to filter out, and demanding the company pays for the wonderful privilege of cleaning it up properly this time. That takes a huge extra-sour bite out of your profits!

How does this relate?

As the previously cited BBC article shows, this issue has been gaining mainstream attention for over a year – and these flaws in the development process were prevalent long before the media seized hold of the problem. Unless something is changed, this cycle of post-release public QA will continue, not only across the web, but throughout other industries.

What’s the solution?

After a long time in the games industry, and over three years undertaking QA at Beyond, I can safely say that the quality of everyone’s work here is pretty darn high. But as we grow, we need to stay vigilant, avoid the trapdoor, plan and scope like mad, keep the vision, avoid the temptations, and max the envelope.

In short, the solution is to make sure QA is part of every stage of the development process across every industry. Although issues raised by QA may sometimes seem easy to overlook, addressing them when they are first flagged, rather than post-release, saves time, money and reputation in the long run.

We all know this, of course, but sometimes we get caught up and forget to stick to our principles. So, this is QA, being constantly reminded by the media to remind you why we have QA, and lastly, answer the opening question: why the orange stopped halfway up the hill… because it ran out of juice.