General:Alan Nanes: Testing, 1, 2, 3

This is a developer diary for The Elder Scrolls III: Morrowind: Game of the Year Edition. The diary was originally posted to the old elderscrolls.com website on 22 August 2003.

---

Alan Nanes - Elder Scrolls QA Lead

How do you test a game?

It is amazing at how simple a question can have such a broad answer, which involves a lot of time, energy, and planning. This very question is the one that we deal with every single day in the Bethesda Softworks Quality Assurance (QA) department (team motto: we break games). And depending on the scale of the project, it can either be a pleasure to deal with, or cause you to run screaming for the hills.

Now, I know what you're thinking. "It's easy! Just load it up and play it, dude! How hard can that be?" Okay, good point. But what most people don't realize is that there is a massive framework that must be created to support testing a game. If you just dive in head first, there is no telling what creepy crawly bug you may miss that can bring the whole game crashing down around you like a house of cards. The best way to go avoid that is to create a testing plan that outlines what systems will be looked at, and in what order they will be addressed. Don't get me wrong; we do indeed "just play the game." I mean, if we didn't, we may miss the whole point: that the game is supposed to be fun.

Instead of me yammering on like a college professor talking about boring test plans and flow charts, lets put it all into perspective. One of the primary challenges in QA right now is testing the upcoming release "Morrowind: Game of the Year Edition"(GotY). Should be a piece of cake. Just take Morrowind, add a cup of Tribunal, a sprig of Bloodmoon, mix it up in a bowl, bake until golden brown, and serve to the gaming public piping hot, right? Not by a long shot. Follow along with me, and I shall lead you down the Hallowed Halls of Testing.

The first step was to acknowledge the difference between "original" Xbox Morrowind and what we envisioned as the finished product. What was going to be added to that original version to make it truly a Game of the Year? Well, the powers that be (ok…ok… Todd and Ashley) decided they wanted to add all of Tribunal, all of Bloodmoon, some features that later versions of Morrowind PC had, tweak it, and put it all together. What does that mean for us in QA? Well, we have to make sure that it is seamless. You drop the GotY disc in your Xbox, and you instantly have access to all of Vvardenfell, Mournhold, and Solstheim right from the get go. To make this all work on the Xbox properly, the programmers had to work their magic and get it all together on one disc.

Which brings me to the next task. We test every single quest line; from getting that package to Caius Cosades all the way to the very end of Bloodmoon (not that you have to play it in that order of course). Even though these things were tested before, we have to make sure that nothing became "bashed" when the whole kit-and-caboodle was tossed together. Nothing would be more upsetting than to release the game, and a whole slew of quests don't function.

Our Bug Tracking Software

In a few cases, changes must to be made to the game structure to iron out possible frustrations that can occur when having all that content together. A great example is the Dark Brotherhood attacking the player when he rests (from the Tribunal Main Quest). Ashley and Todd felt that it would be annoying for those of you who are new to our game (or starting a new game) being attacked over and over right at first level. So the script was changed (you'll have to wait and see what they did instead). Even though we are very familiar with testing that feature as it was, we now have to be ready to change all the test cases involving it. It's this type of fluidity in the design process that we have to adapt to.

The next headache for us, but great for you, the loyal readers and game players, was to test the transfer of "original" Morrowind Xbox saves to GotY. We have to be sure that when you load that massive save game into GotY, that it picks up right where you left off. We also need to make sure that all your piles and piles of loot you've been hoarding (you know who you are) make the trip seamlessly as well. This involves testing as many saves as we can get our hands on. Since no two people play the game exactly alike, the more saves we have the better.

Testing Morrowind Game of the Year for Xbox

Now the fun part. All of the new artwork has to be looked at. Every single piece of clothing, armor, weapons, unique items, creatures, and locations have to be artwork checked. After all, we want to make sure it looks all nice and shiny when you happen across the massive amount of new content that Tribunal and Bloodmoon adds to GotY. So, we load some characters into the game world, and have a sort of strange Vvardenfell fashion show. Try on the Dark Brotherhood armor. Hold that Stahlrim Mace and swing it. Run around with the Helseth robes on. Go check out those pesky Spriggans. And so on. Each and every item is checked off on a list to make sure they get our stamp of approval.

Lastly, but not least, comes the part of testing that you may have originally thought is all we do. When we feel that the game has had all of its basic functions and components checked, we sit back, kick up our legs, open up a bag of chips (I prefer Sour Cream and Onion chips myself), and play the game. And play it. And play it. We empty our minds of all that we know regarding how the game engine works, or how the scripts work, or how the creatures were rendered. We become a game player. Just like you. Is the game fun? Does something not feel right? The same questions you would probably think to yourself at home, we are now thinking as we sit in our cubicles. We consider this step just as important as all of the rest. In fact, we keep playing the game right up until that golden master disc is shipped off to the duplication facility.

We take our role very seriously. After all, we are the last department to touch the game before you do. We have our fun, but much of the process involves extreme detail work and heavy loads of grunt work. Even though we strike terror into the hearts of many a designer, programmer, and artist (I have fond memories during Tribunal and Bloodmoon testing of giving Mark Nelson minor heart attacks by simply knocking on his cubicle and saying "Hey, Mark?"), we are an integral part of the development team. So, with that in mind, back into GotY I go. Now, where did I put those potato chips….