Hi everyone, my name is Paul, although you may also know me online as X1 Commander. I was one of the testers on Armillo, and I’ve been asked to post some thoughts about Armillo and the gaming industry. I hope you find what I share enlightening and informative. If you’d like to learn more about me, please visit my Deviantart page or Youtube channel.
First, a brief introduction is in order. I am a former QA Tester from Electronic Arts of Canada, hired to test games like NBA LIVE, March Madness, Need for Speed and half a dozen other titles. I worked at the Burnaby and Vancouver studios in 2003, and again from 2007-2009. I was one of the many talented testers laid off during the great EA exodus of 2009, and for over 5 years, worked as a Computer Technician for a Computer repair company in lower mainland. About a year ago however I saw a post that called out to me. I am a participant in the Full Indie meetups in Vancouver, and a post was shared by Yanni, one of the co-founders of Fuzzy Wuzzy Games. He was looking for some testers to test their new game, Armillo. Having been wanting to return to the industry for years, I quickly seized the opportunity and replied. Before long I was testing Armillo, doing what I love to do. It was a bit different from what I was used to, but the feelings of being useful again was hard to ignore. So let’s get started on how we tested Armillo.
I’m going to run you through each type of game testing styles that I have experienced over the years, just so you have an understanding of how things have changed from when I first started testing. Each is valid in their own way, but the strengths and weaknesses should be taken into consideration if you’re ever going to be involved with making your own game.
When I was at EA, we were part of very large teams, and as testers we ranged from 5-20 per project. That number would sometimes go even higher if the game was behind its testing milestones, or behind schedule. We were given our own focus on the titles tested, and we were expected to be specialists on our focus. My particular focus for the first NBA and NCAA games was on Presentation, essentially all of the in-game graphic elements I had to test and make sure looked and worked properly. This meant that for every player, stadium, NPC, and prop in the game I had to examine in the game. To do that, I would put every possible player into the game and examine them. All 500+ players, including classic players from decades ago had to be checked and signed off on. Later I would go onto Compliance testing, which involves testing how the game works when pushed to extreme conditions, such as controllers being pulled at critical moments, memory cards being pulled during saves, pushing buttons when going to the consoles home menu, etc. Anyways, in the EA environment, we work as a team, but separate from the developer, who are elsewhere, either in the same studio but on another floor, or in NBA 2008s case, Nova Scotia. This means that there was little interaction with the team, so not much in the way of camaraderie was ever developed. It worked, but it’s very impersonal, so its hard to build relationships beyond the testers you work with, and you only do 1 job, testing a game. It’s strength lies in that a large number of testers can find a lot more bugs than a small group can, and having focused testing on the game helps to avoid duplicate bugs. It’s weakness is that it’s difficult to stand out in such a large team, and you will find it difficult to get the opportunity to try new things, such as programming, or design.
I next worked at a small company called Deep Fried Entertainment, on the WiiWare title, Shadowplay. This was a complete change for me from the compartmentalized style of EA. Sitting next to the developer, I had direct access to him whenever I had questions I needed answers to, or could show him right away what issues I had found. This allowed for quick fixes to the game and also for myself to lend my thoughts on how to improve the game. In truth, I was the test team, and for Shadowplay, a large team made no sense. There was the programmer, artist, 2 designers, a manager and a tester, and the music and sound effects were outsourced, so the team never went beyond 7 people. Being that small, I had the chance to try different things that I could never have the chance to try at EA. I designed a puzzle for the game for example. I also created artwork for promoting the game. I helped to implement the Wii Motion Plus into the game, resulting, in my opinion, a superior game experience when compared to the standard Wii Remote and Nunchuck combo. The strength in this style of testing was that I could wear multiple hats on one job, and try to expand my strengths. The weakness is that since I was the main tester, it was sometimes easy to overwork myself and lose focus on what I was really doing.
So that brings me to Armillo from Fuzzy Wuzzy Games for the Nintendo Wii U eShop, available now. What was that like? Well, given that it had been about 5 years since I had last tested a game, it took a short while to get back into the swing of things, but once I got used to it, I really enjoyed it. How Fuzzy Wuzzy structured its testing was a bit of a combination of EA and Deep Fried, and I was able to partake in much of the communication, which made for a huge difference in how I felt at the end of the project. Actually, most of the people at Fuzzy Wuzzy Games are veterans from EA who decided to keep together after going their separate ways from the company, and thus, the structure was very familiar to me in many ways, but different in others. With EA and Deep Fried, I worked in a studio/office environment, which was in Vancouver, while I lived in Burnaby and Coquitlam during those years. With FWG, I worked from home through the internet. In the time between 2009 and 2014, Nintendo had thankfully relaxed its policy of only giving Devkits to developers with their own office. This meant that FWG and other indie developers did not have to incur the extra costs of renting office space. For a small company like FWG, that meant that more money could be devoted to game development rather than rent, which is one of the strengths of this style of development.
Using the internet, the game was downloadable through a cloud service called Copy and we could create our bug reports using a web service called Assembla, making an office unnecessary. We would have weekly meetings via Google hangout and often emails would be sent back and forth, keeping all involved in the loop. I was one of the lead testers on Armillo, and was also one of the most experienced testers on the team, so I was asked to write a test-plan for the game. I will share that document in the next post, as I think it’s instructive to share at this point. Anyways, using that document as a starting point, we began to test Armillo in earnest, knowing that the game was expected to ship in May. Unfortunately, the weakness in the ‘working from home’ model is the part about being at home. I soon discovered that testing from home is very challenging and full of unexpected distractions. At an office, you can focus on your work, but at home, its easy to fall into the trap of doing housework, or getting a snack, or going to the bathroom or wanting to watch a movie, or go to our real jobs. I live in a 3 room basement suite, so anytime anyone does anything here, I hear it. Did I mention that this was a volunteer job that I did in my spare time?
Regardless, we had a game to ship, and we were going to make it the best it could be, even if we had to work late nights to do it. I am a strong advocate for quality, and I believe passionately that regardless of the size of the game, or the price it cost, the customer is entitled to getting a high quality experience every time he or she plays the game. It ticks me off when I see games released before they are ready, because even if they have a day 1 patch ready, the customer still has to download that patch to get the experience he was expecting when he purchased the game, and I believe that it’s especially unfair to those customers with limited or no internet access, especially if the game is unable to be fully enjoyed without the day 1 patch.
We worked well into the night often, and on weekends and during holidays too. It was a grueling process, but by June, the game was ready. On July 3rd, Armillo was launched and the game went on to be well received by a number of reviewers, and managed to be highly rated by the players too. I’ve always been proud of my work on the games I tested, but Armillo really stands out as the game that made me proud to be a part of.
