Home > Design, Discussion, My experience > Iterate Often, But Get the Bugs Out First

Iterate Often, But Get the Bugs Out First

March 18th, 2009
Goto comments Leave a comment

Being a first time game developer, I am finding that the final stages of game development, namely bug fixing and adding final polish (game menus, transitions, etc) take much longer than I could have anticipated.  It is only now that I am beginning to appreciate some of the complexities behind product development and launch.  From a technical point of view, it is difficult to know on the front end just how much time will be necessary to debug properly but also from a psychological point of view I am beginning to understand just how hard it is to finish well.  At some point, when you have done the most interesting part of app development (where progress on a day to day basis is much faster and noticeable) you are stuck sanding down the rough edges and a sort of development fatigue sets in.  With so little to show in terms of progress, i.e. spending days changing menu layouts or touching up fonts, the developers and the entire team get eager to just push it out the door already and see what happens.  This is where we, and I assume most developers, debate the merits of fine tuning their app versus just launching it in the market, where it can start to make money and give some immediate feedback.

Although conventional wisdom is to iterate frequently, the friction for submitting new updates to the App store is so low that developers seem to be taking this strategy to the extreme as an excuse to launch prematurely. These apps freeze, quit, or just don’t work as advertised and many times users write negative reviews as a result.  While some developers would argue that they get a ranking bump from new updates, and I have seen some applications that submit updates every 4 days or so for the 1st month post launch as a strategy, I think this is damaging to the brand they are trying to create and only serves to piss off the early users.

If your app is free the bar is set low, but if you are charging even $0.99 it might make sense to wait until you get the major bugs out.  Sometimes though it seems to come down to a game of chicken, where the financial urge and market opportunity considerations create a push to launch that runs directly in contrast to the need for more time to refine and debug.  The guys at ngmoco’s have a great post about why they choose to do their first update to Rolando now, 3 months after they launched the game on December 18th.  Clearly this is the ideal approach but ngmoco has a privileged position, having proven themselves early while also having the financial and structural flexibility to be able to run rigorous quality assurance protocols pre-launch (said to have fixed 1000 bugs before launch!).  It is a testament to their process that they have not had to submit any updates since release until this update.

For the rest of us amateur developers though the question is not so simple.  While I do not think we should emulate ngmoco’s strategy, as it is financially risky for an amateur developer to spend so much time and money on testing a totally unproven idea, I think it is to be patient and make sure the major bugs are dealt with before launching to paying customers.  If you are creating a crapware app that has little functionality and is simple then sure, launch it early, but if you have any pretensions of developing a community or following behind your game it might be wise to have all of your users, even the early ones, have a positive experience with your app from day one.

Another approach is to launch your app as a fully functional but with an extremely limited feature set.  The perfect example lies in the current No. 1 in the paid App store, Pocket God. Bolt Creative, the makers of the game, decided to launch after only 1 week of development and subsequently released 1 update per week for 10 weeks, adding a new feature each week.  While this strategy worked well by building a community and fostering an interactive ongoing relationship with the user, it looks like it strained the early users on the front end.  In a recent interview, one of the co-founders of Bolt Creative Dave Castelnuovo said that, at the beginning, they “felt really guilty releasing [the Pocket God] app for $.99 because people said there really wasn’t enough there.”  While I am sure all of the subsequent 10 updates made users happy, it is a risky approach that only works if your ongoing updates are good and if your idea does not get poached in the process.

From my point of view we thought about releasing iCombat with only 1 player mode and less options but in the end we decided to forgo the short term benefits at the risk of exposing our concept.  The tradeoff we saw was that, with such a quick turnover rate to developing new apps, we could be releasing our idea to the world to only have someone with a bigger more experienced team, steal our idea and do it better than us.  We chose to risk the 1st mover advantage as well as short term gain to come to market the first time with a better brand and a more defensible product. Clearly the market is evolving and many models will work, so only time will tell if tour approach was a wise idea when we launch in the coming couple of weeks.

Update:  We just submitted iCombat to Apple on Monday (2 days ago), while there were one or two design considerations we were still debating and figuring out how to perfect, we finally decided to let the users decide which features they liked most.  From our perspective, and maybe from also writing this post and thinking more about the Pocket God example, it just didn’t make much sense to keep working on certain design issues that we weren’t even sure the user cared about.

Related articles by Zemanta
Reblog this post [with Zemanta]

  1. No comments yet.
  1. No trackbacks yet.