Pandemonium

August 13, 2013

XapParse, five years on

Filed under: Uncategorized — bittermanandy @ 10:06 pm

Aaargh, I just lost my post… so I’m writing it again… here’s the short and slightly-pissed-off version.

About five years ago I wrote a bit of C# code for parsing XAP format XACT files, for use (for example) in an XNA build process. I called it XapParse, and made it available on CodePlex.

Five years on, XapParse is (a) embarrassingly bad, I really didn’t know C# very well back then; and (b) probably too limited for anyone to actually use (even though (c), no-one will anyway, because XNA is dead and the Microsoft Thought Police will probably hunt me down just for talking about it, and MonoGame, which is awesome, isn’t supporting XACT, which is disappointing but understandable).

But whether anyone uses it or not, it was still pretty terrible and had my name on it and it was keeping me awake at night. So I rewrote it.

It turns out CodePlex and SVN are also both pretty bad, and an hour of trying to upload the new version (and getting nothing but obscure impenetrable error messages) ended with me rage-deleting XapParse from CodePlex. Hmm.

Anyway… that leaves me with an updated version of XapParse and nowhere to put it. So I’ve put it here. Even though I know no-one will use it, thanks to the aforementioned tragic death of XNA. But anyway, it’s right there if you want it; maybe someone will find it useful, there’s certainly a lot you could do with it if you were so inclined.

Peace.

September 19, 2010

Efficient Development – part five

Filed under: Games Development — bittermanandy @ 6:27 pm

So, as promised: a more positive post, and this one also offers something more concrete than recent posts have tended towards: another component in the Kensei.Dev library, about which I have written before.

I recently found myself needing to add logging to Pandemonium, to aid with debugging. The idea of logging is to write text to the output window and/or a file, to explain what the code is doing. It is then possible the examine the logs to see what happened and when, to identify for example why something went wrong. .NET provides Diagnostics.Trace but it is somewhat limited in scope, so a more powerful logging framework can offer much more control over what is logged.

Initially I planned to create my own logging framework from the ground up, but I suspected that I would merely be repeating what is already available – and indeed about five seconds on Google pointed me in the direction of this. DotNetLog (also known as BitFactory.Logging, also known as Termite) is quite an elegant framework: you can send output to any number of loggers of which there are many different types, and each can be formatted individually. It costs $5 but that’s almost as good as being free, so I wasn’t too concerned by that.

If there is one unfortunate restriction to DotNetLog, it is that it doesn’t work on Xbox 360 (you may have gathered I’m not at all bothered about Windows Phone 7) as it isn’t consciously intended for XNA games. Therefore, I decided that rather than litter my code with references to BitFactory.Logger (all surrounded with #if WINDOWS) I would create a Kensei.Dev.Logger static class, which would be the only area of code that would refer to BitFactory at all. There are a number of other advantages to such a facade, such as those mentioned here (an article from which I borrowed a number of ideas).

Within just a few minutes Kensei.Dev.Logger was up and running, and sending log output to both the console (visible in the Output window in Visual Studio), for monitoring while the game is running, and a file, for analysis later. Generally of course, the console logging would be disabled in your Release build, and file logging would also be disabled but with the option of being enabled via a command line switch, so if your players are having technical problems you can tell them to enable logging, play the game until the problem occurs, then send you the logs.

So far so ordinary – nothing new as of yet. But I had a few further ideas of how this logging framework could be extended to be more useful.

FrameCount and LocalFrameCount

By default, DotNetLog writes out each log entry in a format similar to the following:

[Pandemonium] -- {Kensei} --<Info> -- 19/09/2010 17:13:44 -- Initialised logger.

That’s the application name, followed by the log category, then the log level, then the time of the log entry, and finally the actual log message itself. It’s really easy to write a Formatter of your own (derived from BitFactory.LogEntryFormatter) to display this in a more pleasing format, but what really bugged me was that the time shown just isn’t very useful. A Windows service might want to know the date and time to the nearest second, but for a game such information is simply not very helpful. It would usually be much more useful to track the number of frames since the game was started (or the most recent level change) and log that, instead of the time.

Initially it seemed there was no way to do this, but a quick email to the framework’s authors (and an equally quick reply) showed the way forward. First, I had to define my own FrameBasedLogEntry type, derived from LogEntry:

private class FrameBasedLogEntry : LogEntry

{

    // Constructors not shown

    public uint FrameCount { get; set; }

    public uint LocalFrameCount { get; set; }

}

Then, I had to create my own Logger and override the function that normally created a LogEntry, getting it to create a FrameBasedLogEntry instead, filling in the FrameCount and LocalFrameCount values (with the total number of frames since the game started, and the number of frames since the last level change, respectively).

private class FrameBasedCompositeLogger : CompositeLogger

{

    protected override LogEntry CreateLogEntry

        ( LogSeverity aSeverity, object aCategory, object anObject )

    {

        return new FrameBasedLogEntry(

            aSeverity, Application, aCategory, anObject.ToString(),

            Logger.FrameCount, Logger.LocalFrameCount );

    }

}

Finally, I had to create my own FrameBasedLogEntryFormatter derived from LogEntryFormatter, and tell the various Loggers I was using, to use it for formatting.

private class FrameBasedLogEntryFormatter : LogEntryFormatter

{

    protected override string AsString( LogEntry aLogEntry )

    {

        FrameBasedLogEntry frameLogEntry = aLogEntry as FrameBasedLogEntry;

 

        if ( frameLogEntry != null )

        {

            return String.Format( “[{0}/{1}] [{2}] [{3}] {4}”,

                frameLogEntry.FrameCount,

                frameLogEntry.LocalFrameCount,

                frameLogEntry.SeverityString,

                frameLogEntry.Category ?? “”,

                frameLogEntry.Message ?? “” );

        }

        else

        {

            return String.Format( “{0} – [{1}] [{2}] {3}”,

                aLogEntry.Date.ToShortTimeString(),

                aLogEntry.SeverityString,

                aLogEntry.Category ?? “”,

                aLogEntry.Message ?? “” );

        }

    }

}

Easy! Now all log entries were formatted with the frame count instead of the time, something like as follows:

[47563/1489] [Info] [Kensei] Silver star 42 collected.

LoggerForm

The really clever bit was still to come. You may remember from my earlier discussion of Kensei.Dev.Options and Kensei.Dev.Command that these components offer dialog boxes, so that changing an option is as simple as clicking a checkbox, and entering a command simply involves typing it and hitting Return – all while minimally affecting the game itself. I wanted to do the same for Kensei.Dev.Logger, and furthermore to easily allow the option to disable certain categories from displaying in the logger at all (so you know if the bug you are looking for is in your audio code, for example, you can disable all logging except from the “Audio” category).

You can view my code later to see exactly how I built up the form (using the standard form designer as a basis, of course) but the interesting part was to create a new DialogBoxLogger type, derived from BitFactory.Logging.Logger. Instead of overriding the WriteToLog function, which would only allow me to modify a preformatted string, I overrode the DoLog function, to enable me to work with the LogEntry object itself (really of course a FrameBasedLogEntry). This sent the LogEntry to the form, where I was able to use the Category information to populate a CheckedListBox, and only output the log entry if the corresponding category was checked; and also to change the colour of the log entry to make errors more obvious.
 The Logger dialog shows all logging and allows different categories to be enabled.

I confess, some of the above logging is not completely genuine; but hopefully this screenshot is enough to demonstrate what a powerful and clever tool this is. The whole time I’m debugging my game, I can have the logger dialog running alongside it so I can see exactly what is happening in the game at any one moment. At the same time, logging is still being routed to a log file in the executable folder, so if everything falls apart I can go back and work out what went wrong at my leisure:

[0/0] [Info] [Kensei] Starting LegendsOfXaerolyte version 1.0.0.0 at 19/09/2010 17:50:56.
[0/0] [Info] [Xaerolyte] Hello!
[987/987] [Info] [Kensei] New severity threshold: Debug
[2872/2872] [Debug] [Kensei] Beginning writing Pandemonium blog
[4129/4129] [Debug] [Kensei] Adding links to blog
[6359/6359] [Info] [Pandemonium] Loading game
[8124/8124] [Info] [Pandemonium] Changing level: The River Styx
[8688/8688] [Info] [Kensei] CommandResetLocalFrameCount
[8688/8688] [Info] [Kensei] Resetting local frame count at 19/09/2010 17:53:24
[10315/1627] [Info] [Pandemonium] Loaded level The River Styx
[12559/3871] [Error] [Audio] No sounds available!
[14088/5400] [Info] [AI] Calculating pi...
[18000/9312] [Info] [AI] Calculated pi to three trillion decimal places. Result: 3.14159265358979323846264338327950288419716939937510 ...
[21484/12796] [Debug] [Physics] Player bounced off rock

One possible improvement to the Dev Logger dialog would be to use a tab control, and enable different combinations of categories (and levels) on each tab. Thus you could have the first tab showing all normal logging, the second showing only the “AI” category but including verbose (Debug) logging, the third showing only errors from all categories, and so forth. I chose not to do that at this time, though may come back to it later.

Conclusion

It should be pretty obvious what a useful feature logging is. The DotNetLog logging framework is not only complete in its own right, and easy to wrap in a facade, it is also easily extensible and deliberately structured in such a way that users (eg. me!) can use it in ways that the original authors never expected. Judicious use of inheritance allowed me to take the framework and remould it in a way more appropriate for me, without needing (or indeed, being able) to rebuild the assembly itself.

I make Logger.cs available for download for you to do with as you will, in fact, you can officially do what the fuck you want with it. If you downloaded the rest of the Kensei.Dev code I made available previously it should fit right in with minimal effort (I have tweaked Kensei.Dev a bit since then, but not in any major way as far as I can recall); if not, it should be fairly simple to strip out the references to other Kensei.Dev components and/or replace them with your own equivalents as you see fit. Enjoy, and as always, please leave a comment if you have any feedback, questions or suggestions!

September 17, 2010

Windows Phone 7? Schmindows schmone schmeven

Filed under: XNA — bittermanandy @ 12:51 am

WARNING – rant follows. Those of a sensitive disposition should look away now.

I know this blog has lain dormant for long, but I’ve been working on my game just lately and really getting back into XNA. The chance to work with a halfway sane codebase has been a real relief from my day job, and I’ve really been enjoying it. So I feel bad about this rant, but I have to get this off my chest. It is undeniably one-sided and filled with a certain amount of invective; if you don’t like that kind of thing, it might be an idea to skip this article. There are no coding tips in this article, just heartfelt opinion from an angry man. You have been warned.

XNA is a wonderful and glorious thing of great beauty. It makes game development easy on Windows and more-or-less straightforward on Xbox (though with close-to-zero financial benefit from what I gather). Today XNA Game Studio 4.0 has been released, and what’s the one thing that they keep going on about? Windows Phone 7.

So much so, in fact, that XNA is now part of the Windows Phone Developer Tools. You can’t get away from it! And if you were to visit to the XNA Creators Club website (the first link up there) you’d be forgiven for thinking that it’s the Windows Phone 7 Gaming Creators Club website. Where the hell is all the Windows and Xbox stuff? It’s gone – oh no wait! There’s one tiny little box in the right-hand column that you have to be actually looking for to even notice.

It’s crystal clear where Microsoft are going with this. Windows Phone 7 is their target platform for XNA (Zune appears to have been quietly taken out and shot – hey Microsoft, you can’t create an international brand if you only release it on one continent) and the stuff that is actually cool, easy Windows and Xbox 360 development, has been reduced to nothing more than a forgotten sideshow.

Understand this very clearly. I am a Microsoft fanboy in any meaningful sense of the word. I even still use Internet Explorer despite the fact the whole world is telling me Firefox is better – hey, I like IE, OK? I do. I even worked for Microsoft for getting on for five years, and I enjoyed it, and in another life I might have found a way to stay.

But  here’s the thing. I owned a Windows Mobile 5.0 phone, and it should have been amazing. But that thing was pure hell. Simple tasks – like synching Internet Favourites – failed at the most basic level (one would disappear every time the phone synched, until they were all gone). As a smartphone it was supposed to run my life for me, but when half of my calendar appointments lose an hour if they are in the morning during daylight savings time, with the result that I no longer know when my appointments are, how is it realistically going to do that? That phone, that mobile operating system, that should have dominated the world, instead was a half-assed pile of steaming cow turd that failed on just about every level. I spent hours on the Connect forums for weeks trying to get that phone working, and it just didn’t.

Then the iPhone came along, and it looked really, really nice, but it’s Apple, and I’m a self-confessed Microsoft fanboy. Nevertheless my contract had expired so I could finally get a new phone – but it’s OK! I was told. Windows Mobile 6.1 is much better than 5.0! So (knowing I was making a mistake, but there it was) I got a Windows Mobile 6.1 phone… only to find that not a single one of the issues I had reported in Windows Mobile 5.0 had been fixed. Not one, literally. So again I spent hundreds of hours on the Microsoft support forums trying to sort things out, only to be told in every case “oh yeah, I never use that feature, it doesn’t really work properly”. And it was a two year contract that I couldn’t afford to buy my way out of, so I was stuck with it, and it was the worst phone I have ever owned. I could sit here and write for hours about what made it so bad, I really could.

Happily, I’ve just got an Android (phew, I avoided Apple!) and it’s a thing of beauty. It’s miraculous. I can actually open an internet site in the internet browser and it will actually render! If I then save it as a bookmark, I can come back a week later and the bookmark is still there! Doesn’t sound so amazing? You’ve obviously never had a Windows Mobile 5.0 or 6.1 phone then, because you’d appreciate these things! But now, Windows Phone 7 is on its way… well, no way on Earth was I waiting for that. Not a chance. I have spent hundreds of pounds and hundreds of hours of my life trying to work around basic bugs in Microsoft phone operating systems, and I am very well aware that WP7 is a complete restart compared to WM5 and WM6 (though that didn’t help the KIN…) but sorry, I simply don’t believe that Microsoft know what sane human beings want out of a phone, any longer. I gave them two chances and was punished for it with bug-ridden software and total apathy on their support forums. There will be no third chance. (I am also fairly sure that I am far from the only one to have been so badly burned by WM5 and WM6 – meanwhile, the rest of the world has an iPhone or Android – which by the way now have such magical futuristic features as copy-paste and multitasking, and the excuse “iPhone didn’t have it at launch four years ago” cuts no ice with me).

Long story short: I’m a Microsoft fanboy and even if Windows Phone 7 performed fellatio on demand, I still wouldn’t get one. No non-MS-fanboys are even aware that it’s an option, or if they are they’ll probably get confused with the old WM5 and WM6 phones which were awful. And yet Windows Phone 7 has become the overwhelming focus of XNA. I can see why MS might consider it strategically desirable but this upsets me. As far as I can tell there haven’t been many Windows and 360 features lost in XNA 4.0 (…right?) but if all we’re going to get from now on is Windows Phone 7 shoved down our throats every time we visit the website or the forums, and if all the new features the XNA team are working on are heavily WP7 focussed, XNA itself might get dragged down with Windows Phone 7 when it dies, and that will be a tragedy.

There. I said it. I told you this post would be something of a diatribe, and it got undeniably emotional at times, but I’ve spent the last several years crippled by a godawful mobile phone operating system and that’s the kind of thing that inspires nerd rage. XNA could really make a difference to game development now and in the future, but not if it’s so woefully mistargeted at a platform even Microsoft fanboys have no interest in.

That the next article will be less of a rant and much, much more positive. Pinky promise.

September 10, 2010

What is good code?

Filed under: Personal,Tools and Software Development — bittermanandy @ 12:11 am

I’ve been thinking a lot about good code lately, if only because I’ve been stuck in the unfortunate situation of having to deal with bad code. Without going into the gory details, the people who wrote the bad code were convinced it was good so I had to spend a lot of time and energy explaining why it was not good, and, therefore, what good code is. (With indifferent success, it has to be said; there are none so blind as those who will not see).

A digression: my plan when I started this blog was to keep the articles frequent, regular, game-focussed and specific. Obviously things have been neither frequent nor regular for some time now, and this article continues the ‘recent’ trend of being neither game-focussed nor particularly specific. What can I say? Plans change, life circumstances change, and work on Pandemonium (the game) is on hiatus; at least this isn’t another “how to get into the games industry” article (which is for a blogger what crates are for games designers). I hope you still find it interesting, and as always feedback is very welcome and gratefully received.

So: what is good code? Can we even place a value judgement on code as “good” or “bad”? After all, there are very often many different ways to achieve the result you’re looking for; code is part art and part science, and there’s not a single non-trivial program in all of coding history that is entirely unable to be improved in any way. (Even “Hello, world!” could be localised…). Well, to consider it from the converse point of view, if code crashes or gives the wrong results, it must be bad code; therefore, it is reasonable to conclude that if it ‘works’ (however that is defined), it may be good code. There is however much more to it than that, as we shall see, and there may often be debate, discussion, and extensive philosophising as to exactly what is “good”.

The following list may not be exhaustive, though I think it’s a pretty good start. In general, each item is given in a rough order of priority, those things that I consider most important listed first – but it is critical to understand that what is “important” can vary greatly depending on context. For example you will see that I normally prize readability ahead of performance, but if your profiling reveals that one function is dropping your frame rate from 60Hz to 10Hz, you have no choice but to optimise it, even at the expense of readability.

Good code is…

…correct. It is amazing how often programmers will miss this one when discussing good code. Games programmers, in particular, have an unnerving tendency to mention “fast” as the first thing they think of when talking on this subject. This is a nonsense. The code must do what it is intended to do, correctly, or else it cannot possibly be good.

There is an important implication here. Firstly, the phrase what it is intended to do implies that good code begins with requirements, specification and design – all before a single line of code is written! Exactly how you generate the code design is itself a subject worthy of lengthy discussion, but outside the scope of this article.

Consider: correct isn’t really enough. Provably correct is much, much better. More on this later.

…readable. How long does it take to type enough text to fill, say, a page-long function? Probably only a few minutes. Obviously there’s a significant amount of time invested in working out what to type, but it doesn’t take long. However you are certain to need to read it later - to review it before you commit it into source control, when QA find a bug, when you need to explain it to a colleague, when a colleague needs to understand what it does without you there to explain. Code must therefore be written to WORM – Write Once, Read Many, and I think this is the second most important requirement after correctness.

What contributes toward code being readable? There are many factors – judicious use of whitespace, sensible function and variable names, short functions, use of good patterns and avoidance of antipatterns – but probably the most critical factor is consistency. I don’t really care if you use three spaces or four to indent, but in the name of all that is holy don’t use three sometimes, four other times and five other times still! Style guides, automatic formatting, and tools like StyleCop are helpful in ensuring consistency across a team – but it’s probably more important to stay consistent within a file. So if you’re editing someone else’s code, be sure to match their style. If things are really bad, set aside time to refactor later; but otherwise, match what is already there.

…testable. This requirement is one that has been bubbling up my list of priorities over the years, such that I now consider it one of the most critical factors in “good code”. I have always believed in the adage that “if it’s not tested, it’s broken” – in other words, unless you know and have proved that the code is correct (responds correctly to good input and fails elegantly with bad input) there’s probably some corner case you’ve missed and it will come back to bite you. I used to assume that sufficient QA coverage would be enough. Not any more – code must (wherever humanly possible) be covered by automated unit and soak tests, which must be added to with every bug found and fixed. This is a lesson I have learned through bitter experience. The codebase I am currently working on is untested and bordering on untestable, and it is almost impossible to make changes (even fixes) without breaking something subtle – and there’s no way of catching that subtle breakage until it is too late (ie. the customer has been affected by it).

Interestingly enough, as unit tests are still relatively new to me, I’m still exploring methods of doing it with which I am comfortable. I definitely have more to learn here. I would also add that I do not strictly practice Test-Driven Design myself, though I can see that it might a good idea in principle; however, even though I don’t currently write the tests before the code, I now always consider how the tests can be written while writing the code (which isn’t TDD but it’s close enough to see it on a sunny day) and I think every programmer should do at least the same.

…well-documented. By this I mean not only separate documents listing the requirements, and user guides, and such like; but also comments within the code itself. In general, code should be self-commenting: CalculateDamage() is a better function name than f(), and RemainingHitPoints is a better variable name than hp. In both cases, it is easy to infer what the code does. Comments should not normally describe what the code does, or even how (both of which should normally be understandable from the code itself unless it is highly optimised), but they should be used to explain why the code does what it does in the way that it does it. I have heard it suggested that every comment should contain the word “because”. That’s not a bad guideline.

…robust and reliable. I had a disagreement with someone (a non-coding manager) a while back when he contended that a “genius coder” was someone who had a blazingly brilliant idea, implemented it rapidly, then moved on to the next blazingly brilliant idea, even if the first idea wasn’t completely finished yet - as other coders could then take up the slack. I reject this suggestion utterly. A real coding genius, a guru, a free electron – call them what you will, we all know the kind of programmer I’m talking about, and I don’t claim for one second to be one myself – such a person does indeed implement blazingly brilliant ideas but they make sure it works.

Good code doesn’t crash (for starters). Neither does it scribble over random areas of memory, trash the stack, or give unpredictably different results for the same input. Indeed, restrictions on input are clearly specified, and assertions and/or error handling are used to anticipate bad input and deal with it appropriately (good code is usually tolerant of bad input unless there is a reason for it not to be, such as performance or security). When good code is used by other people, they can be confident that it will work as advertised, and not throw any unexpected spanners in their works.

…maintainable. The simple truth of the matter is this: all code has bugs. At some point, you (or someone else) will need to come back to your code and fix those bugs. Or, you may need to extend or change it to reflect changes in specifications. Do yourself (and others) a favour, and write your code in such a way that it can be easily maintained.

Happily, if your code is readable, well-documented and covered by unit tests it will probably be easy to maintain, but by explicitly remembering that you will probably have to come back to this code while you design and write it, you can make decisions that will make maintenance even easier.

…extensible, flexible and reusable. Once you’ve gone to all the trouble of writing code, do you really want to put yourself through the hassle of writing it again next time you need to do the same thing? Or writing something almost the same but that varies in some minor particular? I would hope not. With good code, it is possible (perhaps using templates or generics) for other people to use the code for purposes that the author may never have expected, except insofar as he deliberately made it extensible. With good code, it is possible for other people to write code that this code then uses, even though it was written first. A study of good software libraries (perhaps most obviously the STL) is enlightening as to how this can be achieved, and how effective it can be.

There is, of course, a risk of over-engineering – remember YAGNI (You Ain’t Gonna Need It): don’t waste time writing code you’ll never need. As always, this is contextual. An IDE like Visual Studio benefits greatly from a plug-in system that allows other people to write software to extend the software itself. Is it worth writing a plugin system for your own game editor? I suspect not.

…efficient, performant and scalable, in terms of memory, CPU, system resources, processor/thread count, internet bandwidth… as much as it’s nice to be able to pretend such things are infinite sometimes, in truth of course they are not. Good code is not wasteful with such things, and makes promises about how much of each it requires (for example sometimes, you can use more memory to make things easier for the CPU; but on an embedded system that memory may not be available, so an understanding of the target platform is required).

In my experience, performance concerns are most often addressed during the design stage, but when it comes to writing the code itself, using appropriate algorithms and data structures (alongside basic guidelines like avoiding allocations/garbage, etc) is usually all that is required. In my experience and by my estimation, 90% of code is not performance critical and so long as you use a sensible algorithm, that will be sufficient. If you do not understand algorithmic complexity, you are not a programmer – whereas an understanding of the effects of branch misprediction, cache misses, false sharing etc. is something that most coders don’t need to spend too much brainpower on, most of the time. (There is still that 10% where those kinds of things do matter, of course; and games programmers find themselves in that 10% more often than most so they tend to have a skewed view of this).

I’m tempted to write considerably more under this heading, but I’m going to stand by my claim that most of the time, “reasonably performant” is enough to qualify for good code. Don’t throw away performance needlessly and you’ll usually be fine.

…secure. I have been lucky enough in my professional career that I have usually been working on products that do not have to worry overmuch about being hacked. Console games and proprietary custom software for a specific company are not usually prime targets for hackers, who are more likely to aim for websites, operating systems or “serious” software where they can either obtain valuable data or cause havoc for lulz.

Or so most people suppose, anyway. I remember breathing a sigh of relief shortly after the release of Kameo because I’d written (among other things) the savegame code for that game, and it was the savegame code in some other game (I forget which, now) that allowed one of the first successful hacks of the Xbox 360: someone managed to edit a savegame file in such a way that the code read off the end, and hey presto, they found a way to launch pirated games. Although I have read a bit about writing secure code I’m sure this is something I could improve on, and I suspect I’m not alone – good code is secure, even if the consequences of being insecure don’t involve giving away people’s credit card numbers.

…discoverable, by which I mean that when someone else comes across your code for the first time, they can grok it quickly and easily. This is something I’d not consciously considered until recently, which is odd given that so many of the problems I’ve been having with the current codebase have been a result of finding that I need to investigate some code, and immediately thinking “WTF is this doing!?”. Had the code been discoverable, the last few months of my career would have been considerably easier. Of course, again, if your code is readable, tested, well-documented and maintainable, it should probably be straightforward for your team-mates to pick it up. But it is always a good idea to think to yourself, “if a reasonably intelligent, reasonably experienced programmer (who was new to this project) had to debug this code, would it take them long to figure out what is going on?”

…simple. Sorry, I’ve had to go back and edit this post because somehow I managed to miss this the first time around! Good code is simple. Even complex good code is comprised of simple building blocks. Good code hides or cuts through the complexity in the problem, to provide a simple solution – the sign of a true coding genius is that he makes hard problems look easy, and solves them in such a way that anyone can understand how it was done (after the fact). Simplicity is not really a goal in its own right, though; it’s just that by means of being simple, code is more readable, discoverable, testable, and maintainable, as well as being more likely to be robust, secure and correct! So if you keep your code simple (as simple as possible, but no simpler), it is more likely to be good code – but that is by no means sufficient in and of itself.

Well, I reckon that’s probably enough to be going on with. Many of the above considerations are worthy of an article in their own right (not that I intend to write such articles any time soon, or indeed at all) but I think I have written enough for tonight! Have I missed anything? Have I got anything in the wrong order? (I guarantee that some people will argue performance considerations should be higher up the list, and in some scenarios they would be correct, but as a rule of thumb I think the above order is generally best). Is there anything I’ve listed that is not actually a requirement for “good code”? Does anything need further clarification? Let me know, in the comments.

April 13, 2009

How not to render 3D graphics

Filed under: Uncategorized — bittermanandy @ 4:38 pm

Heh. Just had to link this because I’ve been there too many times! I think I’ve probably encountered most of these:

http://dmalcolm.livejournal.com/2433.html

… with the addition of “you’re using XNA and have forgotten it’s a right-handed coordinate system”, thanks to Parm for reminding me of that one!

April 10, 2009

A View to a Thrill, Part One: Camera Concepts

Filed under: Games Development,Personal — bittermanandy @ 8:19 pm

I think it’s about time I wrote another article, don’t you?

The funny thing is that I’ve had a draft sitting half-completed for a few weeks now, and this isn’t it! In that draft, I’ve written about some nice (some may say, essential) features for your development tools, because that’s what I’ve been working on for Pandemonium lately. However, a post on the XNA Creators Forums the other day caught my eye, and reminded me of when I used to work on the in-game camera for Kameo, and then I put the disc in my 360 for old-times’ sake, and that inspired me to start this mini-series on third person cameras, based on what I learned during the development of that game. (I’ll almost certainly finish the tools article next, then come back to this mini-series, and so on like that for a while, dipping in an out of each. Note that I don’t promise this mini-series will finish, or even progress, quickly! But over time, it should add up to have some pretty cool ideas).

Incidentally, for those of you who haven’t played it, you can almost certainly pick up Kameo in the bargain bin nowadays and it’s well worth a look. It’s not without its flaws (some of the adventure sequences are a bit dull and the story doesn’t always make complete sense), but it’s still one of the most vibrant and refreshing titles available on the Xbox 360. The combat sequences are pretty good fun, it’s got some really memorable bosses, a phenomenal soundtrack, and (especially given that it was a launch title) it looks very pretty – I defy anyone not to watch dawn break over the Enchanted Kingdom and not go “wow”. It got some criticism at the time for being a bit short for the money, but by criminy it’s eight quid on Amazon now! Probably even cheaper somewhere else. Get involved.

The Very Beginning

Anyway with that bit of blatant and shameless advertising out of the way, let’s talk cameras, starting at the very beginning. The camera is what shows the player what is going on in any 3D game. The player never sees it, and they can’t always control it, but without it you’d be stuck. It is exactly analogous to a movie camera – it’s there to show the audience (the player) what the actors (the player’s avatar and other in-game characters) are doing. Everything in the game world that you see on the screen, you see from the camera’s point of view.

The mapping from the 3D game world to your 2D television screen is called a projection; I won’t go too deeply into the maths just yet (and in fact in XNA, DirectX, OpenGL and other 3D SDKs, functions exist to perform much of the maths for you) but basically a projection transforms a 3D vector in world space to a 2D vector in screen space. Fundamentally speaking there are two types of projection (hence, two types of game camera).

An Orthographic camera performs a parallel projection. Basically, this has the effect that something 2 metres tall 100 metres away, looks the same size on screen as something 2 metres tall 5 metres away, and the viewer can only tell which is closer by seeing which is drawn in front of the other. This is not how we are used to seeing the real world, so isn’t much used in games nowadays. That said, the old isometric games show what an orthographic projection looks like (even if they’re not truly rendered in 3D), and it’s sometimes used for user interfaces, maps and such like. In fact – if you’ve ever used a 3D modelling tool like Max, Maya, Milkshape, Blender etc., you’ve almost certainly used an orthographic projection in the “side-on” or “top-down” views. I won’t go into too much detail about these here though; if you’re interested, look up Matrix.CreateOrthographic in the XNA documentation.

Much more common in games is the Perspective camera, and that’s the type of camera I’m going to concentrate on in this series. In this projection, items closer to the camera are drawn much larger than items the same size that are further away, which matches human eyesight.

The Perspective Projection

We need to know a certain amount of information in order to be able to perform the perspective transformation. First is the field of view. This value represents how wide an angle the camera can see, and by convention is usually given as the vertical angle – the vertical field of view, or sometimes abbreviated to fovY. (Some modelling tools use the horizontal field of view instead, so be careful). Next is the aspect ratio. Computer screens are rarely square, and never circular (like the human eye can see), so the aspect ratio gives the width of the projection compared to the height of the projection.

Deciding on what values to use will depend on your game, but as a starting point, consider that the human eye has something like a 150 degree vertical field of view, and an aspect ratio of something like 1.15, ie. you can see more horizontally than vertically. However – nobody ever sits so close to a computer screen that it fills their entire field of view, and the angle from your eye to the top and bottom of your monitor is usually something like 45 degrees, so that’s a good starting point for your field of view; and the aspect ratio is easy. If your monitor or television is “standard” 4:3, the aspect ratio is 4/3 = 1.3333, while if it’s “widescreen” 16:9, the aspect ratio is 16/9 = 1.7777, etc. (And therefore, if the camera view only fills part of the screen in your game, the aspect ratio will simply be the width of the view window divided by the height). We’ll talk more later about what effect changing these values has – you can get some desirable special effects, and some undesirable bugs, by altering them.

Important note: I’ve used degrees for the field of view above, because that’s what most people are familiar with. Most 3D systems, including XNA, use radians for angles instead – and if you get funny results, it’s probably because you’re using the wrong units! You can use MathHelper.ToRadians to convert from degrees to radians, or some special values have predefined constants, for example 45 degrees is the same as MathHelper.PiOver4 radians.

There’s a couple more values needed for a perspective projection but, unlike field of view and aspect ratio, they don’t have a counterpart in the real world. Your eyes can see things right next to them, even touching them (I had a wasp walk over my eye, once – it was terrifying!) and, if nothing gets in the way and the light has time to get to you, infinitely far away as well. That gives an infinitely large range – and computers are very bad at handling infinity. To make life easier for the computer, we define a minimum and maximum distance – the near clip and far clip values – outside of which, we don’t want to draw anything. Choosing these values is very much game-dependent. In a game where a human is the main character, a near clip of 0.1 metres and a far clip of 1000 metres might be OK. In a game based around spaceships travelling light years in a few seconds, that wouldn’t work! Again, there is a relationship between the near clip and far clip distances that you don’t want to get wrong, and we’ll talk about that later.

I like to think of this as being like sitting in a dark room with one window. You sit in the room and look out. You can’t see anything in the room, it’s too dark (even if there’s something between you and the window – yeah, the analogy breaks a bit there) so the distance between you and the window is the near clip distance. Outside the room, you can only see things that pass behind the window – anything in the vertical field of view (the angle between your eyes and the top and bottom of the window) and the field of view modified by the aspect ratio (the ratio of the width of the window divided by the height. In your imagination, draw a line from your eye to the edges and corner of the window. Can you see that this would form a pyramid? Now extend the lines, out into the world beyond the window. Stop at your far clip distance. Now you have a really big pyramid. Take off the smaller pyramid (inside the room) and you’re left with a shape called a frustum, in this case, a view frustum. Everything inside this frustrum can be seen through the window – so everything in the view frustum in your game will be drawn to the screen. It looks a lot like the red shape shown here (based on original diagram here):

The view frustum of a perspective projection

The view frustum of a perspective projection

(Note, I’ve not shown the horizontal field of view, it should be obvious. However, the maths to calculate it isn’t… it’s not fovY * aspectRatio, as you might guess. It’s actually 2 * atan( tan( fovY/2) * aspectRatio ), for reasons of geometry. Aren’t you glad you wondered, now?)

The maths to perform the projection from items in the frustum onto the screen is a bit complicated, luckily, it’s all done with Vector3′s and Matrix’s, and XNA provides a function to generate the projection matrix (which performs the transformations of your perspective projection): Matrix.CreatePerspectiveFieldOfView, which takes as arguments the field of view, aspect ratio, near clip, and far clip, exactly as just described.

Defining the Camera View

So now we know how a view frustum performs a projection to draw things in the game world, onto the screen. But which things? Or to put it another way: how do we position the view frustum to show what we want? Answer: by defining the position and orientation of the camera.

It’s pretty obvious what the position means. If it were a “real” camera, like in a movie, it’s the point in space where the camera (or perhaps more specifically, the camera lens) is. You just need to make sure you put it somewhere sensible so the player can see what’s going on. (How? We’ll get to that!)

The orientation is fairly straightforward too. It’s the direction the camera is looking in – this gives us the camera’s forward vector, literally, the direction you’d move in to make the camera go forward. Obviously, you’ll want this to point at whatever the player wants to see, and again, we’ll talk about how later… Of course, you can hold a camera upside down (and, if you’re making a flight sim, your planes can fly upside down!) so we also need to know which way up the camera is being held: the up vector. Finally, it’s sometimes useful to know the right vector – think of your head. Your eyes are looking in front of you (the forward vector), the top of your head is usually towards the ceiling, unless you’re doing yoga or something (the up vector), so the right vector is pointing out of your right ear. These three vectors are mutually orthogonal, which is a posh way of saying there’s ninety degrees (or MathHelper.PiOver2 radians!) between each of them. (See diagram here).

 The tricky bit comes in deciding how to represent the orientation. (It’s important to understand the following are all different representations, and for any given orientation, it’s possible to represent it in any of the following ways). We can either simply pick a point in space, and say “the camera is looking at this point” – this is the target or look-at point.

Or, we can assume that the “natural” orientation of a camera is for the forward vector to be looking down the Z axis (known as -Z, -Vector3.UnitZ, or Vector3.Forward – remember XNA uses a right-handed coordinate system!) and the up vector is directly upward (known as +Y, Vector3.UnitY, or Vector3.Up). Of course this implies that the right vector is along the X axis (+X, Vector3.UnitX, or Vector3.Right). From this “natural” orientation, we can rotate it – either using angle values for yaw (rotating horizontally around the Y axis), pitch (rotating vertically around the X axis), and roll (rotating side to side around the Z axis), known as Euler angles (pronounced “oiler”), or by using a single Quaternion value. Each of these methods – Euler and Quaternion – have advantages and disadvantages (Euler is easy to understand, but can encounter issues in certain orientations that quaternions, to all intents and purposes, don’t) and which to use depends on your game. If you’re making a flight sim or other game in which the camera can roll a lot, use quaternions. Otherwise, either can be made to work, and Euler angles are certainly easier to conceptualise.

We can combine the position and orientation into a single view matrix for convenience, and no prizes for guessing that XNA makes it easy, via Matrix.CreateLookAt, Matrix.CreateFromYawPitchRoll, and Matrix.CreateFromQuaternion.

Our projection matrix and view matrix now tell us everything we need to know. When we went to draw something, we pass these matrices to the shader (sometimes along with the World matrix, which represents how the game world or object being drawn is transformed; in the case of the game world, it is usually Matrix.Identity) and it performs the complex transformation maths for us. Note that the projection matrix might not change much during gameplay, but that the view matrix might change in every frame that the camera moves.

So all that’s left is to work out where to put the camera, ie. what values to pass in, in order to get the appropriate view matrix. Should be easy! Right? Well… the number of games reviews where I’ve read “game X has a really bad camera”, and the fact I spent probably 40% of my time for three years coding Kameo’s camera (happily no reviews I saw complained about it), suggest it’s not that simple. (Don’t get too scared. Kameo had 11 player characters and over 20 camera modes. Your game will probably be a lot simpler!) So how can we decide where to position the camera?

Deciding the Camera Position

At a very high level, there are basically three ways to decide how to position and orientate the camera, and they arise from the way that in most cases, you either have something you want to look at, or somewhere you want to look from, or both. (If you have neither, and don’t know where you are or what you want to look at, well, good luck with that!).

First consider the case where you know where the camera is, but you don’t know what you want to look at, just a direction. The most obvious example of this is the first-person shooter. The camera is inside the player character’s head, and the player decides in which direction to look and move. Or, imagine a driving game or flight sim where the camera is placed inside the car or plane, looking out of the front window. This kind of camera is very simple: the position is known, and you can use Matrix.CreateFromYawPitchRoll (for example in a FPS) or Matrix.CreateFromQuaternion (for example in a flight-sim interior camera) to work out the rest of the view matrix.

Example: Halo.

Secondly, imagine you know where the camera is, and also know what to look at. An example might be a track-side camera in a racing game: it’s fixed in position, but as a car races past it turns to follow it. Or, imagine a security camera that follows the player (a bit like in Ico). It’s even easier if the camera is fixed in both position and orientation, like in the early Resident Evil games. Again, this kind of camera is very simple: the position is known, and you can use Matrix.CreateLookAt to work out the rest. (Of course, track-side cameras should have some inertia to follow the cars more naturally, and the camera in Ico has quite a few more complex features as well, but the basic functionality is very straightforward).

Example: Ico.

The final category (and by far the most interesting) is where you know what you want to look at, but not the position you want to look at it from. Many game genres fall into this category.

Sometimes, it’s fairly easy to work out where the camera should be. In an arcade racing game, the obvious target is the player’s car; and the player will want to see where they are driving. So, you need to position the camera somewhere behind and above the car, looking somewhere in front of it. Of course, you’ll want the camera to follow the car as it turns, and if the player reverses into a wall you need to make sure the camera doesn’t go on the other side of the wall, but still, this is pretty straightforward. (An example from a non-racing game might include the early Tomb Raiders, and I think the Gears of War over-the-shoulder camera probably belongs here as well).

Example: Project Gotham Racing 4.

Another example of this category is to be found in the RTS genre. Here, the camera looks at the units or terrain the player is interested in, but the player can control where it looks at them from. Assuming you position the camera high enough not to bump into buildings and trees, again, this is quite straightforward – store the look-at point and move it around the battlefield, then let the player control the angle they want to view it from, and position the camera a fixed distance in that direction.

Example: Age of Empires 3.

So finally we come to the most interesting cameras of all, and the ones I will be concentrating on in this mini-series: the third-person camera. In games of this kind, the camera and the player are controlled independently. The camera must move around the game world, giving a good view of the player character, while moving around and among the terrain the player moves around and among. There are some very difficult (and hence interesting) problems to overcome – how do you stop the camera going through walls? How do you ensure there is always a good view of the player? To what extent should you let the player control the camera position – and even more importantly, to what extent should you require them to? Games with cameras like this include platformers, like Mario or Banjo Kazooie; parkour-style action games like Prince of Persia; and adventures or RPGs like Fable, or, of course, Kameo.

Check out this screenshot of Kameo in action. She’s standing among the tall grass now, cheekily glancing back at the camera – but she could move in any direction and the camera would have to follow her. She could go for a swim, run around the tree, move right up to the edge of the cliff, go into the house; climb to the very top of the castle and jump off it. No matter what she does, the camera has to follow her, and show her doing it in a way the player can understand and react to. That’s not easy! But it is absolutely fascinating. I hope this series will demonstrate that, and you’ll find it entertaining and useful.

Anyway I’m moving house next week – so not sure when the next update will be – so, until next time, farewell from me…!

February 9, 2009

Mean salary

Filed under: Games Industry — bittermanandy @ 2:06 pm

Just a quickie… interesting follow up to my recent article, which was followed by some comments about salaries in the games industry.

Develop have got their annual survey salary out, and list the following:

Average Yearly Salaries:
Lead Programmer – £41,250
Programmer – £25,810
Junior Programmer – £18,928

A few thoughts spring to mind:

1. Wow. Now I remember why I left the games industry.

2. £26K for a programmer is less than £29K for an artist or £27 for a junior audio engineer! That disagrees very strongly with my anecdotal experience. I don’t know of any artists or audio guys paid more than their programmer colleagues, and didn’t think that was the normal state of affairs at any games company anywhere. (Design is a tricky one. Junior designers get shafted, but lead designers / directors can make a fortune). Surprising.

3. £19K average for a junior programmer (implying many get less)? That’s appalling! I was paid £20K as a junior seven years ago! Also, the last company I was involved with hiring at, offered around £23K as standard for new junior hires.

I’m not going to say the Develop survey is wrong, because they’ve had a wide range of respondents whereas I’ve only got my own personal limited anecdotal experience to draw from. What I will say is, if those Develop figures are right, there’s a lot of people out there getting completely stitched up. They’d be earning a lot more in another industry. Is that worth giving up, in order to fulfil their childhood dream of making games for a living? That’s a question only they can answer. For a while, my answer was “yes”. Now, though, I stand my updated decision of making software for a living, and making games for fun.

January 22, 2009

It’s not the size that counts

Filed under: Games Development,Games Industry,Personal,XNA — bittermanandy @ 10:37 pm

Indulge me as I walk down memory lane…

It was summer 2002 and I’d just landed my first professional job. (I’m still not sure how. I wasn’t very good back then. I like to think they saw that I’d become good, and I like to think I am good now. It might just have been luck!). I’d spent some time in the RnD department, to learn the ropes, become familiar with devkits (console hardware with added stuff to enable game development), and understand a bit about how the shared game engine worked; and then the time came to move onto a game team, of which there were five or six to choose from. The choice wasn’t freely given. Each team had different needs, so the question was asked: what did I want to work on? What did I want to specialise in?

I didn’t really know how to answer that. I was vaguely aware of the division between systems programming and gameplay programming, and knew that I preferred the former (I actually think the latter is better solved by a good data-driven system and a good designer, though I have worked with gameplay programmers who produced excellent results); but which system? And how would my choice affect my working day?

Games teams on “AAA” titles nowadays can easily have several tens of programmers. Clearly, if they all worked on whatever they fancied from day to day it would be a disaster. Each programmer therefore gets designated an area of responsibility. Generally speaking, the programming lead and the most senior programmers determine the overall architecture of the game early in development (or it may be mandated by the engine, particularly if it is middleware), and the programming team splits into several sub-teams, each led by a senior programmer who reports to the lead programmer. Examples of roles in the team (in no particular order) include:

Graphics: the celebrities of programming because they get to write code that produces awesome looking screenshots (or at least… code that lets the artists do so). Every time the publishers come for a visit, they’ll get led into the graphics programmers’ office and shown all the latest particle-laden explosions on flashy HDTVs. Graphics programmers spend a lot of time writing shaders, optimising the renderer, and talking with artists.
Networking: modern games are immensely complex and with multiplayer online being practically compulsory nowadays, every game will have network specialists. They tend to spend all their time trying to teach other programmers how to write code that doesn’t break the online mode, for example by sending 600KB packets every frame or updating something on the local client but not the game server. They usually look a bit stressed.
Physics: even with middleware like Havok (or in XNA, JiglibX and the like) available, physics remains one of the most complicated things in a game because it affects just about everything else. One game I worked on had five physics programmers.
AI: most games have enemies or some kind of non-player entity. While scripting and other designer-facing tools mean AI is not as hard-coded as it once was, someone’s got to write the code that interprets the scripts – that’s the job of the AI coder.
Audio: given that the only two ways your game can influence the player is via the screen and the speakers, audio is half of every game. Unfortunately it’s the second half (because it doesn’t look good in screenshots) and all too often audio is neglected. Done well, it can turn a good game into a mind-shatteringly atmospheric epic. Audio coders spend a lot of time talking to the musicians and SFX engineers, and they’re usually slightly bitter that the graphics programmers get all the plaudits (and flashy HDTVs).
Tools: there might be twenty programmers on a team, but there might be ten designers, fifty artists, and five audio engineers (as well as testers, producers, marketing, translators…). You can’t just give them a copy of Photoshop and Maya and tell them to get on with it. Every game needs specific tools that enables these people to get their assets into the game and tweaked until fun and in my experience, the better the tools the better the finished game.  Historically tools were DOS-based and unreliable; increasingly, they’re now written in C#/.NET and actually work more often than not. Tools programmers are the unsung heroes of the programming team. Their work is almost never seen by the public, but without them, the game itself won’t get seen by the public either.
Systems: asset loading. The game camera. Multithreading. Save games. Text, and menus. Achievements. DVD file layout. TCR compliance (rules that the console makers require you to obey before your game can be released). Support for steering wheels, dance mats, webcams and chatpads. The build process (putting together versions of the game to give to artists, management and testing). A veritable pot pourri of tasks that no game can go without. Some of these tasks will be given to the most senior programmers because they’re critical to the game’s success. Others will be given to the most junior programmers because they’re relatively self-contained and can be developed in isolation. Most don’t get noticed, until you try to make a game without them!

There’s more, but that’s a good initial summary. (Just think – when you write an XNA game, you’re responsible for all of the above! Lucky XNA itself is brilliant at doing it loads of it for you). My first ever task on a game team was to write the game camera, and looking back I’m pretty proud of how it turned out. Very soon afterward I also took on the audio programming. Eventually, on that first game (I worked on it for three years – some were working on it for twice that), I was responsible for, or otherwise involved with, text and menus, localisation (making the game support other languages), asset loading, the build process, save games and TCR compliance. Later I’d work on asset optimisation and arranging files on the DVD, and probably some other stuff I’ve forgotten. So it became clear: my specialisation was that I was a Jack-of-all-trades. Hurrah!

With all these people working on different things, it’s critical that they don’t interfere with one another by writing over one another’s changes. This is achieved by use of a Source Control System. Basically, this keeps track of every code file and asset in the game, keeps records of how they change over time, and tries to ensure that if two people make changes at once, those changes are seamlessly merged together. Different teams use different products and approach this in different ways. Some examples using the codenames of games I worked on:

Game Two: used CVS for source control. This is an horrific abomination that should be scourged from the earth. A coder would make all the changes he wanted, send out an email to the team saying “please don’t commit”, commit all the files he’d changed, and send another email saying “OK to commit”. Inevitably he’d have missed something so the next coder to update would have to come and ask him to fix the problem before they could continue. Worst of all, the artists couldn’t get anything into the game without giving it to a programmer to commit it for them. This was frustrating for everyone involved and meant that changes to a level, for example, could take two or three days to get into the game. Putting together a build was an eight hour manual process; all too often it wouldn’t start until 5pm the night before a deadline. I’m not completely sure how we managed to get the game finished, and I’m stunned that it turned out as good as it did despite everything. This is How Not To Do It.

Pocket/Pikelet: an improvement in every area, these teams used Source Depot (basically the same as Perforce) for source control. No more emails to control who could commit – a tool that lived in every developer’s System Tray would lock out commits while someone was going through the process of updating, building the game, running the unit tests and committing. A separate build machine would then automatically update to that version, run the tests again and if they were passed make the build available to all the non-programmers – who, incidentally, had the tools available to do all their work without needing to go through a programmer. It was brilliant, broken builds and artist downtime were unheard of – the game was bulletproof throughout development. There was only one small problem. Updating, building, running unit tests and committing took half an hour or more. In a normal eight hour day, only sixteen programmers could do it – at most. With twenty programmers on the team, there’s an obvious problem and it got very frantic near deadlines – and leaving at least a day between committing meant that you’d commit too much at once, introducing bugs (which would fail the unit tests and delay your commit even longer). This was a problem, but in general this was the best system I’ve had the pleasure of working in.

Polished Turd (not the real codename, just what I call it): again this used Perforce for source control, except this time without a formal commit queue like Pocket/Pikelet. The commit queue had originally been introduced to stop CVS from breaking everything, but Perforce is so much better than CVS that actually things rarely break even if you’re free and easy with committing. It meant that you could fix a bug, commit your changes, fix another bug, commit your changes, and iterate very rapidly through your work. You’d update over lunch and overnight, or when you noticed someone had committed something you need. If we’d only had the same team build system, unit tests, build suite, and artists tools from Pocket/Pikelet this would have been the ideal way of working. (Unfortunately all those things didn’t exist so the game was unstable, the artists couldn’t do their job, and every deadline was a mad scramble to put together get something vaguely playable that lasted more than five minutes between crashes).

Now, most people reading this will be hobbyists working in XNA on their own. At the moment, that applies to me too. However, I’m uncomfortably aware that the big blue blobs and soulless grey polygons that currently represent the actors and props in my game won’t inspire other people to play my game – and I’m closer to being autistic than artistic. So sooner or later, I’m going to need other team members to make models, textures, animations and sound for me. (Any volunteers?) And, while I’m currently planning to save all the coding duties for myself (with the XNA Framework itself and the multitude of excellent community libraries out there, this is a realisitic proposition in a non-trivial game for really the first time this side of about 1998) many of you will no doubt be thinking of forming small coding teams, to split the work among you. (I don’t blame you. Just look at the list above. Even a simple game has a lot to do…) So – how well does XNA support medium-to-large game teams? And what would such a team require?

Source control is absolutely essential the moment your team grows larger than one. And, with a team of one or two, there’s only one choice – Perforce, which is free for up to two users, and so good I use it even though no-one’s forcing me to. Unfortunately, as soon as your team size hits three people, it’s something like $700 a license. Ouch. Worth absolutely every penny and more if you’re a pro developer with your outgoings covered by a publisher, but out of reach of everyone else. I am told that Subversion is good for a free product, but I look at the lack of atomic changelists and cringe. Probably the next best if you can’t afford Perforce though. There’s nothing here that’s different for XNA than if you were using C++ or any other language.

It will be essential to have a configuration of your game that allows your artists to put their assets into the game without waiting for you to do it for them. The XNA content pipeline is a wonderful, wonderful thing of great beauty, but by default it is tied into Visual Studio – the content pipeline build occurs just before you run your game, and that’s it. An artist doesn’t want Visual Studio, and he doesn’t want to restart the game every time he changes a texture – he should be able to save off the texture file and see it change in-game. So you need to provide them with a version of the game that runs on its own (perhaps in a WinForm) and a tool that hooks into the content pipeline and will let them build assets while the game is running, notify the game, and the game then reloads that asset. This is pretty easy for assets that already exist in a content project, but the tool will need to support adding to the content project (hidden from the artist) and even creating new ones. All this needs to hook into your source control system without your artist needing to learn to type “p4 -d -a -q -z” or whatever.

The content pipeline causes a few other problems too. (Though don’t think for a second that I’m knocking it. What it does is amazing). Big games tend to generate thousands, even hundreds of thousands of assets – and in the content pipeline, each of those is a different file. People often don’t realise that opening a file, even from a hard drive, can sometimes take as long as a quarter of a second. That doesn’t sound like much until you multiply it by a hundred thousand. (You can very easily see this for yourself by using WinZip to zip up one large file of say 10MB, then compare it to zipping up a thousand files of 10KB each – I guarantee the latter will be much slower). To give an XNA-related example: Kameo, Pocket and Pikelet all used XACT, the same audio tool as provided with XNA (and which I’ve used very extensively). Pocket, in particular, had well over 200 soundbanks which all needed loading at startup. Even though they were only between just 1 and 4KB each, this took a long time to load from the DVD - so long that the 20 second load time mandated by Xbox 360 TCRs seemed an impossibility. The fix was pretty simple: we altered the build process to package all those soundbanks into a single file, loaded the file into memory, and loaded the soundbanks from that memory. Instantly the problem was solved. The same solution won’t work in XNA: there’s no method to load soundbanks from memory. It expects each soundbank as a separate file, and that’s that. A big game in XNA would really struggle under such restrictions. (Loading on a background thread won’t help, 20 seconds of disk access is 20 seconds of disk access and that’s that, though admittedly an XNA game would be on a hard drive not a DVD which helps a lot). You’d have to plan ahead very carefully – in terms of the game design, not just code; which is hard, because designers don’t understand arbitrary restrictions from code, and nor should they have to - to ensure that there was never a need to load so many individual files all at once.

With these caveats in mind I’m convinced a medium-to-large game can realistically be made in XNA. The “poor performance” of C# compared to C++ is for the most part a myth, though to get the absolute 100% optimum out of the hardware might require C++ – but you could get at least 95% of the way there with C#, and 98% of all games don’t need 100% performance. It’s unfortunate that the biggest win for hobbyists and small-team endeavours – the brilliant content pipeline – appears to be the biggest limiting factor for large teams. The issue of when and how assets are built, and how they can be rapidly iterated without restarting the game, is definitely solvable, though with a fair bit of coding effort; the fact that so much of the pipeline relies on a one asset, one file relationship is trickier. A hundred-thousand-asset game would need a hundred thousand files, and loading times would be horrific.

So really large games might not be easy in XNA. Happily (?) few large games nowadays are single platform, and since XNA is not available on PS3 and Gamecube, I don’t think many large teams will be evaluating XNA anyway. Medium teams, independents, and small hobbyist teams can definitely have a field day with it (though profitability of XBLCG at least has yet to be proven). Just remember that the limiting factor for such games is content, and put a lot of effort into making your artists’ lives easy. If they can make a model, put it in your game, play with it, tweak it, play with it again, tweak it, play with it, and check it into source control, they’ll beg to be allowed to make more. If they have to send you the model and wait two days for you to send them a build with it in, they’ll give up within a week. Frankly – even if you’re working on your own, and doing all the code and art yourself, a little bit of work to make good tools will make your own life easier; and isn’t that what we all want?

January 21, 2009

Any requests?

Filed under: Games Development,Personal,XNA — bittermanandy @ 1:51 pm

Wow, it seems like an age since I posted. I guess that’s because it is! Fable 2 took up far more of my time than I expected, I’ve barely touched Nuts and Bolts, and I still have to complete Tomb Raider and Prince of Persia before March, when Empire: Total War is released. This doesn’t leave much time for any meaningful development on Pandemonium, though I have recently chucked together a few little prototypes of stuff I might one day make into fully-developed games.

Anyway I don’t know when it will be that I’ve done enough on Pandemonium to make an update worthwhile. But I don’t want to leave the blog stagnant, it’s still getting a fair number of hits every day and people must have expectations… so I’ll put the question out there: is there anything (preferably XNA-related) you want to read about? Check out the stuff I’ve already written to get an idea of the kinds of things I know. Clue: I’m not the man to ask about whizz-bang graphical effects, but if there’s anything else you’d like to see, just ask.

December 10, 2008

The Masonic Handshake

Filed under: Games Industry,Personal — bittermanandy @ 11:43 pm

I’ve had a few people contact me to ask, “do you have any advice for how to get into the games industry?” Besides being ironic given that I’m no longer a part of the industry myself, this is a phrase that I really despise, because – well, I’ll get to that in a minute.

I’m going to start by making a couple of assumptions. I’m going to assume that the the person asking the question is really asking, “how do I get a job as a games programmer?” The reason for this assumption is that it’s the only thing I can speak with any experience about. Modern games (at least, console and AAA PC games, which is what people usually mean by “the games industry”; casual or flash games, not so much) involve teams of tens or even hundreds, involving programmers, artists, designers, scriptwriters, musicians, testers, producers, marketing, and a whole bunch of other roles. I’ve worked with all of the above, but I’d be guessing if I tried to tell you how they ended up there.

The second assumption I’ll make is that working as a games programmer is something you really want. The pay is low, compared to non-games programming. The hours are longer (though this is starting change) and the respect from management lower. Most games – and hence, the games worked on by most people – aren’t the amazing high-quality AAA 96%-rated blockbusters like Half-Life 2 that we’d all love to be a part of; in fact, most of the work you do will be soul-destroying just like in any other job. If you’re lucky, you’ll get to work at a company that lets you leave at 5pm, in which case you will get to see the sun; but even if you don’t have to crunch you could easily go months without seeing those strange creatures they call “females”. In fact, for all these reasons and more, I no longer work in games myself. But be clear about this – I’m glad I did it (for about six years). Knowing you’re doing a job you love, and finally seeing your baby on the shelf at Game and hearing someone say “yeah I played that, it’s quite good actually”, is a buzz of fulfilment like no other. Decide for yourself whether the balance works for you. It did for me, for a long while.

Anyway, where was I? Oh yes – “getting into the games industry”. I hate that phrase. It suggests there’s something magical about it, as if you need to know a special password or shake hands with the interviewer in a particular way. The truth is that games have matured into an industry like any other and you’ll get a job programming games the same way you’d get any other job.

Let’s pretend I’m going for a job as an architect. Should I:

(a) spend all day looking at my favourite building, but not really bother about any others because they suck, take a course in Social Studies because you only have to go to three lectures a week, then wonder why architecture companies don’t even respond to my applications even when I put “…it’s what I’ve always wanted to do, and I’m a fast learner!” on my CV; or

(b) learn how to be an architect, get qualified, put together a portfolio, apply for jobs, and make sure I nail the interview?

Third assumption: you answered (b).

Get Competent

If you’re going to be a games programmer, you need to be a good programmer. Sounds obvious really. I’m tempted to say, “it’s that simple”, but of course it’s not. The thing is, if you’ve yet got a job as a professional programmer, you’re probably not aware (even if you think you are!) of what being a good programmer means. That’s OK. The good news is: no-one expects you to.

The simple fact of the matter is that learning to program well comes only with experience. And, if you don’t have QA teams tearing your work to shreds, or customer complaining your software crashes, you may never realise when you’re making mistakes. (Don’t let that stop you coding – a lot – in your spare time. Any experience is better than none). When I first began applying for jobs, I rated my coding skills as 8/10. Looking back, I was more like 2/10. I’m not sure I’m more than 7/10 now, more than six years later. That’s fine – companies looking to employ new hires only really expect 2/10. So why, you might wonder, bother putting any effort into learning to code at all? You’ll learn it all on the job, right? Wrong. Getting to 2/10 is hard (remember the default state is zero!). If you don’t understand pointers, if you can’t fix compiler errors and warnings, if you can’t identify and eliminates bugs in your own code and other peoples’, if you can’t make estimates about the performance of a given algorithm: either learn that stuff or give up now. No company will expect a new hire to be the finished product. But no company will hire someone who hasn’t bothered to learn what they need to know to do the job, either.

What language should you be programming in? I’d love to say C#, but I can’t. Not yet anyway, perhaps in a few years. Currently, most companies only use C# for tools (and some not even then). The overwhelming majority of games companies use C++, and most of the rest use C. (Or, mobile developers probably use Java?) A few devs are starting to use C#, but most don’t and sadly won’t. There are a few very good reasons for this, despite the fact that C++ is a dinosaur and C++0x is going to be at least ten years too late. The first very good reason is that companies currently have millions of lines of C++ and tens of C++ programmers, and they’re not about to throw either of those away just because someone like me claims C# is more productive. The second very good reason is that there is no C# compiler for PS3 or Wii – and that’s a winning argument for companies intending to make games for those platforms. So if you want to make games,  I recommend C# and XNA. But if you want to get a job as a games programmer, I’d be doing you a disservice to say anything other than to learn C++.

You should also have a passing familiarity with two or three other languages (even if only to know that “the major differences between C++ and <language> are…”), an understanding of computer science principles, and above all, the ability to work as part of a team, including with team members who are non-programmers.

Get Qualified

Already, I’m sure some of you out there are screaming at me about this. “You don’t need a degree to be a good programmer!” they’re shouting. That’s not in dispute. “Just because you have a piece of paper, doesn’t mean you are a good programmer!” they’re adding. Completely agree with that too. However, nowadays, if you don’t have a degree in some kind of computing-related field, you probably won’t get past the recruiter’s first cut. Yes, there are exceptions, but they are exceptional! You could take a gamble, but the odds are desperately slim and you’d be much better off getting a degree. (The only time I’d say that doesn’t apply is if you’ve already been working as a professional programmer for years, in which case, yes, experience counts at least as much as a degree. But for a school-leaver or graduate, a degree is, to all intents and purposes, essential).

But – a degree in Computer Science or similar traditional field, or a new-fangled degree in Computer Games Development, perhaps even from a games-specialist University? Well… I cannot deny my prejudice here. My degree was in Computer Science, and I learned things on that course that I don’t think I’d have learned on a games-specific course that since proved invaluable. There is also the consideration that, should things not turn out as you hope, a CompSci degree will get you a job in a non-games company. I’m not sure that a Games degree would do the same. Or, look at it another way: any company will hire someone with a CompSci degree, some companies won’t hire people with a Games degree. That may change over the years, in fact I expect it to, but again – play the odds!

I should mention that at least two excellent people I worked with had Games degrees, so I wouldn’t dismiss someone with a Games degree out of hand. But, I got the impression that they were brilliant despite their Games degree, not because of it. I’ve also seen a lot of applicants with Games degrees who were simply awful and had clearly been let down by their course. It’s up to you, in the end – a Games degree or a CompSci degree is entirely your choice, and either can be made to work - but I’d still suggest that, for now at least, the latter is the better option.

Put Together a Portfolio

So you’re a good programmer and you have a piece of paper to prove it. Excellent! Regrettably, companies will still need convincing. There’s only one way to do that – happily, it coincides with the first piece of advice, “get competent” – and that’s to write some code to show to a prospective employer. (Writing the code will help you become competent, and as you become more competent you will have better code to show to employers).

What kind of code? Well, you’ll need to show a few things really. First, a simple, yet complete, polished, and bug-free game. When I say “simple”, I mean something like Tetris – but when I say complete, I mean with menus, high-score tables, multiplayer, pause, an options screen… everything you get in a pro game. Bug-free should speak for itself… Secondly, you’ll need a more advanced game that may be incomplete but demonstrates more demanding problem-solving. It should involve two or preferably more of the following: 3D graphics, 3D audio, physics, networking, AI, data driven development, and save/load. Finally, you’ll need one game component (from the list above, or “something else”) that is highly polished and well developed and could be plugged into someone else’s game if they chose to use it, so should also be clearly documented with a sensible API.

You should provide this code with your application, either on CD if it’s by snail mail, or on a website  if your application is in electronic form. Provide your complete source code, and make crystal clear exactly which parts were written by you and which by someone else. Also provide the built versions of the software (on the CD, or as a separate download from the code) but remember that actually it’s the code that’s important – recruiters probably won’t run the game and certainly won’t play it for more than five minutes to check it doesn’t crash, but they will be interested in whether your code is well-written. Only ever show your latest, best code – quality beats quantity every time.

Apply For Jobs

First impressions count for a lot: make sure yours is positive. Ensure your CV and covering letter are polished, clear and professional. There’s a lot of websites with good advice which I won’t attempt to repeat here.

Where should you apply? Well, there’s a lot of options. Do some research (subscribe to Develop and Game Developer, and join Gamasutra, GameDev.net, GameDevMap.com and probably a bunch of other places), and try not to close too many doors. When I was a graduate, I applied to forty games companies. I got three interviews, one of which was cancelled the day before the interview itself, and one job offer, which became two a few months later, after I’d already started working for the first. (In contrast, when I started looking to leave Rare, just six applications garnered me six interviews and five job offers – that’s the power of experience!).

Remember that large companies (EA, Microsoft, Sony, Nintendo, etc) are more likely to recruit new hires than small companies, who may not be able to afford to employ someone who will, inevitably, not be very productive for at least the first six to twelve months. Bear in mind the corporate culture of such companies – though remember you might not be able to afford to be choosy. Medium-sized companies sometimes offer a less corporate culture while still hiring graduates; some are more noted for recruiting out of University than others, for example Rare, Blitz and Codemasters in the UK. (Again consider the implications. Are they hiring graduates because they are able to recognise untapped potential – which is a very good thing – or to get a cheap source of labour – which is a bad thing? You’ll need to work that out for yourself at the interview). Whatever happens, don’t apply blind. I’d encourage you to send lots of applications, but you do need to understand what you’re applying for and why, before you do.

Ace The Interview

Again, there’s plenty of websites that can teach good interview technique, so do some research. I can tell you that having done some interviews myself, there are many things I’d be looking for, but basically I’d be looking to answer these questions: (1) will they be any good at the job? (2) will they fit into the team? (3) would I want to work with them myself? Three yes-es and I’d make an offer, any no-s and I’d give a polite “no thanks”.

To answer (1), as a programmer, you should absolutely expect to be given a programming test at (or possibly before) an interview. I’ve taken a lot of these – erm, about twenty-eight altogether? – and they vary a lot. Some – the worst – ask you obscure questions about bizarre corner cases of C++ in an effort to find out what you don’t know (that the examiner does); others ask you to debug code and write some of your own, to find out what you do know and how well you know it. Most are written, some involve giving you a laptop with a game project on it and asking you to fix the bugs. Some involve general programming knowledge, others are strictly C++ only. Whatever happens, all you can do is your best, and if you’re competent, you’ll do well enough to pass.

(2) and (3) are much more up to you. Remember that they’re looking for someone who is competent and confident, but not arrogant. One of my favourite interview techniques is to say something the candidate should know is wrong. If they agree with me, they don’t know their stuff – no hire. If they quietly say nothing, they might know it’s wrong but aren’t confident – I’d try to give them the chance to show that, but it would leave a question mark. If they argue with me or (as one candidate did) call me stupid, it shows they might have trouble fitting into a team environment, especially with temperamental non-technical types – and I won’t be offering them a job. If, however, they calmly explain that I am mistaken, and in fact the truth is this, which they know because they once did that – well, that’s a model answer.

With all that said, there’s a couple of things you need to bear in mind. Firstly, the company has gone through potentially hundreds of CVs and code samples and chosen you as someone worth inviting to interview. They may have spoken to you on the phone, and they’ve certainly set aside time for one or (usually) more people to take time out from developing a game to interview you. They wouldn’t do that if they didn’t want you to do well. If you’ve been invited to an interview – when you get there, relax. The interviewers will want to hire you, or they wouldn’t have asked to meet you: you just need to convince them that they’ll be right to do so. Secondly – remember that in a sense you are interviewing them, too, and not just when they ask “do you have any questions for us?” You should try to get a sense of the working environment, see if you get on with the interviewers, and think deeply about whether, if an offer is forthcoming, you’d want to work there. I had five very good years at Rare and I have generally positive memories, but my second games job was a big mistake, the biggest of my life – one that I could have avoided if I’d been smarter. At some stage you’re going to have to take the plunge, but if there’s a nagging voice in your head saying “…are you sure?”, make sure you can answer “yes” before accepting the job.

Getting In

With that lot under your belt you’ll soon be getting hundreds of job offers. Good luck! Hopefully you’ll be able to choose a company that is right for you. Making games for a living is a lot of fun and I’m very glad I did it. One day, I may even go back – who knows what the future holds? If you’re visiting this blog chances are you already make XNA games as a hobby and, yes, there are few things better than being paid to do something you’d be doing for fun anyway. Just remember that ultimately it’s a job like any other and you’ll do fine.

Oh yeah- speaking of which - money! Again, do some research (principally in Develop and Game Developer magazines which have annual salary surveys) to find the range of starter salaries in your area. Last I was hiring in games, about 18 months ago, £23-25K was a decent offer for a graduate programmer in the UK (outside London). Unfortunately, as a graduate you’re unlikely to have much leverage to ask for more, but equally, don’t quietly let yourself get shafted just to take a job, any job. You should also consider working hours, benefits, location, company size and stability, and whether there is a career path. In what proportions? Well, that’s up to you!

Next Page »

The Rubric Theme Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.