Heh. Just had to link this because I’ve been there too many times! I think I’ve probably encountered most of these:

http://dmalcolm.livejournal.com/2433.html

… with the addition of “you’re using XNA and have forgotten it’s a right-handed coordinate system”, thanks to Parm for reminding me of that one!

I think it’s about time I wrote another article, don’t you?

The funny thing is that I’ve had a draft sitting half-completed for a few weeks now, and this isn’t it! In that draft, I’ve written about some nice (some may say, essential) features for your development tools, because that’s what I’ve been working on for Pandemonium lately. However, a post on the XNA Creators Forums the other day caught my eye, and reminded me of when I used to work on the in-game camera for Kameo, and then I put the disc in my 360 for old-times’ sake, and that inspired me to start this mini-series on third person cameras, based on what I learned during the development of that game. (I’ll almost certainly finish the tools article next, then come back to this mini-series, and so on like that for a while, dipping in an out of each. Note that I don’t promise this mini-series will finish, or even progress, quickly! But over time, it should add up to have some pretty cool ideas).

Incidentally, for those of you who haven’t played it, you can almost certainly pick up Kameo in the bargain bin nowadays and it’s well worth a look. It’s not without its flaws (some of the adventure sequences are a bit dull and the story doesn’t always make complete sense), but it’s still one of the most vibrant and refreshing titles available on the Xbox 360. The combat sequences are pretty good fun, it’s got some really memorable bosses, a phenomenal soundtrack, and (especially given that it was a launch title) it looks very pretty – I defy anyone not to watch dawn break over the Enchanted Kingdom and not go “wow”. It got some criticism at the time for being a bit short for the money, but by criminy it’s eight quid on Amazon now! Probably even cheaper somewhere else. Get involved.

The Very Beginning

Anyway with that bit of blatant and shameless advertising out of the way, let’s talk cameras, starting at the very beginning. The camera is what shows the player what is going on in any 3D game. The player never sees it, and they can’t always control it, but without it you’d be stuck. It is exactly analogous to a movie camera – it’s there to show the audience (the player) what the actors (the player’s avatar and other in-game characters) are doing. Everything in the game world that you see on the screen, you see from the camera’s point of view.

The mapping from the 3D game world to your 2D television screen is called a projection; I won’t go too deeply into the maths just yet (and in fact in XNA, DirectX, OpenGL and other 3D SDKs, functions exist to perform much of the maths for you) but basically a projection transforms a 3D vector in world space to a 2D vector in screen space. Fundamentally speaking there are two types of projection (hence, two types of game camera).

An Orthographic camera performs a parallel projection. Basically, this has the effect that something 2 metres tall 100 metres away, looks the same size on screen as something 2 metres tall 5 metres away, and the viewer can only tell which is closer by seeing which is drawn in front of the other. This is not how we are used to seeing the real world, so isn’t much used in games nowadays. That said, the old isometric games show what an orthographic projection looks like (even if they’re not truly rendered in 3D), and it’s sometimes used for user interfaces, maps and such like. In fact – if you’ve ever used a 3D modelling tool like Max, Maya, Milkshape, Blender etc., you’ve almost certainly used an orthographic projection in the “side-on” or “top-down” views. I won’t go into too much detail about these here though; if you’re interested, look up Matrix.CreateOrthographic in the XNA documentation.

Much more common in games is the Perspective camera, and that’s the type of camera I’m going to concentrate on in this series. In this projection, items closer to the camera are drawn much larger than items the same size that are further away, which matches human eyesight.

The Perspective Projection

We need to know a certain amount of information in order to be able to perform the perspective transformation. First is the field of view. This value represents how wide an angle the camera can see, and by convention is usually given as the vertical angle – the vertical field of view, or sometimes abbreviated to fovY. (Some modelling tools use the horizontal field of view instead, so be careful). Next is the aspect ratio. Computer screens are rarely square, and never circular (like the human eye can see), so the aspect ratio gives the width of the projection compared to the height of the projection.

Deciding on what values to use will depend on your game, but as a starting point, consider that the human eye has something like a 150 degree vertical field of view, and an aspect ratio of something like 1.15, ie. you can see more horizontally than vertically. However – nobody ever sits so close to a computer screen that it fills their entire field of view, and the angle from your eye to the top and bottom of your monitor is usually something like 45 degrees, so that’s a good starting point for your field of view; and the aspect ratio is easy. If your monitor or television is “standard” 4:3, the aspect ratio is 4/3 = 1.3333, while if it’s “widescreen” 16:9, the aspect ratio is 16/9 = 1.7777, etc. (And therefore, if the camera view only fills part of the screen in your game, the aspect ratio will simply be the width of the view window divided by the height). We’ll talk more later about what effect changing these values has – you can get some desirable special effects, and some undesirable bugs, by altering them.

Important note: I’ve used degrees for the field of view above, because that’s what most people are familiar with. Most 3D systems, including XNA, use radians for angles instead – and if you get funny results, it’s probably because you’re using the wrong units! You can use MathHelper.ToRadians to convert from degrees to radians, or some special values have predefined constants, for example 45 degrees is the same as MathHelper.PiOver4 radians.

There’s a couple more values needed for a perspective projection but, unlike field of view and aspect ratio, they don’t have a counterpart in the real world. Your eyes can see things right next to them, even touching them (I had a wasp walk over my eye, once – it was terrifying!) and, if nothing gets in the way and the light has time to get to you, infinitely far away as well. That gives an infinitely large range – and computers are very bad at handling infinity. To make life easier for the computer, we define a minimum and maximum distance – the near clip and far clip values – outside of which, we don’t want to draw anything. Choosing these values is very much game-dependent. In a game where a human is the main character, a near clip of 0.1 metres and a far clip of 1000 metres might be OK. In a game based around spaceships travelling light years in a few seconds, that wouldn’t work! Again, there is a relationship between the near clip and far clip distances that you don’t want to get wrong, and we’ll talk about that later.

I like to think of this as being like sitting in a dark room with one window. You sit in the room and look out. You can’t see anything in the room, it’s too dark (even if there’s something between you and the window – yeah, the analogy breaks a bit there) so the distance between you and the window is the near clip distance. Outside the room, you can only see things that pass behind the window – anything in the vertical field of view (the angle between your eyes and the top and bottom of the window) and the field of view modified by the aspect ratio (the ratio of the width of the window divided by the height. In your imagination, draw a line from your eye to the edges and corner of the window. Can you see that this would form a pyramid? Now extend the lines, out into the world beyond the window. Stop at your far clip distance. Now you have a really big pyramid. Take off the smaller pyramid (inside the room) and you’re left with a shape called a frustum, in this case, a view frustum. Everything inside this frustrum can be seen through the window – so everything in the view frustum in your game will be drawn to the screen. It looks a lot like the red shape shown here (based on original diagram here):

The view frustum of a perspective projection

(Note, I’ve not shown the horizontal field of view, it should be obvious. However, the maths to calculate it isn’t… it’s not fovY * aspectRatio, as you might guess. It’s actually 2 * atan( tan( fovY/2) * aspectRatio ), for reasons of geometry. Aren’t you glad you wondered, now?)

The maths to perform the projection from items in the frustum onto the screen is a bit complicated, luckily, it’s all done with Vector3’s and Matrix’s, and XNA provides a function to generate the projection matrix (which performs the transformations of your perspective projection): Matrix.CreatePerspectiveFieldOfView, which takes as arguments the field of view, aspect ratio, near clip, and far clip, exactly as just described.

Defining the Camera View

So now we know how a view frustum performs a projection to draw things in the game world, onto the screen. But which things? Or to put it another way: how do we position the view frustum to show what we want? Answer: by defining the position and orientation of the camera.

It’s pretty obvious what the position means. If it were a “real” camera, like in a movie, it’s the point in space where the camera (or perhaps more specifically, the camera lens) is. You just need to make sure you put it somewhere sensible so the player can see what’s going on. (How? We’ll get to that!)

The orientation is fairly straightforward too. It’s the direction the camera is looking in – this gives us the camera’s forward vector, literally, the direction you’d move in to make the camera go forward. Obviously, you’ll want this to point at whatever the player wants to see, and again, we’ll talk about how later… Of course, you can hold a camera upside down (and, if you’re making a flight sim, your planes can fly upside down!) so we also need to know which way up the camera is being held: the up vector. Finally, it’s sometimes useful to know the right vector – think of your head. Your eyes are looking in front of you (the forward vector), the top of your head is usually towards the ceiling, unless you’re doing yoga or something (the up vector), so the right vector is pointing out of your right ear. These three vectors are mutually orthogonal, which is a posh way of saying there’s ninety degrees (or MathHelper.PiOver2 radians!) between each of them. (See diagram here).

 The tricky bit comes in deciding how to represent the orientation. (It’s important to understand the following are all different representations, and for any given orientation, it’s possible to represent it in any of the following ways). We can either simply pick a point in space, and say “the camera is looking at this point” – this is the target or look-at point.

Or, we can assume that the “natural” orientation of a camera is for the forward vector to be looking down the Z axis (known as -Z, -Vector3.UnitZ, or Vector3.Forward – remember XNA uses a right-handed coordinate system!) and the up vector is directly upward (known as +Y, Vector3.UnitY, or Vector3.Up). Of course this implies that the right vector is along the X axis (+X, Vector3.UnitX, or Vector3.Right). From this “natural” orientation, we can rotate it – either using angle values for yaw (rotating horizontally around the Y axis), pitch (rotating vertically around the X axis), and roll (rotating side to side around the Z axis), known as Euler angles (pronounced “oiler”), or by using a single Quaternion value. Each of these methods – Euler and Quaternion – have advantages and disadvantages (Euler is easy to understand, but can encounter issues in certain orientations that quaternions, to all intents and purposes, don’t) and which to use depends on your game. If you’re making a flight sim or other game in which the camera can roll a lot, use quaternions. Otherwise, either can be made to work, and Euler angles are certainly easier to conceptualise.

We can combine the position and orientation into a single view matrix for convenience, and no prizes for guessing that XNA makes it easy, via Matrix.CreateLookAt, Matrix.CreateFromYawPitchRoll, and Matrix.CreateFromQuaternion.

Our projection matrix and view matrix now tell us everything we need to know. When we went to draw something, we pass these matrices to the shader (sometimes along with the World matrix, which represents how the game world or object being drawn is transformed; in the case of the game world, it is usually Matrix.Identity) and it performs the complex transformation maths for us. Note that the projection matrix might not change much during gameplay, but that the view matrix might change in every frame that the camera moves.

So all that’s left is to work out where to put the camera, ie. what values to pass in, in order to get the appropriate view matrix. Should be easy! Right? Well… the number of games reviews where I’ve read “game X has a really bad camera”, and the fact I spent probably 40% of my time for three years coding Kameo’s camera (happily no reviews I saw complained about it), suggest it’s not that simple. (Don’t get too scared. Kameo had 11 player characters and over 20 camera modes. Your game will probably be a lot simpler!) So how can we decide where to position the camera?

Deciding the Camera Position

At a very high level, there are basically three ways to decide how to position and orientate the camera, and they arise from the way that in most cases, you either have something you want to look at, or somewhere you want to look from, or both. (If you have neither, and don’t know where you are or what you want to look at, well, good luck with that!).

First consider the case where you know where the camera is, but you don’t know what you want to look at, just a direction. The most obvious example of this is the first-person shooter. The camera is inside the player character’s head, and the player decides in which direction to look and move. Or, imagine a driving game or flight sim where the camera is placed inside the car or plane, looking out of the front window. This kind of camera is very simple: the position is known, and you can use Matrix.CreateFromYawPitchRoll (for example in a FPS) or Matrix.CreateFromQuaternion (for example in a flight-sim interior camera) to work out the rest of the view matrix.

Example: Halo.

Secondly, imagine you know where the camera is, and also know what to look at. An example might be a track-side camera in a racing game: it’s fixed in position, but as a car races past it turns to follow it. Or, imagine a security camera that follows the player (a bit like in Ico). It’s even easier if the camera is fixed in both position and orientation, like in the early Resident Evil games. Again, this kind of camera is very simple: the position is known, and you can use Matrix.CreateLookAt to work out the rest. (Of course, track-side cameras should have some inertia to follow the cars more naturally, and the camera in Ico has quite a few more complex features as well, but the basic functionality is very straightforward).

Example: Ico.

The final category (and by far the most interesting) is where you know what you want to look at, but not the position you want to look at it from. Many game genres fall into this category.

Sometimes, it’s fairly easy to work out where the camera should be. In an arcade racing game, the obvious target is the player’s car; and the player will want to see where they are driving. So, you need to position the camera somewhere behind and above the car, looking somewhere in front of it. Of course, you’ll want the camera to follow the car as it turns, and if the player reverses into a wall you need to make sure the camera doesn’t go on the other side of the wall, but still, this is pretty straightforward. (An example from a non-racing game might include the early Tomb Raiders, and I think the Gears of War over-the-shoulder camera probably belongs here as well).

Example: Project Gotham Racing 4.

Another example of this category is to be found in the RTS genre. Here, the camera looks at the units or terrain the player is interested in, but the player can control where it looks at them from. Assuming you position the camera high enough not to bump into buildings and trees, again, this is quite straightforward – store the look-at point and move it around the battlefield, then let the player control the angle they want to view it from, and position the camera a fixed distance in that direction.

Example: Age of Empires 3.

So finally we come to the most interesting cameras of all, and the ones I will be concentrating on in this mini-series: the third-person camera. In games of this kind, the camera and the player are controlled independently. The camera must move around the game world, giving a good view of the player character, while moving around and among the terrain the player moves around and among. There are some very difficult (and hence interesting) problems to overcome – how do you stop the camera going through walls? How do you ensure there is always a good view of the player? To what extent should you let the player control the camera position – and even more importantly, to what extent should you require them to? Games with cameras like this include platformers, like Mario or Banjo Kazooie; parkour-style action games like Prince of Persia; and adventures or RPGs like Fable, or, of course, Kameo.

Check out this screenshot of Kameo in action. She’s standing among the tall grass now, cheekily glancing back at the camera – but she could move in any direction and the camera would have to follow her. She could go for a swim, run around the tree, move right up to the edge of the cliff, go into the house; climb to the very top of the castle and jump off it. No matter what she does, the camera has to follow her, and show her doing it in a way the player can understand and react to. That’s not easy! But it is absolutely fascinating. I hope this series will demonstrate that, and you’ll find it entertaining and useful.

Anyway I’m moving house next week – so not sure when the next update will be – so, until next time, farewell from me…!

Just a quickie… interesting follow up to my recent article, which was followed by some comments about salaries in the games industry.

Develop have got their annual survey salary out, and list the following:

Average Yearly Salaries:
Lead Programmer – £41,250
Programmer – £25,810
Junior Programmer – £18,928

A few thoughts spring to mind:

1. Wow. Now I remember why I left the games industry.

2. £26K for a programmer is less than £29K for an artist or £27 for a junior audio engineer! That disagrees very strongly with my anecdotal experience. I don’t know of any artists or audio guys paid more than their programmer colleagues, and didn’t think that was the normal state of affairs at any games company anywhere. (Design is a tricky one. Junior designers get shafted, but lead designers / directors can make a fortune). Surprising.

3. £19K average for a junior programmer (implying many get less)? That’s appalling! I was paid £20K as a junior seven years ago! Also, the last company I was involved with hiring at, offered around £23K as standard for new junior hires.

I’m not going to say the Develop survey is wrong, because they’ve had a wide range of respondents whereas I’ve only got my own personal limited anecdotal experience to draw from. What I will say is, if those Develop figures are right, there’s a lot of people out there getting completely stitched up. They’d be earning a lot more in another industry. Is that worth giving up, in order to fulfil their childhood dream of making games for a living? That’s a question only they can answer. For a while, my answer was “yes”. Now, though, I stand my updated decision of making software for a living, and making games for fun.

Indulge me as I walk down memory lane…

It was summer 2002 and I’d just landed my first professional job. (I’m still not sure how. I wasn’t very good back then. I like to think they saw that I’d become good, and I like to think I am good now. It might just have been luck!). I’d spent some time in the RnD department, to learn the ropes, become familiar with devkits (console hardware with added stuff to enable game development), and understand a bit about how the shared game engine worked; and then the time came to move onto a game team, of which there were five or six to choose from. The choice wasn’t freely given. Each team had different needs, so the question was asked: what did I want to work on? What did I want to specialise in?

I didn’t really know how to answer that. I was vaguely aware of the division between systems programming and gameplay programming, and knew that I preferred the former (I actually think the latter is better solved by a good data-driven system and a good designer, though I have worked with gameplay programmers who produced excellent results); but which system? And how would my choice affect my working day?

Games teams on “AAA” titles nowadays can easily have several tens of programmers. Clearly, if they all worked on whatever they fancied from day to day it would be a disaster. Each programmer therefore gets designated an area of responsibility. Generally speaking, the programming lead and the most senior programmers determine the overall architecture of the game early in development (or it may be mandated by the engine, particularly if it is middleware), and the programming team splits into several sub-teams, each led by a senior programmer who reports to the lead programmer. Examples of roles in the team (in no particular order) include:

Graphics: the celebrities of programming because they get to write code that produces awesome looking screenshots (or at least… code that lets the artists do so). Every time the publishers come for a visit, they’ll get led into the graphics programmers’ office and shown all the latest particle-laden explosions on flashy HDTVs. Graphics programmers spend a lot of time writing shaders, optimising the renderer, and talking with artists.
Networking: modern games are immensely complex and with multiplayer online being practically compulsory nowadays, every game will have network specialists. They tend to spend all their time trying to teach other programmers how to write code that doesn’t break the online mode, for example by sending 600KB packets every frame or updating something on the local client but not the game server. They usually look a bit stressed.
Physics: even with middleware like Havok (or in XNA, JiglibX and the like) available, physics remains one of the most complicated things in a game because it affects just about everything else. One game I worked on had five physics programmers.
AI: most games have enemies or some kind of non-player entity. While scripting and other designer-facing tools mean AI is not as hard-coded as it once was, someone’s got to write the code that interprets the scripts – that’s the job of the AI coder.
Audio: given that the only two ways your game can influence the player is via the screen and the speakers, audio is half of every game. Unfortunately it’s the second half (because it doesn’t look good in screenshots) and all too often audio is neglected. Done well, it can turn a good game into a mind-shatteringly atmospheric epic. Audio coders spend a lot of time talking to the musicians and SFX engineers, and they’re usually slightly bitter that the graphics programmers get all the plaudits (and flashy HDTVs).
Tools: there might be twenty programmers on a team, but there might be ten designers, fifty artists, and five audio engineers (as well as testers, producers, marketing, translators…). You can’t just give them a copy of Photoshop and Maya and tell them to get on with it. Every game needs specific tools that enables these people to get their assets into the game and tweaked until fun and in my experience, the better the tools the better the finished game.  Historically tools were DOS-based and unreliable; increasingly, they’re now written in C#/.NET and actually work more often than not. Tools programmers are the unsung heroes of the programming team. Their work is almost never seen by the public, but without them, the game itself won’t get seen by the public either.
Systems: asset loading. The game camera. Multithreading. Save games. Text, and menus. Achievements. DVD file layout. TCR compliance (rules that the console makers require you to obey before your game can be released). Support for steering wheels, dance mats, webcams and chatpads. The build process (putting together versions of the game to give to artists, management and testing). A veritable pot pourri of tasks that no game can go without. Some of these tasks will be given to the most senior programmers because they’re critical to the game’s success. Others will be given to the most junior programmers because they’re relatively self-contained and can be developed in isolation. Most don’t get noticed, until you try to make a game without them!

There’s more, but that’s a good initial summary. (Just think – when you write an XNA game, you’re responsible for all of the above! Lucky XNA itself is brilliant at doing it loads of it for you). My first ever task on a game team was to write the game camera, and looking back I’m pretty proud of how it turned out. Very soon afterward I also took on the audio programming. Eventually, on that first game (I worked on it for three years – some were working on it for twice that), I was responsible for, or otherwise involved with, text and menus, localisation (making the game support other languages), asset loading, the build process, save games and TCR compliance. Later I’d work on asset optimisation and arranging files on the DVD, and probably some other stuff I’ve forgotten. So it became clear: my specialisation was that I was a Jack-of-all-trades. Hurrah!

With all these people working on different things, it’s critical that they don’t interfere with one another by writing over one another’s changes. This is achieved by use of a Source Control System. Basically, this keeps track of every code file and asset in the game, keeps records of how they change over time, and tries to ensure that if two people make changes at once, those changes are seamlessly merged together. Different teams use different products and approach this in different ways. Some examples using the codenames of games I worked on:

Game Two: used CVS for source control. This is an horrific abomination that should be scourged from the earth. A coder would make all the changes he wanted, send out an email to the team saying “please don’t commit”, commit all the files he’d changed, and send another email saying “OK to commit”. Inevitably he’d have missed something so the next coder to update would have to come and ask him to fix the problem before they could continue. Worst of all, the artists couldn’t get anything into the game without giving it to a programmer to commit it for them. This was frustrating for everyone involved and meant that changes to a level, for example, could take two or three days to get into the game. Putting together a build was an eight hour manual process; all too often it wouldn’t start until 5pm the night before a deadline. I’m not completely sure how we managed to get the game finished, and I’m stunned that it turned out as good as it did despite everything. This is How Not To Do It.

Pocket/Pikelet: an improvement in every area, these teams used Source Depot (basically the same as Perforce) for source control. No more emails to control who could commit – a tool that lived in every developer’s System Tray would lock out commits while someone was going through the process of updating, building the game, running the unit tests and committing. A separate build machine would then automatically update to that version, run the tests again and if they were passed make the build available to all the non-programmers – who, incidentally, had the tools available to do all their work without needing to go through a programmer. It was brilliant, broken builds and artist downtime were unheard of – the game was bulletproof throughout development. There was only one small problem. Updating, building, running unit tests and committing took half an hour or more. In a normal eight hour day, only sixteen programmers could do it – at most. With twenty programmers on the team, there’s an obvious problem and it got very frantic near deadlines – and leaving at least a day between committing meant that you’d commit too much at once, introducing bugs (which would fail the unit tests and delay your commit even longer). This was a problem, but in general this was the best system I’ve had the pleasure of working in.

Polished Turd (not the real codename, just what I call it): again this used Perforce for source control, except this time without a formal commit queue like Pocket/Pikelet. The commit queue had originally been introduced to stop CVS from breaking everything, but Perforce is so much better than CVS that actually things rarely break even if you’re free and easy with committing. It meant that you could fix a bug, commit your changes, fix another bug, commit your changes, and iterate very rapidly through your work. You’d update over lunch and overnight, or when you noticed someone had committed something you need. If we’d only had the same team build system, unit tests, build suite, and artists tools from Pocket/Pikelet this would have been the ideal way of working. (Unfortunately all those things didn’t exist so the game was unstable, the artists couldn’t do their job, and every deadline was a mad scramble to put together get something vaguely playable that lasted more than five minutes between crashes).

Now, most people reading this will be hobbyists working in XNA on their own. At the moment, that applies to me too. However, I’m uncomfortably aware that the big blue blobs and soulless grey polygons that currently represent the actors and props in my game won’t inspire other people to play my game – and I’m closer to being autistic than artistic. So sooner or later, I’m going to need other team members to make models, textures, animations and sound for me. (Any volunteers?) And, while I’m currently planning to save all the coding duties for myself (with the XNA Framework itself and the multitude of excellent community libraries out there, this is a realisitic proposition in a non-trivial game for really the first time this side of about 1998) many of you will no doubt be thinking of forming small coding teams, to split the work among you. (I don’t blame you. Just look at the list above. Even a simple game has a lot to do…) So – how well does XNA support medium-to-large game teams? And what would such a team require?

Source control is absolutely essential the moment your team grows larger than one. And, with a team of one or two, there’s only one choice – Perforce, which is free for up to two users, and so good I use it even though no-one’s forcing me to. Unfortunately, as soon as your team size hits three people, it’s something like $700 a license. Ouch. Worth absolutely every penny and more if you’re a pro developer with your outgoings covered by a publisher, but out of reach of everyone else. I am told that Subversion is good for a free product, but I look at the lack of atomic changelists and cringe. Probably the next best if you can’t afford Perforce though. There’s nothing here that’s different for XNA than if you were using C++ or any other language.

It will be essential to have a configuration of your game that allows your artists to put their assets into the game without waiting for you to do it for them. The XNA content pipeline is a wonderful, wonderful thing of great beauty, but by default it is tied into Visual Studio – the content pipeline build occurs just before you run your game, and that’s it. An artist doesn’t want Visual Studio, and he doesn’t want to restart the game every time he changes a texture – he should be able to save off the texture file and see it change in-game. So you need to provide them with a version of the game that runs on its own (perhaps in a WinForm) and a tool that hooks into the content pipeline and will let them build assets while the game is running, notify the game, and the game then reloads that asset. This is pretty easy for assets that already exist in a content project, but the tool will need to support adding to the content project (hidden from the artist) and even creating new ones. All this needs to hook into your source control system without your artist needing to learn to type “p4 -d -a -q -z” or whatever.

The content pipeline causes a few other problems too. (Though don’t think for a second that I’m knocking it. What it does is amazing). Big games tend to generate thousands, even hundreds of thousands of assets – and in the content pipeline, each of those is a different file. People often don’t realise that opening a file, even from a hard drive, can sometimes take as long as a quarter of a second. That doesn’t sound like much until you multiply it by a hundred thousand. (You can very easily see this for yourself by using WinZip to zip up one large file of say 10MB, then compare it to zipping up a thousand files of 10KB each – I guarantee the latter will be much slower). To give an XNA-related example: Kameo, Pocket and Pikelet all used XACT, the same audio tool as provided with XNA (and which I’ve used very extensively). Pocket, in particular, had well over 200 soundbanks which all needed loading at startup. Even though they were only between just 1 and 4KB each, this took a long time to load from the DVD - so long that the 20 second load time mandated by Xbox 360 TCRs seemed an impossibility. The fix was pretty simple: we altered the build process to package all those soundbanks into a single file, loaded the file into memory, and loaded the soundbanks from that memory. Instantly the problem was solved. The same solution won’t work in XNA: there’s no method to load soundbanks from memory. It expects each soundbank as a separate file, and that’s that. A big game in XNA would really struggle under such restrictions. (Loading on a background thread won’t help, 20 seconds of disk access is 20 seconds of disk access and that’s that, though admittedly an XNA game would be on a hard drive not a DVD which helps a lot). You’d have to plan ahead very carefully – in terms of the game design, not just code; which is hard, because designers don’t understand arbitrary restrictions from code, and nor should they have to - to ensure that there was never a need to load so many individual files all at once.

With these caveats in mind I’m convinced a medium-to-large game can realistically be made in XNA. The “poor performance” of C# compared to C++ is for the most part a myth, though to get the absolute 100% optimum out of the hardware might require C++ – but you could get at least 95% of the way there with C#, and 98% of all games don’t need 100% performance. It’s unfortunate that the biggest win for hobbyists and small-team endeavours – the brilliant content pipeline – appears to be the biggest limiting factor for large teams. The issue of when and how assets are built, and how they can be rapidly iterated without restarting the game, is definitely solvable, though with a fair bit of coding effort; the fact that so much of the pipeline relies on a one asset, one file relationship is trickier. A hundred-thousand-asset game would need a hundred thousand files, and loading times would be horrific.

So really large games might not be easy in XNA. Happily (?) few large games nowadays are single platform, and since XNA is not available on PS3 and Gamecube, I don’t think many large teams will be evaluating XNA anyway. Medium teams, independents, and small hobbyist teams can definitely have a field day with it (though profitability of XBLCG at least has yet to be proven). Just remember that the limiting factor for such games is content, and put a lot of effort into making your artists’ lives easy. If they can make a model, put it in your game, play with it, tweak it, play with it again, tweak it, play with it, and check it into source control, they’ll beg to be allowed to make more. If they have to send you the model and wait two days for you to send them a build with it in, they’ll give up within a week. Frankly – even if you’re working on your own, and doing all the code and art yourself, a little bit of work to make good tools will make your own life easier; and isn’t that what we all want?

Wow, it seems like an age since I posted. I guess that’s because it is! Fable 2 took up far more of my time than I expected, I’ve barely touched Nuts and Bolts, and I still have to complete Tomb Raider and Prince of Persia before March, when Empire: Total War is released. This doesn’t leave much time for any meaningful development on Pandemonium, though I have recently chucked together a few little prototypes of stuff I might one day make into fully-developed games.

Anyway I don’t know when it will be that I’ve done enough on Pandemonium to make an update worthwhile. But I don’t want to leave the blog stagnant, it’s still getting a fair number of hits every day and people must have expectations… so I’ll put the question out there: is there anything (preferably XNA-related) you want to read about? Check out the stuff I’ve already written to get an idea of the kinds of things I know. Clue: I’m not the man to ask about whizz-bang graphical effects, but if there’s anything else you’d like to see, just ask.

I’ve had a few people contact me to ask, “do you have any advice for how to get into the games industry?” Besides being ironic given that I’m no longer a part of the industry myself, this is a phrase that I really despise, because – well, I’ll get to that in a minute.

I’m going to start by making a couple of assumptions. I’m going to assume that the the person asking the question is really asking, “how do I get a job as a games programmer?” The reason for this assumption is that it’s the only thing I can speak with any experience about. Modern games (at least, console and AAA PC games, which is what people usually mean by “the games industry”; casual or flash games, not so much) involve teams of tens or even hundreds, involving programmers, artists, designers, scriptwriters, musicians, testers, producers, marketing, and a whole bunch of other roles. I’ve worked with all of the above, but I’d be guessing if I tried to tell you how they ended up there.

The second assumption I’ll make is that working as a games programmer is something you really want. The pay is low, compared to non-games programming. The hours are longer (though this is starting change) and the respect from management lower. Most games – and hence, the games worked on by most people – aren’t the amazing high-quality AAA 96%-rated blockbusters like Half-Life 2 that we’d all love to be a part of; in fact, most of the work you do will be soul-destroying just like in any other job. If you’re lucky, you’ll get to work at a company that lets you leave at 5pm, in which case you will get to see the sun; but even if you don’t have to crunch you could easily go months without seeing those strange creatures they call “females”. In fact, for all these reasons and more, I no longer work in games myself. But be clear about this – I’m glad I did it (for about six years). Knowing you’re doing a job you love, and finally seeing your baby on the shelf at Game and hearing someone say “yeah I played that, it’s quite good actually”, is a buzz of fulfilment like no other. Decide for yourself whether the balance works for you. It did for me, for a long while.

Anyway, where was I? Oh yes – “getting into the games industry”. I hate that phrase. It suggests there’s something magical about it, as if you need to know a special password or shake hands with the interviewer in a particular way. The truth is that games have matured into an industry like any other and you’ll get a job programming games the same way you’d get any other job.

Let’s pretend I’m going for a job as an architect. Should I:

(a) spend all day looking at my favourite building, but not really bother about any others because they suck, take a course in Social Studies because you only have to go to three lectures a week, then wonder why architecture companies don’t even respond to my applications even when I put “…it’s what I’ve always wanted to do, and I’m a fast learner!” on my CV; or

(b) learn how to be an architect, get qualified, put together a portfolio, apply for jobs, and make sure I nail the interview?

Third assumption: you answered (b).

Get Competent

If you’re going to be a games programmer, you need to be a good programmer. Sounds obvious really. I’m tempted to say, “it’s that simple”, but of course it’s not. The thing is, if you’ve yet got a job as a professional programmer, you’re probably not aware (even if you think you are!) of what being a good programmer means. That’s OK. The good news is: no-one expects you to.

The simple fact of the matter is that learning to program well comes only with experience. And, if you don’t have QA teams tearing your work to shreds, or customer complaining your software crashes, you may never realise when you’re making mistakes. (Don’t let that stop you coding – a lot – in your spare time. Any experience is better than none). When I first began applying for jobs, I rated my coding skills as 8/10. Looking back, I was more like 2/10. I’m not sure I’m more than 7/10 now, more than six years later. That’s fine – companies looking to employ new hires only really expect 2/10. So why, you might wonder, bother putting any effort into learning to code at all? You’ll learn it all on the job, right? Wrong. Getting to 2/10 is hard (remember the default state is zero!). If you don’t understand pointers, if you can’t fix compiler errors and warnings, if you can’t identify and eliminates bugs in your own code and other peoples’, if you can’t make estimates about the performance of a given algorithm: either learn that stuff or give up now. No company will expect a new hire to be the finished product. But no company will hire someone who hasn’t bothered to learn what they need to know to do the job, either.

What language should you be programming in? I’d love to say C#, but I can’t. Not yet anyway, perhaps in a few years. Currently, most companies only use C# for tools (and some not even then). The overwhelming majority of games companies use C++, and most of the rest use C. (Or, mobile developers probably use Java?) A few devs are starting to use C#, but most don’t and sadly won’t. There are a few very good reasons for this, despite the fact that C++ is a dinosaur and C++0x is going to be at least ten years too late. The first very good reason is that companies currently have millions of lines of C++ and tens of C++ programmers, and they’re not about to throw either of those away just because someone like me claims C# is more productive. The second very good reason is that there is no C# compiler for PS3 or Wii – and that’s a winning argument for companies intending to make games for those platforms. So if you want to make games,  I recommend C# and XNA. But if you want to get a job as a games programmer, I’d be doing you a disservice to say anything other than to learn C++.

You should also have a passing familiarity with two or three other languages (even if only to know that “the major differences between C++ and <language> are…”), an understanding of computer science principles, and above all, the ability to work as part of a team, including with team members who are non-programmers.

Get Qualified

Already, I’m sure some of you out there are screaming at me about this. “You don’t need a degree to be a good programmer!” they’re shouting. That’s not in dispute. “Just because you have a piece of paper, doesn’t mean you are a good programmer!” they’re adding. Completely agree with that too. However, nowadays, if you don’t have a degree in some kind of computing-related field, you probably won’t get past the recruiter’s first cut. Yes, there are exceptions, but they are exceptional! You could take a gamble, but the odds are desperately slim and you’d be much better off getting a degree. (The only time I’d say that doesn’t apply is if you’ve already been working as a professional programmer for years, in which case, yes, experience counts at least as much as a degree. But for a school-leaver or graduate, a degree is, to all intents and purposes, essential).

But – a degree in Computer Science or similar traditional field, or a new-fangled degree in Computer Games Development, perhaps even from a games-specialist University? Well… I cannot deny my prejudice here. My degree was in Computer Science, and I learned things on that course that I don’t think I’d have learned on a games-specific course that since proved invaluable. There is also the consideration that, should things not turn out as you hope, a CompSci degree will get you a job in a non-games company. I’m not sure that a Games degree would do the same. Or, look at it another way: any company will hire someone with a CompSci degree, some companies won’t hire people with a Games degree. That may change over the years, in fact I expect it to, but again – play the odds!

I should mention that at least two excellent people I worked with had Games degrees, so I wouldn’t dismiss someone with a Games degree out of hand. But, I got the impression that they were brilliant despite their Games degree, not because of it. I’ve also seen a lot of applicants with Games degrees who were simply awful and had clearly been let down by their course. It’s up to you, in the end – a Games degree or a CompSci degree is entirely your choice, and either can be made to work - but I’d still suggest that, for now at least, the latter is the better option.

Put Together a Portfolio

So you’re a good programmer and you have a piece of paper to prove it. Excellent! Regrettably, companies will still need convincing. There’s only one way to do that – happily, it coincides with the first piece of advice, “get competent” – and that’s to write some code to show to a prospective employer. (Writing the code will help you become competent, and as you become more competent you will have better code to show to employers).

What kind of code? Well, you’ll need to show a few things really. First, a simple, yet complete, polished, and bug-free game. When I say “simple”, I mean something like Tetris – but when I say complete, I mean with menus, high-score tables, multiplayer, pause, an options screen… everything you get in a pro game. Bug-free should speak for itself… Secondly, you’ll need a more advanced game that may be incomplete but demonstrates more demanding problem-solving. It should involve two or preferably more of the following: 3D graphics, 3D audio, physics, networking, AI, data driven development, and save/load. Finally, you’ll need one game component (from the list above, or “something else”) that is highly polished and well developed and could be plugged into someone else’s game if they chose to use it, so should also be clearly documented with a sensible API.

You should provide this code with your application, either on CD if it’s by snail mail, or on a website  if your application is in electronic form. Provide your complete source code, and make crystal clear exactly which parts were written by you and which by someone else. Also provide the built versions of the software (on the CD, or as a separate download from the code) but remember that actually it’s the code that’s important – recruiters probably won’t run the game and certainly won’t play it for more than five minutes to check it doesn’t crash, but they will be interested in whether your code is well-written. Only ever show your latest, best code – quality beats quantity every time.

Apply For Jobs

First impressions count for a lot: make sure yours is positive. Ensure your CV and covering letter are polished, clear and professional. There’s a lot of websites with good advice which I won’t attempt to repeat here.

Where should you apply? Well, there’s a lot of options. Do some research (subscribe to Develop and Game Developer, and join Gamasutra, GameDev.net, GameDevMap.com and probably a bunch of other places), and try not to close too many doors. When I was a graduate, I applied to forty games companies. I got three interviews, one of which was cancelled the day before the interview itself, and one job offer, which became two a few months later, after I’d already started working for the first. (In contrast, when I started looking to leave Rare, just six applications garnered me six interviews and five job offers – that’s the power of experience!).

Remember that large companies (EA, Microsoft, Sony, Nintendo, etc) are more likely to recruit new hires than small companies, who may not be able to afford to employ someone who will, inevitably, not be very productive for at least the first six to twelve months. Bear in mind the corporate culture of such companies – though remember you might not be able to afford to be choosy. Medium-sized companies sometimes offer a less corporate culture while still hiring graduates; some are more noted for recruiting out of University than others, for example Rare, Blitz and Codemasters in the UK. (Again consider the implications. Are they hiring graduates because they are able to recognise untapped potential – which is a very good thing – or to get a cheap source of labour – which is a bad thing? You’ll need to work that out for yourself at the interview). Whatever happens, don’t apply blind. I’d encourage you to send lots of applications, but you do need to understand what you’re applying for and why, before you do.

Ace The Interview

Again, there’s plenty of websites that can teach good interview technique, so do some research. I can tell you that having done some interviews myself, there are many things I’d be looking for, but basically I’d be looking to answer these questions: (1) will they be any good at the job? (2) will they fit into the team? (3) would I want to work with them myself? Three yes-es and I’d make an offer, any no-s and I’d give a polite “no thanks”.

To answer (1), as a programmer, you should absolutely expect to be given a programming test at (or possibly before) an interview. I’ve taken a lot of these – erm, about twenty-eight altogether? – and they vary a lot. Some – the worst – ask you obscure questions about bizarre corner cases of C++ in an effort to find out what you don’t know (that the examiner does); others ask you to debug code and write some of your own, to find out what you do know and how well you know it. Most are written, some involve giving you a laptop with a game project on it and asking you to fix the bugs. Some involve general programming knowledge, others are strictly C++ only. Whatever happens, all you can do is your best, and if you’re competent, you’ll do well enough to pass.

(2) and (3) are much more up to you. Remember that they’re looking for someone who is competent and confident, but not arrogant. One of my favourite interview techniques is to say something the candidate should know is wrong. If they agree with me, they don’t know their stuff – no hire. If they quietly say nothing, they might know it’s wrong but aren’t confident – I’d try to give them the chance to show that, but it would leave a question mark. If they argue with me or (as one candidate did) call me stupid, it shows they might have trouble fitting into a team environment, especially with temperamental non-technical types – and I won’t be offering them a job. If, however, they calmly explain that I am mistaken, and in fact the truth is this, which they know because they once did that – well, that’s a model answer.

With all that said, there’s a couple of things you need to bear in mind. Firstly, the company has gone through potentially hundreds of CVs and code samples and chosen you as someone worth inviting to interview. They may have spoken to you on the phone, and they’ve certainly set aside time for one or (usually) more people to take time out from developing a game to interview you. They wouldn’t do that if they didn’t want you to do well. If you’ve been invited to an interview – when you get there, relax. The interviewers will want to hire you, or they wouldn’t have asked to meet you: you just need to convince them that they’ll be right to do so. Secondly – remember that in a sense you are interviewing them, too, and not just when they ask “do you have any questions for us?” You should try to get a sense of the working environment, see if you get on with the interviewers, and think deeply about whether, if an offer is forthcoming, you’d want to work there. I had five very good years at Rare and I have generally positive memories, but my second games job was a big mistake, the biggest of my life – one that I could have avoided if I’d been smarter. At some stage you’re going to have to take the plunge, but if there’s a nagging voice in your head saying “…are you sure?”, make sure you can answer “yes” before accepting the job.

Getting In

With that lot under your belt you’ll soon be getting hundreds of job offers. Good luck! Hopefully you’ll be able to choose a company that is right for you. Making games for a living is a lot of fun and I’m very glad I did it. One day, I may even go back – who knows what the future holds? If you’re visiting this blog chances are you already make XNA games as a hobby and, yes, there are few things better than being paid to do something you’d be doing for fun anyway. Just remember that ultimately it’s a job like any other and you’ll do fine.

Oh yeah- speaking of which - money! Again, do some research (principally in Develop and Game Developer magazines which have annual salary surveys) to find the range of starter salaries in your area. Last I was hiring in games, about 18 months ago, £23-25K was a decent offer for a graduate programmer in the UK (outside London). Unfortunately, as a graduate you’re unlikely to have much leverage to ask for more, but equally, don’t quietly let yourself get shafted just to take a job, any job. You should also consider working hours, benefits, location, company size and stability, and whether there is a career path. In what proportions? Well, that’s up to you!

Wow, what a time. After having barely played a game all year, I spent weeks playing Fable 2 (very good, though could have been even better), and I’m about to get cracking on Banjo Kazooie: Nuts and Bolts (the last game I worked on before leaving Rare) and Banjo Kazooie XBLA. I’ll be getting Tomb Raider Underworld and the new Prince of Persia for Christmas, and sometime in January, I’ll be surrendering every waking moment to Empire: Total War (seriously – I plan to take a week off work when it comes out). I’ve not so much as had a spare second to check out the games on Xbox Live Community Games, which probably makes me a terrible hypocrite who will be punished in the afterlife. Frankly it’s beyond a joke. I don’t even have enough time to play all of these games, all coming out within two months of each other – never mind time for my other hobbies!

Fortunately the other day I managed to squeeze in a few hours with XNA and spent the time fixing up a few odds and ends I’d been meaning to look at for a while. Specifically, I rearranged the structure of my engine and game code, for clarity and useability. It’s well worth emphasising a couple of points here: first, while I’ve built up this structure after a fair bit of experience with professional game engines and a lot of careful thought, there are undoubtedly other ways of doing it. I encourage you to hunt around, look at what others have done, and deeply consider what will work for you. Second, this is very much a work in progress. I will undoubtedly be tweaking things yet further as time goes by, and I’ve had to make a few decisions “blind” – I’m pretty sure they’re the right choices but it’s entirely possible I’ll reverse them later as experience dictates.

So. At the highest level, the XNA code I write is split into two general groups: engine code (shared code, usable by more than one game, written in such a way as to be reasonably general purpose without being so abstract as to be useless); and game code and assets (specific to one particular game, though sometimes code can start life here before being moved into the engine later). In an ideal world, all your code would exist in the engine, and there would be no game-specific code at all. Games would be differentiated only by their assets – the single executable built by the engine would search for an XML file or similar, that would define what other assets to load, and by a combination of scripting and data-driven design, the game would run with no unique code at all. Realistically, that’s not an achievable goal, and a game will always have game-specific code. The trick is to find a balance.

(First a plea. Don’t “write an engine”. The XNA Framework is so close to being an engine that such a task is pointless, and you’ll also find that no-one will use it – because you won’t have solved the problems you’d encounter if you wrote a game. Hobbyist DirectX programmers all set out to write an engine, and barely any of them ever complete a game. In fact, it wouldn’t be bad advice to write a game first, then only when it’s finished, go back and look at how it can be separated into engine and game code, ready for your next game).

Anyway, let’s have a look at the folder structure of the engine and the games in turn. You’ll want to store all your stuff in a single folder – mine are in Documents\Code\XNA – Visual Studio will encourage you to put everything under the Documents\Visual Studio 2008 folder, but that’s a really bad idea because when Visual Studio 2010 and XNA 4.0 come out you’ll just have to move everything, which apart from being generally inconvenient can be a royal pain if you’re using source control!

Engine Folder Layout

My engine is codenamed Kensei, and is therefore stored in the Kensei folder. Here, you will find the Kensei.sln Visual Studio solution file, as well as my Kensei.FxCop project (with CustomDictionary.xml) and a couple of handy batch files which I’ll come to in a minute. Everything else is stored in subfolders, corresponding to the code projects that are referenced in Kensei.sln (which in turn correspond to the assemblies that can be used by my games).

Kensei – the “core” project containing the majority of the code I write. In here exists my audio engine, camera management code, handy development utility code, command pattern implementation, and everything else I’ll come up with as I add features. Each logical unit is stored in a subfolder of this folder, so for example my audio code exists in Documents\Code\XNA\Kensei\Kensei\Audio – but it’s all built together in a single C# project called Kensei. (I’ve worked on games where each logical module had its own project. It just meant we ended up with fifty-odd projects and fifty-odd assemblies where one would do. It might make it easier to keep things decoupled, but adds headaches via complexity, especially as C# makes circular references awkward or worse. Don’t bother).

Kensei.EmbeddedResources – there are a few things in the Kensei engine that require specific assets. For example, Kensei.Dev uses a font for simple text and a shader for simple shape rendering. At first, these assets had to be copied into the content project of each game prototype I created but that quickly became a pain. The answer (as Shawn Hargreaves describes expertly) is embedded resources.

Kensei.Pipeline and Kensei.Pipeline.Runtime – XNA makes working with assets an absolute dream and most non-trivial games will extend the content pipeline to support custom asset types. Kensei.Pipeline extends the content pipeline itself, while Kensei.Pipeline.Runtime acts as a bridge between the content pipeline and the game – again, Shawn explains in full.

DebugContentBuild – it’s an inevitable fact that all code has bugs, so I followed Stephen’s advice and created a project that vastly simplifies the process of debugging my content pipeline extensions. Basically, running this project attaches to MSBuild.exe and runs the content pipeline on the assets in its content project – allowing me to step into my Kensei.Pipeline code and see where it’s going wrong.

(Other Code Projects) – one of the things I love about the XNA community is the easy willingness with which people share their work. Make use of it! I don’t want to spend my hobby time reinventing the wheel or rewriting something someone else has already done perfectly well (unless, of course, it’s the main differentiating point of my game, or something I have a specific interest in doing or learning. Always in-source your USPs, but don’t even attempt to do everything yourself!). Therefore, my Kensei engine solution contains projects for JigLibX, the Curve Editor Control, and a few other public components, as well as code supplied with various books I own. Which books should you own? Start here!

External – of course, some code isn’t supplied with source, but as a standalone assembly. For example, I use Perforce for source control (accept no substitute! It’s the best there is, period – it’s so good, I use it even though I don’t have to, and I’ve not yet come across any other source control I’d say that about) and the rather wonderful P4.NET lets me write the ability to work with Perforce into my tools. This allows my game editor to automatically check out assets that I work on, for instance. This, and other such assemblies, are all stored in the External folder (with a handy text file so I don’t forget where I got them from…).

Now, with all these projects and assemblies floating about, each built into their own build directory, finding them from the individual games becomes awkward! It’s much easier when all the engine assemblies are in the same place. With this in mind, I wrote the following PostBuild.bat batch file. Every project in the solution runs this as a post-build step (details in the script itself). It copies the output files of the project into the bin folder, so when I create a new game, I only have to look in that one folder for all my references.

@echo off
rem ————————————————————————
rem Run this script as a Post Build Step from every Kensei library to be
rem used by a project or game outside of the Kensei solution.
rem It will place all built files into solution\bin\platform\configuration.
rem This will make them easier to find when adding references.
rem ————————————————————————
rem Post Build Step:
rem
rem call “$(SolutionDir)PostBuild.bat” “$(SolutionDir)” “$(PlatformName)” “$(ConfigurationName)” “$(TargetDir)” “$(TargetName)”
rem ————————————————————————

echo —- Post Build Copy —-
echo Copying %4%5.* to %1bin\%2\%3…
xcopy   %4%5.*   %1bin\%2\%3   /Y   /I   /F

Finally, with all these assemblies floating around, it’s nice to be able to clear everything out and know that a build will start completely afresh. (Why does “Clean” do nothing on a C# project? Alright, Visual Studio handles C# a hundred times better than C++, but I still sometimes want to know that everything that isn’t a source file has been deleted). So I wrote a CleanAll.Bat (warning, use at your own risk, especially if you make use of folders named ‘obj’ or ‘bin’):

@echo off
rem Script to completely start afresh without *anything* still hanging around from previous builds.
rem This is useful to, for example, check what would happen if it was checked out on a clean machine.

echo ——————————–
echo Delete Visual Studio local files
echo ——————————–

del /Q /S *.csproj.user
del /Q /S /AH *.suo

echo ————————————
echo Delete intermediate and output files
echo ————————————

FOR /R %%i IN (*) DO IF EXIST “%%i\..\obj” rmdir /Q /S “%%i\..\obj”
FOR /R %%i IN (*) DO IF EXIST “%%i\..\bin” rmdir /Q /S “%%i\..\bin”

echo.

That’s pretty much it for the Kensei engine (well, more or less, as you’ll see in a moment). After seven or eight years with playing around with Visual Studio and trying to arrange large game projects in a way that preserves my sanity, I’m pretty confident that this layout is the easiest to work with.

Game Folder Layout

The whole point of the above is to make creating new games prototypes quick and easy. I’ve got ten or twelve game ideas in my head right now, although Pandemonium remains my main focus, and every time I get a new one, I like to be able to spend one evening slapping something together that just works with a SpriteBatch (for 2D) or Kensei.Dev.Shapes (for 2D and 3D) without spending any time at all plugging the lower-level stuff together. So, all my games exist in the Documents\Code\XNA\Games folder, and reference assemblies in the Documents\Code\XNA\Kensei\bin and External folders. Taking Pandemonium as an example, Documents\Code\XNA\Games\Pandemonium contains (as you’ll have come to expect) the Pandemonium.sln Visual Studio solution, and the Pandemonium.FxCop file (and CustomDictionary.xml) as well. Then there are the following projects in the following folders – again you should be able to guess what they are, and they should be fairly self explanatory when read in combination with the above:

Pandemonium – the game itself. Of course, the content project and all assets are stored in here too.
Pandemonium.Pipeline – content pipeline extensions for the game.
Pandemonium.Pipeline.Runtime – a bridge between the two.

It really is as simple as that, and if I were just writing a quick prototype there would probably only be the first one.

Engine.Game

There’s one project I didn’t mention above in the Kensei engine. I’ve only just started doing things this way, and although so far it’s working well I’m a little concerned it may prove inflexible in the long run. I’d encourage you to experiment with it – after all, if a particular game finds it hard work fitting into this model, just don’t use it. It will save you a lot of time on those games that work alright with it.

Basically, I’ve created a project named Kensei.Game, and it contains a single class, Kensei.Game, which derives from Microsoft.Xna.Framework.Game. (Unimaginative names, I’m afraid…) If you’re wondering why it’s a separate project and not part of the Kensei project, that’s because it uses code from all the different assemblies and it was easier to keep it apart than try to prevent them from referencing one another in a circular manner. The idea is, that all of my games can inherit from Kensei.Game instead of (more directly) Microsoft.Xna.Framework.Game, and Kensei.Game can handle all of the things that each individual game would have to do for themselves.

Allow me to provide an example. Kensei.Game provides the following (sealed!) implementation of the Draw method: 

What this means is that each individual game I write has much, much less to do in it’s (abstract) DrawGame method. I can’t forget to call base.Draw, and I always get the benefits of the same Kensei.Dev options and Kensei.Dev.DevText and Kensei.Dev.Shapes drawing – completely for free. The DrawGame method in Pandemonium is simply this (admittedly, it’s very much work in progress and there’s not much to draw yet, but you should get the idea of how little code there is to write per-game, or per-game-state when it’s a little more developed):

Update, Initialize and other methods are similarly simplified for the individual game, with as much complexity and common behaviour as is feasible moved into the engine.

Incidentally, when you have a base method that needs to do things before and/or after the derived method, I much prefer this pattern of having the base method (Draw here) as public, and have the base method call into protected virtual (or abstract) methods (DrawGame here). This is much superior than having a single protected function and relying on your more-derived class to remember to call the base version of the method. It’s too easy to forget, and too easy to get subtle bugs as a result! Of course, there is a cost. It may be, as I develop my games, that I discover I need to call some game-specific code after base.Draw (or base.Update, or what have you). That would become difficult or impossible in this pattern. If things turn out that way, I’ll have to change it. For now, I’m enjoying the simplicity of my game-specific methods. I can, quite literally, create a new XNA project, change the main class to inherit from Kensei.Game instead of Microsoft.Xna.Framework.Game, and I’ve got all the functionality of my Kensei engine instantly available. It’s very powerful.

(My original plan was going to be even more powerful; I’d create a simple project using Kensei.Game, and with all the necessary references to the Kensei\bin assemblies, and use that as the basis of a Visual Studio Project Template. Then, when creating a new project, I’d select “New… Kensei Game” instead of “New… Microsoft XNA Framework 3.0 Game”. It nearly worked! Everything was perfect, in fact, except that the Export Template wizard didn’t pick up the content project. That’s very unfortunate, and means I can’t use this trick as I’d wanted, but if anyone knows a way around it, please let me know.)

I think that’s enough for now. I hope what I’ve written makes sense, and gives you some ideas on how to structure your own engine and games, code and assets. I don’t claim for a second that this is the only way of doing it, and in fact if you’ve got some better ideas, I’d love to hear about them – leave a comment. This has been a fairly high-level view of the problem and my solution to it, and there are so many tradeoffs to be made at a lower level – something for a future article perhaps, though I’ve tried a number of different structures over the years, and have heard of still more without trying them out yet, and have yet to settle on a favourite. Something to experiment with, for sure. Anyway – let me know if this was useful to you.

I was planning to spend the next couple of nights upgrading my Kensei engine and Pandemonium game to XNA 3.0, and making notes of the problems I encountered, so that I may write an article explaining how I overcame them and others may more easily overcome the same problems if they encounter them too.

Unfortunately, I can’t do that – because if I did, the article would consist of the following: “It just worked”.

Yes, it really was as easy as loading up VS 2008 and loading my existing solutions into it. It took a few minutes, to be sure, but I only had to press OK a few times and everything worked. It was so easy that I didn’t believe it at first – I did a batch build of all projects, then a batch rebuild of all projects, then double checked all the references to ensure it really was using XNA 3.0 and hadn’t just pretended to upgrade from 2.0. But it was true – the upgrade really did take zero effort.

So I thought I’d kick things up a notch – bam! – and investigate what Microsoft had seen fit to provide us with in an area that was sorely lacking in 2.0: installation and distribution of my game on Windows. Please understand that this is only a preliminary look at what’s offered, and I might easily have missed some subtleties. Basically you’ve got three choices:

Package as CCGame: this particular option seems to be quietly glossed over in the docs so I suspect it’s just a hangover from XNA 2.0. It’s got the same old drawbacks, in that the recipient has to have the XNA redistributable already installed, and it doesn’t offer any choice of installation location or creating shortcuts on the desktop or Start menu. However, on the plus side, the final output is a single compressed file. It’s much easier to give someone one file than several files and folders.

Setup Project: assuming you have VS 2008 Professional or higher, you can now create a VS setup project that creates an MSI file. (Instructions are also provided to use a third party installer such as Wise, should you so desire). This would also create a single file, as well as giving total control over installation folders, shortcuts, the ability to show licensing agreements; but the downside is that it doesn’t automatically recognise content project files as dependencies, which would need to be added individually. There could be thousands of these in a typical game. Not going to happen. You’ll drive yourself mad trying to keep it in synch with your game. Personally I’ll be avoiding this option like the plague.

ClickOnce: obviously the favoured choice at Microsoft, it’s got a lot of good things going for it. You can set version numbers (and make them update automatically), it installs the .NET and XNA prerequisites for you, you can massage which files get included and define download locations or whether to autorun the CD you burn it onto… brilliant. Plus you get all the expected shortcuts in the Start menu (but not the Vista Games folder, oddly). The downside? The final output isn’t a single file! It’s an exe, and a manifest, and a folder containing all your game files with the extensions changed to .deploy. I don’t really get what that’s supposed to save you. Sure, you can zip it up – but then your user has to open a zip file and run the right exe. You could just as easily zip your game from the bin folder and tell them to run the XNA installer separately, you’ve not saved anything.

Overall these options are a big leap forward from XNA 2.0 but I still wish they’d taken just one more step and either made ClickOnce create just one file, or made the Setup Project more intelligent. At least now there’s a better-than-evens chance that whoever you give your game to will be able to install and play it first time, but it’s still not quite as simple as it could be. It’s possible that I’m missing something, but where the upgrade from 2.0 to 3.0 was a perfectly smooth 10/10 experience, I’d still have to give the Windows distributable options no more than 8/10.

One thing I do like isn’t strictly an XNA feature at all, but C# 2008. Select View -> Code Definition Window, and every time you select a symbol in your code (class, function, namespace, enum…) the window will update to show you the code definition of that symbol. It’s priceless and I officially love it. I’m looking forward to using all the other new features of the IDE and the language.

(Though bizarrely, the Open Containing Folder feature on the active item tab only worked the first time I used it, and never since. If anyone can tell me why I’d be very grateful as I’d use that all the time).

You twisted my arm.

Let’s very quickly go over what you should know already (remember, this isn’t necessarily a beginner’s guide to garbage, rather an attempt to explain why what you know about garbage is like it is, and what it means for your code):

1. Value types (ints, floats, and other in-built types, as well as any user-defined struct type, eg. Microsoft.Xna.Framework.Vector3 or a struct you define yourself) are created on the stack, so you don’t need to worry about garbage.

2. Reference types (any class) are created on the heap, so should be used carefully as they will be garbage collected, which can be bad for the frame rate of your game if it happens at an inconvenient time. Part one explains why, in more detail than you really need to know.

As you may be able to guess, it isn’t quite that simple.

First, it’s not completely accurate to say that value types are always created on the stack. They will be, if they are created within a function, but not if they are themselves a member of a class, in which case they will exist (as part of the class object) on the heap. It’s a minor and fairly obvious point but it’s important to be correct.

Second, it’s not completely accurate to say that value types never generate garbage. They can, of course, contain reference members, in which case creating a new struct object can indirectly create a new class object, which generates garbage exactly as though you’d created the class object yourself. It’s fair to say that this is a slightly unusual thing to do (I’m not sure I can think of an example?) but it’s entirely legal and something to watch out for.

Third, remember how I said value types are created on the stack unless they’re members of a class, and how that was a minor point? Actually it’s not that minor. Value types contained within reference types are allocated on the heap. For example, arrays always catch out new C#/XNA programmers:

Slightly confusingly, passing value types by reference (using ‘ref’ or ‘out’) does not “turn them into” reference types. It simply means that the object’s value can (or must, with ‘out’) be changed within the function and those changes are reflected in the original object that exists outside the function. It also means that no temporary object is created to act as the function parameter. For example, v1 below is a copy of another Vector3; creating a new object and copying onto it can be marginally slower than just passing by reference, as with v2, and the difference becomes more pronounced the larger the object. (Remember a reference is eight bytes in size in 32-bit code like XNA, so any struct eight bytes or smaller won’t benefit from passing by reference at all). For this reason, functions like Vector3.CatmullRom, that take many struct parameters, are provided in two flavours: one, which is more convenient to use, that takes (copies) the inputs by value and returns the result; and another, which is more performant, that takes the inputs by reference and the result is an out parameter. It’s a pattern you may want to use in your own code though 90% of the time you’ll just call the more convenient version.

Of course, reference types are always passed by reference anyway (which means a function can always change them; that’s one thing I miss about C++: const!) so using the ‘ref’ keyword for reference types is pointless, though legal. (FXCop rightly points out your mistake though).

There’s another difference between value and reference types – more specifically, between structs and classes – than simply where they live. Reference types can inherit from other reference types (and you therefore get inheritance, polymorphism, and all the other clever object-oriented stuff) while value types cannot inherit at all. And yet – you are probably aware that any and every type in C# is considered to ultimately derive from System.Object. You will undoubtedly have seen functions like this (particularly in .NET 1.x, before generics came along):

If obj can be anything, including a value type, like an int, a float, or a Vector3 – but value types can’t derive from anything – how come you’re allowed to call G( 3 )? Well, when such a call is made, a temporary object which does derive from System.Object and is known as a “box” is created on the heap, and a copy of the value type object placed inside it. Putting the value type in the box is called “boxing”, and taking it back out (via a cast, or the ‘as’ operator) is known as “unboxing”. Critically, the temporary box object is garbage. This means that using value types in functions or containers that rely on type System.Object generate garbage. As a result, boxing is probably the second most common cause of unexpected garbage. To avoid it, avoid using or writing functions or classes that use System.Object – prefer generics instead – though you can still get caught out if you’re not careful, as my previous post on Reflector showed.

So, usually, you will want to use reference types (classes) for your data that stays in memory for a significant amount of time. You’ll want to be careful with when you allocate it (remember, a garbage collection can only ever happen on an allocation). Value types (structs) are most useful for small, lightweight data. Think about Vector3 – it only contains three floats and has a handful of methods. The XNA team could have written an abstract base class Vector and specialised it with Vector2, Vector3, Vector4, and VectorN, but what a piece of over-engineering that would have been! More to the point, using vectors (very common objects in 3D games) would have generated masses of garbage. As a struct instead of a class, Vector3 is much more elegant – you can write things like SetPosition( new Vector3( 1.0f, 2.0f, 3.0f ) ), a thousand times a frame if you like, and know that there’s no chance of garbage. On the other hand, your main character object is bound to be of class type. It’s likely to derive from things (interfaces if not classes) and needs to be kept in memory, which means on the heap.

This also implies that if you have an object of class type, you should keep it around if you’re likely to be able to reuse it. For example, this kind of trick is useful all over the place:

Just a couple more things to think about, though I’m a bit short on space. Some types can be considered as “atomic”. Basically what that means is, that if they change, they become a different object. So a Vector3 is non-atomic – you can change v.X and you still have the same Vector3, just in a slightly different place. But if you have a user defined type PlayerDetails, and you change any of the fields, you’ve got a completely different “thing” to deal with:

The player details for me are fundamentally a different “thing” to the player details for Fred Bloggs. (Language fails me slightly, here, and I’m also not sure I’ve picked a great example). Furthermore, if anything unexpected happens in the middle – the object gets viewed on another thread, or setting LastName throws an exception – the object can be seen in an invalid state. To prevent this, such atomic types should be immutable (I’m just not going to stop linking to those books!), which means you can’t change any single field, and once an object has been created, it never changes. This is really good software engineering (it allows you to write more correct and more secure code) but can lead to surprising results:

That short code snipper generates not one, but seven objects: the final message object contains “Player 1 wins by 6 points!” for example, while the strings “Player “, “1″, “Player 1″, “Player 1 wins by “, “6″, and “Player 1 wins by 6″ are all garbage! System.String is an immutable, atomic, reference type. This was absolutely the right choice by the .NET architects (strings in .NET are wondrous things of great beauty compared to strings in C++) but leads careless game programmers to watch helplessly as their frame rate plummets. Code like the above is probably the number one cause of unwanted garbage – watch out for it. If you must build strings piece by piece, use a StringBuilder object. (And, if you find yourself creating a type that is atomic and immutable, consider creating a non-atomic mutable MyTypeBuilder class to work with it).

I know what you’re thinking. “It would generate less garbage if you didn’t call playerNum.ToString() and points.ToString(), and just passed in playerNum and points!” Nice try, but no. System.String.operator+= works with System.Object types, and calls System.Object.ToString(). That means, if you passed playerNum into it, it would box playerNum and still generate the extra string – boxing and strings, the two worst garbage generators happening at once – you’ve made your garbage problem worse, not better!

There’s a lot to understand about garbage but I hope the last couple of articles have opened your eyes somewhat. I’ve approached it in a slightly unorthodox way – most writers will start with an article like this one, explaining the dos and don’ts, and probably never even cover the topics in the previous article which showed the whys and wherefores. Personally, I like to know what’s going on “under the hood”. You can memorise the rules relating to garbage and write good code, but until you understand the reasons behind the rules you might not write brilliant code. There is one important thing related to garbage I’ve not covered – that’s the Dispose pattern. This book (there it is again!) explains what it is and how to implement it, much better than I can; the only thing it doesn’t really cover is the detail of how objects waiting for disposal relates back to the mechanics of garbage collection I covered in Part One, so I might go over that at some point but I’m not promising anything.

I hope this appeases those of you who wanted a follow-up to Part One, I misjudged what people wanted and apologise for that. There is one way to make sure it doesn’t happen again – your feedback is always welcome, let me know what’s useful, if anything wasn’t clear, and what you’d like me to write about in future (though I reserve the right to choose not to .

“There’s four and twenty million doors, on life’s endless corridor” – Oasis

Oooooookaaaaaaaaaay, it seems that Part One wasn’t very interesting as nobody commented on it and not many people even read it. I personally think that the way C# manages garbage is fascinating and immensely clever. Perhaps I’m the only one!

So I was going to go into a similar level of detail for much more garbage-related stuff, and spread it out across three or four posts; but that seems a bit pointless given the previous response, a lot of effort for not much gain. I could instead write about it at a very high level, but you can find better resources elsewhere. Sorry… bit of a let-down this post. I’ll try to make up for it with the next one.

I’m planning to get what Pandemonium (such as it is so far) running on XNA 3.0 over the next few days, hopefully I’ll be able to write a bit about that. I’m looking forward to getting some actual gameplay up and running, not just the basic systems, which I intend to show as I go. I have a few other topic ideas too. Of course, if there’s anything you’d like me to cover, say the word. Check out the kind of things I’ve written about so far to see the kind of level I’m writing at (and for).

Back to normal good service soon…