Tools of the Trade: Part Three – CLRProfiler

Over the last few days I’ve been encountering a growing realisation of just how many excellent C#/XNA/programming tools there are out there, that I use regularly; and therefore how large this mini-series will become, if I continue with this pattern of one article per tool. Still, there remain a few that I think are important enough to talk about in detail, before I summarise the rest more concisely (though leaving open the possibility of returning to them later).

So by now, you will hopefully be keeping the compiler warning level as high as possible, checking your code for style-based or semantic errors with FxCop, and when necessary, checking how the assemblies you’re interacting with do the things they do with Reflector. Now, perhaps, you’re starting to wonder about what the code you’re writing is doing at run time.

Probably the most notable thing your XNA/C# game  will be doing compared to, say, C++, which is (or at least, has been for many years) the de facto language used in the games industry, is dealing with memory in a completely different way. C# is a managed language. This means that where in C++ you use “new” to allocate memory and “delete” to deallocate it (with the possibility of leaks, scribbles and access violations), in C# you use “new” to get a reference to an object, and the garbage collector tidies it up for you when you’re finished with it.

There is a lot that I could write about the garbage collector. It’s probably fair to say that dealing with garbage correctly is key to programming in C#. In fact, until two minutes ago this article was about 1000 words longer, until I realised that I’d written a whole bunch of stuff more relevant to GC itself than the tool the article is about, and even then had either gone into too much or not enough detail depending on your point of view. (Perhaps something for a future article, or even series, if there is demand). In summary:

- GC on Windows is not a good thing and you should try to avoid generating garbage where reasonably convenient.
- GC on Xbox is a very bad thing, and if your game is intended for Xbox 360 you should go out of your way to avoid it everywhere possible.

The other day I was working on Pandemonium and noticed that my frame rate had plummetted from 1800 Hz in the morning to 200 Hz by the evening. Now, 200Hz is more than enough – for a finished game! But I only had one character running around a simple background so it seemed a bit low. Losing 1600 Hz in a day is not a good day. Luckily, using some of the Kensei.Dev.Options I’ve shown before, it took just a couple of mouse clicks to realise that I was suffering lots of garbage collections. This was confirmed when I used another Kensei.Dev.Option to show more details:

    if ( Options.GetOption( “Profile.ShowGarbage” ) )

    {

        Kensei.Dev.DevText.Print( “Max Garbage Generation: “ + GC.MaxGeneration.ToString(), Color.LimeGreen );

#if !XBOX

        for ( int i = 0; i <= GC.MaxGeneration; ++i )

        {

            Kensei.Dev.DevText.Print( “Generation “ + i.ToString() + “: “ + GC.CollectionCount( i ).ToString() + ” collections”, Color.LimeGreen );

        }

#endif

        Kensei.Dev.DevText.Print( “Total Memory: “ + GC.GetTotalMemory( false ).ToString( “n0″ ), Color.LimeGreen );

    }

It was clear that something was generating lots of garbage. Suspicion soon fell on my background hits implementation, as that’s what I’d worked on that day. It was deliberately simple, just an array of triangles, each with a BoundingBox, against which I would test lines and swept spheres. I know an octree or kd-tree would be better but that’s something I can address later. It might not be a very quick method, but it shouldn’t be generating garbage.

Visual inspection of the code revealed nothing so to find the cause, I fired up the CLRProfiler (top tip: make sure to use version 2.0 and run as Administrator) and asked it to run my game. The first thing you should note is that it is invasive: my frame rate dropped from about 200 Hz without it, to less than 10 Hz with it running.

The CLR Profiler

So, clear the “Profiling Active” checkbox, only then “Start Application”, get yourself to the part of your game that’s slow, and only then tick “Profiling Active” for a few seconds before clearing it again. That will generate a report.

A CLR Profiler report

109 Generation 0 garbage collections is quite a lot in the few seconds I ran the game for. Ideally, a game would have zero garbage collections during gameplay. (That’s not possible if you’re using XACT for sound, but it’s a good ideal to get as close to as possible). The Time Line button in the centre gives the best view of when GCs happen and why.

The CLR Profiler Timeline

The coloured peaks show garbage, which you can see was increasing very sharply before dropping as each collection occurred. The vertical black line can be moved to show what the garbage consists of at each time slice; here I placed it just before a GC, and it shows me that 2MB (99% of my garbage) was from System.SZArrayHelper.SZGenericArrayEnumerator<T>. The rest was basically strings generated by the Kensei.Dev prints, listed above, and can be ignored. But – where did this SZGenericArrayEnumerator come from?

Back on the report page, I was interested in Heap Statistics, and wanted to see what had been allocated, so clicked Allocation Graph at the top. This shows a diagram indicating where all the garbage comes from. The highest level is on the left, and you go progressively further right to get more detail. So, Program::Main is the first significant box on the left and will have created 100% of the garbage - so all the garbage in your program was created by your program, go figure – but what’s lower down (ie. to the right)?

The Allocation Graph

As expected, 99% of the garbage is SZGenericArrayEnumerator – which comes from SZArrayHelper::GetEnumerator – which comes in turn from BoundingBox::CreateFromPoints. There was only one place this appeared in my code, and yes, it was bad. In order to provide a (small) speedup to my collision detection, which I’ve already mentioned I knew to be non-optional but wasn’t ready to spend time on yet, I’d done a Line-BoundingBox check to reject misses before doing a Line-Triangle check. And, being lazy and naughty, I’d created a new BoundingBox every test:

    foreach ( Triangle triangle in data.Triangles )

    {

        BoundingBox box = BoundingBox.CreateFromPoints( triangle.Vertices );

Just to be clear, here. foreach has gotten a really bad reputation for garbage because in CLR 1.0 the enumerator that foreach uses to traverse the container was of reference type. Some people, wrongly, claim that you should avoid it for that reason. That’s nonsense – in CLR 2.0 onwards, foreach is safer, easier to read, and at least as performant as a for loop. You should definitely prefer foreach in almost all cases. All my garbage was coming not from foreach, but from BoundingBox.CreateFromPoints.

The fix was easy – instead of creating a new BoundingBox for each Triangle for each hit test, I’d store the BoundingBox with the Triangle when they were created in the Content Pipeline. No more garbage, or at least, none at runtime; and the Content Pipeline doesn’t care about garbage. Really, I ought to have done it that way in the first place, so a definite slap on the wrist for me.

Only one thing still niggled. BoundingBox is a value type (created on the stack) so why was BoundingBox.CreateFromPoints creating garbage? The answer comes when you look back at the screenshot of Reflector in the previous article. BoundingBox.CreateFromPoints has a foreach loop in the middle of it, and foreach creates garbage.

“But Andy, wait!” I hear you cry. “You just said foreach doesn’t create garbage, in fact, you said that was nonsense!” Well, yes, though I did insert the word “almost” in a key position. The truth is that foreach does not create garbage for arrays, Lists, LinkedLists and any container other than Collection<T>. However, BoundingBox.CreateFromPoints has been designed to handle any container via the IEnumerable interface, which means the enumerator has to be boxed. Boxing means it is moved onto the heap, and is therefore garbage.

I’m still a little surprised that an array-specialised version of BoundingBox.CreateFromPoints isn’t provided but then, I guess it’s not exactly difficult to write if you desperately need it - especially given that Reflector shows how! In any case, I hope that this demonstration of how CLRProfiler helped me with a garbage problem, has shown how it can help you, too. Leave a comment and let me know how helpful you’re finding these articles.

“First, solve the problem. Then, write the code.” – John Johnson

Tools of the Trade – Part Two: Reflector

I had originally been intending to write about something completely different in this second part of the Tools of the Trade mini-series, but I encountered a problem today that I simply could not have solved without Reflector. It reminded me of just how cool this tool is, and I just had to make sure everyone else knew how cool it is too.

Following on from my article about FxCop, I decided to have a crack at writing some custom rules to use alongside the default Microsoft ones. The first sign that this might not be as easy as I had assumed it to be was when I realised there was a total absence of documentation about it. There weren’t even many articles – the best ones I did find are here, here and here. I will cut a long story short by saying that the task of setting up my custom rules was more arduous than I expected. (That said, it’s working now and I’m very happy with the results).

I eventually reached the stage where I believed my custom rules library was ready to add into FxCop, but unfortunately when I attempted to do so, it complained that the XML file I had embedded as a resource in my Kensei.FxCopRules assembly was incorrectly formatted. And that’s about all it said – the error messages were not exactly verbose in their description of the problem. I spent a significant amount of time trying to work out what was wrong.

After some head-scratching, I realised that the default rules libraries that ship alongside FxCop must (obviously) contain a correctly-formatted XML file. If only there were some way for me to examine those XML files in the assemblies themselves! In the Bad Old Days, this might have been possible by loading the assembly into a binary viewer and scouring it by eye, looking for readable phrases; otherwise it would be a lengthy and painful process of reverse engineering. Happily this is Space Year 2008 and .NET has made everything so much easier.

Enter Reflector.

This wonderful little tool uses reflection (surprisingly enough) to analyze any .NET assembly you care to throw at it, and expose its innards in glorious detail. All I had to do to solve my problem was open up one of the FxCop assemblies, browse through it until I found the Resources section, find the XML file and click “View Resource”:

An FxCop rules library XML file viewed in Reflector.

And there it was. A correctly-formatted FxCop custom rules XML file laid out in all its glory. From there it was mere seconds to see where I had been going wrong, and just a couple of minutes to fix it. I now have custom FxCop rules up and running and catching errors in my code – and I genuinely believe it would have been, if not impossible, at least very difficult and time-consuming without Reflector.

Of course, Reflector’s awesomeness doesn’t stop there. As well as showing you resources that have been embedded into assemblies, it can show you the code they contain as well. And I don’t mean “code” as in machine code, assembler, or even MSIL – I mean it can show you what is for all intents and purposes the same C# the assembly was originally written in. And it will do it for any assembly you care to throw at it. That’s pretty damn cool. To spell it out – if, say, you’re writing an XNA game and are curious as to how the actual XNA Framework code was written, just fire up Reflector and fill your boots:

The Microsoft.XNA.Framework assembly in Reflector.

If that’s not the single coolest tool anyone has written in the history of computing, I don’t know what is. And, again – just like FxCop, and indeed XNA and (should you not have the full version) Visual C# 2005 Express, it’s completely free. (Incidentally, if you’re wondering why I was inspecting BoundingBox.CreateFromPoints – all will be revealed in Part Three of this mini-series).

I feel as though I should spend longer explaining why Reflector is so amazing but honestly, if what you’ve seen already doesn’t excite you I’m not sure what will. (Perhaps the add-ins will convince you?) In any case, it is unquestionably a tool that any XNA (or indeed C# or .NET) programmer should be familiar with, and you will probably find that you use it often. I certainly do.

“Like the carpenter, the samurai looks after his own tools.” – Miyamato Musashi

Tools of the Trade – Part One: FxCop

The established wisdom – and there’s undoubtedly a lot of truth in it – is that catching a bug late in the development cycle can cost as much as 1000 times as much to fix as the same bug would have cost, if you’d caught it earlier. Really bad bugs can even (depending on what you’re making) make you miss your Christmas launch date, fail to reach launch altogether, force a product recall after launch, or even open you up to legal action. Some bugs have cost their authors literally millions of pounds, and the hardware bug that causes the Xbox 360 “Red Ring of Death” cost Microsoft billions. Of course, “cost” could equally mean time instead of money – but in either case, I’d rather spend ten minutes fixing a bug just after I wrote it, than wait till just before releasing my game and realise I have to spend hours, days or weeks finding and fixing the same bug.

The first weapon in your armoury to find and fix bugs early is to turn the compiler warnings up to the most practical maximum, which in the case of Visual Studio means Warning Level 4, or /W4. (If you’re stuck using Visual C++ you might be surprised to learn that some amazingly useful, in fact, critical warnings are actually disabled by default at level 4). You should always ensure your code compiles without warnings at /W4. You may think some warnings are unnecessary; fix them anyway. It won’t hurt, and might even help without you realising.

Once you’ve got your warnings turned up and fixed, the next step is to use static analysis tools like FxCop. Static analysis tools find bugs in your code without you even needing to run it. In fact, the next version of FxCop isn’t even a separate tool, it’s a compiler option (/analyze). These tools parse your code (or in FxCop’s case, examine the MSIL assemblies) and compare what you’ve written against a set of rules that help to identify dangerous patterns in your code.

With FxCop installed, you simply need to open it up, start a new project, and point it at where to find the assemblies (.dll or .exe) for your code. It works equally well with either XNA game code or standard .NET non-game code. Hit “Analyze”, and a few seconds later it will come up with a list of problems that it has detected, along with an explanation about why they are problems and suggestions on how to fix them.

You will probably be shocked and surprised by how many problems it finds in your code. The good news is, it’s probably not as bad as it looks; the bad news is, that’s because FxCop is slightly on the noisy side, though you have complete custom control over which rules you enable (and you can even define your own). For example, FxCop regularly complains at me about the following:

IdentifiersShouldBeSpelledCorrectly: Correct the spelling of ‘Kensei’ in namespace name ‘Kensei’.

Well, first of all, Kensei is the correct spelling, it’s the codename of my engine. Secondly, I’m not completely sure that I care if the spelling is correct – nobody is going to play my game and think, “it’s a good game, but he misspelled one of his namespace names”. Thirdly, I’ve followed the instructions to create my own user dictionary for FxCop to add “Kensei” to it, and couldn’t get it to work; and I’m certainly not going to try and think up a whole new codename for my project just to keep FxCop quiet. So, I usually disable this rule, or at least exclude the specific occurrence of the warning.

There are some warnings that are a little bit trickier and need more thought. Like this one:

MarkAssembliesWithClsCompliant: Mark ‘Kensei.dll’ with CLSCompliant(true) because it exposes externally visible types.

Really…? ‘CLS compliant’ means, basically, that the assembly can be used by other languages in the Common Language Specification that aren’t C#, like Managed C++ or Visual Basic. I guess, on one level, that seems like a pretty noble thing to aim for, though I wouldn’t expect to ever use those other languages given that I love C# so much. But it’s possible I might make parts of my code available to other people, and they might want to use those other languages. So should I follow this advice or not?

In this case the answer comes when I do what it says to do. Marking the assembly as CLSCompliant results in a number of compiler warnings, because I’ve been using types that are not CLS compliant. That’s fair enough, except that these types include things like GameTime, which is a core XNA type. The XNA team chose not to make their assemblies CLS compliant; so there is absolutely no point in me trying to do so.

Anyway, by now you’re probably wondering why I’m talking about FxCop when so far I’ve just described some things it does less than amazingly well. As it happens, I think it’s a wonderful little tool, especially since it’s completely free. I run it on all my code – the Kensei engine, and my games, including Pandemonium – regularly; at minimum, after each “milestone” I’ve set myself. I’ll then fix the problems it highlights, either by changing the code as recommended or by marking a warning as Excluded, to say that the infraction was deliberate. So… why? Here’s why (a real-life example, though the names have been changed to protect the innocent):

    public class BaseClass

    {

        public BaseClass()

        {

            InitValues();

        }

 

        protected virtual void InitValues()

        {

            // …

        }

 

        // Rest of class implementation

    }

 

    public class DerivedClass : BaseClass

    {

        public DerivedClass()

        {

            // …

        }

 

        protected override void InitValues()

        {

            // …

            base.InitValues();

        }

 

        // Rest of class implementation

    }

Spotted the error yet? You probably should have done – I’ve certainly stripped it down far enough! When I wrote this, the BaseClass was in BaseClass.cs in a project in my Kensei solution, and the DerivedClass was in DerivedClass.cs in my Pandemonium solution, and both files were pushing 1000 lines of code. So the problem was a lot less obvious… but no less real for all that. FxCop spotted it straight away:

DoNotCallOverridableMethodsInConstructors: ‘BaseClass.BaseClass()’ contains a call chain that results in a call to a virtual method defined by the class. … Virtual methods defined on the class should not be called from constructors. If a derived class has overridden the method, the derived class version will be called (before the derived class constructor is called).

In other words, when I did something like this:

    DerivedClass derived = new DerivedClass();

… it would (conceptually; this isn’t completely accurate) first allocate memory on the heap, then call the BaseClass constructor, the BaseClass constructor would call the DerivedClass virtual InitValues method, which would call into the BaseClass InitValues method, and only then would it call the DerivedClass constructor before giving me back the reference to the object. That is to say – it would be calling a member function intended for a DerivedClass object, on an object that wasn’t a DerivedClass at all, only a BaseClass.

Most of you will be well aware that this is a Bad Thing. I certainly was already. Such behaviour is not polymorphism (a good and noble ideal), it’s just broken and wrong. It could rely on members being in a certain state that weren’t in that state yet. It could call out to other functions in other objects, and end up interacting with them despite being only partially constructed. It could, frankly, do just about anything, and almost none of the things it could do would be good. (For more on what is allowed in C#, see the C# Standard). However, despite the fact I’ve been programming for [mumblemumblemumble] years in total, and for six of those years I’ve been paid to do it, I still made this mistake. Well, I am only human after all – and we humans make mistakes.

It’s a mistake that FxCop spotted and I’ve since fixed it. It spotted it almost as soon as I’d written it, and it took five minutes to fix. Late in the dev cycle, I might only have spotted it after hours trying to work out why my DerivedClass objects were behaving strangely, and so much other code might have become dependant on that behaviour that it might have taken hours longer to fix. I could have lost days or weeks to one silly little bug – and believe me, I’ve seen exactly that happen before now.

There are many other genuinely code-breaking bugs that the compiler can’t spot but FxCop will pick up immediately. (Have you implemented the Dispose pattern correctly throughout your program? Are you really sure about that? How about private members and functions that are never used?) If you’re willing to spend money, there are other static analysers out there that check correctness and performance at an even lower level, therefore finding even more subtle (or critical!) bugs, and all without actually running a single cycle of your code. (Indeed, if you’re using C++ for some reason and not using Lint, BoundsChecker, PREFast or something similar, I bet real money your code is broken). But for C# and/or XNA, FxCop is free, easy to use, easy to customise, and if you’ve not used it yet I strongly recommend that you start right now.

“If debugging is the process of removing bugs, then programming must be the process of putting them in.” – Edsger Dijkstra

Come Play With Me

The first few articles on this blog have been written with the intent of sharing ideas, information and best practices with everyone else out there with an interest in developing games. It sounds like at least some of you are finding it useful, and I’ve been getting some great feedback, as well as ideas on how to make it even better.

I hope you will forgive me if I take a moment to be rather more selfish, and shamelessly publicise my in-development game, Pandemonium. That said, as well as showing off (the very early stages of) my game, this video also demonstrates several of the techniques I’ve talked about previously, most notably Kensei.Dev.Options and using PropertyGrids to rapidly tweak game variables.

I know, it’s not exactly full of eye candy! At this stage it doesn’t worry me. You can probably tell that Pandemonium is going to be a third-person platform game, very much in the vein of the classic Mario 64 or Banjo Kazooie games. The key to this genre of games (as with all games, really!) is the character control – if the player doesn’t enjoy moving their character around, they’ll never enjoy the rest of the game; and I’ve therefore decided to concentrate on honing the control first, before I start worrying about graphics, models, shaders and other such trivialities.

To this end, I’ve knocked together a small test level, which I’ve called the Playground, that contains a whole bunch of geometry for my avatar to run and jump around. I’ll use this level to refine how my character moves, and establish early on how far it can jump, how fast it can run, and other constraints that will make later level design much easier. Over time, as I add more features to the game, the Playground will get expanded; by the end of development, I’ll be able to load up the Playground and instantly test or demonstrate anything the player can do on any level of the game proper.

I mentioned SketchUp (the 3D design tool I used to create the Playground) before, and I used Bulent’s Screen Recorder to capture the video.

“I think I can make an entirely new game experience, and if I can’t do it, some other game designer will.” – Shigeru Miyamato

At Your Command

Previously I wrote at some length about tips and tricks you can use to ease your game development. The focus for these articles was code-facing techniques to help you identify and diagnose bugs, and easily provide data about the current state of whichever facet of the game world you’re currently interested in. Code is only half of what makes a game – less than half, really, and a smaller proportion the bigger the game gets; most of what makes a modern game is art, design, and music content - and when it comes to generating content, you need good tools. Today I would like to talk about a feature of your game editor tools that you really shouldn’t try to live without: infinite undo and redo. Once again, I’ll provide an implementation from the Kensei library, at the end.

I won’t lie to you; there’s rather more work involved in setting up undo and redo than in, say, using Kensei.Dev.GetOption(), but I can guarantee it’s worth the effort, even if you’re in a team of one (can you imagine using Visual Studio without being able to hit Ctrl-Z to undo?) but especially if you’re in a team of many (particularly arty, creative, temperamental types, who are liable to hit you over the head with a shovel shouting “UNDO THIS, ASSHOLE!” if their tools are awkward to use – and understandably so). In fact, it’s fair to say that “tools programmer” is now a full time position at most modern games companies; you may not have time to work full-time on tools if you’re doing this for a hobby, but the ability to undo and redo is a feature that is sure to save you time in the long run.

I first read about how to implement infinite undo/redo in the so-called Gang of Four book, which came as something of a relief as until that point there had been nothing in it of any use whatsoever beyond stating the obvious. Gamma et. al. designate this the “Command Pattern”, because at its core lies the concept of encapsulating user commands as objects; to me, that’s not a great name as you can encapsulate user commands as objects and still not have infinite undo and redo. Nevertheless, to avoid confusion I will be using the established terminology in this article and accompanying code.

Let’s imagine your game has an Editor mode (or perhaps you have a separate Editor tool; it’s not important). Let’s further imagine that one of the key variables in your game is the Health of your Player. Taking the line of least resistance, you provide a mechanism (menu option? Dialog box? Command prompt? Property Grid or Reflection? Keyboard shortcut? All of the above, each calling the same function?) for the user to edit the starting value:

    LevelData.Player.Health = userInput;

Pretty straightforward. Until, that is, one day while using your Editor, you decide the player’s health is too low, so you use your Editor to increase it to 250, but then you have second thoughts, and can’t remember if it used to be 210 or 220. The first step in the solution, as noted above, is to encapsulate user input as an object:

    abstract public class ICommand

    {

        abstract public void Execute();

    }

 

    class SetPlayerHealth : ICommand

    {

        public int newHealth;

 

        public SetPlayerHealth( int health )

        {

            newHealth = health;

        }

 

        public override void Execute()

        {

            LevelData.Player.Health = newHealth;

        }

    }

 

    // … Loads of other code …

 

    ICommand cmd = new SetPlayerHealth( newHealth ); 

    cmd.Execute();

A few points to note here: firstly, throughout, I’m using public variables only to keep things simple. Secondly, you can probably guess already that you have to be very careful with the accessibility of your interface; if the code elsewhere can also set LevelData.Player.Health directly, it will be too easy for the code to change it without using a SetPlayerHealth object. Thirdly, you may still wish to wrap the command object behind a function so the external call remains the same. Fourthly, “Execute” is the term used by the GoF, so I use the same term here; I guess they thought “Do” wasn’t pretentious enough.

We’ve not really changed very much yet though. The trick is that, instead of calling Execute ourselves, we push it onto a stack owned by the Editor, and it’s the stack that is responsible for calling Execute:

    public class CommandStack

    {

        public Stack<ICommand> commands = new Stack<ICommand>();

 

        public void AddCommand( ICommand command )

        {

            command.Execute();

            commands.Push( command );

        }

    }

Now that we have a stack, any time the user hits Ctrl-Z we can take the last command off the top of the stack, reverse its effects (ie. undo it), and do that as many times as we like until we get all the way back to where we started, if we want to. (The stack can grow as large as the memory in our PC, hence, “infinite” undo/redo). Of course, we’ve not yet defined how to reverse a command, which we will do now:

    abstract public class ICommand

    {

        abstract public void Execute();

        abstract public void Unexecute();

    }

 

    class SetPlayerHealth : ICommand

    {

        public int newHealth;

        public int oldHealth;

 

        public SetPlayerHealth( int health )

        {

            // Store old state before changing it to the requested new state

            oldHealth = LevelData.Player.Health;

            newHealth = health;

        }

 

        public override void Execute()

        {

            // Set the new state

            LevelData.Player.Health = newHealth;

        }

 

        public override void Unexecute()

        {

            // Restore the old state

            LevelData.Player.Health = oldHealth;

        }

    }

 

    public class CommandStack

    {

        public Stack<ICommand> commands = new Stack<ICommand>();

 

        public void AddCommand( ICommand command )       

        {

            command.Execute();

            commands.Push( command );

        }

 

        public void Undo()

        {

            ICommand command = commands.Pop();

            command.Unexecute();           

        }

    }

Stripped of all irrelevant detail that’s all there is to it. But we don’t need to stop there; there is some very low-hanging fruit that we can pluck to improve this feature yet further.

The most obvious next step is to support redo as well as undo. Very simply, this requires adding a second stack (redoCommands) to our CommandStack. Every time we pop a command off the main stack, after calling Unexecute, we push it onto the redo stack; and to redo, we simply pop off the redo stack, Execute it, and push it onto the main stack. If a new command is added, we empty the redo stack, as you can’t redo something when you’ve done something different since undoing it. If that doesn’t make sense, draw yourself a diagram, think about how Undo/Redo works in Visual Studio, and it should all become clear.

Speaking of Visual Studio – if you type in “abcdef” then hit Ctrl-Z, it will delete the whole string rather than leave you with “abcde”. Evidently, the “add text” command behind the scenes in Visual Studio isn’t limited to single characters. Instead, with the first key press, “add text: a” is added to the command stack, then when the second key is pressed, rather than adding a new command, the top command is modified to represent “add text: ab”. I’m not sure if there is an official term for this; I call it “merging” commands. It’s not too hard to implement, but the code I provide in a moment shows how anyway.

I’ll not go into detail on some of these other features, though again the supplied code implements them; but it would be nice to tie the Command Stack to the Editor’s Save system, so we can provide a “Save changes before closing?” dialog box when the user clicks “Exit” or similar. This is simply a case of recording the size of the stack every time the Editor saves the data file, and if we Do, Undo or Redo away from that size, mark the command stack as dirty. Additionally, some commands are too complex or destructive for us to be able to Undo them. In that case, we’ll need to warn the user (this part is left up to the calling code) then, once the command is executed, clear the undo and redo stacks – they will no longer be valid.

You can download Kensei.CommandStack here. You are, as always, free to do anything you like with it, including use it or not use it, as long as you don’t blame me if something goes wrong. Or, you can use the techniques I’ve discussed to come up with something different – perhaps even something better. Please tell me about it if you do.

I would just like to mention some final caveats:

• This is an all-or-nothing system. If some parts of your Editor implement the Command Pattern and other parts do not, bad things will happen. At best, you will confuse your user. At worst, you may leave your data files in an invalid state with no way to recover from the damage.
• Execute() and Unexecute() must be exact opposites of one another with no side effects. An Undo system that does something other than actually Undoing is no use at all. This is an ideal situation for unit tests; I recommend that you do as I say, not as I do, as Kensei.CommandStack doesn’t come with a unit test suite. Sorry.
• It may be tempting to say, “aha! To Unexecute, I simply need to restore the game world state to what it was before I Executed. So if I just store the whole world state in my command objects, that will be all I need to be able to Unexecute!” Don’t be tempted. It’s not unreasonable for game world data to approach 5MB in a simple game; in a more complex game, it can be 100MB or more. (Compare that to the 20 bytes or so required for SetPlayerHealth, above). Just ten “Executes” later, with each storing the whole game state, and you’ve used 1GB of memory in your CommandStack. Even if you don’t run out completely, Windows will eventually start swapping memory with the hard disk and your tool will be unusably slow. Not an improvement. Store the changes (deltas) only, and change only what you need to, to Execute and Unexecute.

I hope this article has helped. I’d love to hear your comments if it did, or, even more so if it didn’t. I wrote at the beginning that there is more work in this than the “naive” way of writing an Editor, and that is true. But I hope you can see that it is not that much more work, and given how simple it is to understand, implementing infinite Undo/Redo in your tools is well worth the effort.

“In the midst of this our mortal life, I found me in a gloomy wood, astray.” – Dante

The Path to Wisdom

I’d originally been planning to show off the the (very!) early stages of my game, Pandemonium, today; but I hit a minor bump in the road and it’s not quite ready for showing yet. (Top tip: if CLRProfiler.exe keeps displaying a “Waiting for the .NET Framework” message when you run your game, then keeps crashing when you shut it down, try running as Administrator. Took me hours to work that one out…)

Instead I think it’s a good time to tell you about the Five Books You Must Own If You’re Writing XNA Games…

Game Coding Complete, 2nd Edition (Mike McShaffry)

You might think this a strange selection as XNA doesn’t even get a mention, not even once. All the code samples are in C++ and DirectX. So why is it the first book on the list?

The very simple reason is that whether you use C++ or C# is, really, just an implementation detail (though obviously, C# is better! . What you will get in this book is a massive 884 pages absolutely jam-packed with valuable information about games, making games, and the games industry from someone who has been there and done it all. I struggle to think of a single topic (XNA excepted) that the author doesn’t discuss with familiarity and wisdom, in an easy style. Design? Coding? Scheduling? Testing? Algorithms? Maths? Graphics? Audio? Networking? 2D and 3D? It’s all there. This is, I believe, the single best book about games development that has ever been written. You need it on your bookshelf – no, on your desk, within easy reach.

Microsoft XNA Unleashed (Chad Carter)

The range of XNA books available is slowly growing, and I’ve got most of them. (Not all of them are good. I’ve been through the pain of reading some of them so you don’t have to…). I keep coming back to XNA Unleashed for a few reasons. Firstly, it covers pretty much the whole range of what XNA can do, and takes great care to ensure that the earlier stages are described in great detail, making it an ideal first purchase if you’re just getting into XNA. The second reason is that the supplied code samples (on CD and downloadable) are particularly clear and well-written, and include a whole bunch of classes and methods that you’ll find yourself coming back to time and again. Recommended.

XNA Game Programming Recipes (Riemer Grootjans)

I wasn’t sure what to expect from this book when I ordered it. The idea of arranging the contents into recipes just seemed a strange way of doing it, it didn’t seem to match up to the way I read books.

All I can say is that it is a format that works brilliantly. The author writes in a style that is clear, simple and to the point. (This does mean the absolute basics don’t get treated in as much detail as in XNA Unleashed, but it is otherwise a much better book for intermediate or advanced coders). Each recipe is either self-contained or references the work that it builds upon, and there are some stunning ideas here, all explained very clearly and concisely.  It’s the kind of book you can flick through, open up a random page, read for a bit and think: “…that’s brilliant!” Even on my first skim through I noticed a couple of recipes that exactly solved issues I’d been pondering in my head for a while.

If I had to pick fault, it would be that the book could have used more diagrams. This is a minor complaint, though, because the whole point is to run it and see it in action – then use it yourself in your own games. I would also have liked to see more discussion about audio. Still, as a handy reference to remind you of how specific XNA concepts work, or as a source of inspiration for truly imaginative solutions to realistic problems, this book succeeds on almost every level.

Effective C# (Scott Wagner)

There’s a well-known saying in coding circles: “C is powerful but lets you shoot yourself in the foot. C++ lets you re-use the bullet”. Fortunately Effective C++ by Scott Meyers has come to be considered a compulsory text for any C++ programmer, because it shows you how to use the language properly.

I am very glad to say that the author has achieved with this book what Meyers did for C++. It is, simply, the definitive guide to developing C# code that doesn’t just run, but runs correctly, effectively and efficiently. I found it particularly appropriate for someone like me, coming to C# from a C++ background. I look back at the early code I wrote when I started playing with XNA and I wince – I’ve learned since then, and it was this book that taught me. To spell it out: if you’re writing in C# and haven’t read this yet, your programs are probably fundamentally flawed.

The only drawback is that it only covers .NET 1.1, and the framework changed a lot in 2.0 and later. I am reliably told there is a second edition due later this year that will cover newer versions. I will be buying the second edition as well. (This, and Game Coding Complete mentioned above, are the only books I’ve ever been able to say that about).

Develop and Game Developer

I know, technically these are magazines not books, but to make up for it I’ve listed two.

Together these publications cover just about everything that is new in the games business. Particularly of note are the Post-mortems in Game Developer, which provide a uniquely in-depth insight into the challenges that particular games encountered during their development. Both magazines also provide extensive job listings and offer regular columns on each of the major games development disciplines: coding, design, art, sound, and business.

If you’re a professional in the games industry (or possibly a student? not sure) you can subscribe for free if you live in the country of publication, but both are worth the subscription price even if you can’t take advantage of that offer.

I hope this helps – I know when I get started with something I’m always keen to buy a few books so I can gain the benefit of experience without quite as much trial-and-error as having to make mistakes on my own. With XNA in particular, there are a few books that really aren’t that useful; I don’t think it would be fair to call them out, but with any luck these recommendations of which ones to buy should help keep you from wasting money on turkeys.

(Disclaimer: I have no connection in any form to any of the authors or publishers of these books or magazines).

“A clever fighter is one who not only wins, but excels in winning with ease.” – Sun Tzu