Pandemonium

December 6, 2008

Foundations

Filed under: Games Development,XNA — bittermanandy @ 12:04 am

Wow, what a time. After having barely played a game all year, I spent weeks playing Fable 2 (very good, though could have been even better), and I’m about to get cracking on Banjo Kazooie: Nuts and Bolts (the last game I worked on before leaving Rare) and Banjo Kazooie XBLA. I’ll be getting Tomb Raider Underworld and the new Prince of Persia for Christmas, and sometime in January, I’ll be surrendering every waking moment to Empire: Total War (seriously – I plan to take a week off work when it comes out). I’ve not so much as had a spare second to check out the games on Xbox Live Community Games, which probably makes me a terrible hypocrite who will be punished in the afterlife. Frankly it’s beyond a joke. I don’t even have enough time to play all of these games, all coming out within two months of each other – never mind time for my other hobbies!

Fortunately the other day I managed to squeeze in a few hours with XNA and spent the time fixing up a few odds and ends I’d been meaning to look at for a while. Specifically, I rearranged the structure of my engine and game code, for clarity and useability. It’s well worth emphasising a couple of points here: first, while I’ve built up this structure after a fair bit of experience with professional game engines and a lot of careful thought, there are undoubtedly other ways of doing it. I encourage you to hunt around, look at what others have done, and deeply consider what will work for you. Second, this is very much a work in progress. I will undoubtedly be tweaking things yet further as time goes by, and I’ve had to make a few decisions “blind” – I’m pretty sure they’re the right choices but it’s entirely possible I’ll reverse them later as experience dictates.

So. At the highest level, the XNA code I write is split into two general groups: engine code (shared code, usable by more than one game, written in such a way as to be reasonably general purpose without being so abstract as to be useless); and game code and assets (specific to one particular game, though sometimes code can start life here before being moved into the engine later). In an ideal world, all your code would exist in the engine, and there would be no game-specific code at all. Games would be differentiated only by their assets – the single executable built by the engine would search for an XML file or similar, that would define what other assets to load, and by a combination of scripting and data-driven design, the game would run with no unique code at all. Realistically, that’s not an achievable goal, and a game will always have game-specific code. The trick is to find a balance.

(First a plea. Don’t “write an engine”. The XNA Framework is so close to being an engine that such a task is pointless, and you’ll also find that no-one will use it – because you won’t have solved the problems you’d encounter if you wrote a game. Hobbyist DirectX programmers all set out to write an engine, and barely any of them ever complete a game. In fact, it wouldn’t be bad advice to write a game first, then only when it’s finished, go back and look at how it can be separated into engine and game code, ready for your next game).

Anyway, let’s have a look at the folder structure of the engine and the games in turn. You’ll want to store all your stuff in a single folder – mine are in Documents\Code\XNA – Visual Studio will encourage you to put everything under the Documents\Visual Studio 2008 folder, but that’s a really bad idea because when Visual Studio 2010 and XNA 4.0 come out you’ll just have to move everything, which apart from being generally inconvenient can be a royal pain if you’re using source control!

Engine Folder Layout

My engine is codenamed Kensei, and is therefore stored in the Kensei folder. Here, you will find the Kensei.sln Visual Studio solution file, as well as my Kensei.FxCop project (with CustomDictionary.xml) and a couple of handy batch files which I’ll come to in a minute. Everything else is stored in subfolders, corresponding to the code projects that are referenced in Kensei.sln (which in turn correspond to the assemblies that can be used by my games).

Kensei – the “core” project containing the majority of the code I write. In here exists my audio engine, camera management code, handy development utility code, command pattern implementation, and everything else I’ll come up with as I add features. Each logical unit is stored in a subfolder of this folder, so for example my audio code exists in Documents\Code\XNA\Kensei\Kensei\Audio – but it’s all built together in a single C# project called Kensei. (I’ve worked on games where each logical module had its own project. It just meant we ended up with fifty-odd projects and fifty-odd assemblies where one would do. It might make it easier to keep things decoupled, but adds headaches via complexity, especially as C# makes circular references awkward or worse. Don’t bother).

Kensei.EmbeddedResources – there are a few things in the Kensei engine that require specific assets. For example, Kensei.Dev uses a font for simple text and a shader for simple shape rendering. At first, these assets had to be copied into the content project of each game prototype I created but that quickly became a pain. The answer (as Shawn Hargreaves describes expertly) is embedded resources.

Kensei.Pipeline and Kensei.Pipeline.Runtime – XNA makes working with assets an absolute dream and most non-trivial games will extend the content pipeline to support custom asset types. Kensei.Pipeline extends the content pipeline itself, while Kensei.Pipeline.Runtime acts as a bridge between the content pipeline and the game – again, Shawn explains in full.

DebugContentBuild – it’s an inevitable fact that all code has bugs, so I followed Stephen’s advice and created a project that vastly simplifies the process of debugging my content pipeline extensions. Basically, running this project attaches to MSBuild.exe and runs the content pipeline on the assets in its content project – allowing me to step into my Kensei.Pipeline code and see where it’s going wrong.

(Other Code Projects) – one of the things I love about the XNA community is the easy willingness with which people share their work. Make use of it! I don’t want to spend my hobby time reinventing the wheel or rewriting something someone else has already done perfectly well (unless, of course, it’s the main differentiating point of my game, or something I have a specific interest in doing or learning. Always in-source your USPs, but don’t even attempt to do everything yourself!). Therefore, my Kensei engine solution contains projects for JigLibX, the Curve Editor Control, and a few other public components, as well as code supplied with various books I own. Which books should you own? Start here!

External – of course, some code isn’t supplied with source, but as a standalone assembly. For example, I use Perforce for source control (accept no substitute! It’s the best there is, period – it’s so good, I use it even though I don’t have to, and I’ve not yet come across any other source control I’d say that about) and the rather wonderful P4.NET lets me write the ability to work with Perforce into my tools. This allows my game editor to automatically check out assets that I work on, for instance. This, and other such assemblies, are all stored in the External folder (with a handy text file so I don’t forget where I got them from…).

Now, with all these projects and assemblies floating about, each built into their own build directory, finding them from the individual games becomes awkward! It’s much easier when all the engine assemblies are in the same place. With this in mind, I wrote the following PostBuild.bat batch file. Every project in the solution runs this as a post-build step (details in the script itself). It copies the output files of the project into the bin folder, so when I create a new game, I only have to look in that one folder for all my references.

@echo off
rem ————————————————————————
rem Run this script as a Post Build Step from every Kensei library to be
rem used by a project or game outside of the Kensei solution.
rem It will place all built files into solution\bin\platform\configuration.
rem This will make them easier to find when adding references.
rem ————————————————————————
rem Post Build Step:
rem
rem call “$(SolutionDir)PostBuild.bat” “$(SolutionDir)” “$(PlatformName)” “$(ConfigurationName)” “$(TargetDir)” “$(TargetName)”
rem ————————————————————————

echo —- Post Build Copy —-
echo Copying %4%5.* to %1bin\%2\%3…
xcopy   %4%5.*   %1bin\%2\%3   /Y   /I   /F

Finally, with all these assemblies floating around, it’s nice to be able to clear everything out and know that a build will start completely afresh. (Why does “Clean” do nothing on a C# project? Alright, Visual Studio handles C# a hundred times better than C++, but I still sometimes want to know that everything that isn’t a source file has been deleted). So I wrote a CleanAll.Bat (warning, use at your own risk, especially if you make use of folders named ‘obj’ or ‘bin’):

@echo off
rem Script to completely start afresh without *anything* still hanging around from previous builds.
rem This is useful to, for example, check what would happen if it was checked out on a clean machine.

echo ——————————–
echo Delete Visual Studio local files
echo ——————————–

del /Q /S *.csproj.user
del /Q /S /AH *.suo

echo ————————————
echo Delete intermediate and output files
echo ————————————

FOR /R %%i IN (*) DO IF EXIST “%%i\..\obj” rmdir /Q /S “%%i\..\obj”
FOR /R %%i IN (*) DO IF EXIST “%%i\..\bin” rmdir /Q /S “%%i\..\bin”

echo.

That’s pretty much it for the Kensei engine (well, more or less, as you’ll see in a moment). After seven or eight years with playing around with Visual Studio and trying to arrange large game projects in a way that preserves my sanity, I’m pretty confident that this layout is the easiest to work with.

Game Folder Layout

The whole point of the above is to make creating new games prototypes quick and easy. I’ve got ten or twelve game ideas in my head right now, although Pandemonium remains my main focus, and every time I get a new one, I like to be able to spend one evening slapping something together that just works with a SpriteBatch (for 2D) or Kensei.Dev.Shapes (for 2D and 3D) without spending any time at all plugging the lower-level stuff together. So, all my games exist in the Documents\Code\XNA\Games folder, and reference assemblies in the Documents\Code\XNA\Kensei\bin and External folders. Taking Pandemonium as an example, Documents\Code\XNA\Games\Pandemonium contains (as you’ll have come to expect) the Pandemonium.sln Visual Studio solution, and the Pandemonium.FxCop file (and CustomDictionary.xml) as well. Then there are the following projects in the following folders – again you should be able to guess what they are, and they should be fairly self explanatory when read in combination with the above:

Pandemonium – the game itself. Of course, the content project and all assets are stored in here too.
Pandemonium.Pipeline – content pipeline extensions for the game.
Pandemonium.Pipeline.Runtime – a bridge between the two.

It really is as simple as that, and if I were just writing a quick prototype there would probably only be the first one.

Engine.Game

There’s one project I didn’t mention above in the Kensei engine. I’ve only just started doing things this way, and although so far it’s working well I’m a little concerned it may prove inflexible in the long run. I’d encourage you to experiment with it – after all, if a particular game finds it hard work fitting into this model, just don’t use it. It will save you a lot of time on those games that work alright with it.

Basically, I’ve created a project named Kensei.Game, and it contains a single class, Kensei.Game, which derives from Microsoft.Xna.Framework.Game. (Unimaginative names, I’m afraid…) If you’re wondering why it’s a separate project and not part of the Kensei project, that’s because it uses code from all the different assemblies and it was easier to keep it apart than try to prevent them from referencing one another in a circular manner. The idea is, that all of my games can inherit from Kensei.Game instead of (more directly) Microsoft.Xna.Framework.Game, and Kensei.Game can handle all of the things that each individual game would have to do for themselves.

Allow me to provide an example. Kensei.Game provides the following (sealed!) implementation of the Draw method: 

/// <summary>

/// This is called when the game should draw itself.

/// </summary>

/// <param name=”gameTime”>Provides a snapshot of timing values.</param>

protected sealed override void Draw( GameTime gameTime )

{

    // Prepare the game for drawing

 

    if ( Kensei.Dev.Options.GetOption( “Draw.Wireframe” ) )

    {

        GraphicsDevice.RenderState.FillMode = FillMode.WireFrame;

    }

 

    if ( Kensei.Dev.Options.GetOption( “Draw.CullModeNone” ) )

    {

        GraphicsDevice.RenderState.CullMode = CullMode.None;

    }

 

    Color clearColour = Color.Black;

 

    if ( Kensei.Dev.Options.GetOption( “Draw.CornflowerBlue”, Kensei.Dev.Options.BehaviourIfNotPresent.ReturnTrue ) )

    {

        clearColour = Color.CornflowerBlue;

    }

 

    GraphicsDevice.Clear( ClearOptions.Target | ClearOptions.DepthBuffer, clearColour, 1.0f, 0 );

 

    GraphicsDevice.RenderState.DepthBufferEnable = true;

    GraphicsDevice.RenderState.AlphaBlendEnable = false;

    GraphicsDevice.RenderState.AlphaTestEnable = false;

 

    // Perform game-specific drawing

 

    DrawGame( gameTime );

 

    // Complete drawing

 

    Kensei.Dev.Manager.Draw( GraphicsDevice,

        WorldViewProjectionMatrix,

        Window.ClientBounds.Width,

        Window.ClientBounds.Height ); 

    base.Draw( gameTime );

}

 

/// <summary>

/// This is called when the game should draw itself.

/// </summary>

/// <param name=”gameTime”>Provides a snapshot of timing values.</param>

protected abstract void DrawGame( GameTime gameTime );

What this means is that each individual game I write has much, much less to do in it’s (abstract) DrawGame method. I can’t forget to call base.Draw, and I always get the benefits of the same Kensei.Dev options and Kensei.Dev.DevText and Kensei.Dev.Shapes drawing – completely for free. The DrawGame method in Pandemonium is simply this (admittedly, it’s very much work in progress and there’s not much to draw yet, but you should get the idea of how little code there is to write per-game, or per-game-state when it’s a little more developed):

/// <summary>

/// This is called when the game should draw itself.

/// </summary>

/// <param name=”gameTime”>Provides a snapshot of timing values.</param>

protected override void DrawGame( GameTime gameTime )

{

    m_background.Draw( m_camPosition, ViewMatrix, ProjectionMatrix );

    m_angel.Draw( m_camPosition, ViewMatrix, ProjectionMatrix );

}

Update, Initialize and other methods are similarly simplified for the individual game, with as much complexity and common behaviour as is feasible moved into the engine.

Incidentally, when you have a base method that needs to do things before and/or after the derived method, I much prefer this pattern of having the base method (Draw here) as public, and have the base method call into protected virtual (or abstract) methods (DrawGame here). This is much superior than having a single protected function and relying on your more-derived class to remember to call the base version of the method. It’s too easy to forget, and too easy to get subtle bugs as a result! Of course, there is a cost. It may be, as I develop my games, that I discover I need to call some game-specific code after base.Draw (or base.Update, or what have you). That would become difficult or impossible in this pattern. If things turn out that way, I’ll have to change it. For now, I’m enjoying the simplicity of my game-specific methods. I can, quite literally, create a new XNA project, change the main class to inherit from Kensei.Game instead of Microsoft.Xna.Framework.Game, and I’ve got all the functionality of my Kensei engine instantly available. It’s very powerful.

(My original plan was going to be even more powerful; I’d create a simple project using Kensei.Game, and with all the necessary references to the Kensei\bin assemblies, and use that as the basis of a Visual Studio Project Template. Then, when creating a new project, I’d select “New… Kensei Game” instead of “New… Microsoft XNA Framework 3.0 Game”. It nearly worked! Everything was perfect, in fact, except that the Export Template wizard didn’t pick up the content project. That’s very unfortunate, and means I can’t use this trick as I’d wanted, but if anyone knows a way around it, please let me know.)

I think that’s enough for now. I hope what I’ve written makes sense, and gives you some ideas on how to structure your own engine and games, code and assets. I don’t claim for a second that this is the only way of doing it, and in fact if you’ve got some better ideas, I’d love to hear about them – leave a comment. This has been a fairly high-level view of the problem and my solution to it, and there are so many tradeoffs to be made at a lower level – something for a future article perhaps, though I’ve tried a number of different structures over the years, and have heard of still more without trying them out yet, and have yet to settle on a favourite. Something to experiment with, for sure. Anyway – let me know if this was useful to you.

Advertisements

9 Comments »

  1. Quick question.

    Why are your game projects referencing the engine’s binaries (Documents\Code\XNA\Kensei\bin) instead of just referencing the engine’s solution files directly?

    Comment by ian — December 6, 2008 @ 1:28 am | Reply

  2. Thanks for the article. It’s good to see what other people are doing. I hadn’t noticed the curves editor before I’d read that. That could be very useful to me. I’m thinking about getting Effective C# on your recommendation. Additionally I hand’t thought about embedding perforce support into my editor. That could be very nice especially since it’s not supported in Visual C# Express.

    Comment by Dale — December 6, 2008 @ 3:48 am | Reply

  3. Hey Andy,
    It so happens that I do know how you can create proper templates for XNA Game Studio projects. (I am also keenly aware of the lack of extensibility in the template export wizard that forces you to follow these manual steps…) This post was written for XGS 2.0, but XGS 3.0 supports the 2.0 templates so the steps are the same.

    Template Article

    Comment by Stephen Styrchak — December 6, 2008 @ 6:08 am | Reply

  4. Thanks for the comments guys.

    > Why are your game projects referencing the engine’s binaries (Documents\Code\XNA\Kensei\bin) instead of just referencing the engine’s solution files directly?

    Mainly because then I only have to build them once. Ten or twenty game ideas in my head => ten or twenty game solutions => building the engine assemblies ten or twenty times if the engine source is included in the game project! It also makes it less likely that game-specific code will spill over into the engine code, and means that any given game solution has only four projects (three plus one for Xbox), not several tens.

    The other method is entirely workable, of course, and has the advantage that the assemblies are even easier to find (you can add a Project Reference, instead of Browsing for them); I just prefer doing it this way.

    Comment by bittermanandy — December 6, 2008 @ 10:57 am | Reply

  5. Keep up the good work, I especially like the idea of abstracting your engine from the game itself, a different view of extending the XNA framework.
    Look forward to more.

    Comment by Simon (Darkside) Jackson — December 8, 2008 @ 12:40 am | Reply

  6. A very interesting read, thanks. I’m going to be playing with XNA properly soon (once I’ve cleaned a few other projects off my plate), so these articles have been very useful.

    Comment by Muttley — December 10, 2008 @ 3:16 pm | Reply

  7. Just a follow-up to Stephen’s note about project templates; he’s written a tool for exporting XNA 3.0 game projects to templates, greatly simplifying the process, which can be found here.

    Comment by bittermanandy — December 22, 2008 @ 12:13 pm | Reply

  8. Hi andy,

    I’ve been following your posts for a while now. Really enjoy the depth that you go into, don’t worry about frequency of posts its the content that counts!

    I’m about to start in earnest with my own game ideas and would like to use your game engine framework as the basis, should i download the last post of your source, around August last year I think, or will you be releasing an updated (restructred as per this article) version soon?

    Thanks,

    Justin

    Comment by Justin (FatalFrenchy) French — February 27, 2009 @ 12:25 pm | Reply

    • Hi Justin, thanks for the good feedback (and as it happens I’ve had a draft article lined up for a week or so, just not got around to polishing it yet!).

      I’ve never actually released the Kensei engine proper, nor am I ever likely to, because I won’t be able to support it, and can’t guarantee it will be useful to anyone but me. What I have done (and will continue to do) is release individual components as I finish them, which people are more than welcome to use. I think you’re referring to my Kensei.Dev library – feel free to download it and use it or not as you wish, but be clear that it’s not a game engine or even anything close, just one component that a game engine would really benefit from.

      One of the major reasons for this is that I don’t want to write an engine, I want to write games. So the common, game-agnostic code that I write, I call the “Kensei engine” but really it’s just the minimal subset of an engine that is needed to support the games I write. It might be inadequate, incomplete or inappropriate for the games other people write so I’d rather let them do their own thing and use my advice, or not, as they wish – rather than attempt to use a large monolothic engine that probably wouldn’t do what they want.

      It’s entirely practical to write games direct from XNA, or by developing an “engine subset” like I’m doing, or by using someone else’s continuously-developed and well–supported engine of which there are several available (if you’re a paid-up XNA CC member you can even use the TorqueX engine for free I believe). Those are by far your best options rather than relying on me!

      Good luck with your game ideas, let me know how you get on.

      Comment by bittermanandy — February 27, 2009 @ 6:54 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: