Battlefront III TDD

Wii version available here.
TDD.png
==Overview==
===General Description of Game Design===
The game is a first-person/third-person action title that pits factions against each other in famous battlefields featured in the Star Wars movies. The player has a wide range of abilities and choices over how to fight, such as on-foot with a selection of weapons or lightsabre, in a ground vehicle, or soaring over terrain and space in a fighter-craft. The game introduces the Vertical Battlefront, which allows players to seamlessly travel between epic battles occurring simultaneously on ground, in space, and across the decks of whole capital ship interiors.
The main design goals that will have an impact on technology are:
*Seamless yet concise transition from ground to space without disempowering the player
*Visual feedback of events occurring on opposite sides of the atmosphere
*Large numbers of units fighting in the battles at all times
*Vehicles requiring expansive outdoor environments
*Capital ships moving through space whilst units are onboard
===Technical Requirements imposed by Game Design===
The world size will be large, as it encompasses both a planet’s surface and the area of space above it. In space, the world needs to be big enough to accommodate the sheer scale of capital ships. These ships can be up to 1km in length and possibly moving, albeit relatively slowly. The player will be able to see a large section of the planet’s surface from space, but most of this can be modeled simply. Only the playable part of the surface, which the player will be flying over to land on and embark, needs to be modeled in detail.
The playable ground section does need to be big enough to accommodate flying craft traveling at high speeds however. More specifically, players should be able to fly in a particular direction for at least 15 seconds without hitting a world boundary, as otherwise the playing area will feel too restrictive.
This requirement means that the ground world size will be several kilometers, big compared to most games, and the space world size will be even more. The precision of floats may become an issue.
There needs to be a high number of characters and vehicles populating the battlefield to immerse the player. The scope of the vertical battlefront means that multiple battles are occurring simultaneously in different places, and the game still needs to feel heavily populated by units. Each unit requires processing power, so the issues involved here will be focusing the active AI units around the players and simulating events happening outside the players’ locality, to give the impression that the entire world is busy.
The high number of units to be rendered will need level-of-details (LODs) generating, either automatically or manually.
The ability to seamlessly travel between the ground, space, and inside capital ship interiors means that there will be a high amount of geometry and textures required for each level. In order to provide the best possible visuals at any one part in the level, geometry and textures will have to be streamed into memory from the storage device. The speed of the transition between these areas is limited by the speed of the flying craft, which will help control the streaming issues. However, the game is an action title, so it is imperative that the transition from ground to space does not take too much time.
Capital ships will contain whole interiors containing units, command points, and AI navigation information. One design goal is that capital ships can move around space and this will require moving, or “appearing” to move, all the objects contained on the ship.
Capital ship interiors can be kilometers in length, which can be traversed quickly in a space craft but would take too long on-foot. In order for the player to experience key locations spread throughout the ship in a fun manner, it will be necessary for the distances involved whilst inside the ship on-foot to be shorter than in reality. However, where there are windows or portals to the exterior, the view needs to be representative of what would be seen if the distances had not been shortened.
Split-screen play requires that up to 4 players can be in different sections of the vertical battlefield on one console. However, this severely limits the advantages of streaming, which will be used extensively in single-player, as 4 times as much memory and disc access can be required. It also creates a potential need for more AI units to be active at a time, to populate the different parts of the environment the players can habit, as opposed to focusing the AI around just one player.
Likewise, the online multiplayer mode allows players to be spread over the whole vertical battlefront. This isn’t such a problem for streaming, as each client’s machine only needs to stream the data needed for that client’s view. However, there is still the need for more AI units to be active when the players are spread out. Additionally, the actions of each separate client need to be transmitted to the server, and the server needs to send information to each client. The more players and AI units active, the higher the bandwidth required to do this.
===Hardware Specifications===
PS3 - Standard consumer console unit.
*CPU - Cell: 1xPPU (dual core) + 6xSPU @ 3.2Ghz
*GPU - RSX: core @ 550Mhz, mem @ 700Mhz
*500+ Mverts/s
*8+ Gpixels/s
*1920 x 1080 max res (1080p)
*256MB XDR (main mem) up to 25.6GB/s
*256MB GDDR3 (video mem) up to 22.4GB/s
*Blu-ray drive (up to 54Gb dual layer) (DVD/CD support)
*Bluetooth wireless controllers (up to 7)
*Wi-Hi
*2+ USB slots
*1Gbit ethernet
*Hard disk support (2.5")
*Memory stick (possibly SD and compact flash too)
*PAL/NTSC (SD)
Xbox 360 - Standard consumer console unit.
*CPU - 3xPPC (dual core) @ 3.2Ghz
*GPU @ 500Mhz
*500 Mverts/s
*4 Gpixels/s and 8 Gtexels/s
*1920 x 1080 max res (1080i)
*512MB UMA (main mem) up to 22.4GB/s
*10MB EDRAM (video mem) up to 32GB/s (or possibly 256GB/s)
*12x DVD drive (up to 9Gb usable dual layer)
*Wireless controllers (up to 4)
*2 controller ports
*10/100Mbit ethernet
*Hard disk support (2.5")
*64Mb (initially) memory units (MUs) x 2
*PAL/NTSC (SD)
On both PS3 and Xbox 360, the intent is to support High-definition at 720P at 16:9 aspect ratio, using 1280 x 720 resolution. 4:3 aspect ratio will be supported for legacy displays. It is expected that 2XAA will be used, and alternative anti-aliasing methods will be explored.
===Development Environment===
Typically, PCs with a spec close to the consoles above are used for development. Playstation Dual Shock 2 controllers are used with USB converters as control devices.
Xbox 360, Playstation 3, and Wii development kits are also allocated to some team members. Playstation 3 debug kits are also becoming increasingly useful for development and these will be allocated to a many team members.
Operating Systems
*Windows XP
*Linux Fedora Core 6
Programming Languages
*C and C++ used primarily
Programmer Tools
*Sourcecode editor: Vim 6.4 - 7.1
*Compiler: Gcc 4.1.1 (Linux)
*Compiler: Gcc 4.1.1 (Playstation 3)
*Compiler: Microsoft 32-bit C/C++ Optimizing Compiler Version 14.00.6502 for PowerPC (Xbox 360)
*Compiler: Radix Version 4.2 build 125 (Wii)
*Compiler: Microsoft cl 14.00.50727.42 (Windows)
*Debugger: Gdb 6.5.8 (Linux)
*Debugger: SN Pro DG 2.8.32.6 (Playstation 3)
*Debugger: Microsoft Visual C++ 2005 service pack 1 (Xbox 360)
*Debugger: Codewarrior IDE version 5.7.1764 (Wii)
*Debugger: Visual Studio 2005 (Windows)
Source Control
*Sourcecode: SVN 1.4.5
*Assets: Perforce
Infrastructure
All artists work on Windows machines. There are additional machines reserved for lightmap generation.
Backups are made nightly of the work partition of every team members machine.
The programmers use Vim for source editing, as it works under both Linux and Windows. Although it requires some initial learning to get familiar with the keyboard commands, it can be very fast to use after this. Visual Studio is not used for editing on Windows because there are incompatibilities between the two applications that cause formatting issues.
===Multi-platform development plan===
Xbox 360, Wii, PC, and Playstation 3 platforms are to be developed concurrently, for release at the same time. All game code is platform independent, and utilizes the core engine technology, parts of which are specific to particular platforms. All assets are exported in a platform independent format, and converted to the appropriate platform as the last stage of the asset pipeline.
===Coding standards and code reviews===
The coding standards document is a set of rules on our Wiki that all the programmers adhere to (see Appendix). It considers code readability and ease of maintenance, as well as common or particularly nasty errors to avoid. New code modules are reviewed by lead or senior programmers as they are committed, and by peers as they use the module.
===Major project dates===
:
Major Milestone	
Date	
Duration
Treatment	
June 2006	
1 month
Pre-TDR	
July 2006	
1 month
DDR	
July 2006	
1 month
TDR	
August 2006	
1 month
Prototype(s)	
July 2006	
2 months
FPP	
December 2006	
5 months
FPM	
March 2007	
3 months
E3 Demo 2007	
April 2007	
1 month
Pre-Alpha	
February 2008	
10 months
Alpha	
March 2008	
1 month
E3 Demo 2008	
April 2008	
1 month
Beta	
August 2008	
3 months
Gold	
October 2008	
===High-level engineering schedule===
:
Deliverable	
Date
VBF Prototype	
July 2006
Xbox 360 (1st build)	
August 2006
Network Prototype	
September 2006
PS3 (1st build)	
October 2006
NTSC & PAL (1st build)	
February 2007
Online Beta Test	
August 2007
Pre-certification	
June 2008
Note: Longer submission times (6 weeks) will be needed due to new TRC’s and TCR’s of the next-generation platforms.
==Top Technical Risks==
===World size===
One of the technical requirements is the need for a large ground area that from space appears to be a planet. This has not been done before with the current engine, so is likely to involve some work.
The planet will be modelled as a section of a larger sphere using a low number of polygons, which will give the desired impression from a distance. An area of this will be modelled in high detail, and this will be the area that players can travel towards in spacecraft. For ground areas, a world size of 4km x 4km is proposed. Ground troops will be restricted to the center 1-2km sq, allowing higher detail in this area. Furthermore, the outer 500m of the world will be for rendering purposes only, with no units able to traverse this area.
A space world size of 4-8km sq is proposed, 4km above the surface, with a further 2km above this playable.
The large scale may cause world-coordinate and z buffer precision issues towards the edges of the map.
Mitigations:
*Pre-production will involve a prototype using the proposed world size. This will also allow any other concerns with the world size to be identified.
Note: It would be possible to split the ground and space up, and keep each in their own space, with a local origin, putting the point of least accuracy at the mid-point where it’s needed the least. However, in space combat inaccuracies are less noticeable because of the minimum speed that objects can move at. It was found that the inaccuracies at that range are only noticeable on-foot, so it was felt preferable to use transformation portals on capital ship hangars to transform the player back near the origin. Transformation portals would be used in any case for moving capital ships, and so one world space for both ground and space simplifies a lot of systems.
===High number of units===
It is proposed that 50 units will be active at any one time, split between soldiers, ground vehicles and fighter craft as appropriate, focused around the players. This is a risk, because with advanced AI and animation, each unit impacts performance and to a lesser degree memory usage. In order to keep the battle seemingly busy over the entire vertical battlefront, the game will simulate higher numbers of units by dynamically deactivating units far from the player, and activating new units near the player.
Over split-screen and networked games, this is even more of an issue, because players can be spread over the vertical battlefront, hence many more units will need to be updated and broadcast. On network games with non-dedicated servers, this may involve distributing the workload over clients, which is not a simple task.
Mitigations:
*A large number of units will be tested in the game early on, to give us time to optimise where necessary.
*We will also be making level-of-detail algorithms integral to the game, both in rendering, and in other systems such as AI, navigation and animation.
*Split-screen is being implemented early on, and once implemented will continue to be tested regularly.
*The network prototype is due early in the schedule in order to find and address problems quickly.
===Editable Terrain===
Several levels will require a terrain, so the editable terrain system needs to provide a large ground area for players to traverse in vehicles, and to be viewable from spacecraft in orbit. It needs to allow artists and designers to create, populate and tweak large areas efficiently. This will require new tools and a new build process to be established.
Certain terrain levels require a high density of static objects in the environment, such as trees, rocks or buildings. Rendering and storing these in an efficient manner will be an additional task.
The terrain should also make full use of next-generation rendering techniques to ensure the game stands out. These requirements mean the terrain system needs a lot of work and tweaking, which makes it a risk.
Mitigations:
*It will be developed early, and used to generate the terrain in the pre-production prototype, so any problems with the process can be identified immediately.
*Key personnel will be developing it.
===Terrain Rendering===
Megatexturing is a next generation texturing technique, and is also being used in Quake Wars: Enemy Territory. It allows artists to paint unique textures across the whole terrain. It is lower pixel density than traditional texturing techniques, but detail maps can be applied at a higher resolution.
The mega-texture will be compressed on the disc to speed up load times, and decompressed in a separate thread.
Lightmaps are not currently baked onto the mega-texture. Lightmaps are applied in the lightmapper which is run offline (as lightmaps can take several hours to complete). If static objects are moved in the art editor, the lightmapper can be ran again and it automatically bakes the new lightmap.
Moving static objects will have an affect on the art pipeline, as the mega-texture around the object may need to be repainted (e.g. if a roadway was painted, and a building moves, the roadway may need to be moved too).
One disadvantage with the mega-texturing is that uv coordinates are distributed uniformly across the terrain from a topological point of view. This means that steep slopes have a lower pixel density than flat slopes. These are particularly obvious because they tend to be perpendicular to the players view direction.
Mitigations:
*High density overlay textures will be used to give high detail in particular areas, usually those where players traverse the most. This requires both terrain editor and terrain rendering code, and will be completed for MS10.
===AI Planning===
The AI Planning system is currently designed to work in single player, and is story based. We will need to change it over to work in multi-player environment with different game modes. The AI needs to be tweaked throughout the development of the game.
Mitigations:
*This task is being started early on in development.
*Reusing components from the existing AI framework will help to make this task less time consuming.
===Texture Streaming===
The large number of textures required for the entire ground and space battle means that it will be necessary to stream them in as the player moves around the game area. This can be difficult since each texture needs be streamed in before it needs to be used. It also presents a challenge in 4 player split screen, since each of the players can require a separate set of textures.
The main problems involved are:
*Predicting which data needs to be streamed in.
*Whether the amount of data needed can be streamed in quickly enough.
*Fitting 4 separate sets of textures in memory for split-screen play.
Mitigations:
*A low resolution version of every texture for the level will be kept in memory, so textures will never appear to be missing.
*In split-screen play, the screen resolution for each player is smaller, so lower resolution textures can be used and therefore more texture sets can be allocated within in the same amount of memory.
*This task has been placed early on the schedule so the results can be evaluated.
===Transformation Portals===
The player needs to be able to traverse the capital ships in a reasonable amount of time whilst on foot. Since these can be a kilometre long, the actual distances involved in the interiors will be set based on gameplay, but portals looking out of the ships will give the player the impression they are the correct distances. Transformation Portals are used to do this, and they require some low level engine changes.
Mitigations:
*This technology is being researched and implemented early on in development.
===Performance===
Currently the framerate is too low.
Mitigations:
*A target of 20fps with 50 units in the entire VBF exists for MS11 (PC) and MS12 (console).
Framerate targets leading up to MS12:
	
PC	
Console
MS9 (January 2007)	
8 fps	
5 fps
MS10 (February 2007)	
15 fps	
10 fps
MS11 (March 2007)	
20 fps	
15 fps
MS12 (April 2007)	
-	
20 fps
This shall be achieved by profiling the game, identifying the sections of code that are the most expensive, improving these, and then re-profiling. In addition to this, there are some areas that we suspect will lead to important performance gains.
The visibility system uses portals and is geared towards indoor areas. It needs to be overhauled to suit our outdoor areas, including better occlusion integration, and knowing how to deal with visibility between ground and space.
There is scope to use more object LODs to improve rendering performance. The LODs occupy more memory though, so there will always be a trade-off. The pixel shaders can also be sped up for objects far away by turning off the the passes that cannot be noticed at a distance (eg. Normal and detail maps).
With 50 units, the animation can be expensive. We can speed this up by simplifying the animation for characters at a distance. This can involve animating less bones, using less bone weights per vertex, and timeslicing the animation.
The AI can use LOD techniques at a distance, such as simpler pathfinding, and timeslicing its decisions.
===Console-hosted network / online===
The target for MS12 is minimum 8 players over the VBF running at 20fps online.
Target network performance:
Bandwidth per player	
5kbps
Server upstream required for 8 players	
256kbps
Packets sent per player	
10 per second
Packets client receives	
70 per second
Dropped packets	
0.01%
Latency	
10 – 150 ms
===Splitscreen===
Separate maps will be needed for VBF split-screen because players on the same console can be in different parts of the VBF. This means streaming cannot be used to the same extent, and so more objects / background needs to be in memory at a time. Low-resolution versions of the textures can be used in split-screen, because the players view is smaller. However, the geometry needs to be simplified in order to fit more objects in memory, and this is a manual process by an artist The simplified geometry also has the benefit of improving the rendering speed.
It is possible that less AI will feature in split-screen for performance reasons. As the players can be in several places over the VBF, less LODing of the AI and animation can be done.
The current target for testing a VBF split-screen map is by May 2007.
==Metrics==
===Memory maps===
The target hardware has 512MB memory. The following figures are estimates only based on another game in development and future projections, as actual amounts change continuously as systems change or optimizations are made.
Main memory
:
OS	
96 MB
Game	
50 MB
Terrain heightmap	
8 MB
Vertex buffers	
50 MB
Physics	
25 MB
Sound	
15 MB
Animation	
5 MB
Other	
7 MB
Total	
256 MB
Video memory
:
Render buffers	
100 MB
Streamed textures	
100 MB
Static textures	
30 MB
Terrain	
20 MB
Other	
6 MB
Total	
256 MB
CPU usage map / breakdown
:
Rendering	
45 %
AI / Animation	
25 %
Physics	
10 %
Sound	
5 %
Network	
5 %
Game logic	
5 %
Other	
5 %
GPU usage map / breakdown
:
Terrain / environment	
25 %
Objects	
20 %
Shadows	
20 %
Clouds	
15 %
Atmosphere	
5 %
Filters	
5 %
HDR	
5 %
Effects	
5 %
===Asset budgets===
Rendering
In terms of performance, both the polygon count and the number of shader batches are important. On current PC hardware, it is typically more important to keep the batch count low than the polygon count. On console hardware this is less of a problem.
The overall goal is to render up to 300,000 polygons per frame, and up to 500 batches. However, this includes all render passes including dynamic shadows (which often require a high number of polygons, but use a cheap pixel shader).
In contrast, the pixel shader used for most opaque polygons is complex as it calculates both the ambient and the lighting for each pixel, and performs multiple other uses. This was found to be more efficient than using simple shaders but having to change the shader frequently.
Environment
The polygon budget for the environment is 40,000 – 60,000 at any one time, and around 100 batches. The environment is made up of the terrain, buildings (including static objects), and foliage. The budget is generally split evenly between these 3 systems, but if one or the other is not necessary (for example, capital ship interiors do not require terrain or foliage) this is available to the remaining systems.
However, it is worth noting that the terrain and foliage rendering is more efficient due to their specialization, so fluctuation in their poly counts do not have as much effect on performance.
The total polygon budget for the whole environment is 300,000. Up to 64 2048x2048 can be used, each with normal and specular maps.
Characters
Cutscene characters use 4000-5000 polygons, with around 80 bones for full facial animation.
In-game characters generally use 3000-3500 polygons, with around 20-25 bones. These have limited facial animation.
Both versions use a set of diffuse, normal and specular maps at 2048x2048 for the body, and a set at 1024x1024 shared between the face and arms.
4 bone weight limits can be used, and up to 4 blends. The number of blends will reduce as characters get further away.
Weapons
The 1ˢᵗ person version will be higher poly (3000-4000) due to its proximity to the camera, and because the full character does not need to be drawn.
The 3ʳᵈ person version will be 500 polys, or up to 1000 for large guns (eg. Rocket launcher). 
Both versions share a single set of diffuse, normal, and specular maps at 2048x2048.
Vehicles
The budget for vehicle exteriors is 4000 – 5000 polygons, including the 3ʳᵈ-person cockpit. A single set of 2048x2048 diffuse, normal and specular maps can be used. Certain large and iconic vehicles have special budgets set, which can be up to 8000 (eg. AT-AT, Millenium Falcon). These are allowed an additional texture.
Each vehicle has a 1ˢᵗ-person cockpit which again is 4000-5000 polygons, with a set of 1024x1024 maps.
Capital ship exteriors
The budget for capital ship exteriors is 10,000 polygons. An additional 15,000 can be added and flagged as detail, which will only be turned on when close to the ship.
2-4 sets of 2048x2048 maps can be used for the exterior.
Audio
It is likely that up to 40-50 concurrent voices will be allowed.
Physics
Up to roughly 30 rigid bodies can be active at any time. Up to 5 rag dolls are likely to be needed at one time. These figures are subject to change at a later date once the gameplay has advanced sufficiently to measure. The systems will attempt to degrade gracefully. For instance, characters will have animated deaths in addition to rag-doll, so where the physics load is high or the dying character is at a distance, rag-doll will not be used.
Various vehicles use the physics to different degrees. Legged vehicles such as AT-ATs are more computationally intensive than hover vehicles, which themselves are more expensive than flying vehicles. The more legs on a vehicle, the more expensive they are. It is expected that around 40 vehicles will be active at a time.
AI / Gameplay budgets
The game will allow a minimum of 50 units in a slice of the vertical battlefront, and simulate events in the other slices at a lower level-of-detail and hence cost. It is a high-priority goal to have more units than this, but this cannot be confirmed until a representative playable level is available and level-of-detail is integrated into all systems.
Given this number, the units could be split as follows:
	
Infantry	
Ground vehicles	
Space craft
Ground slice	
30	
10	
10
Space slice	
8	
0	
40
Capital ship	
40	
0	
10
Space and capital ship slices share some resources as there is visibility between them.
In addition to the active units, a number of tricks commonly used in games will be utilized to make the battle seem higher populated.
One example is very low-poly models in the distant background with simple behaviour. The player generally does not notice the simple behaviour as there is enough action in the foreground behaving in a believable manner.
Another example is to create randomized particle effects such as shell explosions, ship explosions, laser fire, flak and rocket trails around the battle. These can be effective because units in the far distance would not be particularly visible in reality, but bright light caused by explosions and weapon fire would be. These are set to avoid actual units and do not affect gameplay which makes them computationally cheap. Another advantage is that they don’t need to be broadcast over the network.
===Performance===
Potential bottlenecks
The number of shader batches is a potential bottleneck. The complex pixel shaders are also a concern, as this means the pixel shader processors are always busy but the vertex shader processors will be underused.
Heavily layered scenes (such as those dense in foliage) are a potential bottleneck, as fill-rate is slow on the target hardware, so overdraw needs to be kept to a minimum.
A high-level potential bottleneck is that a lot of planet detail could be rendered even when at a distance from the planet. This will require all rendering systems to incorporate level-of-detail rendering.
Split screen can mean combinations of views that are extremely unfavourable to the rendering can be produced. Level-of-detail rendering may have to be increased when in split screen.
Optimization tools
The in-game profiler gives hierarchical view of functions and the time spent in each. There is also an in-game stats viewer which lists objects being rendered, shows number of polygons drawn, and gives information on shader batches.
The following tools are also used:
*SN Tuner (PS3)
*Microsoft PIX (Xbox / Windows)
*Nvidia NVPerf and NVShaderPerf performance tools
*Intel VTune Performance Analyzer
Additionally it is intended to greatly extend the statistics available to designers and programmers from our ingame tools. This will help keep the game within quota during development reducing the required optimisation period.
Optimization strategies
There are different methods of optimization for CPU and GPU. The multi-processor nature of the consoles means that porting systems off the main processor to other processors is essential to get the best performance, and this is one of the main challenges of next-generation development. The Wii has only a single core, and so the game processing will have to be scaled back appropriately.
For CPU optimizations, it is necessary to use profiling to identify functions or processes that are using a high percentage of the total CPU time, and address these first. It is important to assess the efficiency of the underlying algorithm before trying to optimize the code itself. Switching to a more efficient algorithm will usually provide better gains.
For GPU optimizations, it is useful to assess a shader’s cost by swapping to a simpler shader (both for pixel and vertex shaders) at runtime and judging the effect this has on performance.
Making use of the ingame statistics is important for both programmers and artists. Artists building environments are required to use the statistics to test visibility and polycount issues, with expensive views automatically issuing on-screen warnings.
As a general optimization, updating and rendering objects or environment that is not visible should be avoided where possible. Likewise, updating and rendering objects in the distance at a higher detail than necessary should be avoided. Strategies for achieving this are discussed throughout the document.
The frame rate target will be 30fps.
==Technical Design Overview==
The Engine is written in C and is designed to be completely standalone, it's only interactions being with the Framework and the Hardware. It is built as a separate library and then linked to the game. Parts of the engine (the renderer, file system, input and sound systems) are platform dependent. These parts have a common interface across platforms, and this interface is then used by the platform independent parts of the Engine.
The Framework is a set of C++ classes, which make writing a game easier. The framework makes direct use of the engine. Other game code derives from or adds onto the framework code, and is generally specific to a title. Both the framework and the derived code make use of the platform independent section of the engine, and they are entirely platform independent.
Framework code calls game code, and game code calls Framework code – the boundaries between the two are (to a certain extent) non-existent. Both the game code and the framework are heavily data driven. When an object is created, classes are created (serialized) using data from pre-processed text files. The text files describe what class is to be created, and what it's data members are initialized to. This has several benefits. It means changes can be made to tweak gameplay whilst the game is running (the files can be reloaded on the fly), and means that composition changes can be made without having to recompile the game – for example, designers can change the type of camera and physics used on a vehicle, as well as tweaking any data used by the vehicle without needing a programmer.
===Subsystems===
SubSys.jpg
===Platform-specific details===
====Playstation 3====
PS3 has 1 primary processor (PPU) and 6 secondary processors (SPUs) available to the game. One of the biggest challenges is fully utilizing all of these for the best performance, by sending tasks and data from the PPU to the SPUs, which themselves process the data before sending it back.
There are a number of tasks that utilise the SPUs, including character skinning, sound processing, particle systems, terrain LODing, visibility, animation, AI high-level pathfinding, video playback, and data decompression. The Havok physics library also makes use of multiple SPUs.
Sony currently support tools on both Linux and Windows, but they are increasingly leaning towards a Windows only development solution. It is therefore likely that most Playstation 3 development will be carried out on Windows.
====Xbox 360====
Xbox 360 has 3 processors with two cores each, so it is important to split tasks amongst them for optimal performance.
As all the processors are the same and have full access to main memory, writing concurrent code on the Xbox 360 is generally much easier than on the Playstation 3. All tasks that run on the SPU on the Playstation 3 run on a separate core on the Xbox 360.
====Wii====
Compared to the other two console version Wii presents a lot of additional technical challenges. The three biggest challenges are that it is has a single core processor with a much reduced clock rate, the shader system is not as programmable as the other consoles and the amount of memory available is only 88Mb which is approximately a fifth available on other platforms. Additionally the disc size is single layered DVD (half the space available on Xbox 360).
The aim of the Wii version is to match the experience of the other consoles as much as possible but enhanced by the addition of the Wii's unique controller. The Wii has technical limitations but these limitations are well known and can be worked around.
====PC====
MINIMUM SYSTEM REQUIREMENTS
COMPUTER: 100% DirectX 9.0c compatible computer required.
OPERATING SYSTEM: Windows XP or Windows Vista required.
CPU: Intel Pentium 4 or AMD Athlon 3.0 GHz or faster required.
MEMORY: 1GB RAM or higher required.
GRAPHICS CARD: 256 MB 3D graphics card with Vertex Shader and Pixel Shader (2.0) capability required
SOUND CARD: 100% DirectX 9.0c compatible Sound Card required.
DVD-ROM: DVD-ROM drive or faster required.
INPUT DEVICE: Keyboard and mouse required. Joystick or gamepad recommended.
DirectX: Microsoft DirectX 9.0c is included on this CD.
*NOTE: DirectX may require the "latest" drivers for your particular hardware.
===Middleware===
====Havok====
The Havok physics library will be used on all platforms. It is being successfully used by Battlefront III currently.
====OpenAL====
The OpenAL (Open Audio Library) is currently used to provide sound on the Windows and Linux builds of the game, it is not used on the consoles.
====Ogg Vorbis====
The Ogg Vorbis library is currently used on the Windows and Linux platforms for streamed sounds. It was chosen largely because it was a free and open standard, which many tools work with. There are currently no plans to use the format in the console versions of the game.
===Game Loop===
GLoop.jpg
===Data Flow Diagram===
DFlow.png
==Technical Design Detail==
===Rendering===
The rendering system is an existing system, which is used by both games under development at Free Radical. Many parts of the system are finished, but some new sub-systems will be necessary for Battlefront III, and information on these (and some existing sub-systems) follows.
====Wii Specifics====
The Wii has a weaker GPU than the other platforms but still very capable if used efficiently. The biggest change from the other platforms is that it doesn't support fully programmable shaders which the current renderer is largely dependent on. This will require a separate renderer to be developed specifically for the Wii.
The render budget the Wii will aim for will be 200,000 polys running at 30 frames per second. This reduction in render polys will be made by reducing the poly budget of objects by 30%. The rest will be achieved by a combination of culling shadows at a shorter distance, bringing LODs in earlier, limiting the number of shadow projections by using clip map shadows and removing the 1 meter per vertex resolution area from the terrain renderer.
Bump mapping will be kept to a minimum. While it is technically possible, it increases the size of the assets and is approximately 3 times as expensive to render than regular objects. However if there is specific areas where bump mapping is a required (for instance the heroes faces) then it will be supported in these cases.
====PC Specifics====
The minimum resolution supported will be 800x600, but the other main resolutions to be tested with are 1024x768, 1152x720 (widescreen), 1280x720 (widescreen), and 1280x1024. The aim is to allow any resolution upwards from 800x600 that is supported by the display device, so that users can balance the ideal resolution and resulting framerate for their machine. However, there may be a maximum buffer size that the game can support and be tested at, so there may end up being a maximum resolution that is allowed. The user interface and HUD will need to scale appropriately with resolution.
The PC version uses OpenGL for all rendering, not Direct3D. Any DirectX 10 features on Vista will be accessed using OpenGL extensions.
The main graphics cards that the game will support are Nvidia and ATI cards. Currently, compatibility with ATI cards is limited, so this needs to be addressed.
====All other SKUs====
=====Materials=====
Each surface is assigned a material, which is a combination of shader properties. A shader property is typically a texture (24-bit DXTC and VQ compressed) with variables to control how the texture is applied. The source textures are 24-bit TGA files, which are compressed by the tools.
The most commonly used shader properties are:
*Diffuse map.. This is the base colour texture
*Specular map. This specifies where the object is shiny, along with a variable to control the glossiness.
*Normal map. This allows per-pixel normals to give the impression of increased detail and better lighting. These normals are relative to the vertex normals, which allows better re-use of the normal maps. The normal maps only store 2 of the 3 values from the normal vector (the 3rd is calculated in the shader as the result must be a unit length vector).
**The two values are stored as 8-bit values in the RGB and alpha channels respectively, as DXTC compresses these channels separately. High precision (16-bit) normal maps can also be used, but the extra quality these allow can only be noticed on very smooth objects close-up, so these are unlikely to be used on this project due to their high memory cost.
*Parallax map. This simulates per-pixel depth, and so gives the impression of much higher polygon meshes when viewed at an angle.
*Environment map. This is a cube map which makes an object appear reflective, and is mainly used on very shiny objects (e.g vehicles and droids). If no map is specified, it uses the global environment map (which is set on a per-level basis). This allows them to be location specific, or non-location specific, which is more likely to be used when an object is instanced several places in the environment.
**The cube map is specified by the artist as 6 separate sides arranged in a cross shape. This is cut-up automatically into the individual sides as part of the convert process.
*Incandescence map. This is rendered with a glow applied, and is useful for very bright parts of an object, such as neon lights or vehicle jets.
The materials sub-system is already relatively mature, and should only require small changes/tweaks.
=====Props/Characters/Vehicles=====
Any object that is not static during gameplay is a prop, and is lit as described for dynamic objects in the Lighting section. Each prop has a world-space matrix, and a set of object-space matrices for its different parts which allows each part to animate. Characters are rendered with skinning to smoothly bridge polygons between two parts with different matrices.
The prop rendering systems already exist. The lighting and other polish will require some work, but is low risk.
=====Environment=====
The backgrounds are static meshes which use an instancing system to allow for multiple occurrences of the same mesh in a scene. They are lit with pre-calculated lightmaps as described in the Lighting section. All interior sections (including inside the capital ships) and some outdoor levels (such as Coruscant and Bespin) will be implemented in this manner. Other outdoor levels require terrain, which is discussed in the Terrain section. The background rendering code is already written, and should require no major changes.
The sky is rendered using a atmospheric shader, which combines two sky boxes, a ground-based sky box, and a space sky box. The sky boxes are stored as cube maps in a high dynamic range format (RGBE). The atmospheric shader combines the boxes differently based on the height of the camera above the ground, and the angle against the horizon, to represent the way light travels through a planet’s atmosphere.
=====Visibility=====
Indoor areas will be split into sectors / rooms (the terrain will be a single sector). Portals allow visibility between sectors. Frustum checking will disregard objects and terrain chunks not in front of the camera.
Occluders can be manually placed inside high terrain to cull props from being rendered. When the view is near the ground, this will prevent having to render expensive parts of the background, and when the view is higher, coarser detail will be used.
In order to render larger scenes without a significant drop in performance, portals can be setup to render to a texture. This texture will then be applied to a polygon in place of the portal (known as an Imposter Portal).
=====Terrain=====
The terrain system is for rendering a large heightmap based terrain with a high level of detail in the foreground but with a low polygon count when viewed from a distance.
To simplify the terrain system some assumptions were made. It is assumed that the terrain is square, and can be represented by a heightmap. Terrain is also assumed to be continuous and smooth, and needs to be highly detailed for the innermost 1km square. Gameplay can only take place inside the inner 1km square, so no collisions will be required outside of this range.
Some of the environments require caves within the terrain itself. The caves themselves will be polygonal meshes built by artists, and attached to the terrain by flagging certain terrain quads as holes at a 1 metre resolution. In order to hide the seam between the terrain and a cave entrance, it is likely that a frame will have to be modelled around the entrance that can intersect with the terrain (e.g boulders or a metal doorway).
=====Wii-specific Terrain=====
The terrain geometry will be a similar clip map based system to that currently employed on the other platforms. However, the maximum vertex resolution will be reduced to 2 meters per vertex. This will have the double benefit of reducing the number of polygons drawn (but removing the top most shell) and also the memory usage of the height map. By reducing each clip map shell to a 48 by 48 square (instead of 64 by 64 on other platforms) the total drawn polygon count will be almost halved compared to the other versions. This will involve earlier morphing (25% closer than other platforms) but could be mitigated by smoothing out the height map data for Wii.
=====Foliage=====
Foliage will be required for several of the outdoor environments. It is painted onto the surface of meshes using the world editor. Different brushes can be used for different foliage objects, which will allow a variety of grass, bushes, and flowers.
The foliage is rendered denser according to the distance from camera. For performance reasons, no foliage will be rendered when the viewpoint is beyond a certain distance from the terrain. This may require the terrain base texture to be appropriately coloured.
=====Detail Geometry=====
The detail geometry system will be used for static objects which can be instanced and re-used frequently throughout the environment (such as trees), as these can be batched together and rendered more efficiently. These are lit as described for static objects in the Lighting section.
Systems for rendering both foliage and detail geometry already exist. Some lighting changes will be made to give a more polished look to them.
=====Level-of-detail (LOD)=====
The game design allows the player to view environments and its objects up close on the surface, or from far away in space. In order to display a large number of objects without adversely affecting performance, the LOD system needs to allow assets to display at high detail close up, and very low detail at a distance. It needs to seamlessly blend between the levels, as the player can approach from a distance without any camera cuts. It also needs to be relatively inexpensive with respect to processor time and memory use.
The terrain renderer natively uses LOD. The foliage renderer also already uses LOD, and draws denser foliage at close distances.
=====Lighting / shadows=====
For all static geometry, light maps are generated offline with the in-house photon-mapping tool. This is an artist-run tool which allows lighting to be baked onto the background from lights placed in the background scene. This process can be distributed across a number of machines, vastly speeding up the render time of light maps. Light map UVs are calculated within Maya, allowing alterations to be made without necessitating a light map build. Monte Carlo integration is chosen as it allows settings to be balanced with render time and visual quality.
The lightmap is stored in a proprietary HDR format, which stores the total colour (and hence intensity) of light hitting that pixel, but not the directions of the lights. For geometry with a lightmap attached, each vertex has a normal stored, indicating the direction of the most influential light., which is used to correctly apply the lightmap over the normal and specular maps. For outdoor scenes, the most influential light tends to be the sun.
*Dynamic geometry has direct and indirect lighting calculated in different ways:
For each dynamic object, direct lighting is calculated using a maximum of 2 light sources. Outside, the sun is generally one of these light sources. Other light sources are placed by artists or created inside particle effects (e.g. explosions).
*Indirect lighting (ambient) is calculated offline in the light mapper tool. Tiny cube maps are generated, sampling the incoming ambient at regular spatial intervals throughout the environment. For each dynamic object, the closest cube maps are interpolated between and applied.
Shadow maps are generated every frame for dynamic objects from the closest light, and projected onto geometry as a separate render pass. Objects are also self-shadowing with this technique. The resolution of the shadow maps decrease further away from the camera. Where unnoticeable, shadows are not created at all for props far away from the camera.
The dynamic lighting is mature and used in-game.
=====Wii-specific Lighting / shadows=====
The model of lighting is proposed is a combination of artist produced vertex lighting and decals, real time directional specular, environment hemisphere maps and real time vertex lighting.
Before rendering of a prop begins, a 64x64 sphere map will be produced for each view and for each prop that takes up a large amount of the screen. The sphere map is always orientated towards the viewer. For non bump mapped objects the sphere map only consists of the specular component. This is made up of an environment map (attenuated by a stored shadow map) and specular directional lights. These are generated by using an indirect texture to transform from sphere normals into the reflected normals. If the surface is reflective then these reflected normals are then used as a texture coordinate to lookup into a hemisphere environment map of the current sky box. Specular directional lights are added in passes by using a matrix to dot the reflected normals with the light direction together with a texture to provide a sharp spot.
When rendering props the normal is transformed into the sphere map space and the component facing the camera is zeroed to generate the 2D sphere map coordinates. It should be impossible to see normals facing away from the camera. Diffuse lights that are close to the player (such as light from the gun and other other lighting effects) can use up to 4 hardware lights which are available to use with zero extra cost to the renderer.
Assuming that there are 20 visible props needing their own sphere map per frame, we will require 320K to store the sphere maps.
The lighting system will decrease in quality based on how close the model is to the viewer. Models in the distance will use a much simpler lighting model, decreasing the cost to render.
The order in which the lighting LODs will be...
Global hemisphere map gets used
Specular map turns off
Revert to fully hardware lit by only diffuse hardware lights.
Background lighting will be mainly artist made using a combination of vertex lighting and decals. The hemisphere method of specular lighting is only based on the object's normals. This gives unsatisfactory results on backgrounds which have a large number of large flat surfaces. Therefore background specular is done using an alternative method. By using a single directional specular light for the background the background polygons can be split into those that accept specular and those that don't. This saving allows a more complex specular scheme to be used. Backgrounds have the reflected normals calculated at convert time. From this reflected normal a screen space offset for the centre of the specular spot is calculated. This offset is added to a screen space texture of a specular highlight using an indirect TEV stage. Materials with different specular values are accounted for by shrinking or expanding the screen space specular spot. This allows a good one pass approximation to real specular. Hardware lights will be available for free as with the props for doing background lighting effects such as bullets.
In the other versions of the game shadowing is preformed by each casting object being rendered into it's own shadow buffer. Then everything affected by this shadow gets re-rendered with the shadow buffer projected onto it. This calls for objects to be repeatedly redraw when projecting. This can be partially mitigated by rendering only the polygons of the object affected by the shadow but this takes CPU time to determine. For Wii, it is proposed that a clip map approach is used. This method uses predefined shadow buffers that cover an area around the viewer. This has the advantage of limiting the amount of times objects have to be rendered to project shadows and allows for interesting dynamic shadows (such as moving trees or rolling clouds) to be added for very little cost.
Two shadow buffers are used. The first extends up to 16 meters away from the players (although could be extended) with a resolution of 32 texels per meter. The second extends to 128 meters but has only a 4 texel per meter resolution. The inner shadow buffer has objects rendered into it from the light's position and then blurred to give nice soft projected shadows. The outer shadow buffer has only diffuse blobs the same size of the object's bounds rendered into it. This allows for much more objects to contribute to the shadow buffer than if they all had to be rendered. Projection onto the background only has to be done once and can be part of the normal background render. The inner buffer blends into the outer buffer providing a seamless transition between the shadow levels. Objects no not receive shadows.
===Scene Management===
Scenes are objects used to keep track of the game state. For example, there is a LoadingScene object (which carries out the loading process for a level), a FrontendScene (which is responsible for the menus and fx used in the main menus), and a GameScene (which deals with the gameplay itself).
When the game starts up, a Scene Manager is used to start the appropriate scene. In the final version of the game, this will likely be a scene which loads the frontend scene and plays intro movies, but during development it is often a scene which loads straight into a level. The scene manager also handles the game flow when a level ends, switching from one level to the loading scene, then to the frontend/next level.
The game scene can also contain components, which are used to control flow of the game. This will allow implementation of game modes (Deathmatch, Capture The Flag, etc) using scripting. Scripts will contain game callbacks so they can be notified of game events (game finishing, player dying, flag captured, etc) and react accordingly. Components can be used to share code between game modes as necessary – timers and scoring systems, for example, and the scripts can make use of these.
The Scene Manager and some scenes (in-game, front-end, and loading) already exists, as do some components. New scenes and components will need to be added during development, but these are low risk. The system currently has not been used in a shipped title, but is entirely functional and will have been shipped in Haze before Battlefront is released.
===Game Objects===
The game object system is existing technology and has been deployed in Haze.
Game objects are referred to as props. They exist in both the framework and the engine layers. The engine layer provides limited support for them, managing their creation, deletion, rendering, and a few other aspects.
Most props are actually wrappers for artist built models, which are referred to as obs. Each ob is loaded once (an obdef), and instances of it are created (obinsts). These are almost entirely dealt with in the engine layer, and most in-game props have an ob. The obinst contains matrices, deals with animations, skeletons, switchflags (flags to turn parts of models on and off), lods, and other information about individual objects. The obdef contains information about the geometry, textures used, the number of parts, and other information (about glass, cloth, ladders, materials, physics, etcetera).
Props have a link to an obinst (not always used), a bounding box (used for basic frustum culling of props), and a matrix (which contains the prop’s world position and rotation). They also have a render callback so that special case rendering can be done on a per-prop basis.
The framework has a class called CGameProp. This is a wrapper/utility class that provides methods for manipulating engine props. There are a series of classes that extend CGameProp to provide different types of prop (around 50 or 60 types at current count); CVehicleProp, CPlayerProp, CDoorProp, CGun, and CPhysicsProp being regularly used examples.
Each prop serialises itself during the creation process. It typically reads in values describing it’s initial position and orientation from a resfile. The resfile can be changed and re-loaded whilst the game is running, which allows us to test and alter level designs rapidly.
There is also a system that allows props to serialise component classes in. The basic CGameProp class has only a single component, a CNetworkComponent, but most subclasses have many different components. A CVehicleProp contains about 30, which include a CSoundComponent (makes engine noises), a CFXComponent (adds trails, dust, tyre marks), a CCameraComponent (changes how the camera follows the vehicle), and a CHealthComponent (which keeps track of how much health the vehicle has remaining).
These components are created when the owner prop serialises them in, based on text resfiles, and each typically has a set of values that can be changed (how much health the vehicle has when undamaged, for example). The resfiles can be changed and re-loaded on the fly, which allows us to test and change components quickly. For example, if several camera components have already been written, a vehicle camera could be changed to any of these and tweaked without having to re-compile and run the game.
===Animation===
The animation system is existing code and has been deployed in Haze. It allows the export of an animation created in Maya, converts it for use in-game, and then plays the animation back on an in-game object. The system has an engine and game side. The engine side is responsible for loading animation files and returning the data in the form of matrices. These matrices can then be applied to an objects matrices. The game side has code for blending different sets of animation matrices together to allow two animations to be played on the same object at the same time.
Animations are blended via a directed acyclic graph (DAG), where each node within the graph can use parameters determined by the game. This allows a common set of code to be used in many ways. Use of 2-chain IK on top of the final output allows correction for final display.
In future, animations may be stored using B-spline curves, allowing good compression while still retaining the quality of output. This system could allow larger space savings in terms of memory and media space, but may not be necessary, depending on the amount of room used by animations. Work has not been started on this, and will not be done unless it is necessary.
Currently animations can be compressed by leaving out keyframes – the exact amount varies depending on the animation, and is determined by a set of heuristics and error tolerance checks. The tools that build the animation from the source carry this process out, and the game plays the resulting animation with no special case code. Further compression is achieved by quantizing values used in the animation to a lower precision. These lower precision values can then be stored in less space.
Free Radical’s in house Upper Body Inverse Kinematics System (known as UBIKS) is used to apply inverse kinematics to characters upper body, and is largely used so characters can point their gun at a target independently to the rest of the animation being played on the character.
The system in the engine has improvements over previous versions of the system. Previous versions used a single animation to control the up/down part of the pointing and then rotated this left/right to aim to the left and right. It now blends 4 from a grid of 9 animations together to produce the same effect, giving animators much better control over the end result.
More Inverse Kinematics code will be required by Battlefront game (code for walking vehicles, for instance). This system will use a 2 chain IK solver, which will produce a single result. Constraints and stiffness information will be controlled from Maya. This is a relatively simple system to write and maintain (and indeed will be deliberately kept simple for performance reasons), but should produce the desired results. Work on this system is scheduled to be carried out relatively early in order to make sure the result looks convincing, but there should be little risk, since we have working IK code in other areas of the game.
====Animation Editor====
A new tool for editing animation trees and their parameters has recently been developed by the tools team. This tool allows animators more control over the various aspects of how source animations are combined into the overall in game result.
====ABM: Animation Blending Machine====
The animation code will generate a list of animation blending instructions (also known as 'opcodes') which can be evaluated by a virtual machine called the 'animation blending machine', or ABM. It is conceptually very similar to OpenGL in that the calls emitted by the code don't immediately take effect, but are queued up for later processing. This processing is then distributed across multiple CPU cores reducing the load on the main thread and allowing for increasingly sophisticated animation blends without harming the framerate.
===Cutscenes===
The system is largely animation driven, and the game code responsible for it relatively simple. Animators create an animation in Maya. It contains information about the necessary characters and props movement, and also about the camera position and orientation. Since cutscenes are often relatively long this animation can be large, so it is normally streamed in from disc and played on the fly.
Using animated cutscenes often results in a more professional and polished cutscene than would be possible using scripting given the same development time.
Whilst the animation files can be large, they are often smaller than the movie file that would otherwise be necessary. This should make it easier to fit the game onto smaller media, but at the cost of slightly longer load times.
===Camera System===
The camera system is already implemented and has been deployed in Haze. It is split into two parts; an engine part and a framework part. The engine part is a basic camera system which can render views from given positions and orientations. The framework code then updates these positions and orientations.
Typically, the framework side has one camera per player. This camera is updated every frame, and the type of camera changed based upon the players preferences and actions (if the player gets in a vehicle the camera will change types, or the player may switch between first and third person cameras).
There is also a free camera used for debugging purposes; this can be flown around the level using a keyboard and mouse.
===Particle system / VFX===
====Particles====
The particle system is already implemented and has been deployed in Haze. It consists of particle emitters, which have a world position, area of emission, and other properties. Particles are emitted from these and updated during their lifetime. These are usually rendered as billboarded sprites. The emitter stores most of the common properties of the particles, so that each particle only uses a small amount of memory.
An in-house editor has been written for creating and editing particles using the core game renderer. This allows particle effects to be altered and visualized as they will appear in-game and also tweaked differently for particular levels or graphical filters.
Specific shaders are written for particles to allow effects such as heat-haze and refraction, and the system features a range of built-in emitter properties and particle behaviours. The particles are integrated with the physics systems to allow effects as sparks ricocheting off characters and background geometry, to suitable sized splashes when objects hit water.
====Weather====
The weather system populates areas within the camera frustum with weather particles such as rain and snow that are affected by gravity and wind for efficient yet realistic results. This is combined with billboarded sprites that move towards, away from or across the camera, depending on the wind direction. These sprites feature a noise texture that give the impression of sweeping drafts of wind.
====Clouds====
Volumetric clouds that can be flown through will be scattered about the sky for several of the environments. Each cloud initially generates a 3D noise texture using Perlin noise. This 3D texture is projected in the shader onto a series of billboard-aligned planes which the camera can pass through. Lighting is also calculated in the shader, although for performance reasons this assumes the light is always directly above the cloud, which in practise is unnoticeable.
A very important optimization is the use of imposters. Clouds in the distance are rendered to texture (the imposter buffer), and only the texture is drawn onto the screen. The imposter buffer is only updated when the cloud is approached or the camera angle changes sufficiently.
Clouds are generally fluffy and don’t require high definition which allows the imposter buffer to be fairly low resolution. This also helps the non-imposter clouds, as all the fill-rate intensive planes can first be rendered onto a quarter size buffer. The buffer is then applied to the full screen.
The procedural nature of the clouds allows the artists to tweak the parameters dynamically. It also allows the clouds to change during gameplay, although this requires updating the imposters which is expensive. One solution would be to only update a small number of cloud imposters per frame, so that the entire cloud cover gradually changes over the course of a few seconds.
====Other====
The visual effects system also provides several effect primitives such as tracers, trails and electricity which will be utilized for the sci-fi weapon and lightsaber effects.
====Wii-specific Effects====
Blending between the ground sky and space sky is done as part of the fog calculation on other platforms. However, this will not be practical on Wii. The sky box blending will be a separate process to the fogging of objects. Fogging of objects will be done using the hardware fog with the near, far plane and fog colour set per object to blend in with the sky box.
The space sky box is a textured cube centered on the player. To save on texture space the space sky box is first rendered using a low resolution cube map containing nebula. Then each of the six faces have a generic star texture tiled over each one. Finally the suns are a separate render on top of this. Blending on the ground sky uses a sphere rendered around the viewer's position. A ground sky hemisphere map is mapped to this. To achieve the blending between the sky boxes, a texture is mapped between the poles of the sphere that contains the pre-calculated amount of blending between the sky boxes. This pre-calculated texture is 2D. In the x direction is the amount of atmosphere passed through for each latitude on the sphere. In the y is the height of the viewer. As the viewer moves through the atmosphere the texture coordinate is shifted in the y direction which has the effect of making more of the space sky box visible. So that the horizon of the ground sky box doesn't follow the viewer up as they move through the atmosphere the ground sky box is shifted down as the viewer moves up.
The planet itself and the clouds are also part of this sphere. The planet is a disc within the sphere that moves up and down to stay aligned with the atmosphere. This disc's texture coordinates scale as the viewer moves up so more of the planet surface can be made visible. The clouds, if any, are added on top of the planet disc. The vertices at the edge are the same as the planet disc but a single vertex in the middle separates it from the planet making the clouds visible from both on the ground and from space and providing some parallax to make the scene more realistic when viewed from above. To hide the transition through this cloud plane, the cloud layer fades out when the viewer gets close.
===Full-screen FX===
The game will make use of several types of full screen effects. Existing effects include a High Dynamic Range effect, Blur and Fog.
The High Dynamic Range system works by rendering the scene into a floating point render target. This target is then downsampled and an average brightness for the scene calculated. A shader is applied to blur the bright sections of the scene, providing a ‘bloom’ effect. In order to get a dynamic effect, the brightness is filtered towards the brightness of previous frames.
The blur effect is carried out using a fragment shader.
A fog effect is currently applied using a combination of particles and a shader which reads values from the depth buffer and filters things accordingly (more fog should be visible when objects are further away).
Possible full screen effects may include a Depth of Field effect. This would be for the purpose of adding polish to the game, and not necessary to ship the game, so this would be low risk.
===User Interface===
The current menu system is a new system that was written for Haze. The system uses a GUI editor to allow a user to lay menus out and specify various attributes of interactive widgets in the GUI.
The HUD is largely data driven, with the HUD split into separate components for parts of the display. The components are written in code, with several standard types available for bars and text displays. More complicated components are generally written with bespoke logic, but the components that comprise the HUD and their locations can be altered from data files without needing to rebuild the game.
===Audio===
Source files are stored in WAV format. These are then converted to VAG (adpcm) and atrac3 for the PS3, and WMA for the Xbox 360. During development on PC, the WAV files are used for shorter sounds, and OGG for large sound files. All source files are 48khz 16bit files.
Audio code is split into three separate parts - a platform layer, an engine layer, and a framework layer. The details of the platform layer vary depending on the platform, but all platforms expose the same functions to the engine. The engine layer can then play sound effects on all platforms in the same way. At present the PS3 uses Multistream and the Xbox 360 uses XAudio and X3DAudio.
The engine layer deals with three types of audio; sound fx, streamed sounds, and music. Sound fx are kept in memory and used regularly - gunshots or hit sound effects are often played this way, as are other small sounds. Streamed sounds are longer, less frequently used sound effects, and are played whilst being streamed from disc. These are often used for speech, or similar longer effects. Music is mostly handled in the engine by managing streamed sounds - it has special functions for fading between tunes, pausing, and setting volumes.
The sound system should be able to cope with up to 100 simultaneous voices, and 5.1 channel output. It currently supports pitch shifting and sample skipping, interactive mixing, fades between sounds, and distance gain. Occlusion and environmental effects, interactive music sequencing, sound pathfinding, audio HDR, and complex looping are features still to be added to the sound code.
Reverb and other DSP effects can be implemented in software (handled by Multistream on the PS3). In-game materials have sounds attached to them for different events. Artists place the materials on models and backgrounds, and the sounds are triggered by events. This means footsteps and bullet hits can produce different sound effects on metal and concrete, for example.
The sound code should use approximately 15MB of memory on all platforms.
The most commonly used method of decaying sound volume in games is a simple geometric distance test. However, in the setup editor, interconnected sound volumes can be placed throughout the environment. The volumes are combined to model a flow of sound through the environment; this combined with the ability to tag volumes with additional data will allow us to create more realistic sound design.
Sounds can be placed using the level editors, either as background sounds, or triggered under certain conditions using scripts. More complex sound effects can be created procedurally by altering volumes, pitches and filters in code based on events (e.g. engine sounds can be simulated by changing the pitches of mixed samples as a cars revs increase in the physics code).
Volumes and pitches can be adjusted in data files using proprietary tools, and interactive mixing can be carried out on the fly using tools.
Cutscene SFX can be triggered in-game but may also include pre-mixed streams.
Free Radical has a dedicated audio team who are responsible for writing in house tools to facilitate sound implementation.
===Wii-specific Audio===
The Wii audio hardware has internal support for panning, mixing, envelopes, pitch, reverb and chorus. Effects will be limited to this selection. Sound occlusion/path finding is also expensive and will be dropped from the Wii version.
Sound samples will be mainly 48KHz ADPCM. For music and other long streams 48kHz ADPCM will be streamed from disc. 48kHz ADPCM has the advantage of being able to stream off the disk and played without involvement from the CPU.
Surround sound will be supported via Dolby Pro Logic 2.
===Input===
Input is handled largely through the platform specific controllers (joypads). During debugging, however, it is useful to have access to keyboard and mouse input. Both systems are supported by the engine on PC, with joypads only on consoles.
Each platform has it's own input functions, and these are wrapped by an engine layer. The engine layer is then used by the framework. The framework has a data driven system for setting player controls up. The framework input code can map different joypad inputs to different player controls based on the player’s current state. These can be changed by designers (freeing up programmer time), and on the fly while the game is running - sensitivities can be tweaked without having to rebuild and run the game.
===Wii/PC Specific Input / Controls===
*Wii: Because of the limited number of gestures bespoke gesture recognition code will be written for each one.
*PC:
**Mouse and keyboard support will be added to the game. Every game control will be bindable to custom keys from the Controls menu, and the sensitivity of the mouse will be configurable.
**The main game controllers will be supported, including the X360 controller. The most common controllers will have their mappings preset, but the ability to remap any game control to any button or axis will be provided so that any controller can be configured to work.
**The user interface will be changed where necessary to allow easier mouse navigation, whilst still working with game controllers.
===File I/O===
The engine exposes a set of file management functions to the game, which are a wrapper around platform specific file functions. The file functions give the engine the ability to open files and read or write data from them, and there are some utility functions here to make common operations easier. There is also a streaming API which is used for the majority of file operations so that the main thread remains responsive at all times. The engine layer of the file API is responsible for handling platform specific TRC requirements, such as disc eject or file reading errors.
====Package files====
Files can be packaged together into large archive files, called pack files. If all the files are loaded in one load operation it reduces load times (only one seek should be required, and the file can then be loaded with the disc spinning at high speed). When the engine is looking for a file, it will first check to see if the file is in an open pack file. If it is, this file will be used, otherwise the engine will look for the file on disc. When files are in pack files on disc, the pack file is sorted to reduce the number of drive seeks required for that dataset. Small file reads are automatically extended into larger reads at little to no extra cost. The data from these extra reads is kept in a lookahead buffer and is immediately available if requested.
====Streaming====
Streaming functions are written on a per platform basis. They allow data to be read from files in the background whilst code is running. They are mainly used to read data while the game is running (textures, game objects, music and other sounds are all streamed in from disc as needed in order to save memory).
====Asset compression====
Audio will be in wma, ogg, or atrac3 formats, which give similar results to mp3 compression, so audio will not use much space on the media, nor benefit much from extra external compression.
Object files and textures make up the majority of the data, textures in particular. Different files compress better than others, but from our tests it is estimated that on average object files compress to 50%, and textures compress to 33%.
The compression method will be based on a gzip variant of lz77. This does not give optimal compression size, but gives very fast decompression. Decompression is done automatically and is performed on a separate thread.
Half-floats will be used instead of floats for vertices where possible. These are sufficiently accurate up to 32 metres, so these will be used for any objects or rooms smaller than this.
====Media layout====
The layout of media on disc is still to be decided. It will depend on the amount of data the game is using nearer the end of the project, and cannot be finalized until nearer the time.
====Wii-specifics====
The Wii's maximum disc capacity is a dual layer DVD which is the same available on the Xbox 360. Fitting on a single layer DVD maybe possible however due to the four fold reduction in texture sizes and the lack of HD movies.
===Save games===
The Battlefront save game system is currently unimplemented, but will be loosely based on systems Free Radical has used in previous games (especially TimeSplitters, which had a flexible multi-player save system).
Each save game will have a header chunk, which will store information about the rest of the save game.
There will also be room for players to save a profile, probably allowing room for a fixed number of profiles (most likely 8-16 slots). Each profile will be referenced by a player name, and will contain the player’s preferences and controls. It will also store the player’s statistics (what levels they’ve completed, how accurate they are, what awards they’ve won, etc). It should be possible to save information about a checkpoint reached in the game so the player can resume where they left off at a later date.
It may also become necessary for the player to save Galactic Conquest saves separately from their player data depending on how the design works.
Any other information which is not stored with the player data (volume preferences and similar data) can be stored in the header.
The system is medium risk, and will be worked on relatively early in the project. However, work on the system is best carried out when other gameplay and features are implemented. It also depends on the Sony and Microsoft platform libraries, which may not be mature yet.
===Collision===
Physics collision is handled by the Havok physics library.
Additionally, custom collision detection is performed for line tests used by AI and bullets/lasers. These line tests are done in a separate thread to take advantage of extra processing cores, and generally yield a result the frame after they are started.
===Physics===
====Rigid Bodies====
The physics for the game is built on Havok. The framework contains in-house code that wraps around Havok. In the past this has allowed us to work with different physics engines, such as PhysX and ODE.
Nearly all moving objects in the game consist of dynamic rigid bodies that can be interacted and collided with such as crates, vehicles, characters and pickups.
====Ragdoll Physics====
When a character dies it makes a transition into a ragdoll. The death is initially driven by an animation, but eventually evolves into a physics driven ragdoll.
The transition is determined by contacts between the character and its surrounding geometry. The ragdoll's geometry consists of physics primitives such as spheres and object bounding boxes connected via joints, each with six degrees of freedom. The joints are given realistic constrains, tension and "springiness" to make the ragdoll feel more like a rigid corpse than a wooden ragdoll.
Vehicles are created using the rigid body physics system in the framework code. They are data-driven; many aspects of their handling can be tweaked from data files. Different types of vehicle share code allowing characters to enter, leave, and drive the vehicle, but each has control over its handling. This can be achieved by setting the velocity of the physics object or by applying forces to the object.
Some vehicle props will need to use a different, animation driven system (walking vehicles such as AT-ATs, for example).
The codebase also contains code for wheeled vehicles. These have custom engine properties such as torque curves and gear ratios, as well as custom suspension with non-linear damping, and springs to give more realistic handling. The wheeled vehicle system is mature and has been deployed in Haze.
===AI / Gameplay===
====Scripting====
The propriety custom scripting language (VM) has been used in a previously shipped title hence is reliable and bug-free. This is used to drive events, objectives and object behaviour. The custom nature of the language allows us to expand the feature set based on the requirements of the game and engine, a notable point being the networking of game information. It is a typed C-style language, hence easier to catch compilation errors than a type-less language such as LUA. Commands can also be executed from within the in-game command-line console.
The scripting system uses triggers to start code running. Several types of triggers exist – volume triggers are fired when a player enters the volume, and switches have triggers associated with them. The trigger system is flexible and new triggers can easily be added.
Events are linked to a trigger, so that when a trigger occurs, the event linked to it is started. These can be pieces of code written in C, or pieces of VM code. From within the VM, designers can then create code to handle a complex chain of events – when a switch is pulled the VM can start props playing animations, delete props, create props, add messages to the player’s HUD, and many other things.
====Wii-specific AI====
Depending on feasibility tests, it may be necessary to scale back the combat density, or to more heavily LOD the AI on the Wii version.
===Pathfinding===
The pathfinding system is an existing system, which is used by Haze. The system uses designer specified navigation meshes.
Navigation meshes are a set of polygons defining the areas traversable by AI on foot, and as created using the setup editor. These meshes are grouped by navigation volumes, which are used for quick path finding. Navigation volumes also represent navigable areas for vehicles. For exterior sections, there are obstacle volumes which flying AI avoids.
Navigation meshes are used by the AI system to plan a route from point to point. A system allows the AI to ask for a path from point A to point B, and this is then returned to them as a series of waypoints.
AI characters also use steering behaviours to do local obstacle avoidance. These behaviours are applied as the character traverses along the path calculated by the pathfinder and allow them to steer around other characters, physics objects or other designer tagged objects.
===Network===
The network system is a new system, and is a high risk system due to it's complexity. It will enable interactivity between players on different machines and allow the delivery of game updates and modular content. It will support up to 50 players playing against each other in a single battlefront and 2 player galactic conquest over internet, LAN and system link connections, with dedicated and player-hosted servers.
Key Features:
*Instant Action
*Clans
*Friends
*Squad support
*Lobby
*Leader boards
*Statistics
*Spectator Mode
*Replays
*Downloadable Content
*Voice Chat
*Patching System
*Account management
*Synchronisation
*LAN
*System link
*Dedicated server
*Automated testing support
*Stability
Server migration will not be supported.
The requirements of the system for the network prototype are limited to a small number of opponents playing on one level, including on-foot and vehicular traversal of the terrain and interior of capital ships, the use of flying vehicles to span the vertical battlefront, and shooting, and as such the synchronization of all these elements.
The networking layer will utilize a client/server and peer-to-peer hybrid architecture, as this allows the lowest data throughput over the network infrastructure.
The necessity of game state being distributed around all clients dictates that most of the state data will be contained within the scripting language. The custom nature of the scripting language allows a good understanding and control over the data being transferred automatically.
Other tools, such as packet sniffers, statistical analysis and network condition simulation tools will be used for debugging, performance tuning and stress testing. It is anticipated that most, if not all, of these won't be developed internally, but used off-the-shelf.
====Bandwidth====
In the UK, broadband upstreams are almost exclusively 256kbit and upwards. In America, the most common broadband connections have a 128kbit upstream, but this can drop into the low 100’s. This may have improved by 2008, but 100-128kbps will be the target bandwidth. Players with lower bandwidth than this will not be able to participate in 50-player matches, but rather 16-32.
One common problem in games is that the server upstream needs to be very high, because it is sending information about all the players to every player. For example, if the data for a single player’s movement is roughly 2kbps, and there are 50 players (including the server), the server needs to send out information on 49 players to 49 clients, which will be almost 5Mbps upload. To solve this and reduce the bandwidth requirements, not all updates will go through the server.
Important updates that could be contested will be sent to the server, who will to choose to agree or disagree. A message will be sent to the server only, and the server will then send these to each client (including the one that sent it initially) if it agrees. Typically, these will be sent as a reliable message i.e. resent until a reply is received. These important updates tend to be game state changes that the server and other clients need to know about (e.g. player entered a vehicle, player died).
Less critical updates can be sent from each client directly to every other client. This includes the movement and shooting of the player on the client, which is always deemed to be correct, and so does not require the server to verify it. The movement and shooting currently tends to be 2-3 kbps of unreliable data, which is sent to every other client in the game. For 49 other clients, this would be sending ~100 – 150 kbps. This method effectively distributes the bandwidth load, so the server doesn’t need a massive bandwidth.
An optimisation is to send less updates to players not in close proximity. For example, players on opposite sides of the vertical battlefront will not need to know the exact location of each other, just an estimation. In order for this to be effective, the design of the game must provide incentive for players to be spread out over the vertical battlefront.
Interaction with physics objects such as crates tends to increase the information for the client to send up to 8kbps. For this reason it is important that there aren’t areas with high numbers of physics objects interacting with each other, unless they have no impact on the gameplay.
====NATs====
There is an issue with sending data directly from client to client. Routers or gateways use NAT (Network Address Translation) to hide several local machines behind one network address. The local machines send messages through the NAT to the Internet connection and the NAT is responsible for returning replies to the correct machine.
There are three categories of NAT: open, moderate, and strict. Clients with strict or moderate NATs have limited or no ability to connect to other clients with strict or moderate NATs. Open NATs are the most common type, as these can connect to any NAT, and only this type is Xbox Live Certified.
====Testing====
Network testing and debugging tools can make the process of debugging network games a lot easier. The game and engine have built in fault simulations for testing packet loss and delay. If used regularly during early testing they will save a lot of time fixing bugs later. Free Radical has also previously used packet capture tools like Ethereal to debug problems, and previously worked with a STORM network robustness test. Whilst this requires expensive hardware and software Free Radical doesn’t have, occasional offsite tests would be useful if they could be arranged.
Automated testing will involve AI playing matches over the network. This allows us to easily gather crash information as if a high number of players were competing, and gather gameplay statistics to help identify any bugs. Automated leaving and joining will be incorporated.
====Wii Specifics====
Networking is done using Nintendo's DWC libraries. Although this library is not socket based it is still possible to use it with our existing hardware independent layer. The biggest change is in the lobby system. We are currently planning on supporting server/client and match anybody modes. The match anybody method is very unsuited to our type of game but it is the only way of playing with players you do not know. Anybody matching has to occur before the game starts which means a very long wait if a large number of players aren't online. Also there is presently no way of connecting to a dedicated server which means either limiting the number of players or having one player (randomly selected from the peers) potentially with a lower frame rate.
====PC Specifics====
We plan to use GameSpy as the matchmaking service for the PC version.
===Split-screen support===
Battlefront will support 4 players on a single console using split screens. This presents certain technical challenges. The 4 players are all created using their own screen view window by a player managing class in the framework, and the engine is then responsible for handling rendering the 4 views.
In order to make good use of the console in single player mode, Battlefront streams data into memory from disc while the game is playing. If 4 players are playing at once and can all move to different areas of the level, each will require a different area of the terrain to be visible at high resolution at the same time. This means each player needs a buffer to fill with the streamed terrain data. These buffers (and those for streaming textures in general) will have to be smaller than those used in single player because of the limited amount of memory available, and this will mean using lower texture resolutions in split screen. The smaller screen area that is rendered to should mitigate this – the texel to pixel ratio should remain similar.
Geometry streaming suffers from the same issues, except it is harder to replace high polygon versions with lower polygon versions, because artefacts are introduced even with a smaller screen area. LODs generated automatically suffer from this to a greater extent, as it produces models that look acceptable at a distance but not close up. To mitigate this, artists can manually generate the LODs where necessary, as they can judge where best to lose polygons. LODs are also being built with detail added on and flagged as detail. In these cases, it is very easy to render the geometry without the detail and for it to still look acceptable.
Split-screen also limits the ability to LOD several systems that are easy in single-player, including AI and animation. To mitigate this, the playing areas may need to be reduced for split-screen play. This should not have an impact on development, as levels are already being designed to work at different capacities for network games (ie 16, 32 and 50 player versions).
When in split screen the game will not support picture-in-picture display, partly due to the extra load on streaming and memory, and partly because of the difficulty of displaying the window in a small screen window.
The split screen can be split horizontally or vertically. Most games have split the screen horizontally into top and bottom, but recent trends for widescreen televisions may make a vertical split more visually appealing. Either split should be possible using Free Radical’s technology.
The split screen code is not a self-contained system, and its implementation is split between several other systems. The stress put on other systems (streaming and memory management) are the main risks with the split-screen code. The risk for split screen is mainly related to performance and the compromised in visual quality necessitated to achieve this.
===Wii-specific Split-screen===
Split screen performances will be a problem because the GPU will be utilized so fully by the single screen game. Split screen will be limited to 2 players with AI on single player maps. To make up for the extra cost of the extra view, shadows will be turned off and the far plane moved in to reduce the number of polys being rendered. High quality lighting will also not be used and will fall back to hardware lights to decrease the vertex processing cost. The rendered terrain resolution can also be reduced to help with performance on CPU and GPU.
When running in split screen the increase in GPU cost mainly comes from the extra vertices needing processing. Pixel cost should stay reasonably constant as the same area is being drawn regardless of the number of views. Considering a typical set of vertex data (position, 1 texture coordinate and a normal) it takes 8 cycles for the vertex processor to read in this data limiting the peak transformation rate to 30.38Mvtx/sec. A full scene made up of 200,000 polygons has approximately 400,000 vertices. This means processing 400,000 vertices will take at least 13ms. This is almost half of a frame. Having 4 views would take the time needed to process the vertices to 52ms! With two player split screen 30fps should be possible (by making the savings above). Without reducing the vertex count in the single player game, we cannot see four player split screen running above 15fps, an unacceptably low frame rate. Therefore, a limit on 2 player split screen will be enforced on single player maps. However, it will be possible to have 4 player split screen on the Wii-exclusive content where a lower polygon count can be enforced.
===Other systems===
====Cloth====
A cloth system is used on characters.
====Lip Sync====
Cut-scene characters will have full facial animation. This will be animated in sync with the English language, and dubbed over for foreign languages.
In game, characters will do a much less precise lip sync by semi-randomly blending between animator defined face poses whilst talking. They will also have a bank of face poses for idles, pain and firing. This system was used in Haze.
====Skinning====
On the Playstation 3, Xbox 360 and PC characters are soft skinned. This skinning is done on an SPU on the Playstation 3, and in a separate thread on the Xbox and PC.
====Wii Specifics====
On the Wii skinning poses two main problems: it requires a huge amount of processing and large buffers are needed to contain the results. These problems are exacerbated by the large number of characters that can be on screen in BF3. The Wii provides a cheap alternative to this called stitching. The Wii hardware allows up to 10 model view matrices to be loaded into the graphics processors memory at once. Which matrix to be used to transform a vertex can be selected on a per vertex basis. For each pair of bones, the two bone matrices are uploaded along with eight interpolated matrices. The model is split into parts depending on what pair of bones influence a polygon. Each vertex of the part has an index that specifies what combination of the two bone matrices it is influenced by. Because the skinning weights are quantized, it gives less smooth skinning (it's more like having 8 more bones between every pair) but this is rarely noticeable. Other disadvantages are that each vertex has to have more index data, the graphics FIFO has to accommodate a lot of extra matrix data and each polygon (not just each vertex) can only be influenced by a maximum of two bones. The big advantage is that there is almost zero CPU cost on this method. The PC uses 4 weights per vertex so some models may have to be re-boned.
===Memory management===
The memory management API is a mature system and has been used in Haze. It supports a number of different heap types, including:
*Arbitrary general purpose heap
*Fixed size heap
*Stack based heap
*Handle based defragging heap
*Object allocator
Different heaps are used so that the resources dedicated to each subsystem are easy to track – physics memory is allocated from a different heap to particle effects, for example. Easy tracking of memory use is very important, as it makes memory optimization easier later on in the project.
By default, all heaps are thread safe by use of mutexes. If a given heap is only to be used from a specific thread, it can be marked as not needing the mutex; this increases performance.
The defragging heap will be used for textures. The engine also supports its use for object data should that prove necessary. The defragging process is done either using the GPU, DMAs or CPU copies.
The object allocator is designed to provide minimal fragmentation when allocating lots of small arbitrary sized memory blocks. It is used by the framework for allocating C++ objects and similar small allocations. It was very successful in Haze at keeping memory fragmentation to a minimum.
====Wii Specifics====
The Wii has 88Mb total RAM split between two banks. 24Mb is T1 RAM which is extremely fast. The second bank is 64Mb of standard DDR. As a general rule the 24Mb T1 RAM should be reserved for code and frequently used data. The remaining 64Mb should be used for infrequently used data and assets. Both the CPU and GPU can access both banks. There is a further 3Mb embedded in the GPU but this is reserved for GPU caches and frame buffers.
The Wii vertex data format is a lot more flexible that the other platforms, allowing different index buffers for each parameter (position, texture coordinate etc) allowing for a reduction in repeated data. In addition tangents and bi normals are not needed on most objects due to the lack of normal maps. A further saving can be gained by using a static normal table shared between all assets. These techniques combined leads to an average reduction of 40% on vertex data.
Smaller textures and a lack of normal maps reduce the texture cost significantly. The Wii hardware limits textures sizes to 1024x1024. As a lot of assets currently use 2048 by 2048 color maps which if down sampled to 1024x1024 reduces the memory required by 75%. Specular maps will be reduced to 256x256. If bump mapping is used it will be combined with specular map.
Texture and object streaming will be used more aggressively reducing the memory further.
Any texture (or mipmaps of a texture) smaller than 256x256 are loaded statically. Bigger textures are streamed as needed. To determine what textures need streaming in, each surface is stored with a bounding box and an average texture density. The bounding box is used to calculate how much of the screen the surface is occupying. This value is then multiplied by the texture density to obtain the size of texture needed to achieve 1 texel per screen pixel. If this size is greater than the texture's “requested size” then the requested size is increased. Each frame the requested size is reduced allowing textures that haven't been visible for several frames to be streamed out.
The stream buffer is allocated in blocks, large enough to accommodate a 256x256 compressed color texture. Only 256x256 and 1024x1024 textures are streamed in. 1024x1024 textures use 16 blocks. If a texture is to be streamed in, first the 256x256 is streamed in. If no more 256x256 textures need to be streamed then 1024x1024 textures are streamed in and the respective 256x256 textures are freed. If a 1024x1024 is streamed out then the first 15 blocks are freed to leave a 256x256 texture in the last block.
To make space for the larger textures the 256x256 texture slots are defragmented. This is done by using the DMA functions of the locked caches. These functions give a 330Mbps copy bandwidth in MEM2. Meaning a 256x256 texture can be moved within a microsecond. During periods of defragging around 4 moves are done per frame.
When a block is freed or moved, it old block will not be reallocated for 2 frames so that all references to the texture still in the graphics FIFO can be flushed out.
Memory map
:
System	
Size (Mb)
OS	
1
GFX Fifo	
1
Frame buffers	
1
Terrain height map	
2
Shadow buffer	
2
Animation	
4
Sound	
5
Physics	
5
Object data	
15
Static textures	
15
Streamed textures	
15
Game	
22
TOTAL	
88
===Asset resource / management===
Asset management code is stored with utility code in the engine, and is used by the game to make sure it is loading the correct version of different asset types.
People who create assets to put in the game (generally referred to as artists, but this includes designers, musicians, animators and others) create them and store them in editable file formats. These editable files are checked into Perforce.
The engine’s tools system reads these editable files and converts them into in-game formats. This allows us to change the format of assets for different platforms, but also means we can add new features without artists having to re-export all of their assets.
Whenever an asset file format changes, a version number associated with that format is also changed. This version number is included in the path of the asset the game looks for. If the game looks for an asset that does not exist, the game runs the tools to attempt to convert that asset.
To illustrate this, imagine the game sound file format changes to incorporate a new effect. The old sound file format was version 1, so the new version is 2. When the game is rebuilt with the changes, it will look for a file called ‘sound_v2/gunshot.snd’. This file doesn’t exist at present – only ‘sound_v1/gunshot.snd’ is there, so the game starts the tools to build gunshot.snd. The tools will only produce the latest version of the asset, so will output ‘sound_v2/gunshot.snd’ which is then loaded by the game.
As an optimisation, by default the tools will look to see if other team members on the network have the missing asset, and if so it will copy it from them. This saves everyone on the team having to convert a new asset individually. The first machine that is checked for the asset is the central asset build server. This sever automatically rebuilds assets as they are checked into Perforce and always has the latest converted set of all assets in the game. Users can batch copy their assets from the server if they want to refresh all their assets to the latest versions.
Additionally, the build server supports a simple branching system that allows a separate set of tools and converted tools to exist. This allows for ongoing work on the engine and tools which change assets, and for all assets to be converted and tested in isolation.
This asset management system was successfully used in Haze.
===Core libraries===
The engine contains some commonly used core utility code which is used throughout the game and the engine.
There are a set of math functions, dealing with matrices, vectors, quaternions, trigonometry/angles, clamping and wrapping numbers between limits, and filtering numbers. There is also code to generate noise (including Perlin noise) and random numbers (this contains various utility functions for generating numbers in a range).
Various containers are used in different places. A pool system is often used for tracking large numbers of small objects, since it provides a fast (O(1)) allocation and is simple to set up. Linked lists are used to keep track of many game objects that need to be iterated through. Queues and stacks are also available to use. A sort algorithm makes it easy to use efficient sort algorithms throughout the code.
A string pool can be used to hold large numbers of strings, and a stringtable to link keys with other data. These are combined to create a dictionary class, which can be serialized and stored in plain text format. This dictionary class is used to store the data necessary to run the game. It is convenient to store this in text format during development because it makes it easy to store the files in version control and merge changes together when two people are working on a file at the same time.
An assert module is used to keep track of conditions which could be harmful in code and warn the user when something goes wrong. These are disabled for release builds, but make debugging significantly easier during development.
The code is mature and used throughtout the codebase.
===Localization plan / system===
UTF-8 encoding is used for the language files. This allows UNICODE text to be represented whilst keeping the memory footprint as small as possible. The text is stored in Excel-loadable spreadsheets, which eases the localization process. A command line tool is used to convert from the spreadsheet to an in-game format. The spreadsheet can then be sent for localization. When a new localized version is returned, the normal game build process will produce the new versions of the in-game strings automatically.
Where the strings in the spreadsheet are associated with an audio sample (subtitles), information about the soundid is stored with the string. This makes editing strings, audio and localization easier by providing a central file to tie it all together.
Localization tools are also available which allow an external localization team to change the spreadsheet or localized audio samples and preview them on a build. This means localization and localization testing can be done by a team external to Free Radical. This system was lightly used in Haze, but it would be low risk to make much heavier use of.
===TRC compliance plan===
Several tools are planned to make it easier to test and debug TRC issues.
An in-game display of TV safe areas will be used to make sure important information is only displayed within the appropriate area of the screen.
Memory card debugging code is also planned, to simulate different actions being performed while the game is running. The code will simulate memory card insertion and removal; other actions may be added depending on what the TRCs contain.
===Debugging and testing tools===
Debugging code is built into in-game systems as they are added to the game. Hooks to this debug code are added to runtime debug systems (an in-game menu, a debug manager, and various debug overlays).
The in-game menu can be used to turn systems on and off, and to adjust parameters. It is easily used by people with little technical expertise to alter settings, but requires specific code writing for each use.
The debug manager allows systems to be turned on and off and tweaked using a text file. This is mostly used by programmers to alter settings since it is easily reloaded and added to.
Debug overlays are used to convey information about how particular systems are working. A memory overlay can be used to see how much memory is used by different sub systems, and an audio overlay shows which sounds and music are currently playing.
An in-game profiler allows game code profiling while the game is running. This often proves useful for optimizations, since it can be used to work out in detail why a particular level is running slowly.
===Vista===
Windows Vista OS compatibility needs to be added to the engine, and Vista certification gained. 64-bit compatibility is also required, but this is not a difficult task because a 32-bit application can still be used.
===Games for Windows===
This will be available for both Vista and XP, and is designed to integrate games better and more consistently into Windows. It requires Games Explorer integration, which allows games to launch easily, and Rich Saved Games, where the save game file features information and an image about the status of the game (eg campaign level reached).
===Additional PC features===
A game launcher will be written, that appears when the DVD is inserted, with several options, including Play, Install, Help, and Dedicated Server. The game installer will install the game onto the hard disk. The installation software is likely to be either Microsoft Installer, or the more common choice Install Shield (version 12 or later).
==Production Pipeline and Tools==
===Build process===
====Code Build process====
The code is stored using Subversion version control system. Makefiles are used to produce a build from a fresh checkout.
====Asset Build process====
All assets are stored in Perforce version control system. A single in-house tool called resconvert can build all of these assets to the format used in-game by a given platform (Xbox 360, PS3 and PC all have slightly different asset file formats, and the tool can build whichever format is required). This tool can build all the assets required from a single command, but can also build individual assets or groups of assets if required.
====Automated builds====
Code builds are automatically performed when new code is checked in. If there is a compile error, all users who have committed since that last successful build are altered so the matter can be resolved as quickly as possible.
Asset builds are performed automatically whenever a new asset is checked into Perforce for all platforms. The asset builds are done by a cluster of dedicated asset building machines. Additionally, full nightly asset builds are also done which allows for our tool chain to be completely and fully tested regularly. If errors occur during an automated build, the artists and their lead are automatically notified and sent the error log from the tools. Additionally, the state of all assets is stored in a MySQL database allowing for further querying. Any assets that fail to convert are not stored onto the build server. Team members can sync their assets from the build server whenever they wish.
====Automated Tests====
There are plans to investigate the use of automated testing for Battlefront III. This will involve at a minimum load testing levels regularly, but the plan is to extend the testing to track memory usage, load times and other statistics. A formative system was used in Haze, but more work is needed. This is low risk as it is not critical to the games development.
====Data backup procedure====
Programmers and designers have their machines backed up every night, and artists machines are backed up every 3 hours. Subversion and Perforce are backed up twice a day, and a Perforce checkpoint file is taken once a day to allow the database to be rebuilt. A rebuild of the Perforce system from backup has been tested, and is known to work. Free Radical is currently based across four buildings on one site, and information from each building is backed up to a different building. Subversion and Perforce backups are also to be stored off site on a regular basis, though this is not currently carried out.
====Tool version rollout process====
The artists' tools are built from Subversion on a nightly basis as part of the automated build process. Testing of the tools is done manually. If the changes have caused no obvious bugs, the versions are switched on a central server. When Maya is started up it checks for new tool versions and copies them from the central server. The tools can also be changed manually if necessary.
Tools used for creating in game assets are built as part of the game build process, since these need to be kept synchronized with the current game version. As mentioned previously, these are tested as part of the nightly automated build process. Experimental tool changes can be checked into a separate branch in the asset build system where they can produce a complete set of assets using the new tools. These assets can be synced to any machine for evaluation. If the test is successful, these changes can be promoted into the main repository where the tools and assets will then be available to the entire team.
====Build delivery process====
Builds are to be delivered electronically using rsync.
===Pipelines===
PPlines.jpg
====Version control system====
All of Free Radical's assets are stored using Perforce version control system. This gives us the ability to return to earlier versions of assets and also provides a sensible backup point – all assets must be stored in Perforce, and can be backed up from there.
====Authoring process====
All assets which the game loads are referenced in data files, and managed using a general purpose asset management tool called resconvert.
resconvert reads the asset list, and can rebuild any of these assets from Perforce (or directly from the artist's version). It runs different tools based on the asset type, but means that all assets can be rebuilt using one interface. During development, it means the game can also rebuild assets itself (if any are missing or old).
All assets have a type and version number, and are placed in a directory structure according to these. This feature makes it virtually impossible to load an old asset, which reduces time spent finding bugs. Each converted asset is also stored with a manifest, which describes the input files necessary to build the asset, the generated output files, and the date the asset was built. This allows resconvert to rebuild assets only when the source data changes, but it also means we can copy assets from one developer to another without processing them. All assets are converted every night, and can then be copied to any machine, which saves lengthy waits for assets to convert.
A custom Maya exporter outputs a scene file in XML format, which allows the viewing and extraction of large scenes using off-the-shelf components as appropriate. The exporter supports animations, props and backgrounds.
The converter takes in the bulky XML data for backgrounds and props and exports it into a streamlined format applicable for use by the appropriate hardware.
The imageConvert tool converts the image formats used by the artists into custom formats targeted to the hardware.
The animConvert tool exports animations in a format that can be loaded into the game engine without further time-consuming processing.
====Characters====
Characters are designed and reference material gathered. A low polygon game model is built in Maya. Whilst the model is built it is bound to the skeleton. Testing and tweaking is then done iteratively.
A high polygon model is generated in Z Brush by sculpturing into the low polygon model. This is used to create 2 normal maps, a 2048x2048 texture for the body and a 1024x1024 texture for the face, hands and eyeballs. These normal maps are edited in Photoshop to remove seams and artifacts. Cloth and skin detail is so fine it is generally not modelled in Z Brush and so needs to be painted on at this stage.
The diffuse map is the base texture, often generated by piecing together photos. The specular map is generated to distinguish shiny and dull surfaces. Again 2 of each map are created, a 2048x2048 texture for the body and a 1024x1024 texture for the face, hands and eyeballs. The skeleton binding is then tweaked.
These props are then exported to XML, and built using a tool called the converter. This process converts things to in-game formats.
====Backgrounds and Props====
Backgrounds and props are built and textured using a combination of Maya and Photoshop. A Maya script exports the files to an XML format, which can be read by toolchains. A tool called Bacon can be used to 'bake' on radiosity lightmaps. The XML file contains references to all the textures used by the model, and the actual data for the model itself. A tool called converter parses this XML, creating platform dependent versions of the model and textures in an in-house file format (the model files are generally referred to as 'ob' files). The amount of time it takes to rebuild an asset (so an artist can see the impact of a change) depends on the size of the assets – a capital ship takes significantly longer to rebuild than a starfighter or a gun. In order to improve this, changes are being made so assets can be worked with in smaller parts.
Artists can also preview versions of their artwork using a cut down version of the engine, called the QuickViewer (also referred to as 'The Stage'), which can reduce preview times as the asset doesn't need to be fully processed – stages such as texture compression can be skipped, for instance.
====Terrain====
The terrain assets are also built from resconvert. They are built using a proprietary World-Editor tool, which integrates terrain and art assets, as well as effects. This allows designers and artists to edit the heightfield and see changes to this in real-time. It allows artists to edit their appearance by painting on materials.
The world editor can export a version of the terrain to be lightmapped (by Bacon). The lightmaps take several hours, but this exporter allows either all the static objects on the terrain to be re-lightmapped too, none of them, or a particular selection, so the artist decide what needs to be redone and hence saves time.
The world editor is also responsible for positioning buildings, foliage and other props which do not move round the level. This can be partially automated – foliage can be made to appear only in particular parts of the level (only where the level is grassy, for instance), and more complicated positioning rules can be added for buildings. The user still maintains complete control over this process and can hand edit things where necessary.
Non terrain levels can be built using the world editor too. Each section of the environment can be built as a separate mesh, and imported, positioned and saved in the world editor. Portals can be connected using the world editor.
An in-house tool called terrainConvert then processes these assets to produce platform dependent versions, which can be loaded by the game.
====Animation====
Animations are created in Maya, and then exported using a Maya script to an XML format. This is processed by a tool called animConvert, which builds the animation data into a platform dependent format and compresses it. Once added to the asset list, animations are generally more difficult to preview in-game than artwork, but they can still be previewed using the QuickViewer. This process is currently being streamlined to provide a faster turnaround.
====Sound and Music====
Sound and music are created using a variety of tools, based on the musician’s preferences. However, since there are industry standard file formats, this isn't a problem. Sound files can be read from WAV files, and converted into platform dependent files (normally some form of ADPCM compressed files for in-memory sound effects, and OGG/WMA/other compression formats for streamed files). Sounds and music can be previewed in-game using a debug tool, and turnaround on this is usually very quick.
====Game asset integration====
Every asset is listed in a set of resfiles, which are text files containing structured information about each asset. New assets are currently entered by hand by a designer, specifying the asset type, location, and associating the appropriate attributes with the asset. This process will be moved into the art and setup editors, so that this can be done by the artist who created the asset.
====Level Layout====
Designers can prototype parts of levels rapidly by using QRadiance, a freeware Quake 3 editor. The editor allows designers to “block-out” layouts, and then produces files which can be loaded into the game quickly, without the designers having to worry about texturing or adding detail to the level. Once the level is suitably advanced from a gameplay perspective, the level can be converted to a Maya format so an artist can work on the level. The artist can replace a room at a time with the finished version without breaking the playable version.
====Texturing====
Textures are created in Photoshop, before being applied to a model in Maya, or added directly to Perforce to be used for particles and various other effects.
====Lighting====
Point lights, directional and volume lights are set in Maya. These are used by the bacon lightmapper to generate the lightmap and the cubemap ambient lighting. They are also exported with the environment to apply lighting and shadows on dynamic objects.
====Materials / shaders====
Materials are specified in Maya plug-in, allowing an artist to specify textures for diffuse, specular and normal maps and several others. Each surface in a model is associated with a material. When the model is converted, it selects the correct shader from this information.
====Cutscenes====
Cutscenes are completely animated in Maya. All characters and moving objects in the scene are animated along a timeline, and the camera is directed. These are exported in xml format and converted using the animConverter tool for use in-game.
====Physics / collision====
Collision geometry is generated automatically in the converter from a model’s mesh and saved as a separate file from the model. This is in the format the physics system is expecting. Additional geometry can be tagged for collision only, and this will never be rendered.
Cloth is modelled in Maya as normal geometry, but put under a node labelled CLOTH, which is automatically output by the converter as cloth. Vertices that are not supposed to move (i.e. attached) are flagged by the artist.
====Game logic====
Game logic can be edited using a system of events and vm scripts using the in-game editor. These are saved to text resfiles, which are committed to Subversion, and loaded by the game as data files.
====Special FX====
Special effects are currently both placed in the environment and tweaked using the particle editor in our World Editor, rendering exactly the particles as they are seen in-game.
====Conventions====
The assets directory is split into animation, sound, characters, weapons, vehicles, backgrounds, capital ships, and prop folders. These are further split as determined by the lead for each discipline. Where appropriate, subfolders are created called cis, imp, reb, rep to stand for the CIS, imperials, rebels, and republic respectively. The asset names also incorporate these abbreviations.
Example:
*bf / capital_ships / cis_commandship_interior / cis_commandship_interior.mb
*bf / capital_ships / cis_commandship_interior / cis_commandship_interior.xml
===Tools===
====Off-the-shelf Tools====
*Maya
*ZBrush 2
*Adobe Photoshop
*Perforce
*Steinberg Nuendo 3
*Steinberg Cubase SX 3
*Adobe Audition 2
*Sony Media Sound Forge 8
*Basehead (Sound Effect Database)
*Reaktor 5
====In-house Tools====
The following tools all exist already, and can be used to get working assets in-game. Screenshots are provided for those with a visual interface.
* The Stage / Quickviewer - this is the main preview tool used by all departments during game development. It provides a simple way to visualize an art asset, both for checking construction conformance and shader setup. It helps assess performance implications in a more controlled setting. Artists can preview their art in this tool from a single button within Maya. Textures can be changed dynamically from within Maya or Photoshop and will automatically update in the stage, allowing instant appraisal of changes.
The stage provides a host of extra features for checking art, such as disabling shader passes. Several preset lighting setups can be flicked through, and custom lights can be added. Parts of the model can be turned on and off to check the meshes, and animations can be played.
* GUI Editor – the GUI editor can be used to define all in-game menus (both the front end and actual game menus).
* Editor – The editor can be used to tune different aspects of the setup of the game. AI (placing and scripting characters, editing navigation information), and props (placing props in the level, scripting their behaviour, and altering their properties) can be changed. The editor is built on top of the game framework code, which makes it possible for designers to test their work without exiting the editor and starting the game.
* Profiler – The game features in built profiling code, which is designed to allow measuring of code performance and tuning it accordingly. This works on PC, PS3 and Xbox 360, and is built in a platform independent manner to allow porting to different platforms.
* Bacon lightmapper – A program to generate lightmaps based on Monte Carlo integration. Artists run this tool, which allows lighting to be baked onto the background from lights placed in the scene. The lightmapper can be distributed across a number of machines, which reduces the length of time it takes to make a lightmap.
* World editor – This package integrates 3 editing tools, the art editor, the terrain editor, and the particle editor, featuring a shared GUI.
Multiple users are not able to edit the same terrain or background at the same time, although they are able to edit the background components separately (in Maya). This is not be a problem, because even though multiple artists work together on a background, only one person is generally responsible for the terrain heightmap and placing together the components.
====External Tools====
=====MySQL=====
In house tools can use MySQL database tables to store information. Using a database makes it more difficult to version control information, so the tables are normally only used for session information.
For example, if a section of the terrain can only be edited by one user at a time, a lock can be created. The user’s session will be recorded in a MySQL table. When a user tries to access a new part of the terrain, the editor then checks the database for any existing locks on the terrain. If there are no locks, a lock is added to the table referencing the users session. If a lock is present, the user cannot access the terrain section until the previous user finishes with the terrain.
=====GTK=====
Tools which require a graphical user interface, such as the in-game editor currently use an open-source library called GTK to draw and handle their windowing events. This is convenient because GTK runs on Windows and Linux, and the library is well documented and widely used by the open-source community.
GTK has several failings. It is easy to write buggy code in GTK, because the library is not re-entrant. It can be difficult to arrange GUI widgets as the programmers wants because the library is often unintuitive. There have also been problems when running editors on new machines where the library cannot find the GTK dll files.
Having used GTK for some time and decided there are problems, work has begun on a Free Radical GUI library which is smaller and easier to maintain than GTK, which will give us more control over future tools. This new in-house GUI is being used to construct the new terrain and art editors.
===Asset Pipeline Summary===
Asset Pipeline	
Exists	
Tested	
Data driven
Static Models	
Yes	
Yes	
Yes
Dynamic Models	
Yes	
Yes	
Yes
Animated Models	
Yes	
Yes	
Yes
Animations	
Yes	
Yes	
Yes
Scenes/Backgrounds	
Yes	
Yes	
Yes
Collision Data	
Yes	
Yes	
Yes
Audio FX	
Yes	
Yes	
Yes
Audio Streams	
Yes	
Yes	
Yes
Movies	
Yes	
Yes	
Yes
Lighting	
Yes	
Yes	
Yes
Particle Systems	
Yes	
Yes	
Yes
Scripting	
Yes	
Yes	
Yes
HUD	
Yes	
Yes	
Yes
Frontend	
Yes	
Yes	
Yes
Splines	
Yes	
Yes	
No
Shaders	
Yes	
Yes	
Yes
Visual FX	
Yes	
Yes	
Yes
Game Data	
Yes	
Yes	
Yes
Cutscenes	
Yes	
Yes	
Yes
Terrain	
Yes	
Yes	
Yes
==Appendix A - Framework Overview==
===Introduction===
This document briefly outlines the approach taken to creating the framework and tool chain used by the game. The emphasis is on the basic philosophies and concepts rather than the specific implementation.
===What is the 'framework'? ===
The framework is a set of C++ classes that allow you to write a game more easily. It's a bit like a games makers' library but because it's in C++ the flow of control and information is more tightly coupled with the game.
You are more likely to use the framework by deriving from bits of it and fusing your game specific code in with the main library of reusable code, thus framework calls game code and game code calls framework and the boundaries between the two are, to a certain extent, non-existent.
===Why is a framework a good idea? ===
There are a number of advantages to using a framework to build your game.
====Reusability====
Probably one of the main benefits is re-usability. Any code that is written to work with the framework can potentially be used by any other game that uses the framework. This isn't referring to copying and pasting some code into one game from another, but literally adding the bits from one game to another by simply linking to the classes you want to use. A well written module will have limited dependencies on other modules increasing the likelihood that it can be independently deployed.
====Scalability====
The framework has been designed with scalability in mind. The game will instantiate only the objects from the framework that it needs. If a game doesn't need to use certain features it doesn't even need to link to them.
Much of this is achieved through a class registration system, where modules such as props, components (more on these later) and managers are registered during start up, and then instantiated on demand by actions such as loading the setup for a level. If you don't register a module then you don't need to link to it.
This gives us the ability to have massively different games running off the same framework, sharing just the bits of it they need to.
====Open Data Format====
The framework provides a common data file format called the resfile. Resfiles are a flexible and easy to understand text file format that provide the same level of descriptive power as XML; whilst avoiding XML's convoluted syntax.
All level setups, manager configuration settings and asset conversion information are contained in resfiles. The framework has functions to manipulate resfiles making them very easy to use for arbitrary data storage.
The standardisation on the open resfile format means the game itself is no longer needed to extract information from data files. This opens up the possibility of offline game data processing utilities, such as utilities to analyse asset usage or make batch changes to data files.
====Data Driven====
The framework also supports a very strong serialisation model. Any game object can be converted to a hierarchical data structure of key/value pairs called a dictionary. The key is any arbitrary string, and the currently supported value types include ints, floats, strings, arrays and other dictionaries. Data structures like this are sometimes referred to as associative arrays.
These dictionaries can be trivially loaded from and saved out to resfiles. This makes it very easy to create an instantiated C++ object from a data file description on disk, in fact a resfile is just a dictionary flattened out to a text file.
It was mentioned earlier that the C++ classes are all registered so they can be easily created by name. The dictionaries contain the names of all the objects to be created, as well as all the data values they will need to set themselves up. This means that by simply changing a data file, such as a level setup, you can change all the values for an object, or even which C++ class is created to implement that object.
If a game executable was compiled that registered all framework and game specific classes, it could actually be made to run any framework-based game by just pointing it at the appropriate data folder - without the need to recompile.
The strong serialisation model also means that all code to load and save an object is embedded within it, drastically reducing any errors due to loading and saving code being out of sync. It also allows an object to be saved out to a dictionary at any point, meaning display and editing of the dictionary of any serialisable object is possible in game and in real time. The same serialisation code that is used to load and save from a setup file could also be used to load and save saved games or checkpoints, massively reducing the amount of work needed to implement this feature.
The combined use of a strong data driven architecture with an open data file format will also make it easier to create a 'mod-community' for the PC platform.
====Modularity====
The framework promotes strong code modularity by using a components based architecture. The easiest way to think of a component is an object that performs a service for another object.
A NPC prop for example, is comprised of a render component, a navigation component, a health component, an animation component and a state machine component. Each of these components is completely responsible for a particular service to the prop. The prop 'glues' all these components together via it's tick. It will call various functions in each of its components to get the job done. Each of the components can also call the others.
As the components have limited responsibility they are often very simple and easy to reuse. It also means that you can completely change how a part of a prop is implemented by just changing the data file to create a different component. Eg a cartoon rendering component instead of a standard render component, or a navigation component that uses a completely different set of data. Neither of these changes need require code alterations.
====Quick Development====
Due to reusable nature of components, it becomes very easy to quickly develop new props. For instance, adding a new AI spider that would crawl about the level could reuse generic state machine, physics and navigation components, meaning you only have to write the new code and, more importantly, you reuse the existing code without polluting it in any way. If you do need to make a change to they way a standard component works, you just create a new C++ subclass and override the function you want, then change your data file to create your new subclass.
It might take longer to create the code initially, as you have to think about how to factor your prop into components that are as independent as possible, but once it's done you will save time with future development and gain easier debugging due to components being simple and easy to swap out and isolate problems with.
====Unified Editor====
The framework also comes with an editor that lets you set up props in the world. In the framework a base prop is very small, it essentially just has a position and rotation and not a lot else. By making all parts of your setup into props you gain the ability to edit them in the editor all together in one place.
It may seem a little surreal to be changing things like alarms or event triggers into props, but these props won't have any rendering components or anything, they'll just exist so they can be referred to and edited in a standard manner.
The editor can runtime query objects in a scene as to what components they have, this allows the editor to use a custom interface for more effectively displaying the setup. For instance, the event prop mentioned earlier can be configured to have a volume based trigger and a message sending action. The editor then detects that this prop has an editable volume allowing it to edit it visually. It also detects that the prop can send messages and lets lines be visually dragged between props to display the communication channels. Of course, because the editor is implementing these custom user interfaces based on the presence of components, the exact same editing code can be invoked on any prop that has that component, not just event props. This will reduce the amount of work required to develop an editor for a new prop.
====Asset Toolchain====
The framework, engine and tools work together to provide a robust toolchain for the management of game assets. All information on an asset and how to convert it are stored in resfiles. A tool, resconvert, is responsible for processing these resfiles and calling the relevant tool to convert the asset.
The file format version of the resulting file is automatically encoded into the asset file path as it is converted. When the engine attempts to load an asset, it encodes the version of the file format that it understands into the path. If the asset file is not found by that path then it means the correct version of that asset isn't available locally. There is no distinction between having an old file format version of an asset and never having had the asset. It will no longer be possible to accidentally load an outdated file into the engine.
If an asset is found to be missing, the engine will ask if you want to remake the asset. If you do, it runs resconvert to make the asset. If resconvert finds the correct version of the asset on the nightly build server, it will simply copy the asset onto your machine - taking a fraction of the time of actually converting it. If the asset is not on the nightly build then it converts it locally.
This works for all assets: animations, objects, fonts, misc textures etc. It is possible to run the game with no assets at all, and have it simply generate the assets it needs as it goes.
Resconvert also has Perforce integration, meaning that it can remake objects from any date or label that have been put into Perforce.
==Appendix B - Framework Technical==
===Introduction===
This document gives an introduction to the basic framework classes and functions. It explains dictionaries, scenes, props, components, memory management and managers and the relationships between them. It also covers basic engine modules and how they interface with the framework.
===Dictionaries===
A dictionary is a list of key - value pairs. The key is any arbitrary string, and the currently supported value types are int, float, string, array and dictionary. Dictionaries like this are sometimes referred to as associative arrays.
As dictionaries can contain other dictionaries, it is easy to construct hierarchical data structures from them. Look up of values is done from a string key, but dictionaries internally use hash tables meaning this is typically very fast; however you would probably want to avoid many dictionary lookups in time critical code.
Dictionaries are implemented in the framework by the abstract CDictionary class. The most commonly used implementation of CDictionary is CHashDictionary.
Dictionaries can be easily loaded and saved to resfiles by the functions dictionarySaveFile() and dictionaryParseFile(). Objects can also be serialised to and from dictionaries making them ideal for setup and configuration files.
===Templates===
A dictionary can also have a template associated with it. A template is a dictionary of default values that will be used for keys that aren't specified by the dictionary itself. In most cases, you will specify a template for any dictionary you create as it minimises the amount of data you have to record. The template is implemented in the framework by the CTemplate class.
In a level setup file, you would try and put any data that is not unique for a prop instance into a template. That way, when specifying a prop, you only ever need specify the template, the position and the rotation. All the rest of the data to define the prop will be found in the template. This allows you to easily change settings for all props in a setup just by changing the templates.
Screenshot_24.png
Values inherited from a template can be overridden just by specifying a new value for the key in the dictionary.
Templates can also inherit from a parent template, allowing you to easily construct hierarchies of templates. For example, all props' templates in a setup file will have the prop template at their root. The template hierarchy doesn't need to follow the class hierarchy, they are completely independent. However, often they tend to mirror each other just due to ease of organisation.
===Metadata===
Templates can also contain information on fields within the template, this is called meta data. A template could for instance define that the field health be a float value between 0-1. Meta data could be used to allow dictionaries to be validated by separate tools before they are loaded into the game. Tools such as the editor can use this information to apply more meaningful user interfaces to the data; in this example, the editor could display the float value as a slider bar between 0-1. 
Screenshot_25.png
===Props / Components===
Props in the framework are much smaller structures than the props of the previous engine. This is because all the code that implements functionality for a prop is placed into components.
A component is an C++ class that performs a particular task or provides a particular service. A prop uses components to gain functionality. For example, a standard crate would have a physics component, a health component and a render component. The prop is responsible for 'gluing' the components together to produce the overall result. The idea is to make each individual component as self contained as possible, thus making it easier to reuse or replace.
===Prop Duality===
The main prop class in the framework is the CGameProp class. It is a subclass of the more general CBaseObject class. A CBaseObject is an class that has a type id (a four char code currently) and functions such as GetComponent() and Serialise(). The prop is a CBaseObject of type 'prop'.
The CGameProp creates and manages an engine counterpart called an engine prop. Engine props are very minimal data structures containing just a trans/rot matrix and little else. An engine prop can have an obinst, but this is created and attached separately. The obinst is created and attached to the engine prop by the CObinstRenderComponent, which most props use as their render component. However, using a different render component allows for procedurally generated rendering of props, such as the CParticleRenderComponent.
===Component Access===
A component is implemented by the CComponent class. Like CGameProp, it is a subclass of CBaseObject. All components are subclasses of this CComponent class. The CRenderComponent for instance, is a subclass with the type 'rend'. 
Screenshot_26.png
This form should always be preferred over the previous because of the safe casting and because it allows for an additional layer of redirection before asking the prop for its component, which is good for debugging.
There are no required components, any component request can come back NULL. Obviously you can make certain assumptions when you know what you're dealing with, but for an arbitrary prop you need to be careful.
When interacting with a prop, the desired approach is to ask it for the relevant component that implements that feature, and then interact with the component. So to damage a prop, you would ask it for its health component and then, if it returns one, call the Damage() function.
The prop can have a customised health component that ignores certain types of damage, or handles switch flag states as the health is reduced, or plays animations etc.
===Registration===
All base objects that you wish to create from dictionaries need to be registered. The registration maps a string known as the class-id to a function that will create a new object of that type. Here is an example registration function:
Cplusplus1.png
A sample class registration function
This function would be called once at program start up. It creates a static CObjectAllocator which is responsible for allocating all CBaseObjects from pools and grouping similarly sized objects into the same pool. It also creates a CObjectFactory which is responsible for mapping the class-id to the function to create the object of that type. The class-id is defined in any CBaseObject class header using the CLASS_ID macro.
===Serialisation===
As mentioned previously, all base objects can be serialised to and from dictionaries. This is used when creating objects from dictionaries, such as when loading a setup file. To create an object from a dictionary use dictionaryLoadObject() or the type safe wrapper dictionaryCreateTypedObject().
dictionaryLoadObject() first retrieves the key 'class-id' from the CDictionary passed. It then uses the CObjectFactory class to create an object from that id. This is followed by calling the objects Serialise() function, which is passed the dictionary so the object can load the rest of the data itself. The dictionary is put inside a CSerialiseStore class before it is passed to Serialise(). The CSerialiseStore class provides utility functions for loading data from and saving it to the dictionary. Here is an example Serialise() function: 
Cplusplus2.png
This example shows a Serialise() function that serialises two values, a float and a string. It then serialises a sub-component using a sub-dictionary inside the main dictionary. When loading, this will allocate the object if m_componentPtr is NULL, if it's already allocated it will simply call its Serialise(). Components are serialised to/from their own dictionaries inside the main dictionary, in this example the sub-dictionary is called 
"mysubcomponent". This would load from/save to a dictionary resembling the following: 
Cplusplus3.png
Example dictionary that could be produced or used to load the above object
The Serialise() func above calls the inherited implementation of Serialise() first, this ensures that the data needed by the parent class is serialised. Using these serialisation methods it is very easy to change the settings for an object on disk, or change it to use different classes for itself or any components by just changing the class-id key.
===Runtime Creation===
When creating objects on the fly, a dictionary must also be used in order to create the object. In these cases it makes sense to create the majority of the dictionary in advance and just do minor alterations to it before creating the object with it. Different implementations of the CDictionary class which are less flexible but faster could also be used for more efficient runtime creation of objects.
A utility function propMgrCreateTemplateAtPos() exists to create a prop at a position in the world using only a template name.
===Managers===
A manager is a module that is responsible for managing and implementing a game wide feature. Examples are CNavMgr (a service used by the AI), CParticleSystemMgr (for creation and manipulation of particles) and CPhysicsMgr (for creation of rigid bodies and interacting with physics in general).
The idea of having data files define what class of object is created and all the object's settings can be extended to managers too by making the managers into 'singleton' classes. A singleton class is a class of which there should only one instance of at any time. To use the manager, you call a static function, conventionally called GetMgr(), which returns a pointer to the current instance of the class; you can then call its functions.
An example use of a manager class. This is using the physics mgr to do a line test: 
Cplusplus4.png
Having the manager as a singleton brings numerous benefits:
====Subclassing / Customisation====
You can subclass the manager class and override some of its functions to alter how the manager works for your game. Then all you need do is create your customised manager instead of the normal manager.
Along the same lines, you could even have completely different implementations of the same manager and switch between them during development. For example, the CPhysicsMgr is an abstract class. It used to have one concrete subclass, CODEPhysicsMgr, which implemented all the physics mgr functions using the Open Dynamics Engine physics library. This library was found to be too inefficient, so the physics manager was switched to a different implementation using Ageia PhysX (Novodex), CNovodexPhysicsMgr. Incidentally, the two tiered approach to physics is followed throughout the framework, eg CSimplePhysicsComponent / CNovodexSimplePhysicsComponent, CPhysicsMovementComponent / CNovodexPhyicsMovementComponent etc.
====Memory Management====
As all the manager state data is encapsulated into member variables in your class; deleting the manager class automatically frees ALL memory associated with your manager.
====Serialisation====
By making your manager singleton class into a CBaseObject subclass, you gain the ability to Serialise() the manager. This means the manager can be data driven by resfiles in the same way as the rest of the framework, this includes being able to change which class is created for the manager by changing the class-id in the resfile.
There is also a helper function called mgrLoad() which creates the manager from a resfile and then sets up a watch on the manager's resfile. If the resfile is edited while the game is running, the manager automatically reloads its settings from the resfile giving real time feedback with no extra work. Once a real time dictionary editor is available, you will also be able to edit the manager's settings and get live feedback, again without any extra work.
===Scenes===
A scene is similar to a game state in the old engine. Scenes are objects that are responsible for the current game state, for example there is a CFrontendScene, CLoadingScene, CGameScene and a CEditorScene. A scene can push another scene into a queue along with parameters and then the scene manager will switch scenes.
For instance, main() would push a CFrontendScene into the queue. This would start ticking and handle all the menus. When a level is selected the CFrontendScene would push a CLoadingScene with parameters to load the selected level. The CFrontendScene would then be deleted and the CLoadingScene would start ticking. It would load the level and display a progress bar. When the loading is complete it would push a CGameScene to begin the game.
The scene should control all aspects of the game, including all ticking. The CGameScene for example ticks all the props and physics, but the editor scene doesn't, so the props don't move around whilst you're editing.
The management of scenes is controlled by CSceneManager and all scenes derive from the base CScene class.
===Memory Management===
Memory management is provided by the engine. The engine has different heaps that you can memAlloc() from. Each heap, referenced by a MemGroupHandle can be a different size and employ a different allocation algorithm.
When calling the C++ new operator to create an object, you should specify which heap you want to allocate it from:
CMyObject *ob = new (g_heapGame) CMyObject(2,5);
Most objects should be allocated from pools rather than allocating directly from the heap. A pool is just an array of objects but has a very fast ( O(1) ) allocation and freeing algorithm. All CBaseObject subclasses are automatically pooled by the CObjectAllocator class. Non CBaseObject classes, such as CBullet, provide their own implementation of new and delete which allocate from a pool in the CBulletMgr class.
===Debug Memory ===
Any memory or objects allocated for debugging purposes should be placed into a separate g_debugHeap. This allows us to more accurately measure memory use during development, without having to compile out all debugging code.
===Temporary Memory===
There is also a g_scratchStack heap which acts like local memory allocations did in the old engine. You can allocate from it for short periods of time ( < a frame ) and you must free in the opposite order than you allocated in.
===Messaging===
Any prop can have a CMsgHandlerComponent. This is a simple component that has a function capable of receiving a generic message formed from a message id and two void* parameters.
You can send any prop a message using the CMsgDispatcher class. There are currently two functions, MsgDirect() which takes a CGameProp* and MsgByPropID() which takes a prop id (aka setupref in the old engine). If the prop has a CMsgHandlerComponent it will receive the message and possibly handle it, otherwise it will return a result to say the message wasn't handled.
A prop can change how it responds to messages by using subclasses of the CMsgHandlerComponent. Message handlers are currently written in a macro language which ensures a valid result is returned and also automatically allow queries on which messages a prop can handle. The aim is to write a lot of setup based message handlers in a scripting language, using a special message handling component that any prop can use, which forwards the messages to the script.
A prop can also have a CSendMsgComponent, which is a component that can be configured to send a message to a prop when it is triggered. The editor can work with the CSendMsgComponent and the CMsgHandlerComponents of props to visually edit the flow of messages between props in the world.
==Appendix C - Coding Standards==
This document describes a set of standards that all programmers will be expected to follow. This is a working document that will be updated as and when necessary.
===File naming convention===
The engine code is split into a number of modules, the C/CPP for which can be found in engine/source and the headers are in engine/include/engine.
The game/framework code is also split into modules, which are in game/source/framework, game/source/common, and game/source/PROJECT directories (the source and headers are in the same folders).
===Module naming conventions===
For the engine, modules should define the following function calls:
*Reset() - called at level start. Allocate memory for module use here, reset should call restart whenever possible
*Restart() - called at level restart. Clear memory and reset all module variables.
*Init() - one time initialisation at the start of the game.
*Tick() - called every frame to do necessary calculations.
Wherever possible, all exported function names should be prefixed with the module name, e.g. given a module called objInst, functions should be called objInstNew(), objInstSetAnim(), etc.
Local (i.e. not exported) functions do not contain the module name. These should be static, as this reduces compile and link times.
===The Engine, Framework, and Game Code===
The code is organised in three main sections.
*The Engine - C library available to all games. Platform specific.
*The Framework - C++ classes available to all games. Cross platform.
*Game Specific Code - Cross platform C++.
The engine is separate to the game, and should be seen as a server that provides a set of services. The framework and game code make use of these services.
All platform specific code must reside on the engine side, and the framework should never be required to know the format of a data asset. The best way of accomplishing this is the through hiding of structures (using headers that are internal to the engine) and handles.
The engine should not contain any hard-coded paths or filenames. These should be passed into the engine from the game code, where it can be loaded from resfiles.
===Using Subversion===
Before a Subversion commit:
#Update your local copy using the 'update' script. You may want to use the 'upcheck' script first to see what will be updated.
#
Once updated, resolve any conflicts that arise. Be careful when resolving conflicts – make sure you don’t undo a change that someone has just made.
#Recompile & check everything still works.
#Go though all your modified files & check for unnecessary code, test code, changes you forgot about & generally tidy up. ‘svngdiff’ and ‘svn diff’ are very useful for these purposes.
#Recompile & check everything still works.
#
Commit your changes with a sensible log message. This should briefly describe the work done. If you ever have to commit a change on someone else’s machine (especially on a build machine) be sure to mention your username in the commit log.
All programmers should commit their code frequently (at least weekly and preferably more often). Only in exceptional cases will this not be possible.
===General Code===
*Try to keep related fields grouped logically in structures, especially big structures like the player.
*Don’t leave dozens of comments to yourself about the job you’ve just done. You should satisfy yourself that your new changes work correctly and then remove unnecessary commented out code. Once code is committed, you can retrace your steps later by using 'svngdiff' if you need to get an old version back.
*A comment should be placed in the code when it’s not obvious what the code is doing.
*Don’t create a revision history at the top of the file – Subversion could do this automatically if we wanted it. Revision information can be retrieved from Subversion using ‘svnlog’ if you need this.
*Try to use numeric constants as little as possible where their meaning may not be obvious. e.g.
**DON’T if (num >= 300-5)
**If a const int exists, use it. Better, where valid, use data (from resfiles or similar).
*Do not use malloc or related memory functions. Use the in-game memAlloc functions provided (in engine/include/engine/mem/mem.h). Allocate as much as possible upfront (in restart functions, not during a frame). Where possible, use pools at runtime. If memory is allocated whilst the game is running, this will fragment the game heap. Streamed data will be handled by a separate memory system.
*Handle failed creation of props. Some game assets will be streamed, and the code needs to handle this.
*Use handles whenever possible.
*Caching member variables to locals can help complier to optimise code.
*Naming convention:
**m_*-> class member variable
**s_*-> static variable
**k_*-> const variable
**Prepend classes with ‘C’
**Prepend enums with ‘E’
**Prepend typedefs with ‘T’
**Prepend struts or unions with ‘S’
**Member functions start with upper case - CObject::MemberFunction
** Member/local variables start with lower case - CObject::m_data s32 data;
**Extern functions start with lower case - void libraryInit();
*C headers should always be wrapped with
**
Screenshot_31.png
*Use frd typedefs eg u32, f32, f64, etc in structures/classes which are intended for binary save. Use TPtrAsInt if you want an int that can fit a pointer in it. Casting from ptr <-> int is not 64 bit friendly if you don't.
*Brackets should follow ifs, whiles, etc.
**
Cplusplus5.png
*There should be no trailing semicolon when using inline functions e.g.
**DON'T extern inline int f(int x) {return x;};
**DO extern inline int f(int x) {return x;}
**Also all extern inline functions should be in .inl files, static inline in src files
*In switch statements, don’t use ranges:
**DON’T case 1 … 10:
*Don’t use variable sized local arrays (some compilers don’t like them; e.g. Visual C). This includes:
**
Cplusplus6.png
*Avoid use of doubles unless it is absolutely necessary (place f’s at end of numbers e.g. x=0.0f, atanf , sqrtf etc). The complier will assume doubles unless f’s are placed, and they are generally very slow compared to floats.
*Keep tabs to 4 spaces width and use brackets on a separate line. This aids readability on all machines (often at the end of projects fixes will have to be made on other people’s code!).
*Included files should use quotes:
**DON’T #include <myheader.h>
**DO #include “myheaderdir/myheader.h”
*For clarity, always use brackets and spaces where confusion may arise. e.g.
**DON’T i=j*k+l*m;
**DO i = (j*k) + (1*m);
**Remember – other people may have to edit your code.
*printf’s to console should not be committed. Use the trace function instead where possible. Important game information is likely to be missed otherwise.
*Zero length arrays are not cross platform. Don’t use them!
*The use of globals should be minimised as far as possible, and all should be grouped in a global structure in one place! The game will clear this structure so that on successive running of the game it behaves exactly the same way. This is particularly important for replays. Similarly static variables in a file scoped outside of functions should be placed in a structure for clearing & initialising.
*Compiler warnings, in much the same way as printf’s cluttering the output, build up quickly & prevent bugs from being caught. They need to be resolved. Warning levels are set at levels so that no warnings are acceptable.
*Use external include guards in header files – it reduces the number of file accesses (which in turn reduces compile time).
*Misc textures and any ‘programmer introduced art assets’ should be placed in perforce.
*Unnecessary nested #includes i.e. #includes in header files. Use forward declarations instead and pass classes/structs by reference or pointer. Nested includes are only required when there is a need for an enum or a typedef in another header file. It can be better in this case to group related enum/typedefs into a separate header.
*Where possible, implicit pointer casting should be avoided so that unsafe casts are not the result. Simple type punning example:
**
Screenshot_35.png
===C++ rules===
*Don’t use stl, if we require portions of the functionality we will write our own.
*Avoid the Diamond of death (inheritance structure) at all costs. (This will more likely lead to memory issues & code calling ambiguity).
*Don’t overuse templates (thin templates okay).
*Keep functionality with the class – i.e. no exhaustive list of Get/Set functions in each class ( this is NOT OOP ).
*A base class destructor should always be virtual. This ensures that all sub class destructors are called as required.
*Be careful with function overloading. There should be no ambiguity.
*Don’t use exceptions. Don’t inherit base objects as virtual.
*Avoid overridden operations.
*Prefer composition to inheritance.
*Code in header files is a really bad idea (since the compiler creates multiple copies and the linker has to remove redundant copies), it's more difficult to debug, causes lots of redundant compiles and it's difficult to read the interface of a class. Don’t do it! (There are a couple of reasons when this required - inline functions, and template functions and classes. The solutions to these are to minimize the use of inline functions and forward declare templates. Inline funtions can actually be slower than non-inline functions, they should be no more than a few lines long, if they contain a loop over a variable or large number of iterations they're pointless, if they contain multiple function calls they're pointless, if they're called infrequently they're pointless. It's clearer if inline member funtions are put after the class declaration).
===Component rules===
*Try & split reusable features off into separate components.
*Consider when adding new features to a component, whether to subclass it to avoid breaking or bloating the base class. On the flip-side don’t subclass unless there is sufficient functionality in the subclass.
*When first creating a component, consider whether it makes sense to be split into several hierarchy levels. This is often the case when the behaviour can be separated from the implementation. For example:
**Physics component
***The base component is an abstract base class with functions like ‘ApplyForceAtPoint’, ‘GetVelocity’. Separate subclass implements these functions in the physics engine. This means all users of the component don’t need to know or care what the physics engine is, isolating the code that needs changing, if we switch physics engines.
**Bullets
**Games will often want to associate arbitrary data with bullets. For this reason, there are two bullets, a framework bullet & a game bullet. The game bullet is a subclass of the framework bullet, meaning future game specific extensions don’t impinge on the framework. General changes go into the framework bullet.
*Try & create manager classes to wrap functionality for the rest of the code. Eg CBulletMgr, CPhysicsMgr, CStimMgr, CphysicsMgr etc. Serialising these from a file will also make it easy to change configurations or to subclass the mgr and override parts of its functionality on a per game basis.
*Any class of non-trivial size should have its own .cpp & .h file. These should have the same name as the class. Don’t call your file ManagerOfCool.cpp & your class CcoolManager.
*Follow the same rules as the engine for Reset(), Restart(), and End() where valid.
*InsertIntoWorld() is the point when the prop is inserted into the world not Serialise().
*All code should be written to be data driven whenever possible. Tuning of the game parameters should be fully exposed to designers.
==References==
*SWBF3 TDD v1.3.doc (21/12/2006)
Category:Internal & Technical