Asset Build Systems

I'm required this semester to post a write-up for every assignment I complete in one of my engineering courses to a website... so here's one about building up an Asset Build System for processing mesh and shader files.

Assignment 08: Asset Build Systems

Time Spent: 8 hrs
Git Tag: Assignment_08
Download Executable Archive here
Controls: 
Sphere Movement:

• Up Arrow: Fwd
• Down Arrow: Back
• Left Arrow: Left
• Right Arrow: Right
• Shift: Down
• Enter: Up

Camera:

• W: Fwd
• A: Left
• S: Back
• D: Right
• Q: Down
• E: Up

Purpose of Assignment: 
This assignment was mean to demonstrate the power of using a custom asset build system, and the basics of implementing one. The added code implements a manifest for declaring which assets should be built, and a system for defining specialized builder code for various types of asset files. In the case of this project the power of this specialization is demonstrated by using the build system to appropriately handle compilation of Shaders and meshes to binary files before copying them to the data directory for distribution with the finished game.

How I completed it:
I elected to use a different file extension for both of my authored asset file types: .hrshader and .hrmesh. The -hr- prefix is meant to signify "human readable" as a distinction between the authored code and the compiled or processed binary files. This choice makes sense to me since it provides an obvious distinction in the type of file you are referencing, which may help prevent errant references within the project, and because it may help distinguish the type of file being dealt with in collaborative work where files may be sent back and forth.

I added AssetsToBuild.lua to the BuildAllAssets project in VisualStudio's solution because I feel like that would be the first place I would look if I wanted to know what "building all assets" should consist of. Additionally, the BuildAllAssets project is the one where the AssetsToBuild.lua file is passed as a command line argument to the AssetBuildSystem executable, again leading me to feel like it was appropriate to store it in the project which calls it.

Dependencies:
As it turns out, the build dependencies among the AssetBuildSystem components end up being quite limited. The MeshBuilder, ShaderBuilder, and AssetBuildSystem projects are dependent upon the AssetBuildLibrary, but that's just about it. No other projects have MeshBuilder or ShaderBuilder as build dependencies since no other project relies upon them being built to build successfully. At run time, the AssetBuildSystem.lua script will invoke the MeshBuilder and ShaderBuilder, requiring the executables for them to be present, but that's the closest we get to a dependency.

Why use AssetsToBuild.lua?
This liberates us from adding a bunch of command line arguments to the custom build step of BuildAllAssets every time we add new assets to the build. It provides a source controlled means of deciding which assets should or should not be built, and could even go so far as to use additional parameters to determine the configurations in which a file should be built or not and which Builder should be used to do so. This degree of flexibility and control is very useful as projects grow larger.

The Advantages of Pre-building Assets
The vertex.hrshader file has a size of 5.15 KB, versus the compiled shader which has a size of 1.34 KB in the case of a D3D build and just 778 bytes in the case of the OpenGL build. This makes the advantage of a build system pretty clear: you can work with files in human readable formats during development but the deployed files end up substantially smaller while still being independent, and thus independently distributable, updatable, etc. from the main executable.

It is also worth noting that the file sizes for debug vs release builds are also quite different. In the case of the D3D vertex shader the Release build jumps up in size to 17.3 KB vs 1.34 KB in release. This is due to the inclusion of debug symbols in the shader itself which allow for data to be extracted and analyzed at runtime for debugging. In the release build, not only are the debug symbols omitted, but additional compiler optimization is applied to further reduce size and increase operational speed. Optimizations are not applied in Debug mode because they can potentially alter the control flow of the program, making it difficult to relate run time information back to the authored code in the event that problems arise. The priority in Debug mode is transparency of operation, the priority in Release is small size and operational efficiency.

In the case of OpenGL, the file sizes are also different between debug and release, despite the fact that OpenGL shaders are not compiled to binaries until runtime. In Debug, the vertex shader for OpenGL is 2.17KB vs 778 bytes in release mode. In the case of OpenGL the file size drops from 5.15KB to 2.17KB during Debug builds as the D3D platform specific code is stripped out by the preprocessor. The size drops again from 2.17KB to the final 778 bytes from Release to Debug as all of the comments are removed during the copy operation. Again this signifies the distinction in priority between debug and release: clarity and readability vs size and efficiency.