What's new?

These past few weeks I've been pretty busy at work (yay Leipzig) and haven't been very diligent at home on my own stuff. The plan for the Zone system is pretty much in shambles. I have a number of idea's but each of them has their own set of limitations, which makes it very frustrating to just go with one. At this point I feel so indecisive about it I don't know how I'll ever get it working. Because of this I started working on my new material system. I said I wouldn't work on this until the project is over but what else was I going to do, ha.

For those that may not know, a material is basically a way for an artist to describe the surface properties of a 3D object. While a model describes the shape of the object, the material defines which textures it will use, how light interacts with it, and possibly additional surface flags to describe how it interacts with the environment (i.e. concrete, dirt, grass). Although the term "texture" usually refers to an image applied to a 3D mesh, a material is what actually does define the texture of an object.

Now it's not like my previous material system was insufficient; it was certainly incredibly capable. It's just that I have had an idea for a much more robust and extensible material system for a while now but really didn't want to re-write my material system yet again (the current version is probably the 4th or 5th revision in the history of the STO engine). Most of what I'm doing is design work; fleshing out the formats and detailing the creation process. Right now there's about 3 stages throughout the process of making a material but the final result is a fully optimized engine ready "compiled" output file.

The way it works, there's a base material template (.mtrtmpl) which defines the specifics of what that material will do. Things like the number of passes, LOD info, sampling textures, multiplying colors and implementing lighting equations happens here. This file is setup to mirror most high level shader intrinics (things like dot(), mul(), etc...) and intended to be edited in a visual graph based environment (which I hope to have time to build one day, hee hee) strictly by a graphics programmer or (very) technical artist. The low level instructions are actually referred to as 'Micro-Operators'. 'Macro-Operators' allow for a collection of multiple Micro-Ops.

The second part of the material system is the material definition file (.mtrdef). This file mostly matches the structure of the template and used is to allow the artist to specify which uniform and constant parameters are to be passed to the material. This includes things like colors, scalar values and texture (names). This file is also intended to be edited visually but also more easily than the template file, so only each operator that represents a user specified parameter is displayed. This is basically the artist frontend and thus is intentionally simple and straightforward to use.

The material definition is then taken and compiled into a material object file (.mtro). During this process the template and definition files are evaluated and hlsl (or equivalent high level shader language) code is procedurally synthesized and the material object file created to store any user parameters (like which textures are used and any of the states for the material rendering passes).

One thing I haven't touched on and will try to briefly describe is the way that shader permutations will work with this new system. One of the most irritating things graphics programmers have to do is generate multiple versions of the same shader with slight variations. When I was at RavenSoft for instance, there was a particular game I worked on that required 4 versions of any given shader. A version without skinning, one with one bone skinning, another with two bones, and yet another with four. Now if I wanted to make a version of this shader for when fog was enabled, that would require 4 more shaders!! Now there are ways around this problem if you use the .fx file format (uniform shader parameters for instance), but that's quite a limitation (a non-D3D renderer obviously can't use such a feature). To work around this issue, permutations will be a first class citizen in the material template file (described above). This means when I am defining an operation and it's inputs and outputs, I can also declare it as a permutation operator, define the possible permutations, and based on the game state, the proper one is used. When the material is being compiled all permutations are generated so the run-time costs are negligible; the material system just matches the renderer state to the correct shader permutation for that material. Half-life 2 has a similar system (although not as versatile and very slow) as well as Unreal Engine 3 (also not as versatile, plus they compile these shaders in run-time, which is, well, unwise, imo).

There are still a few rough edges to iron out but most of the major hurdles have been accounted for. One thing I'm doing different than the other material system revisions is building this system in conjunction with the existing material system (MaterialV2). This means I can still choose to use the older system if something went wrong. I highly recommend this approach to anyone else contemplating reworking an existing system, it can lead to a lot less headaches over time.

Alright, I have a lot more to talk about but I'll save it for another time. Have a great labor day everyone!