8.4. Shader Development Tools

In coming years, we should see some exciting advances in the area of tools for shader development. This section describes shader development tools available at the time of this writing.

8.4.1. RenderMonkey

As the era of programmable graphics hardware has unfolded, we've learned that there is more to developing shaders than just developing the code for the shaders themselves. Shaders can be highly customized to the point that they may work as intended only on a single model. Shader source code, textures, geometry, and initial values for uniform variables are all important parts of a production-quality shader. Shader development tools must capture all the essential elements of a shader and allow these elements to easily be modified and maintained.

Another factor in shader development is that the person writing the shader is not necessarily the same person developing the application code that deploys the shader. Often, an artist will be employed to design textures and to contribute to or even to control the shader development process. The collaboration between the artist and programmer is an important one for entertainment-based applications and must be supported by shader development tools.

An integrated development environment (IDE) allows programmers and artists to develop and experiment with shaders outside the environment of the application. This reduces the complexity of the shader development task and encourages rapid prototyping and experimentation. Finished shaders are a form of intellectual property, and maintenance, portability, and easy deployment to a variety of platforms are essential to maximizing the benefit of this type of company asset. The idea behind an IDE is that all the essential elements of the finished shader can be encapsulated, managed, shared, and exported for use in the final application.

ATI first released an IDE called RenderMonkey in 2002. In its initial release, RenderMonkey supported shader development for DirectX vertex shaders and pixel shaders. However, RenderMonkey was architected in such a way that it could easily be extended to support other shader programming languages. In 2004, ATI and 3Dlabs collaborated to produce a version of RenderMonkey that contains support for high-level shader development in OpenGL with the OpenGL Shading Language in addition to the support for DirectX shader development. The RenderMonkey IDE is currently available for free from both companies' Web sites (http://developer.3dlabs.com and http://www.ati.com/developer).

RenderMonkey was designed for extensibility. At its core is a flexible framework that allows easy incorporation of shading languages. It is an environment that is language agnostic, allowing any high-level shading language to be supported by means of plug-ins. It currently supports the pixel shaders and vertex shaders defined in Microsoft's DirectX 8.1 and 9.0, the High-Level Shader Language (HLSL) defined in DirectX 9.0, and the OpenGL Shading Language.

In RenderMonkey, the encapsulation of all the information necessary to re-create a shading effect is called an effect workspace. An effect workspace consists of effects group nodes, variable nodes, and stream mapping nodes. Each effects group is made up of one or more effect nodes, and each effect node is made up of one or more rendering passes. Each rendering pass may contain rendering state, source code for a vertex shader and a fragment shader, geometry, and textures. All the effect data is organized into a tree hierarchy that is visible in the workspace viewer.

Effects group nodes collect related effects into a single container. This is sometimes handy for taming the complexity of dealing with lots of different effects. You might also use an effects group as a container for several similar effects with different performance characteristics (for instance, "best quality," "fast," and "fastest"). The criteria for grouping things within effects groups is entirely up to you.

Effect nodes encompass all the information needed to implement a real-time visual effect. The effect may be composed of multiple passes. Starting state is inherited from a default effect to provide a known starting state for all effects. The default effect can store effect data that is common to all shading effects.

Pass nodes define a drawing operation (i.e., a rendering pass). Each pass inherits data from previous passes within the effect, and the first pass inherits from the default effect. A typical pass contains a vertex shader and fragment shader pair, a render state block, textures, geometry, and nodes of other types (for example, variable nodes). Different geometry can be used in each pass to render things like fur.

Variable nodes define the parameters that are available from within a shader. For the OpenGL Shading Language, variable nodes are the mechanisms for defining uniform variables. Intuitive names and types can be assigned to variable nodes, and the contents of a variable node can be manipulated with a GUI widget.

RenderMonkey is built completely out of plug-ins. Source code for some plug-ins is available, and you are encouraged to write your own plug-ins to perform tasks necessary for your work flow or to support importing and exporting your data with proprietary formats. Existing RenderMonkey modules are listed here.

• Shader editorsThese are modeled on the interface of Microsoft's Visual Studio to provide an intuitive interface; they support editing of vertex and fragment shaders; syntax coloring; creation of OpenGL, HLSL, and assembly language shaders.

• Variable editorsShader parameters can be edited with GUI widgets that "know" the underlying data type for editing; editors exist for colors, vectors, matrices, scalars; custom widgets can be created.

• Artist editorShader parameters relevant to the art designer can be presented in an artist-friendly fashion so that they can be viewed and modified; programmers can select which parameters are artist-editable; changes can be seen in real time.

• PreviewersThese allow real-time viewing of the shading effect; changes to the shader source code or its parameters are immediately reflected in the preview window; view settings are customizable; views (front, back, side, etc.) are preset; DirectX 9.0 and OpenGL Shading Language/OpenGL previews are available.

• ExporterEverything required to recreate a shading effect is encapsulated and written into a single XML file.

• ImporterEverything required to recreate a shading effect is read back from an XML file.

The XML file format was chosen as the basis for encapsulating shader information in RenderMonkey because it is an industry standard, it has a user-readable file format, it is user extensible, and it works with the many free parsers that are available. It is relatively easy to adapt an existing XML parser for your own use or to write a new one. The XML file that encapsulates a shader effect contains all shader source code, all render states, all models, and all texture information. This makes it straightforward to create, manage, and share shader assets.

8.4.2. OpenGL Shading Language Compiler Front End

In June 2003, 3Dlabs released an open source version of its lexical analyzer, parser, and semantic checker (i.e., an OpenGL Shading Language COMPILER FRONT END). This code reads an OpenGL shader and turns it into a token stream. This process is called LEXICAL ANALYSIS. The token stream is then processed to ensure that the program consists of valid statements. This process is referred to as SYNTACTIC ANALYSIS, or parsing. SEMANTIC ANALYSIS is then performed to determine whether the shader conforms to the semantic rules defined or implied by the OpenGL Shading Language specification. The result of this processing is turned into a high-level representation of the original source code. This high-level intermediate language (HIL) is a binary representation of the original source code that can be further processed by a target-specific back end to provide machine code for a particular type of graphics hardware.

It is anticipated that individual hardware vendors will implement the back end needed for their particular hardware. The compiler back end will typically include intellectual property and hardware-specific information that is proprietary to the graphics hardware vendor. It is not anticipated that 3Dlabs or other hardware vendors will make public the source code for their compiler back ends.

Still, the compiler front end provided by 3Dlabs has been, and will continue to be, a useful tool for the development of the OpenGL Shading Language, and it will be useful for other organizations that want to develop an OpenGL Shading Language compiler or tools for shader development. As the language specification was nearing completion, the compiler front end was being developed. Except for the preprocessor (which was derived from another open source preprocessor), it was implemented from scratch with the publicly available system utilities flex and bison. It was not derived from existing code. This made it a clean way to double-check the specification and discover language flaws before the specification was released. Indeed, a number of such flaws were discovered through this effort, and, as a result, the specification was improved before its release.

Because of its clean implementation, the OpenGL Shading Language compiler front end also serves as additional technical documentation about the language. The specification should be your first stop, but if you really want to know the details of what's allowed in the language and what's not, studying the compiler front end will provide a definitive answer.

OpenGL implementors that base their compiler on the compiler front end from 3Dlabs will also be doing a big favor to their end users: The semantics of the OpenGL Shading Language will be checked in the same way for all implementations that use this front end. This will benefit developers as they encounter consistent error-checking between different implementations.

Although few readers of this book will likely end up developing an OpenGL Shading Language compiler, this resource is nevertheless a useful one to know about. The source code for the compiler front end is available for download at the 3Dlabs Web site (http://developer.3dlabs.com).

Using the GLSL compiler front end, 3Dlabs has also written a tool called GLSLvalidate. This tool reads a file containing a shader and uses the compiler front-end to parse it. Errors are reported in the output window. This tool can be executed on any platform; an OpenGL 2.0 driver is not required. This tool is provided as open source by 3Dlabs.

Another tool from 3Dlabs, GLSLParserTest, determines whether your OpenGL implementation properly supports the OpenGL Shading Language specification. It attempts to compile some 200 shaders and compares the results against the reference compiler. Some shaders should compile and some should not. Results are printed, and the information log for any shader can be examined. Again, this tool is provided as open source (see http://developer.3dlabs.com).