14.4. Polynomial Texture Mapping with BRDF Data

This section describes the OpenGL Shading Language BRDF shaders that use the Polynomial Texture Mapping technique developed by Hewlett-Packard. The shaders presented are courtesy of Brad Ritter, Hewlett-Packard. The BRDF data is from Cornell University. It was obtained by measurement of reflections from several types of automotive paints that were supplied by Ford Motor Co.

One reason this type of rendering is important is that it achieves realistic rendering of materials whose reflection characteristics vary as a function of view angle and light direction. Such is the case with these automotive paints. To a car designer, it is extremely important to be able to visualize the final "look" of the car, even when it is painted with a material whose reflection characteristics vary as a function of view angle and light direction. One of the paint samples tested by Cornell, Mystique Lacquer, has the peculiar property that the color of its specular highlight color changes as a function of viewing angle. This material cannot be adequately rendered if only conventional texture-mapping techniques are used.

The textures used in this example are called POLYNOMIAL TEXTURE MAPS, or PTMs. PTMs are essentially light-dependent texture maps; PTMs are described in a 2001 SIGGRAPH paper by Malzbender, Gelb, and Wolters. PTMs reconstruct the color of a surface under varying lighting conditions. When a surface is rendered with a PTM, it takes on different illumination characteristics depending on the direction of the light source. As with bump mapping, this behavior helps viewers by providing perceptual clues about the surface geometry. But PTMs go beyond bump maps in that they capture surface variations resulting from self-shadowing and interreflections. PTMs are generated from real materials and preserve the visual characteristics of the actual materials. Polynomial texture mapping is an image-based technique that does not require bump maps or the modeling of complex geometry.

The image in Color Plate 27A shows two triangles from a PTM demo developed by Hewlett-Packard. The triangle on the upper right has been rendered with a polynomial texture map, and the triangle on the lower left has been rendered with a conventional 2D texture map. The objects that were used in the construction of the texture maps were a metallic bezel with the Hewlett-Packard logo on it and a brushed metal notebook cover with an embossed 3Dlabs logo. As you move the simulated light source in the demo, the conventional texture looks flat and somewhat unrealistic, whereas the PTM texture faithfully reproduces the highlights and surface shadowing that occur on the real-life objects. In the image captured here, the light source is a bit in front and above the two triangles. The PTM shows realistic reflections, but the conventional texture can only reproduce the lighting effect from a single lighting angle (in this case, as if the light were directly in front of the object).

The PTM technique developed by HP requires as input a set of images of the desired object, with the object illuminated by a light source of a different known direction for each image, all captured from the same viewpoint. For each texel of the PTM, these source images are sampled and a least-squares biquadric curve fit is performed to obtain a polynomial that approximates the lighting function for that texel. This part of the process is partly science and partly art (a bit of manual intervention can improve the end results). The biquadric equation generated in this manner allows runtime reconstruction of the lighting function for the source material. The coefficients stored in the PTM are A, B, C, D, E, and F, as shown in this equation:

Au² + Bv² + Cuv + Du + Ev + F

One use of PTMs is for representing materials with surface properties that vary spatially across the surface. Things like brushed metal, woven fabric, wood, and stone are all materials that reflect light differently depending on the viewing angle and light source direction. They may also have interreflections and self-shadowing. The PTM technique captures these details and reproduces them at runtime. There are two variants for PTMs: luminance (LRGB) and RGB. An LRGB PTM uses the biquadric polynomials to determine the brightness of each rendered pixel. Because each texel in an LRGB PTM has its own biquadric polynomial function, the luminance or brightness characteristics of each texel can be unique. An RGB PTM uses a separate biquadric polynomial for each of the three colors: red, green, and blue. This allows objects rendered with an RGB PTM to vary in color as the light position shifts. Thus, color-shifting materials such as holograms can be accurately reconstructed with an RGB PTM.

The key to creating a PTM for these types of spatially varying materials is to capture images of them as lit from a variety of light source directions. Engineers at Hewlett-Packard have developed an instrumenta dome with multiple light sources and a camera mounted at the topto do just that. This device can automatically capture 50 images of the source material from a single fixed camera position as illuminated by light sources in different positions. A photograph of this picture-taking device is shown in Figure 14.4.

The image data collected with this device is the basis for creating a PTM for the real-world texture of a material (e.g., automobile paints). These types of PTMs have four degrees of freedom. Two of these represent the spatially varying characteristics of the material. These two degrees of freedom are controlled by the 2D texture coordinates. The remaining two degrees of freedom represent the light direction. These are the two independent variables in the biquadric polynomial.

A BRDF PTM is slightly different from a spatially varying PTM. BRDF PTMs model homogeneous materialsthat is, they do not vary spatially. BRDF PTMs use two degrees of freedom to represent the light direction, and the remaining two degrees of freedom represent the view direction. The parameterized light direction (Lu,Lv) is used for the independent variables of the biquadric polynomial, and the parameterized view direction (Vu,Vv) is used as the 2D texture coordinate.

No single parameterization works well for all BRDF materials. A further refinement to enhance quality for BRDF PTMs for the materials we are trying to reproduce is to reparameterize the light and view vectors as half angle and difference vectors (Hu,Hv) and (Du,Dv). In the BRDF PTM shaders discussed in the next section, Hu and Hv are the independent variables of the biquadric polynomial, and (Du,Dv) is the 2D texture coordinate. A large part of the vertex shader's function is to calculate (Hu,Hv) and (Du,Dv).

BRDF PTMs can be created as either LRGB or RGB PTMs. The upcoming example shows how an RGB BRDF PTM is rendered with OpenGL shaders. RGBA textures with 8 bits per component are used because the PTM file format and tools developed by HP are based on this format.

14.4.1. Application Setup

To render BRDF surfaces using the following shaders, the application must set up a few uniform variables. The vertex shader must be provided with values for uniform variables that describe the eye direction (i.e., an infinite viewer) and the position of a single light source (i.e., a point light source). The fragment shader requires the application to provide values for scaling and biasing the six polynomial coefficients. (These values were prescaled when the PTM was created to preserve precision, and they must be rescaled with the scale and bias factors that are specific to that PTM.)

The application is expected to provide the surface normal, vertex position, tangent, and binormal in exactly the same way as the BRDF shader discussed in the previous section. Before rendering, the application should also set up seven texture maps: three 2D texture maps to hold the A, B, and C co-efficients for red, green, and blue components of the PTM; three 2D texture maps to hold the D, E, and F coefficients for red, green, and blue components of the PTM; and a 1D texture map to hold a lighting function.

This last texture is set up by the application whenever the lighting state is changed. The light factor texture solves four problems:

14.4.2. Vertex Shader

The BRDF PTM vertex shader is shown in Listing 14.8. This shader produces five varying values:

• gl_Position, as required by every vertex shader

• TexCoord, which is used to access our texture maps to get the two sets of polynomial coefficients

• Du, a float that contains the cosine of the angle between the light direction and the tangent vector

• Dv, a float that contains the cosine of the angle between the light direction and the binormal vector

• LdotN, a float that contains the cosine of the angle between the incoming surface normal and the light direction

The shader assumes a viewer at infinity and one point light source.

Listing 14.8. Vertex shader for rendering BRDF-based polynomial texture maps

// // PTM vertex shader by Brad Ritter, Hewlett-Packard // and Randi Rost, 3Dlabs. // // © Copyright 2003 3Dlabs, Inc., and // Hewlett-Packard Development Company, L.P., // Reproduced with Permission // uniform vec3 LightPos; uniform vec3 EyeDir; attribute vec3 Tangent; attribute vec3 Binormal; varying float Du; varying float Dv; varying float LdotN; varying vec2 TexCoord; void main() { vec3 lightTemp; vec3 halfAngleTemp; vec3 tPrime; vec3 bPrime; // Transform vertex gl_Position = ftransform(); lightTemp = normalize(LightPos - gl_Vertex.xyz); // Calculate the Half Angle vector halfAngleTemp = normalize(EyeDir + lightTemp); // Calculate T' and B' // T' = |T - (T.H)H| tPrime = Tangent - (halfAngleTemp * dot(Tangent, halfAngleTemp)); tPrime = normalize(tPrime); // B' = H x T' bPrime = cross(halfAngleTemp, tPrime); Du = dot(lightTemp, tPrime); Dv = dot(lightTemp, bPrime); // Multiply the Half Angle vector by NOISE_FACTOR // to avoid noisy BRDF data halfAngleTemp = halfAngleTemp * 0.9; // Hu = Dot(HalfAngle, T) // Hv = Dot(HalfAngle, B) // Remap [-1.0..1.0] to [0.0..1.0] TexCoord.s = dot(Tangent, halfAngleTemp) * 0.5 + 0.5; TexCoord.t = dot(Binormal, halfAngleTemp) * 0.5 + 0.5; // "S" Text Coord3: Dot(Light, Normal); LdotN = dot(lightTemp, gl_Normal) * 0.5 + 0.5; }

The light source position and eye direction are passed in as uniform variables by the application. In addition to the standard OpenGL vertex and normal vertex attributes, the application is expected to pass in a tangent and a binormal per vertex, as described in the previous section. These two generic attributes are defined with appropriate names in our vertex shader.

The first line of the vertex shader transforms the incoming vertex value by the current modelview-projection matrix. The next line computes the light source direction for our positional light source by subtracting the vertex position from the light position. Because LightPos is defined as a vec3 and the built-in attribute gl_Vertex is defined as a vec4, we must use the .xyz component selector to obtain the first three elements of gl_Vertex before doing the vector subtraction operation. The result of the vector subtraction is then normalized and stored as our light direction.

The following line of code computes the half angle by summing the eye direction vector and the light direction vector and normalizing the result.

The next few lines of code compute the 2D parameterization of our half angle and difference vector. The goal here is to compute values for u (Du) and v (Dv) that can be plugged into the biquadric equation in our fragment shader. The technique we use is called Gram-Schmidt orthonormalization. H (half angle), T', and B' are the orthogonal axes of a coordinate system. T' and B' maintain a general alignment with the original T (tangent) and B (binormal) vectors. Where T and B lie in the plane of the triangle being rendered, T' and B' are in a plane perpendicular to the half angle vector. More details on the reasons for choosing H, T', and B' to define the coordinate system are available in the paper Interactive Rendering with Arbitrary BRDFs Using Separable Approximations by Jan Kautz and Michael McCool (1999).

BRDF data often has noisy data values for extremely large incidence angles (i.e., close to 180°), so in the next line of code, we avoid the noisy data in a somewhat unscientific manner by applying a scale factor to the half angle. This effectively causes these values to be ignored.

Our vertex shader code then computes values for Hu and Hv and places them in the varying variable TexCoord. These are plugged into our biquadric equation in the fragment shader as the u and v values. These values hold our parameterized difference vector and are used to look up the required polynomial coefficients from the texture maps, so they are mapped into the range [0,1].

Finally, we compute a value that applies the lighting effect. This value is simply the cosine of the angle between the surface normal and the light direction. It is also mapped into the range [0,1] because it is the texture coordinate for accessing a 1D texture to obtain the lighting factor that is used.

14.4.3. Fragment Shader

The fragment shader for our BRDF PTM surface rendering is shown in Listing 14.9.

Listing 14.9. Fragment shader for rendering BRDF-based polynomial texture maps

// // PTM fragment shader by Brad Ritter, Hewlett-Packard // and Randi Rost, 3Dlabs. // // © Copyright 2003 3Dlabs, Inc., and // Hewlett-Packard Development Company, L.P., // Reproduced with Permission // uniform sampler2D ABCred; // = 0 uniform sampler2D DEFred; // = 1 uniform sampler2D ABCgrn; // = 2 uniform sampler2D DEFgrn; // = 3 uniform sampler2D ABCblu; // = 4 uniform sampler2D DEFblu; // = 5 uniform sampler1D Lighttexture; // = 6 uniform vec3 ABCscale, ABCbias; uniform vec3 DEFscale, DEFbias; varying float Du; // passes the computed L*tPrime value varying float Dv; // passes the computed L*bPrime value varying float LdotN; // passes the computed L*Normal value varying vec2 TexCoord; // passes s, t, texture coords void main() { vec3 ABCcoef, DEFcoef; vec3 ptvec; // Read coefficient values for red and apply scale and bias factors ABCcoef = (texture2D(ABCred, TexCoord).rgb - ABCbias) * ABCscale; DEFcoef = (texture2D(DEFred, TexCoord).rgb - DEFbias) * DEFscale; // Compute red polynomial ptvec.r = ABCcoef[0] * Du * Du + ABCcoef[1] * Dv * Dv + ABCcoef[2] * Du * Dv + DEFcoef[0] * Du + DEFcoef[1] * Dv + DEFcoef[2]; // Read coefficient values for green and apply scale and bias factors ABCcoef = (texture2D(ABCgrn, TexCoord).rgb - ABCbias) * ABCscale; DEFcoef = (texture2D(DEFgrn, TexCoord).rgb - DEFbias) * DEFscale; // Compute green polynomial ptvec.g = ABCcoef[0] * Du * Du + ABCcoef[1] * Dv * Dv + ABCcoef[2] * Du * Dv + DEFcoef[0] * Du + DEFcoef[1] * Dv + DEFcoef[2]; // Read coefficient values for blue and apply scale and bias factors ABCcoef = (texture2D(ABCblu, TexCoord).rgb - ABCbias) * ABCscale; DEFcoef = (texture2D(DEFblu, TexCoord).rgb - DEFbias) * DEFscale; // Compute blue polynomial ptvec.b = ABCcoef[0] * Du * Du + ABCcoef[1] * Dv * Dv + ABCcoef[2] * Du * Dv + DEFcoef[0] * Du + DEFcoef[1] * Dv + DEFcoef[2]; // Multiply result * light factor ptvec *= texture1D(Lighttexture, LdotN).rgb; // Assign result to gl_FragColor gl_FragColor = vec4(ptvec, 1.0); }

This shader is relatively straightforward if you've digested the information in the previous three sections. The values in the s and t components of TexCoord hold a 2D parameterization of the difference vector. TexCoord indexes into each of our coefficient textures and retrieves the values for the A, B, C, D, E, and F coefficients. The BRDF PTMs are stored as mipmap textures, and, because we're not providing a bias argument, the computed level-of-detail bias is just used directly. Using vector operations, we scale and bias the six coefficients by using values passed from the application through uniform variables.

We then use these scaled, biased coefficient values together with our parameterized half angle (Du and Dv) in the biquadric polynomial to compute the red value for the surface. We repeat the process to compute the green and blue values as well. We compute the lighting factor by accessing the 1D light texture, using the cosine of the angle between the light direction and the surface normal. Finally, we multiply the lighting factor by our polynomial vector and use an alpha value of 1.0 to produce the final fragment color.

The image in Color Plate 27B shows our BRDF PTM shaders rendering a torus with the BRDF PTM created for the Mystique Lacquer automotive paint. The basic color of this paint is black, but, in the orientation captured for the still image, the specular highlight shows up as mostly white with a reddish-brown tinge on one side of the highlight and a bluish tinge on the other. As the object is moved around or as the light is moved around, our BRDF PTM shaders properly render the shifting highlight color.