12.2. Image-Based Lighting

Back in Chapter 10 we looked at shaders to perform environment mapping. If we're trying to achieve realistic lighting in a computer graphics scene, why not just use an environment map for the lighting? This approach to illumination is called IMAGE-BASED LIGHTING; it has been popularized in recent years by researcher Paul Debevec at the University of Southern California. Churches and auditoriums may have dozens of light sources on the ceiling. Rooms with many windows also have complex lighting environments. It is often easier and much more efficient to sample the lighting in such environments and store the results in one or more environment maps than it is to simulate numerous individual light sources.

The steps involved in image-based lighting are

On his Web site (http://www.debevec.org/), Debevec offers a number of useful things to developers. For one, he has made available a number of images that can be used as high-quality environment maps to provide realistic lighting in a scene. These images are high dynamic range (HDR) images that represent each color component with a 32-bit floating-point value. Such images can represent a much greater range of intensity values than can 8-bit-per-component images. For another, he makes available a tool called HDRShop that manipulates and transforms these environment maps. Through links to his various publications and tutorials, he also provides step-by-step instructions on creating your own environment maps and using them to add realistic lighting effects to computer graphics scenes.

Following Debevec's guidance, I purchased a 2-inch chrome steel ball from McMaster-Carr Supply Company (http://www.mcmaster.com). We used this ball to capture a light probe image from the center of the square outside our office building in downtown Fort Collins, Colorado (Color Plate 10A). We then used HDRShop to create a lat-long environment map (Color Plate 9) and a cube map (Color Plate 10B) of the same scene. The cube map and latlong map can be used to perform environment mapping as described in Chapter 10. That shader simulated a surface with an underlying base color and diffuse reflection characteristics that was covered by a transparent mirror-like layer that reflected the environment flawlessly.

We can simulate other types of objects if we modify the environment maps before they are used. A point on the surface that reflects light in a diffuse fashion reflects light from all the light sources that are in the hemisphere in the direction of the surface normal at that point. We can't really afford to access the environment map a large number of times in our shader. What we can do instead is similar to what we discussed for hemisphere lighting. Starting from our light probe image, we can construct an environment map for diffuse lighting. Each texel in this environment map will contain the weighted average (i.e., the convolution) of other texels in the visible hemisphere as defined by the surface normal that would be used to access that texel in the environment.

Again, HDRShop has exactly what we need. We can use HDRShop to create a lat-long image from our original light probe image. We can then use a command built into HDRShop that performs the necessary convolution. This operation can be time consuming, because at each texel in the image, the contributions from half of the other texels in the image must be considered. Luckily, we don't need a very large image for this purpose. The effect is essentially the same as creating a very blurry image of the original light probe image. Since there is no high frequency content in the computed image, a cube map with faces that are 64 x 64 or 128 x 128 works just fine. An example of a diffuse environment map is shown in Color Plate 10C.

A single texture access into this diffuse environment map provides us with the value needed for our diffuse reflection calculation. What about the specular contribution? A surface that is very shiny will reflect the illumination from a light source just like a mirror. This is what we saw in the environment mapping shader from Chapter 10. A single point on the surface reflects a single point in the environment. For surfaces that are rougher, the highlight defocuses and spreads out. In this case, a single point on the surface reflects several points in the environment, though not the whole visible hemisphere like a diffuse surface. HDRShop lets us blur an environment map by providing a Phong exponenta degree of shininess. A value of 1.0 convolves the environment map to simulate diffuse reflection, and a value of 50 or more convolves the environment map to simulate a somewhat shiny surface. An example of the Old Town Square environment map that has been convolved with a Phong exponent value of 50 is shown in Color Plate 10D.

The shaders that implement these concepts end up being quite simple and quite fast. In the vertex shader, all that is needed is to compute the reflection direction at each vertex. This value and the surface normal are sent to the fragment shader as varying variables. They are interpolated across each polygon, and the interpolated values are used in the fragment shader to access the two environment maps in order to obtain the diffuse and the specular components. The values obtained from the environment maps are combined with the object's base color to arrive at the final color for the fragment. The shaders are shown in Listing 12.2 and Listing 12.3. Examples of images created with this technique are shown in Color Plate 18.

Listing 12.2. Vertex shader for image-based lighting

varying vec3 ReflectDir; varying vec3 Normal; void main() { gl_Position = ftransform(); Normal = normalize(gl_NormalMatrix * gl_Normal); vec4 pos = gl_ModelViewMatrix * gl_Vertex; vec3 eyeDir = pos.xyz; ReflectDir = reflect(eyeDir, Normal); }

Listing 12.3. Fragment shader for image-based lighting

uniform vec3 BaseColor; uniform float SpecularPercent; uniform float DiffusePercent; uniform samplerCube SpecularEnvMap; uniform samplerCube DiffuseEnvMap; varying vec3 ReflectDir; varying vec3 Normal; void main() { // Look up environment map values in cube maps vec3 diffuseColor = vec3(textureCube(DiffuseEnvMap, normalize(Normal))); vec3 specularColor = vec3(textureCube(SpecularEnvMap, normalize(ReflectDir))); // Add lighting to base color and mix vec3 color = mix(BaseColor, diffuseColor*BaseColor, DiffusePercent); color = mix(color, specularColor + color, SpecularPercent); gl_FragColor = vec4(envColor, 1.0); }

The environment maps that are used can reproduce the light from the whole scene. Of course, objects with different specular reflection properties require different specular environment maps. And producing these environment maps requires some manual effort and lengthy preprocessing. But the resulting quality and performance make image-based lighting a great choice in many situations.