13.2. Shadow Maps

Ambient occlusion is quite useful for improving the realism of rigid objects under diffuse lighting conditions, but often a scene will need to incorporate lighting from one or more well-defined light sources. In the real world, we know that strong light sources cause objects to cast well-defined shadows. Producing similar shadows in our computer-generated scenes will make them seem more realistic. How can we accomplish this?

The amount of computer graphics literature that discusses generation of shadows is vast. This is partly because no single shadowing algorithm is optimal for all cases. There are numerous trade-offs in performance, quality, and simplicity. Some algorithms work well for only certain types of shadow-casting objects or shadow-receiving objects. Some algorithms work for certain types of light sources or lighting conditions. Some experimentation and adaptation may be needed to develop a shadow-generation technique that is optimal for a specific set of conditions.

OpenGL and the OpenGL Shading Language include facilities for a generally useful shadowing algorithm called SHADOW MAPPING. In this algorithm, the scene is rendered multiple timesonce for each light source that is capable of causing shadows, and once to generate the final scene, including shadows. Each of the per-light passes is rendered from the point of view of the light source. The results are stored in a texture that is called a SHADOW MAP or a DEPTH MAP. This texture is essentially a visible surface representation from the point of view of the light source. Surfaces that are visible from the point of view of this light source are fully illuminated by the light source. Surfaces that are not visible from the point of view of this light source are in shadow. Each of the textures so generated is accessed during the final rendering pass to create the final scene with shadows from one or more light sources. During the final rendering pass, the distance from the fragment to each light is computed and compared to the depth value in the shadow map for that light. If the distance from the fragment to the light is greater than the depth value in the shadow map, the fragment receives no contribution from that light source (i.e., it is in shadow); otherwise, the fragment is subjected to the lighting computation for that particular light source.

Because this algorithm involves an extra rendering pass for each light source, performance is a concern if a large number of shadow-casting lights are in the scene. But for interactive applications, it is quite often the case that shadows from one or two lights add sufficiently to the realism and comprehensibility of a scene. More than that and you may be adding needless complexity. And, just like other algorithms that use textures, shadow mapping is prone to severe aliasing artifacts unless proper care is taken.

The depth comparison can also lead to problems. Since the values being compared were generated in different passes with different transformation matrices, it is possible to have a small difference in the values. Therefore, you must use a small epsilon value in the comparison. You can use the glPolygonOffset command to bias depth values as the shadow map is being created. You must be careful, though, because too much bias can cause a shadow to become detached from the object casting the shadow.

A way to avoid depth precision problems with illuminated faces is to draw backfaces when you are building the shadow map. This avoids precision problems with illuminated faces because the depth value for these surfaces is usually quite different from the shadow map depth, so there is no possibility of precision problems incorrectly darkening the surface.

Precision problems are still possible on surfaces facing away from the light. You can avoid these problems by testing the surface normalif it points away from the light, then the surface is in shadow. There will still be problems when the back and front faces have similar depth values, such as at the edge of the shadow. A carefully weighted combination of normal test plus depth test can provide artifact-free shadows even on gently rounded objects. However, this approach does not handle occluders that aren't simple closed geometry.

Despite its drawbacks, shadow mapping is still a popular and effective means of generating shadows. A big advantage is that it can be used with arbitrarily complex geometry. It is supported in RenderMan and has been used to generate images for numerous movies and interactive games. OpenGL supports shadow maps (they are called depth component textures in OpenGL) and a full range of depth comparison modes that can be used with them. Shadow mapping can be performed in OpenGL with either fixed functionality or the programmable processing units. The OpenGL Shading Language contains corresponding functions for accessing shadow maps from within a shader (shadow2D, shadow2DProj, and the like).

13.2.1. Application Setup

As mentioned, the application must create a shadow map for each light source by rendering the scene from the point of view of the light source. For objects that are visible from the light's point of view, the resulting texture map contains depth values representing the distance to the light. (The source code for the example program deLight available from the 3Dlabs Web site illustrates specifically how this is done.)

For the final pass of shadow mapping to work properly, the application must create a matrix that transforms incoming vertex positions into projective texture coordinates for each light source. The vertex shader is responsible for performing this transformation, and the fragment shader uses the interpolated projective coordinates to access the shadow map for each light source. To keep things simple, we look at shaders that deal with just a single light source. (You can use arrays and loops to extend the basic algorithm to support multiple light sources.)

We construct the necessary matrix by concatenating

• The modeling matrix (M) that transforms modeling coordinates into world coordinates (this is the same modeling matrix that would be used to render the object normally)

• A view matrix (V_light) that rotates and translates world coordinates into a coordinate system that has the light source at the origin and is looking at the point (0, 0, 0) in world coordinates

• A projection matrix (P_light) that defines the frustum for the view from the light source (i.e., field of view, aspect ratio, near and far clipping planes)

• A scale and bias matrix that takes the clip space coordinates (i.e., values in the range [1,1]) from the previous step into values in the range [0,1] so that they can be used directly as the index for accessing the shadow map.

The equation for the complete transformation looks like this:

We can actually use the OpenGL texture generation capabilities to generate shadows with OpenGL fixed functionality. We store the transformation matrix as a texture transformation matrix to produce the proper texture coordinates for use in the texture access operation. By performing this transformation in a vertex shader, we can have shadows and can add any desired programmable functionality as well. Let's see how this is done in a vertex shader. Philip Rideout developed some shaders for the deLight demo that use shadow mapping. They have been adapted slightly for inclusion in this book.

13.2.2. Vertex Shader

The OpenGL Shading Language defines that all varying variables are interpolated in a perspective-correct manner. We can use this fact to perform the perspective division that is necessary to prevent objectionable artifacts during animation. To get perspective-correct projective texture coordinates, we need to end up with per-fragment values of s/q, t/q, and r/q. This is analogous to homogeneous clip coordinates where we divide through by the homogeneous coordinate w to end up with x/w, y/w, and z/w. Instead of interpolating the (s, t, r, q) projected texture coordinate directly and doing the division in the fragment shader, we divide the first three components by w in the vertex shader and then interpolate s/w, t/w, and r/w. The net result of the perspective-correct interpolation is then (s/w)/(q/w) = s/q and (t/w)/(q/w) = t/q, which is exactly what we want for projective texturing.

The vertex shader in Listing 13.4 shows how this is done. We use ambient occlusion in this shader, so these values are passed in as vertex attributes through the attribute variable Accessibility. These values attenuate the incoming per-vertex color values after a simple diffuse lighting model has been applied. The alpha value is left unmodified by this process. Using light source 0 as defined by OpenGL state makes it convenient for the application to draw shadows by using either the fixed functionality path or the programmable path. The matrix that transforms modeling coordinates to light source coordinates is stored in texture matrix 1 for the same reason and is accessed through the built-in uniform variable gl_TextureMatrix[1]. This matrix transforms the incoming vertex, and the resulting value has its first three components divided by the fourth component to make the interpolated values turn out correctly, as we have just discussed.

Listing 13.4. Vertex shader for generating shadows

attribute float Accessibility; varying vec4 ShadowCoord; // Ambient and diffuse scale factors. const float As = 1.0 / 1.5; const float Ds = 1.0 / 3.0; void main() { vec4 ecPosition = gl_ModelViewMatrix * gl_Vertex; vec3 ecPosition3 = (vec3(ecPosition)) / ecPosition.w; vec3 VP = vec3(gl_LightSource[0].position) - ecPosition3; VP = normalize(VP); vec3 normal = normalize(gl_NormalMatrix * gl_Normal); float diffuse = max(0.0, dot(normal, VP)); float scale = min(1.0, Accessibility * As + diffuse * Ds); vec4 texCoord = gl_TextureMatrix[1] * gl_Vertex; ShadowCoord = texCoord / texCoord.w; gl_FrontColor = vec4(scale * gl_Color.rgb, gl_Color.a); gl_Position = ftransform(); }

13.2.3. Fragment Shader

A simple fragment shader for generating shadows is shown in Listing 13.5. The main function calls a subroutine named lookup to do the shadow map lookup, giving it offsets of 0 in both the x and the y directions. These offset values are added to the interpolated projective texture coordinate, an epsilon value is added, and the result is used to perform a texture access on the shadow map (depth texture) specified by ShadowMap. When shadow2Dproj is used to access a texture, the third component of the texture index (i.e., ShadowCoord.p + Epsilon) is compared with the depth value stored in the shadow map. The comparison function is the one specified for the texture object indicated by ShadowMap. If the comparison results in a value of true, shadow2Dproj returns 1.0; otherwise, it returns 0. If shadow2Dproj returns a value of 0, the lookup function returns a value of 0.75 (shadowed); otherwise, it returns a value of 1.0 (fully illuminated). The value returned by the lookup function is used to attenuate the red, green, and blue components of the fragment's color. Fragments that are fully illuminated are unchanged, while fragments that are shadowed are multiplied by a factor of 0.75 to make them darker.

Listing 13.5. Fragment shader for generating shadows

uniform sampler2DShadow ShadowMap; uniform float Epsilon; varying vec4 ShadowCoord; float lookup(float x, float y) { float depth = shadow2DProj(ShadowMap, ShadowCoord + vec3(x, y, 0) * Epsilon).x; return depth != 1.0 ? 0.75 : 1.0; } void main() { float shadeFactor = lookup(0.0, 0.0); gl_FragColor = vec4(shadeFactor * gl_Color.rgb, gl_Color.a); }

Chances are that as soon as you execute this shader, you will be disappointed by the aliasing artifacts that appear along the edges of the shadows. We can do something about this, and we can customize the shader for a specific viewing situation to get a more pleasing result. Michael Bunnell and Fabio Pellacini describe a method for doing this in an article called Shadow Map Antialiasing in the book GPU Gems. Philip Rideout implemented this technique in GLSL, as shown in Listing 13.6 and Listing 13.7.

The shader in Listing 13.6 adds a couple of things. The first thing is that the main function assigns a value to Illumination based on a Boolean uniform variable. This shader essentially distinguishes between two types of shadowsthose that are generated by the object itself and those that are generated by another object in the scene. The self-shadows are made a little lighter than other cast shadows for aesthetic reasons. The result of this conditional statement is that where the object shadows itself, the shadows are relatively light. And where the object's shadow falls on the floor, the shadows are darker. (See Color Plate 22.)

The second difference is that the shadow map is sampled four times. The purpose of sampling multiple times is to try to do better at determining the shadow boundary. This lets us apply a smoother transition between shadowed and unshadowed regions, thus reducing the jagged edges of the shadow. However, it is incorrect to simply average the Boolean values returned by shadow2D, because this can result in rendering errors. Instead, the returned Boolean value is used to assign a value to Illumination, and then the four computed Illumination values are subsequently averaged.

Listing 13.6. Fragment shader for generating shadows with antialiased edges, using four samples per pixel

uniform sampler2DShadow ShadowMap; uniform float Epsilon; uniform bool SelfShadowed; uniform float SelfShadowedVal; uniform float NonSelfShadowedVal; varying vec3 ShadowCoord; float Illumination; float lookup(float x, float y) { float depth = shadow2D(ShadowMap, ShadowCoord + vec3(x, y, 0) * Epsilon).x; return depth != 1.0 ? Illumination : 1.0; } void main() { // lighten up the self-shadows Illumination = SelfShadowed ? SelfShadowedVal : NonSelfShadowedVal; float sum = 0.0; sum += lookup(-0.5, -0.5); sum += lookup( 0.5, -0.5); sum += lookup(-0.5, 0.5); sum += lookup( 0.5, 0.5); gl_FragColor = vec4(sum * 0.25 * gl_Color.rgb, gl_Color.a); }

This shader can be extended in the obvious way to perform even more samples per pixel and thus improve the quality of the shadow boundaries even more. However, the more texture lookups that we perform in our shader, the slower it will run.

Using a method akin to dithering, we can actually use four samples that are spread somewhat farther apart to achieve a quality of antialiasing that is similar to that of using quite a few more than four samples per pixel. In Listing 13.7 we include code for computing offsets in x and y from the current pixel location. These offsets form a regular dither pattern that is used to access the shadow map. The results of using four dithered samples per pixel provides much better quality than just using four standard samples, though it is not quite as good as using 16 samples per pixel.

Listing 13.7. Fragment shader for generating shadows, using four dithered samples

uniform sampler2DShadow ShadowMap; uniform float Epsilon; uniform bool SelfShadowed; uniform float SelfShadowedVal; uniform float NonSelfShadowedVal; varying vec3 ShadowCoord; float Illumination; float lookup(float x, float y) { float depth = shadow2D(ShadowMap, ShadowCoord + vec3(x, y, 0) * Epsilon).x; return depth != 1.0 ? Illumination : 1.0; } void main() { // lighten up the self-shadows Illumination = SelfShadowed ? SelfShadowedVal : NonSelfShadowedVal; // use modulo to vary the sample pattern vec2 o = mod(floor(gl_FragCoord.xy), 2.0); float sum = 0.0; sum += lookup(vec2(-1.5, 1.5) + o); sum += lookup(vec2( 0.5, 1.5) + o); sum += lookup(vec2(-1.5, -0.5) + o); sum += lookup(vec2( 0.5, -0.5) + o); gl_FragColor = vec4(sum * 0.25 * gl_Color.rgb, gl_Color.a); }

Sample images using these shaders are shown in Color Plate 22. A small area of the shadow has been enlarged by 400% to show the differences in quality at the edge of the shadow.