Section 4.5. Interaction with OpenGL Fixed Functionality

This section offers a little more detail to programmers who are intimately familiar with OpenGL operations and need to know precisely how the programmable capabilities introduced by the OpenGL Shading Language interact with the rest of the OpenGL pipeline. This section is more suitable for seasoned OpenGL programmers than for OpenGL novices.

4.5.1. Two-Sided Color Mode

Vertex shaders can operate in two-sided color mode. Front and back colors can be computed by the vertex shader and written to the gl_FrontColor, gl_BackColor, gl_FrontSecondaryColor, and gl_BackSecondaryColor output variables. If two-sided color mode is enabled after vertex processing, OpenGL fixed functionality chooses which of the front or back colors to forward to the rest of the pipeline. Which side OpenGL picks depends on the primitive type being rendered and the sign of the area of the primitive in window coordinates (see the OpenGL specification for details). If two-sided color mode is disabled, OpenGL always selects the front color outputs. Two-sided color mode is enabled and disabled with glEnable or glDisable with the symbolic value GL_VERTEX_PROGRAM_TWO_SIDE.

The colors resulting from this front/back facing selection step are clamped to the range [0,1] and converted to a fixed-point representation before being interpolated across the primitive that is being rendered. This is normal OpenGL behavior. When higher precision and dynamic range colors are required, the application should use its own user-defined varying variables instead of the four built-in gl_Color ones. The front/back facing selection step is then skipped. However, a built-in variable available in the fragment shader (gl_FrontFacing) indicates whether the current fragment is the result of rasterizing a front or back facing primitive.

4.5.2. Point Size Mode

Vertex shaders can also operate in point size mode. A vertex shader can compute a point size in pixels and assign it to the built-in variable gl_PointSize. If point size mode is enabled, the point size is taken from this variable and used in the rasterization stage; otherwise, it is taken from the value set with the glPointSize command. If gl_PointSize is not written while vertex shader point size mode is enabled, the point size used in the rasterization stage is undefined. Vertex shader point size mode is enabled and disabled with glEnable or glDisable with the symbolic value GL_VERTEX_PROGRAM_POINT_SIZE.

This point size enable is convenient for the majority of applications that do not change the point size within a vertex shader. By default, this mode is disabled, so most vertex shaders for which point size doesn't matter need not write a value to gl_PointSize. The value set by calls to glPointSize is always used by the rasterization stage.

If the primitive is clipped and vertex shader point size mode is enabled, the point size values are also clipped in a manner analogous to color clipping. The potentially clipped point size is used by the fixed functionality part of the pipeline as the derived point size (the distance attenuated point size). Thus, if the application wants points farther away to be smaller, it should compute some kind of distance attenuation in the vertex shader and scale the point size accordingly. If vertex shader point size mode is disabled, the derived point size is taken directly from the value set with the glPointSize command and no distance attenuation is performed. The derived point size is then used, as usual, optionally to alpha-fade the point when multisampling is also enabled. Again, see the OpenGL specification for details.

Distance attenuation should be computed in a vertex shader and cannot be left to the fixed functionality distance attenuation algorithm. This fixed functionality algorithm computes distance attenuation as a function of the distance between the eye at (0, 0, 0, 1) and the vertex position, in eye coordinates. However, the vertex position computed in a vertex shader might not have anything to do with locations in eye coordinates. Therefore, when a vertex shader is active, this fixed functionality algorithm is skipped. A point's alpha-fade, on the other hand, can be computed correctly only with the knowledge of the primitive type. That information is not available to a vertex shader, because it executes before primitive assembly. Consider the case of rendering a triangle and having the back polygon mode set to GL_POINT and the front polygon mode to GL_FILL. The vertex shader should fade only the alpha if the vertex belongs to a back facing triangle. But it cannot do that because it does not know the primitive type.

4.5.3. Clipping

User clipping can be used in conjunction with a vertex shader. The user clip planes are specified as usual with the glClipPlane command. When specified, these clip planes are transformed by the inverse of the current modelview matrix. The vertices resulting from the execution of a vertex shader are evaluated against these transformed clip planes. The vertex shader must provide the position of the vertex in the same space as the user-defined clip planes (typically, eye space). It does that by writing this location to the output variable gl_ClipVertex. If gl_ClipVertex is not specified and user clipping is enabled, the results are undefined.

When a vertex shader that mimics OpenGL's fixed functionality is used, the vertex shader should compute the eye-coordinate position of the vertex and store it in gl_ClipVertex. For example,

When you want to do object space clipping instead, keep in mind that the clip planes are transformed with the inverse of the modelview matrix. For correct object clipping, the modelview matrix needs to be set to the identity matrix when the clip planes are specified.

After user clipping, vertices are clipped against the view volume, as usual. In this operation, the value specified by gl_Position (i.e., the homogeneous vertex position in clip space) is evaluated against the view volume.

4.5.4. Raster Position

If a vertex shader is active when glRasterPos is called, it processes the coordinates provided with the glRasterPos command just as if these coordinates were specified with a glVertex command. The vertex shader is responsible for outputting the values necessary to compute the current raster position data.

The OpenGL state for the current raster position consists of the following seven items:

Window coordinates computed from the value written to gl_Position. These coordinates are treated as if they belong to a point and passed to the clipping stage and then projected to window coordinates.
A valid bit indicating if this point was culled.
The raster distance, which is set to the vertex shader varying variable gl_FogFragCoord.
The raster color, which is set to either the vertex shader varying variable gl_FrontColor or gl_BackColor, depending on the front/back facing selection process.
The raster secondary color, which is set to either the vertex shader varying variable gl_FrontSecondaryColor or gl_BackSecondaryColor, depending on the front/back facing selection process.
One or more raster texture coordinates. These are set to the vertex shader varying variable array gl_TexCoord[].
The raster color index. Because the result of a vertex shader is undefined in color index mode, the raster color index is always set to 1.

If any of the outputs necessary to compute the first six items are not provided, the value(s) for the associated item is undefined.

4.5.5. Position Invariance

For multipass rendering, in which a vertex shader performs some passes and other passes use the fixed functionality pipeline, positional invariance is important. This means that both the vertex shader and fixed functionality compute the exact same vertex position in clip coordinates, given the same vertex position in object coordinates and the same modelview and projection matrices. Positional invariance can be achieved with the built-in function ftransform in a vertex shader as follows:

does not result in positional invariance, because of possible compiler optimizations and potential underlying hardware differences.

4.5.6. Texturing

One of the major improvements to OpenGL made by the OpenGL Shading Language is in the area of texturing. For one thing, texturing operations can be performed in a vertex shader. But the fragment side of the pipeline has improved as well. A fragment shader can potentially have access to more texture image units than the fixed functionality pipeline does. This means that more texture lookups can be performed in a single rendering pass. And with programmability, the results of all those texture lookups can be combined in any way the shader writer sees fit.

The changes to the pipeline have resulted in some clarification to the language used to describe texturing capabilities in OpenGL. The term "texture unit" in OpenGL formerly specified more than one thing. It specified the number of texture coordinates that could be attached to a vertex (now called texture coordinate sets) as well as the number of hardware units that could be used simultaneously for accessing texture maps (now called texture image units). A texture coordinate set encompasses vertex texture coordinate attributes, as well as the texture matrix stack and texture generation state. The symbolic constant GL_MAX_TEXTURE_UNITS can be queried with glGet to obtain a single number that indicates the quantity of both of these items.

For the implementations that support the OpenGL Shading Language, these two things might actually have different values, so the number of available texture coordinate sets is now decoupled from the maximum number of texture image units. Typically, the number of available texture coordinate sets is less than the available texture image units. This should not prove to be a limitation because new texture coordinates can easily be derived from the texture coordinate attributes passed in or the coordinates can be retrieved from a texture map.

For a vertex shader, the maximum number of available texture image units is given by GL_MAX_VERTEX_TEXTURE_IMAGE_UNITS.
For a fragment shader, the maximum number of available texture image units is given by GL_MAX_TEXTURE_IMAGE_UNITS.
The combined number of texture image units used in the vertex and the fragment processing parts of OpenGL (either a fragment shader or fixed function) cannot exceed the limit GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS. If a vertex shader and the fragment processing part of OpenGL both use the same texture image unit, that counts as two units against this limit. This rule exists because an OpenGL implementation might have only GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS actual texture image units implemented, and it might share those units between the vertex and fragment processing parts of OpenGL.
When a fragment shader is not active, OpenGL can still perform multi-texturing. In this case, the maximum available multitexture stages are given by the state variable GL_MAX_TEXTURE_UNITS.
The number of supported texture coordinate sets is given by GL_MAX_TEXTURE_COORDS. This limit applies regardless of whether a vertex shader or fixed-function OpenGL performs vertex processing.

The fixed function hierarchy of texture-enables (GL_TEXTURE_CUBE_MAP, GL_TEXTURE_3D, GL_TEXTURE_2D, and GL_TEXTURE_1D) is ignored by shaders. For example, even if the texture target GL_TEXTURE_1D is enabled for a texture image unit, a sampler can be used to access the GL_TEXTURE_2D target for that texture image unit.

Samplers of type sampler1DShadow or sampler2DShadow must be used to access depth textures (textures with a base internal format of GL_DEPTH_COMPONENT). The texture comparison mode requires the shader to use one of the variants of the shadow1D or shadow2D built-in functions for accessing the texture (see Section 5.7). If these built-in functions are used to access a texture with a base internal format other than GL_DEPTH_COMPONENT, the result is undefined. Similarly, if a texture access function other than one of the shadow variants is used to access a depth texture, the result is undefined.

If a shader uses a sampler to reference a texture object that is not complete (e.g., one of the textures in a mipmap has a different internal format or border width than the others, see the OpenGL specification for a complete list), the texture image unit returns (R, G, B, A) = (0, 0, 0, 1).