< Previous PageNext Page >

Hide TOC

Techniques for Working with Texture Data

Textures add realism to OpenGL objects. They are what makes the objects defined by vertex data take on the material properties of real-world objects, such as wood, brick, metal, and fur. Texture data can originate from many sources, including images. As with vertex data, there are a variety of techniques you can use to minimize the number of times texture data is copied and converted as it's moved throughout the system.

Figure 9-1 Textures add realism to a scene

Textures start as pixel data that flows through an OpenGL program, as shown in Figure 9-2. As with vertex data you can supply pixel data in two ways. The first way, from pixel data to per-pixel operations, is as part of an OpenGL command sequence that is issued by the application and executed immediately (immediate mode). The second is packaged as a named display list that can be preprocessed ahead of time and used later in the program.

Figure 9-2 Texture data path

The precise route that texture data takes from your application to its final destination can impact the performance of your application. The purpose of this chapter is to provide techniques you can use to ensure optimal processing of texture data in your application. This chapter

shows how to use OpenGL extensions to optimize performance
lists optimal data formats and types
provides information on working with textures whose dimensions are not a power of two
describes creating textures from image data
shows how to download textures
discusses using double buffers for texture data

In this section:

Using Extensions to Optimize
Optimal Data Formats and Types
Working with Non–Power-of-Two Textures
Creating Textures from Image Data
Downloading Texture Data
Double Buffering Texture Data
See Also

Using Extensions to Optimize

Without any optimizations, texture data flows through an OpenGL program as shown in Figure 9-3. Data from your application first goes to the OpenGL framework, which may make a copy of the data before handing it to the driver. If your data is not in a native format for the hardware (see “Optimal Data Formats and Types”), the driver may also make a copy of the data to convert it to a hardware-specific format for uploading to video memory. Video memory, in turn, can keep a copy of the data. Theoretically, there could be four copies of your texture data throughout the system.

Figure 9-3 Data copies in an OpenGL program

Data flows at different rates through the system, as shown by the size of the arrows in Figure 9-3. The fastest data transfer happens between VRAM and the GPU. The slowest transfer occurs between the OpenGL driver and VRAM. Data moves between the application and the OpenGL framework, and between the framework and the driver at the same "medium" rate. Eliminating any of the data transfers, but the slowest one in particular, will improve application performance.

There are several extensions you can use to eliminate one or more data copies and control how texture data travels from your application to the GPU:

GL_APPLE_client_storage
GL_APPLE_texture_range along with a storage hint, either GL_STORAGE_CACHED_APPLE or GL_STORAGE_SHARED_APPLE
GL_ARB_texture_rectangle

The sections that follow describe the extensions and show how to use them.

Apple Client Storage

The Apple client storage extension (APPLE_client_storage) lets you provide OpenGL with a pointer to memory that your application allocates and maintains. OpenGL retains a pointer to your data but does not copy the data. Because OpenGL references your data, this extension requires that you retain a copy of your texture data until it is no longer needed. By using this extension you can eliminate the OpenGL framework copy as shown in Figure 9-4. Note that a texture width must be a multiple of 32 bytes for OpenGL to bypass the copy operation from the application to the OpenGL framework.

Figure 9-4 The client storage extension eliminates a data copy

The Apple client storage extension defines a pixel storage parameter, GL_UNPACK_CLIENT_STORAGE_APPLE, that you pass to the OpenGL function glPixelStorei to specify that your application retains storage for textures. The following code sets up client storage:

glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE);

For detailed information, see the OpenGL specification for the Apple client storage extension.

Apple Texture Range and Rectangle Texture

The Apple texture range extension (APPLE_texture_range) lets you define a region of memory used for texture data. Typically you specify an address range that encompasses the storage for a set of textures. This allows the OpenGL driver to optimize memory usage by creating a single memory mapping for all of the textures. You can also provide a hint as to how the data should be stored: cached or shared. The cached hint specifies to cache texture data in video memory. This hint is recommended when you have textures that you plan to use multiple times or that use linear filtering. The shared hint indicates that data should be mapped into a region of memory that enables the GPU to access the texture data directly (via DMA) without the need to copy it. This hint is best when you are using large images only once, perform nearest-neighbor filtering, or need to scale down the size of an image.

The texture range extension defines the following routine for making a single memory mapping for all of the textures used by your application:

void glTextureRangeAPPLE(GLenum target, GLsizei length, GLvoid *pointer);

target is a valid texture target, such as GL_TEXTURE_2D.

length specifies the number of bytes in the address space referred to by the pointer parameter.

*pointer points to the address space that your application provides for texture storage.

You provide the hint parameter and a parameter value to to the OpenGL function glTexParameteri. The possible values for the storage hint parameter (GL_TEXTURE_STORAGE_HINT_APPLE) are GL_STORAGE_CACHED_APPLE or GL_STORAGE_SHARED_APPLE.

Some hardware requires texture dimensions to be a power-of-two before the hardware can upload the data using DMA. The rectangle texture extension (ARB_texture_rectangle) was introduced to allow texture targets for textures of any dimensions—that is, rectangle textures (GL_TEXTURE_RECTANGLE_ARB). You need to use the rectangle texture extension together with the Apple texture range extension to ensure OpenGL uses DMA to access your texture data. These extensions allow you to bypass the OpenGL driver, as shown in Figure 9-5.

Note that OpenGL does not use DMA for a power-of-two texture target (GL_TEXTURE_2D). So, unlike the rectangular texture, the power-of-two texture will incur one additional copy and performance won't be quite as fast. The performance typically isn't an issue because games, which are the applications most likely to use power-of-two textures, load textures at the start of a game or level and don't upload textures in real time as often as applications that use rectangular textures, which usually play video or display images.

The next section has code examples that use the texture range and rectangle textures together with the Apple client storage extension.

Figure 9-5 The texture range extension eliminates a data copy

For detailed information on these extensions, see the OpenGL specification for the Apple texture range extension and the OpenGL specification for the ARB texture rectangle extension.

Combining Extensions

You can use the Apple client storage extension along with the Apple texture range extension to streamline the texture data path in your application. When used together, OpenGL moves texture data directly into video memory, as shown in Figure 9-6. The GPU directly accesses your data (via DMA). The set up is slightly different for rectangular and power-of-two textures. The code examples in this section upload textures to the GPU. You can also use these extensions to download textures, see “Downloading Texture Data.”

Figure 9-6 Combining extensions to eliminate data copies

Combining extensions to eliminate two data copies

Listing 9-1 shows how to use the extensions for a rectangular texture. After enabling the texture rectangle extension you need to bind the rectangular texture to a target. Next, set up the storage hint. Call glPixelStorei to set up the Apple client storage extension. Finally, call the function glTexImage2D with a with a rectangular texture target and a pointer to your texture data.

Note: The texture rectangle extension limits what can be done with rectangular textures. To understand the limitations in detail, read the OpenGL extension for texture rectangles. See “Working with Non–Power-of-Two Textures” for an overview of the limitations and an alternative to using this extension.

Listing 9-1 Using texture extensions for a rectangular texture

glEnable (GL_TEXTURE_RECTANGLE_ARB);

glBindTexture(GL_TEXTURE_RECTANGLE_ARB, id);

glTexParameteri(GL_TEXTURE_RECTANGLE_ARB,

        GL_TEXTURE_STORAGE_HINT_APPLE,

        GL_STORAGE_CACHED_APPLE);

glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE);

glTexImage2D(GL_TEXTURE_RECTANGLE_ARB,

            0, GL_RGBA, sizex, sizey, GL_BGRA,

            GL_UNSIGNED_INT_8_8_8_8_REV,

            myImagePtr);

Setting up a power-of-two texture to use these extensions is similar to what's needed to set up a rectangular texture, as you can see by looking at Listing 9-2. The difference is that the GL_TEXTURE_2D texture target replaces the GL_TEXTURE_RECTANGLE_ARB texture target.

Listing 9-2 Using texture extensions for a power-of-two texture

glBindTexture(GL_TEXTURE_2D, myTextureName);

glTexParameteri(GL_TEXTURE_2D,

        GL_TEXTURE_STORAGE_HINT_APPLE,

        GL_STORAGE_CACHED_APPLE);

glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE);

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA,

            sizex, sizey, GL_BGRA,

            GL_UNSIGNED_INT_8_8_8_8_REV, myImagePtr);

Optimal Data Formats and Types

The best format and data type combinations to use for texture data are:

GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV
GL_BGRA, GL_UNSIGNED_SHORT_1_5_5_5_REV)
GL_YCBCR_422_APPLE, GL_UNSIGNED_SHORT_8_8_REV_APPLE

The combination GL_RGBA and GL_UNSIGNED_BYTE needs to be swizzled by many cards when the data is loaded, so it's not recommended.

Working with Non–Power-of-Two Textures

With more and more frequency, OpenGL is being used to process video and images, which typically have dimensions that are not a power-of-two. Until OpenGL 2.0, the texture rectangle extension (ARB_texture_rectangle) provided the only option for a rectangular texture target. This extension, however, imposes the following restrictions on rectangular textures:

You can't use mipmap filtering with them.
You can use only these wrap modes: GL_CLAMP, GL_CLAMP_TO_EDGE, and GL_CLAMP_TO_BORDER.
The texture cannot have a border.
The texture uses non-normalized texture coordinates. (See Figure 9-7.)

OpenGL 2.0 adds another option for a rectangular texture target through the ARB_texture_non_power_of_two extension, which supports these textures without the limitations of the ARB_texture_rectangle extension. Before using it, you must check to make sure the functionality is available. You'll also want to consult the OpenGL specification for the non—power-of-two extension.

Figure 9-7 Normalized and non-normalized coordinates

If your code runs on a system that does not support either the ARB_texture_rectangle or ARB_texture_non_power_of_two extensions you have these options for working with with rectangular images:

Use the OpenGL function gluScaleImage to scale the image so that it fits in a rectangle whose dimensions are a power of two. The image undoes the scaling effect when you draw the image from the properly sized rectangle back into a polygon that has the correct aspect ratio for the image.
Note: This option can result in the loss of some data. But if your application runs on hardware that doesn't support the ARB_texture_rectangle extension, you may need to use this option.
Segment the image into power-of-two rectangles, as shown in Figure 9-8 by using one image buffer and different texture pointers. Notice how the sides and corners of the image shown in Figure 9-8 are segmented into increasingly smaller rectangles to ensure that every rectangle has dimensions that are a power of two. Special care may be needed at the borders between each segment to avoid filtering artifacts if the texture is scaled or rotated.
The OpenGL Image sample application available on Sample Code > Graphics & Imaging > OpenGL contains segmentation code and demonstrates other OpenGL features that support high-performance image display.

Figure 9-8 An image segmented into power-of-two tiles

Creating Textures from Image Data

OpenGL on the Macintosh provides several options for creating high-quality textures from image data. Mac OS X supports floating-point pixel values, multiple image file formats, and a variety of color spaces. You can import a floating-point image into a floating-point texture. Figure 9-9 shows an image used to texture a cube.

Figure 9-9 Using an image as a texture for a cube

For Cocoa, you need to provide a bitmap representation. You can create an NSBitmapImageRep object from the contents of an NSView object. For either Cocoa or Carbon, you can use the Image I/O framework (see CGImageSource Reference). This framework has support for many different file formats, floating-point data, and a variety of color spaces. Furthermore, it is easy to use. You can import image data as a texture simply by supplying a CFURL object that specifies the location of the texture. There is no need for you to convert the image to an intermediate integer RGB format.

Creating a Texture from a Cocoa View

You can use the NSView class or a subclass of it for texturing in OpenGL. The process is to first store the image data from an NSView object in an NSBitmapImageRep object so that the image data is in a format that can be readily used as texture data by OpenGL. Then, after setting up the texture target, you supply the bitmap data to the OpenGL function glTexImage2D. Note that you must have a valid, current OpenGL context set up.

Note: You can't create an OpenGL texture from image data that's provided by a view created from the following classes: NSProgressIndicator, NSMovieView, and NSOpenGLView. This is because these views do not use the window backing store, which is what the method initWithFocusedViewRect: reads from.

Listing 9-3 shows a routine that uses this process to create a texture from the contents of an NSView object. A detailed explanation for each numbered line of code appears following the listing.

Listing 9-3 Building an OpenGL texture from an NSView object

-(void)myTextureFromView:(NSView*)theView

                textureName:(GLuint*)texName

    NSBitmapImageRep * bitmap = [NSBitmapImageRep alloc]; // 1

    int samplesPerPixel = 0;

    [theView lockFocus]; // 2

    [bitmap initWithFocusedViewRect:[theView bounds]]; // 3

    [theView unlockFocus];

    glPixelStorei(GL_UNPACK_ROW_LENGTH, [bitmap pixelsWide]); // 4

    glPixelStorei (GL_UNPACK_ALIGNMENT, 1); // 5

    if (*texName == 0) // 6

            glGenTextures (1, texName);

    glBindTexture (GL_TEXTURE_RECTANGLE_ARB, *texName); // 7

    glTexParameteri(GL_TEXTURE_RECTANGLE_ARB,

                    GL_TEXTURE_MIN_FILTER, GL_LINEAR); // 8

    samplesPerPixel = [bitmap samplesPerPixel]; // 9

   if(![bitmap isPlanar] &&

       (samplesPerPixel == 3 || samplesPerPixel == 4)) { // 10

        glTexImage2D(GL_TEXTURE_RECTANGLE_ARB,

0,

                     samplesPerPixel == 4 ? GL_RGBA8 : GL_RGB8,

                     [bitmap pixelsWide],

                     [bitmap pixelsHigh],

0,

                     samplesPerPixel == 4 ? GL_RGBA : GL_RGB,

                     GL_UNSIGNED_BYTE,

                    [bitmap bitmapData]);

    } else {

       // Your code to report unsupported bitmap data

   [bitmap release]; // 11

Here's what the code does:

Allocates an NSBitmapImageRep object.
Locks the focus on the the NSView object so that subsequent commands take effect in coordinate system of the NSView object. You must invoke lockFocus before invoking methods that send commands to the window server, which is the case with the next line of code. Later, you must balance a lockFocus message with an unlockFocus message.
Initializes the NSBitmapImageRep object with bitmap data from the current view using the bounds returned by the NSView object passed to the myTextureFromView:textureName routine.
Sets the appropriate unpacking row length for the bitmap.
Sets the byte-aligned unpacking that's needed for bitmaps that are 3 bytes per pixel.
If a texture object is not passed in, generates a new texture object.
Binds the texture name to the texture target.
Sets filtering so that it does not use a mipmap, which would be redundant for the texture rectangle extension.
Gets the number of samples per pixel.
Checks to see if the bitmap is nonplanar and is either a 24-bit RGB bitmap or a 32-bit RGBA bitmap. If so, retrieves the pixel data using the bitmapData method, passing it along with other appropriate parameters to the OpenGL function for specifying a 2D texture image.
Releases the NSBitmapImageRep object when it is no longer needed.

Creating a Texture from a Quartz Image Source

Quartz images (CGImageRef data type) are defined in the Core Graphics framework (ApplicationServices/CoreGraphics.framework/CGImage.h) while the image source data type for reading image data and creating Quartz images from an image source is declared in the Image I/O framework (ApplicationServices/ImageIO.framework/CGImageSource.h). Quartz provides routines that read a wide variety of image data.

To use a Quartz image as a texture source, follow these steps:

Create a Quartz image source by supplying a CFURL object to the function CGImageSourceCreateWithURL.
Create a Quartz image by extracting an image from the image source, using the function CGImageSourceCreateImageAtIndex.
Extract the image dimensions using the function CGImageGetWidth and CGImageGetHeight. You'll need these to calculate the storage required for the texture.
Allocate storage for the texture.
Create a color space for the image data.
Create a Quartz bitmap graphics context for drawing. Make sure to set up the context for pre-multiplied alpha.
Draw the image to the bitmap context.
Release the bitmap context.
Set the pixel storage mode by calling the function glPixelStorei.
Create and bind the texture.
Set up the appropriate texture parameters.
Call glTexImage2D, supplying the image data.
Free the image data.

Listing 9-4 shows a code fragment that performs these steps. Note that you must have a valid, current OpenGL context.

Listing 9-4 Using a Quartz image as a texture source

CGImageSourceRef myImageSourceRef = CGImageSourceCreateWithURL(url, NULL);

CGImageRef myImageRef = CGImageSourceCreateImageAtIndex (myImageSourceRef, 0, NULL);

GLint myTextureName;

size_t width = CGImageGetWidth(myImageRef);

size_t height = CGImageGetHeight(myImageRef);

CGRect rect = {{0, 0}, {width, height}};

void * myData = calloc(width * 4, height);

CGColorSpaceRef space = CGColorSpaceCreateDeviceRGB();

CGContextRef myBitmapContext = CGBitmapContextCreate (myData,

                        width, height, 8,

                        width*4, space,

                        kCGBitmapByteOrder32Host |

                          kCGImageAlphaPremultipliedFirst);

CGContextDrawImage(myBitmapContext, rect, myImageRef);

CGContextRelease(myBitmapContext);

glPixelStorei(GL_UNPACK_ROW_LENGTH, width);

glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

glGenTextures(1, &myTextureName);

glBindTexture(GL_TEXTURE_RECTANGLE_ARB, myTextureName);

glTexParameteri(GL_TEXTURE_RECTANGLE_ARB,

                    GL_TEXTURE_MIN_FILTER, GL_LINEAR);

glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_RGBA8, width, height,

                    0, GL_BGRA_EXT, GL_UNSIGNED_INT_8_8_8_8_REV, myData);

free(myData);

For more information on using Quartz, see Quartz 2D Programming Guide, CGImage Reference, and CGImageSource Reference.

Getting Decompressed Raw Pixel Data from a Source Image

You can use the Image I/O framework together with a Quartz data provider to obtain decompressed raw pixel data from a source image, as shown in Listing 9-5. You can then use the pixel data for your OpenGL texture. The data has the same format as the source image, so you need to make sure that you use a source image that has the layout you need.

Alpha is not premultiplied for the pixel data obtained in Listing 9-5, but alpha is premultiplied for the pixel data you get when using the code described in “Creating a Texture from a Cocoa View” and “Creating a Texture from a Quartz Image Source.”

Listing 9-5 Getting pixel data from a source image

CGImageSourceRef myImageSourceRef = CGImageSourceCreateWithURL(url, NULL);

CGImageRef myImageRef = CGImageSourceCreateImageAtIndex (myImageSourceRef, 0, NULL);

CFDataRef data = CGDataProviderCopyData(CGImageGetDataProvider(myImageRef));

void *pixelData = CFDataGetBytePtr(data);

Downloading Texture Data

A texture download operation uses the same data path as an upload operation except that the data path is reversed. Downloading transfers texture data, using direct memory access (DMA), from VRAM into a texture that can then be accessed directly by your application. You can use the Apple client range, texture range, and texture rectangle extensions for downloading, just as you would for uploading.

To download texture data using the Apple client storage, texture range, and texture rectangle extensions:

Bind a texture name to a texture target.
Set up the extensions
Call the function glCopyTexSubImage2D to copy a texture subimage from the specified window coordinates. This call initiates an asynchronous DMA transfer to system memory the next time you call a flush routine. The CPU doesn't wait for this call to complete.
Call the function glGetTexImage to transfer the texture into system memory. Note that the parameters must match the ones that you used to set up the texture when you called the function glTexImage2D. This call is the synchronization point; it waits until the transfer is finished.

Listing 9-6 shows a code fragment that downloads a rectangular texture that uses cached memory. Your application processes data between the glCopyTexSubImage2D and glGetTexImage calls. How much processing? Enough so that your application does not need to wait for the GPU.

Listing 9-6 Code that downloads texture data

glBindTexture(GL_TEXTURE_RECTANGLE_ARB, myTextureName);

glTexParameteri(GL_TEXTURE_RECTANGLE_ARB, GL_TEXTURE_STORAGE_HINT_APPLE,

                GL_STORAGE_SHARED_APPLE);

glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE);

glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_RGBA,

                sizex, sizey, GL_BGRA,

                GL_UNSIGNED_INT_8_8_8_8_REV, myImagePtr);

glCopyTexSubImage2D(GL_TEXTURE_RECTANGLE_ARB,

                0, 0, 0, 0, 0, image_width, image_height);

glFlush();

// Do other work processing here, using a double or triple buffer

glGetTexImage(GL_TEXTURE_RECTANGLE_ARB, 0, GL_BGRA,

                GL_UNSIGNED_INT_8_8_8_8_REV, pixels);

Double Buffering Texture Data

When you use any technique that allows the GPU to access your texture data directly, such as the texture range extension, it's possible for the GPU and CPU to access the data at the same time. To avoid such a collision, you must synchronize the GPU and the CPU. The simplest way is shown in Figure 9-10. Your application works on the data, flushes it to the GPU and waits until the GPU is finished before working on the data again.

One technique for ensuring that the GPU is finished executing commands before your application sends more data is to insert a token into the command stream and use that to determine when the CPU can touch the data again, as described in “Fence Extension.” Figure 9-10 uses the fence extension command glFinishObject to synchronize buffer updates for a stream of single-buffered texture data. Notice that when the CPU is processing texture data, the GPU is idle. Similarly, when the GPU is processing texture data, the CPU is idle. It's much more efficient for the GPU and CPU to work asynchronously than to work synchronously. Double buffering data is a technique that allows you to process data asynchronously, as shown in Figure 9-11.

Figure 9-10 Single-buffered data

To double buffer data, you must supply two sets of data to work on. Note in Figure 9-11 that while the GPU is rendering one frame of data, the CPU processes the next. After the initial startup, neither processing unit is idle. Using the glFinishObject function provided by the fence extension ensures that buffer updating is synchronized.

Figure 9-11 Double-buffered data