How to Compress and Decompress Sequences of Images
This chapter provides a sample program illustrating the processes used to compress and decompress image sequences. The listing is broken up into a main program and a series of functions. Each function is introduced with a discussion of its purpose.
Functions for saving a sequence of images to a disk and for creating, compressing, and drawing a sequence of images are also discussed.
In this section:
Compressing Sequences
Decompressing Sequences
Defining Key Frame Rates
A Sample Program for Compressing and Decompressing a Sequence of Images
Compressing Sequences
The Image Compression Manager also provides functions that allow your application to compress and decompress sequences of images, such as might constitute a QuickTime movie. The tools provided by the Image Compression Manager focus on image compression and decompression and on the ordering of the images in a sequence, not on timing considerations. Use the Movie Toolbox to handle all the issues relating to the amount of time each image should be shown on the screen. For information on decompressing image sequences, see the next section, Decompressing Sequences.
A series of images can be compressed as a sequence if those images share an image description. That is, each image in the sequence must have the same compressor type, pixel depth, color lookup table, and boundary dimensions. To take best advantage of temporal compression, the images should also be related to each other (like frames in a movie), but this relationship is not necessary for them to be grouped as a sequence. If you create a sequence from completely unrelated images, you may not be able to achieve significant temporal compression.
When compressing image sequences, your application must perform several steps in addition to those required for single-frame image compression. This section describes a typical function for compressing an image sequence. Note that much of the setup processing is the same as that performed for single-frame images.
First, determine the parameters for the compression operation. As with single-image compression, the user may specify these parameters in a dialog box you can supply via the standard image-compression dialog component. Your application may choose to give the user the ability to specify such parameters as the compression algorithm, image quality, and so on. Note that image sequences require additional parameters, such as temporal quality.
Your application may give the user the option of specifying a compression algorithm based on an important performance characteristic. For example, the user may be most concerned with size, speed, or accuracy. The Image Compression Manager allows your application to choose the compressor component that meets the specified criterion.
Your application signals its intention to compress an image sequence by issuing the Image Compression Manager’s CompressSequenceBegin function (see Working With Sequences for more information about this function). At this time your application specifies many of the parameters that govern the sequence-compression operation. When you set the compression parameters and the temporalQuality parameter is not 0, then be sure to set the value of either the codecFlagUpdatePrevious or codecFlagUpdatePreviousComp flag to 1 in the flags parameter of the CompressSequenceBegin function.
Once you have started the sequence, you then compress each image in the sequence by performing the following steps:
Your application must call the Image Compression Manager’s
GetMaxCompressionSizefunction to determine the maximum size of the compressed data that will result from the current image (see Getting Information About Compressed Data for more information about this function). You provide the specified compression parameters to this function. In response, the Image Compression Manager invokes the appropriate compressor component to determine the number of bytes required to store the largest compressed image in the sequence. Your application should then reserve sufficient memory to accommodate that compressed image. You can use this returned value until you change the settings of the compression parameters.Your application must call the
CompressSequenceFramefunction to compress the image (see Working With Sequences for more information about this function). It may be necessary or desirable for your application to change one or more of the compression parameters while processing a sequence. The Image Compression Manager provides several functions that allow your application to modify such parameters as the spatial or temporal quality or the data-unloading function. See Changing Sequence-Compression Parameters for more information about these functions.The Image Compression Manager manages the compression operation and invokes the appropriate compressor. The manager returns the compressed image and its associated image description to your application.
Your application is then free to store the compressed image with the others in the sequence.
After the entire sequence is compressed, you end the process by calling the CDSequenceEnd function.
Decompressing Sequences
The Movie Toolbox handles the details of displaying compressed image sequences that are stored in QuickTime movies. However, if you want to work with sequences in your application, the Image Compression Manager provides tools for decompressing image sequences. As with still-image compression, decompressing sequences requires additional effort on the part of your application. In addition, there are some processing considerations that are particular to sequence decompression. This section describes the steps necessary to decompress an image sequence. Then it discusses several points you should consider before decompressing a sequence.
When decompressing an image sequence, your application must first determine where to display the decompressed sequence. Your application must specify the destination graphics port to the Image Compression Manager. In addition, you may indicate that only a portion of the source image is to be displayed. You describe the desired portion of the image by specifying a rectangle in the coordinate system of the source image. You can determine the size of the source image by examining the image description structure associated with the image (see The Image Description Structure for more information about image description structures).
Your application may also specify that the image is to be mapped into the destination graphics port. The DecompressSequenceBegin function allows your application to specify a mapping matrix for the operation.
Your application can invoke additional effects by specifying a mask region or blend matte for the image. Mask regions and mattes control which pixels in the source image are drawn to the destination. Mask regions must be defined in the destination coordinate system. During decompression the Image Compression Manager displays only those pixels in the source image that correspond to bits in the mask that are set to 1. Mattes contain several bits per pixel and are defined in the coordinate system of the source image. Mattes provide a mechanism for blending pixels from source images.
Your application signals its intention to decompress an image sequence by issuing the Image Compression Manager’s DecompressSequenceBegin function. At this time your application specifies many of the parameters that govern the sequence-decompression operation. The Image Compression Manager, in turn, allocates system resources that are necessary for the operation.
Once you have started the sequence, you then decompress each image in the sequence. Call the DecompressSequenceFrame function to decompress the image. It may be necessary or desirable for your application to change one or more of the decompression parameters while processing a sequence. The Image Compression Manager provides several functions that allow your application to modify such parameters as the accuracy, the transformation matrix, or the data-loading function. See Changing Sequence-Decompression Parameters for more information about these functions.
The Image Compression Manager manages the decompression operation and invokes the appropriate compressor component. The manager returns the decompressed image to the location specified by your application and applies any effects you may have specified.
After the entire sequence is decompressed, you end the process by calling the CDSequenceEnd function.
The Basic Functions to Use
The basic functions used to compress and decompress a sequence of images include
Note that the sequence itself contains no time information, only the order in which images should appear. This book defines several new functions for working with sequences, including functions supporting timecodes and asynchronous decompression.
Decompressing Still Images From a Sequence
Your application can, of course, decompress individual images from a sequence. When doing so, you must be careful to select only those frames that do not depend on other frames. That is, do not decompress frames from a sequence that has been temporally compressed unless you first decompress all the frames in sequence starting from the preceding key frame (see Defining Key Frame Rates for more information on key frames in image sequences). In general, you should decompress images from sequences as sequences, rather than as individual frames.
Using Screen Buffers and Image Buffers
The use of screen buffers has been discontinued in QuickTime. Use only image buffers. A request for a screen buffer will return an image buffer. Do not request a screen buffer in your application.
The Image Compression Manager uses image buffers when decompressing sequences that have been temporally compressed and therefore contain key frames. Image buffers are especially useful when you want to skip to random frames within a sequence. Random frame access in temporally compressed sequences forces the compressor to decompress all the frames between the nearest preceding key frame and the desired frame. Reconstructing the frame in this manner on the screen can result in jerky sequence display. As an alternative, the compressor can reconstruct the frame in the offscreen image buffer and then copy it to the screen when appropriate. Image buffers are allocated at an appropriate depth and size for the decompressor.
Your application can control the use of the image buffer by the compressor component. For example, you can force the compressor to draw images only to the image buffer, not to the screen. In this manner you can use the image buffer to build up sequences without making the process visible. You can also control when the compressor uses the image buffer. You may need to do this when your program is decompressing directly to the screen and suddenly is prevented from doing so (for example, when your window becomes hidden).
Defining Key Frame Rates
The process of temporal compression involves reducing or eliminating temporal redundancy from an image sequence. Temporal compression is most effective when a sequence contains frames that bear significant similarity to adjacent frames. This is typically true of movies and other video sequences. Reconstructing an individual frame within a sequence that has been temporally compressed requires knowledge of the previous frames. This does not present a problem if your application always plays compressed sequences from the beginning. However, if your application needs to start playing a sequence from a random point, or perhaps backward, the decompressor does not have enough information to decompress the frames.
To alleviate this problem, compressors insert key frames in compressed sequences at regular intervals. Key frames define starting points for portions of a temporally compressed sequence. Subsequent frames depend on the previous key frame.
At the start of a sequence compression your application can specify a rate at which the compressor is to insert key frames into the compressed data stream. This key frame rate indicates the maximum number of frames you will accept between key frames. The Image Compression Manager picks the best key frames from the source sequence and at the same time enforces the specified key frame rate (the best key frames are those that are least similar to adjacent frames, such as at scene changes; these frames would have the largest compressed images even if they were not selected as key frames).
During sequence compression your application can change the key frame rate by calling the SetCSequenceKeyFrameRate function. By manipulating the parameters for the sequence, you can force the Image Compression Manager to place a key frame at any arbitrary point in a sequence (set the codecFlagForceKeyFrame flag to 1 in the flags parameter of the CompressSequenceFrame function.
A Sample Program for Compressing and Decompressing a Sequence of Images
The sample program presented in this section illustrates the processes described in the previous sections. The program has been divided into several functions. Listing 6-1 shows the main program.
The data for each frame is written to the data fork of the disk file, preceded by a long word that contains the number of bytes of data for that frame. A description of the compressed images in the sequence is stored in a 'SEQU' resource in the same file with a resource ID of 128 or 129. This description is simply the image description structure maintained by the Image Compression Manager.
The image for each frame of the sequence is drawn into an offscreen graphics world that the SequenceSave function creates in the currWorld variable. SequenceSave calls the DrawOneFrame function (described in the next section) to draw each frame’s image into the currWorld variable. Before any of the frames of the sequence are drawn, the Image Compression Manager is prepared to compress a sequence of images through the CompressSequence function.
Listing 6-1 Compressing and decompressing a sequence of images: The main program
WindowPtr displayWindow; /* window in which to display |
sequence */ |
Rect windowRect; /* rectangle of displayWindow */ |
main (void) |
{ |
WindowPtr displayWindow; |
Rect windowRect; |
InitGraf (&thePort); |
InitFonts (); |
InitWindows (); |
InitMenus (); |
TEInit (); |
InitDialogs (nil); |
SetRect (&windowRect, 0, 0, 256, 256); |
OffsetRect (&windowRect,/* middle of screen */ |
((qd.screenBits.bounds.right - qd.screenBits.bounds.left) - |
windowRect.right) / 2, |
((qd.screenBits.bounds.bottom - qd.screenBits.bounds.top) - |
windowRect.bottom) / 2); |
displayWindow = NewCWindow (nil, &windowRect, |
"\pImage", true, 0, |
(WindowPtr)-1, true, 0); |
if (displayWindow) |
{ |
SetPort (displayWindow); |
SequenceSave (); |
SequencePlay (); |
} |
} |
A Sample Function for Saving a Sequence of Images to a Disk File
The SequenceSave function shown in Listing 6-2 saves a sequence of images to a disk file. This function creates and opens a disk file for the image sequence, calls the CompressSequence function to create and compress the image sequence into the file, and then calls the MakeMyResource function to save the image description resource in the file, so that the sequence can be played back later. For details on CompressSequence, see the next section.
Listing 6-2 Saving a sequence of images to a disk file
void SequenceSave (void) |
{ |
long filePos; |
StandardFileReply fileReply; |
short dfRef = 0; |
OSErr error; |
ImageDescriptionHandle description = nil; |
StandardPutFile ("\p", "\pSequence File", &fileReply); |
if (fileReply.sfGood) |
{ |
if (! (fileReply.sfReplacing)) |
{ |
error = FSpCreate (&fileReply.sfFile, 'SEQM', 'SEQU', |
fileReply.sfScript); |
CheckError (error, "\pFSpCreate"); |
} |
error = FSpOpenDF (&fileReply.sfFile, fsWrPerm, &dfRef); |
CheckError (error, "\pFSpOpenDF"); |
error = SetFPos (dfRef, fsFromStart, 0); |
CheckError (error, "\pSetFPos"); |
CompressSequence (&dfRef, &description); |
error = GetFPos (dfRef, &filePos); |
CheckError (error, "\pGetFPos"); |
error = SetEOF (dfRef, filePos); |
CheckError (error, "\pSetEOF"); |
FSClose (dfRef); |
FlushVol (nil, fileReply.sfFile.vRefNum); |
MakeMyResource (fileReply, description); |
if (description != nil) |
DisposeHandle ((Handle) description); |
} |
} |
void MakeMyResource ( StandardFileReply fileReply, |
ImageDescriptionHandle description) |
{ |
OSErr error; |
short rfRef; |
Handle sequResource; |
FSpCreateResFile (&fileReply.sfFile, 'SEQM', 'SEQU', |
fileReply.sfScript); |
error = ResError(); |
if (error != dupFNErr) |
CheckError (error, "\pFSpCreateResFile"); |
rfRef = FSpOpenResFile (&fileReply.sfFile, fsRdWrPerm); |
CheckError (ResError (), "\pFSpOpenResFile"); |
SetResLoad (false); |
sequResource = Get1Resource ('SEQU', 128); |
if (sequResource) |
RmveResource (sequResource); |
SetResLoad (true); |
sequResource = (Handle) description; |
error = HandToHand (&sequResource); |
CheckError (error, "\pHandToHand"); |
AddResource (sequResource,'SEQU', 128, "\p"); |
CheckError (ResError (), "\pAddResource"); |
UpdateResFile (rfRef); |
CheckError (ResError (), "\pUpdateResFile"); |
CloseResFile (rfRef); |
} |
A Sample Function for Creating, Compressing, and Drawing a Sequence of Images
Listing 6-3 shows the CompressSequence function, which creates and then compresses the image sequence. CompressSequenceBegin informs the Image Compression Manager which compressor (of type codectype ) to use, what the desired compression quality is, the key frame rate, the portion of the image to compress (in this example, the entire image is compressed), and the image to be compressed (in this example, the pixel map, of type PixMap, in the currWorld variable).
CompressSequenceBegin returns a unique number that identifies the sequence for subsequent image-compression routines, and it initializes a new image description structure, which is stored in the handle referenced by the description local variable.
Using a loop, the DrawOneFrame function draws each frame until the last frame is drawn, at which time the function returns the value of false. Each frame that it draws is copied to the window so that it can be seen during the compression sequence.
The CompressSequenceFrame function is used to compress each frame’s image. CompressSequenceFrame tells the Image Compression Manager
which image to compress (in this case, the pixel map of the
currWorldvariable)the portion of that image to compress (in this case, all of it)
whether to update the previous frame’s buffer for frame differencing
the address of the buffer that’s to receive the compressed image data
In updating the previous frame’s buffer for frame differencing, the Image Compression Manager control flag codecFlagUpdatePrevious copies the uncompressed image to the previous frame’s buffer; contrast this with the codecFlagUpdatePreviousComp flag, which copies the compressed image to the previous frame’s buffer. The more lossy the compression, the more the difference between the compressed and uncompressed images.
The CompressSequenceBegin function returns a rating of the similarity between the current frame and the previous frame, but this example ignores this rating. After each frame is compressed, the number of bytes in the compressed image data is written to the disk file, followed by the compressed image data itself.
After all the images in the sequence have been compressed, the CDSequenceEnd function is called to tell the Image Compression Manager that the sequence is over. The data fork of the file is closed, and the image description is written to a 'SEQU' resource.
The DrawOneFrame function draws one frame of the sequence with QuickDraw. The frame’s image is drawn into the rectangle specified by the destRect parameter. The image is a set of color ramps in which the shading goes from light to dark in smooth increments. The color ramps fill the destination rectangle and the current frame number centered within the destination rectangle over the ramps.
The PaintImage function paints a series of vertical color ramps into the rectangle specified by the destRect parameter into the current color graphics port. This is done through a nested loop. The outer loop iterates only twice, and half of the ramps are drawn in the first iteration and half in the second. The inner loop iterates over all the steps in a ramp.
Listing 6-3 Creating and compressing an image sequence
void CompressSequence (short* dfRef, ImageDescriptionHandle* description) |
{ |
GWorldPtr currWorld = nil; |
PixMapHandle currPixMap; |
CGrafPtr savedPort; |
GDHandle savedDevice; |
Handle buffer = nil; |
Ptr bufferAddr; |
long compressedSize; |
long dataLen; |
Rect imageRect; |
ImageSequence sequenceID = 0; |
short frameNum; |
OSErr error; |
CodecType codecKind = 'rle '; |
GetGWorld (&savedPort, &savedDevice); |
imageRect = savedPort->portRect; |
error = NewGWorld (&currWorld, 32, &imageRect, nil, nil, 0); |
CheckError (error, "\pNewGWorld"); |
SetGWorld (currWorld, nil); |
currPixMap = currWorld->portPixMap; |
LockPixels (currPixMap); |
/* |
Allocate an embryonic image description structure and the |
Image Compression Manager will resize. |
*/ |
*description = (ImageDescriptionHandle) NewHandle (4); |
error = CompressSequenceBegin ( |
&sequenceID, |
currPixMap, |
nil, /* tell ICM to allocate previous |
image buffer */ |
&imageRect, |
&imageRect, |
0, /* let ICM choose pixel depth */ |
codecKind, |
(CompressorComponent) anyCodec, |
codecNormalQuality, /* spatial quality */ |
codecNormalQuality, /* temporal quality */ |
5, /* at least 1 key frame every |
5 frames */ |
nil, /* use default color table */ |
codecFlagUpdatePrevious, |
*description ); |
CheckError (error, "\pCompressSequenceBegin"); |
error = GetMaxCompressionSize( |
currPixMap, |
&imageRect, |
0, /* let ICM choose pixel depth */ |
codecNormalQuality, /* spatial quality */ |
codecKind, |
(CompressorComponent) anyCodec, |
&compressedSize ); |
CheckError (error, "\pGetMaxCompressionSize"); |
buffer = NewHandle(compressedSize); |
CheckError (MemError(), "\pNewHandle buffer"); |
MoveHHi (buffer); |
HLock (buffer); |
bufferAddr = StripAddress (*buffer); |
for (frameNum = 1; frameNum <= 10; frameNum++) |
{ |
DrawFrame (&imageRect, frameNum); |
error = CompressSequenceFrame ( |
sequenceID, |
currPixMap, |
&imageRect, |
codecFlagUpdatePrevious, |
bufferAddr, |
&compressedSize, |
nil, |
nil ); |
CheckError (error, "\pCompressSequenceFrame"); |
dataLen = 4; |
error = FSWrite (*dfRef, &dataLen, &compressedSize); |
CheckError (error, "\pFSWrite length"); |
error = FSWrite (*dfRef, &compressedSize, bufferAddr); |
CheckError (error, "\pFSWrite buffer"); |
} |
CDSequenceEnd (sequenceID); |
DisposeGWorld (currWorld); |
SetGWorld (savedPort,savedDevice); |
if (buffer) DisposeHandle ( buffer ); |
} |
void DrawFrame (const Rect *imageRect, long frameNum) |
{ |
Str255 numStr; |
ForeColor( redColor ); |
PaintRect( imageRect ); |
ForeColor( blueColor ); |
NumToString (frameNum, numStr); |
MoveTo ( imageRect->right / 2, imageRect->bottom / 2); |
TextSize ( imageRect->bottom / 3); |
DrawString (numStr); |
} |
A Sample Function for Decompressing and Playing Back a Sequence From a Disk File
The SequencePlay function, shown in Listing 6-4, plays back a sequence of images from a disk file that was created by the SequenceSave function.
The SequencePlay function begins by grabbing the image description structure from the file that the user specified from a 'SEQU' resource ID 128. This structure is needed to decompress the images in the file.
Before these compressed images are read, the Image Compression Manager is told to prepare to decompress a sequence of images through the DecompressSequenceBegin function. This routine tells the Image Compression Manager
how the images were compressed with the image description structure
where to display the decompressed image (the current port in this example)
what part of the image to decompress (all of it)
what transfer mode to use when displaying the image (
srcCopy)whether to buffer the image for frame differences
A loop iterates for each frame in the file. For each frame, a long word with the number of bytes in the frame is read from the file, and then that many bytes are read from the file into a compressed-image buffer. This buffer is passed to DecompressSequenceFrame, which decompresses the image to the screen (the destination doesn’t have to be the screen, but it is in this example). The loop iterates until the end of the file has been reached.
Listing 6-4 Playing back a sequence of images from a disk file
void SequencePlay (void) |
{ |
ImageDescriptionHandle description; |
long compressedSize; |
Handle buffer = nil; |
Ptr bufferAddr; |
long dataLen; |
long lastTicks; |
ImageSequence sequenceID; |
Rect imageRect; |
StandardFileReply fileReply; |
SFTypeList typeList = {'SEQU',0,0,0}; |
short dfRef = 0; /* sequence data fork */ |
short rfRef = 0; /* sequence resource fork */ |
OSErr error; |
StandardGetFile (nil, 1, typeList, &fileReply); |
if (!fileReply.sfGood) return; |
rfRef = FSpOpenResFile (&fileReply.sfFile, fsRdPerm); |
CheckError (ResError (), "\pFSpOpenResFile"); |
description = (ImageDescriptionHandle) |
Get1Resource ('SEQU', 128); |
CheckError (ResError (), "\pGet1Resource"); |
DetachResource ((Handle) description ); |
HNoPurge ((Handle) description ); |
CloseResFile (rfRef); |
error = FSpOpenDF (&fileReply.sfFile, fsRdPerm, &dfRef); |
CheckError (error, "\pFSpOpenDF"); |
buffer = NewHandle (4); |
CheckError (MemError (), "\pNewHandle buffer"); |
SetRect (&imageRect, 0, 0, (**description).width, |
(**description).height); |
error = DecompressSequenceBegin ( |
&sequenceID, |
description, |
nil, /* use the current port */ |
nil, /* go to screen */ |
&imageRect, |
nil, /* no matrix */ |
ditherCopy, |
nil, /* no mask region */ |
codecFlagUseImageBuffer, |
codecNormalQuality, /* accuracy */ |
(CompressorComponent) anyCodec); |
while (true) |
{ |
dataLen = 4; |
error = FSRead (dfRef, &dataLen, &compressedSize); |
if (error == eofErr) |
break; |
CheckError( error, "\pFSRead" ); |
if (compressedSize > GetHandleSize (buffer)) |
{ |
HUnlock (buffer); |
SetHandleSize (buffer, compressedSize); |
CheckError (MemError(), "\pSetHandleSize"); |
} |
HLock (buffer); |
bufferAddr = StripAddress (*buffer); |
error = FSRead (dfRef, &compressedSize, bufferAddr); |
CheckError (error, "\pFSRead"); |
error = DecompressSequenceFrame ( |
sequenceID, |
bufferAddr, |
0, // flags |
nil, |
nil ); |
CheckError (error, "\pDecompressSequenceFrame"); |
Delay (30, &lastTicks); |
} |
CDSequenceEnd (sequenceID); |
if (dfRef) FSClose (dfRef); |
if (buffer) DisposeHandle (buffer); |
if (description) DisposeHandle ((Handle)description); |
} |
© 2005, 2006 Apple Computer, Inc. All Rights Reserved. (Last updated: 2006-01-10)