Getting images in and out from Quartz Composer compositions

Quartz Composer compositions can have inputs and outputs to pass and retrieve values to and from the composition. Those inputs are typed and accept numerical values, text strings or even images (refer to the Quartz Composer Programming Guide for more information on compositions inputs and outputs and regarding how to create them). This technote focuses on how to pass and retrieve efficiently images to a composition.





Passing images to a composition

If the composition has Image inputs, you can pass images to them using the - (BOOL) setValue:(id)value forInputKey:(NSString*)key method from the QCView or QCRenderer classes. You must pass a Quartz Composer compatible type of image. This method will return YES if it can successfully set the image on the input. Quartz Composer will retain the image you pass so you can safely release it afterwards.

Compatible types of images are:

IMPORTANT: Always try to pass images using the closest type from their original sources as this will improve performances e.g. don't convert a CGImageRef to a CIImage or NSImage to pass it to Quartz Composer. Instead, pass the original CGImageRef directly.

Images passed to Quartz Composer as NSImages or CGImageRefs will be resized to fit in the maximum texture size on the video card (typically 2048x2048 or more). On the other hand, CIImages and CVImageBuffers bigger than the maximum texture size are not resized and may fail to display properly.

Depending on where the images to pass to Quartz Composer come from, you should use the following types:

NSImage

Use NSImage to pass images obtained from AppKit. Note that calling -valueForInputKey:(NSString*)key on an Image input after a call to -setValue:(id)value forInputKey:(NSString*)key passing a NSImage, will return a different NSImage instance. The reason is that Quartz Composer performs several operations on the original NSImage like color correction or downsampling when necessary.

WARNING: There is an issue in Quartz Composer when passing a NSImage, that can cause its color profile information to be lost, producing a slight hue shift in the image's pixels. This is especially visible when passing / retrieving an image as an NSImage several times to / from Quartz Composer. As a consequence, NSImages should not be used to pass images to Quartz Composer when color fidelity matters (use CGImageRef instead).

Back to Top 

CGImageRef

Use CGImageRef when loading image files from disk through ImageIO (see /System/Library/Frameworks/ApplicationServices.framework/Frameworks/ImageIO.framework), obtaining images directly from the Quartz 2D API, or to pass raw pixels data that needs color matching.

Listing 1: Creating a CGImageRef from binary data or a file using ImageIO.

CGImageRef CreateCGImageFromData(NSData* data)
{
    CGImageRef        imageRef = NULL;
    CGImageSourceRef  sourceRef;

    sourceRef = CGImageSourceCreateWithData((CFDataRef)data, NULL);
    if(sourceRef) {
        imageRef = CGImageSourceCreateImageAtIndex(sourceRef, 0, NULL);
        CFRelease(sourceRef);
    }

    return imageRef;
}

CGImageRef CreateCGImageFromFile(NSString* path)
{
    NSURL*            url = [NSURL fileURLWithPath:path];
    CGImageRef        imageRef = NULL;
    CGImageSourceRef  sourceRef;

    sourceRef = CGImageSourceCreateWithURL((CFURLRef)url, NULL);
    if(sourceRef) {
        imageRef = CGImageSourceCreateImageAtIndex(sourceRef, 0, NULL);
        CFRelease(sourceRef);
    }

    return imageRef;
}

Back to Top 

CIImage

Use CIImage for images resulting from Core Image filters.

IMPORTANT: Do not wrap CGImageRefs, CVImageBuffers, raw pixels data or OpenGL textures into CIImages to pass them to Quartz Composer as this will lead to a performance hit.

Back to Top 

CVImageBuffer

Use CVImageBuffer for images coming from QuickTime movies, OpenGL rendering or raw pixels data. They are the optimal way to pass images from the following sources:

  • QuickTime movies: create a visual context to render the movie into using the QuickTime 7 APIs (see QuickTime Reference Update). Then pass the CVImageBuffers obtained by QTVisualContextCopyImageForTime() to the appropriate composition's Image input and release them. For best performances, it's highly recommended to use a QuickTime OpenGL texture context that uses the same OpenGL context and pixel format as a the ones used to render the composition (assuming a QCRenderer is used, as the QCView does not provide this information).

  • OpenGL rendering: create a CVOpenGLBufferRef with CVOpenGLBufferCreate(), attach it to an OpenGL context with CVOpenGLBufferAttach(), perform some OpenGL rendering in that context, then call glFlush() to terminate rendering, and eventually pass the CVOpenGLBufferRef to the appropriate composition's Image input and release it (see Core Video Reference for more information on the CVOpenGLBufferRef API).

  • Raw pixel data: create a CVPixelBufferRef with CVPixelBufferCreate() using a pixel format like k32ARGBPixelFormat or k8IndexedGrayPixelFormat for example (planar pixel buffers are not supported by Quartz Composer). Then write to the pixels directly using CVPixelBufferLockBaseAddress(), CVPixelBufferGetBaseAddress() and CVPixelBufferGetBytesPerRow() (don't forget to call CVPixelBufferUnlockBaseAddress() when done), and eventually pass the CVPixelBufferRef to the appropriate composition's Image input and release it (see Core Video Reference for more information on the CVPixelBufferRef API).

Creating and destroying CVOpenGLBufferRefs or CVPixelBufferRef can be a fairly expensive operation. If those buffers have constant dimensions, it's more efficient to recycle them and use appropriate buffer pools like CVOpenGLBufferPoolRef and CVPixelBufferPoolRef (see Core Video Reference for more information on the CVOpenGLBufferPoolRef and CVPixelBufferPoolRef APIs).

Note: Quartz Composer considers the contents of CVImageBuffers immutable, therefor you should not attempt to modify them after passing them to Quartz Composer.

You can find sample code on how to use CVImageBuffers with Quartz Composer at /Developer/Examples/Quartz Composer/Performer.

Back to Top 

Retrieving images from a composition

If the composition has Image outputs, you can retrieve the images on them using the - (id) valueForOutputKey:(NSString*)key method from the QCView or QCRenderer classes. This method will return the image as a NSImage or nil if there is none (or in case of an internal error). Note that -valueForOutputKey: returns a new NSImage instance each time the image on the output has changed, so you can safely retain them, and not copy them, if you need to keep around previous versions of those images.

Note: The NSImages produced by Quartz Composer are in RGB format (even the ones resulting from gray-level images), and use the display colorspace.

You can pass that NSImage to a NSImageView to display it in the user interface. The NSImage can also be saved as a LZW-compressed TIFF file using the following code snippet:

Listing 2: Saving an NSImage as a LZW-compressed TIFF file.

NSImage*          image;
BOOL            success;

image = [myRenderer valueForOutputKey:@"foo"];
if(image) {
    success = [[image TIFFRepresentationUsingCompression:NSTIFFCompressionLZW factor:1.0]
        writeToFile:@"/Users/foo/bar.tiff" atomically:YES];
    if(success == NO) {
        /*
            Handle error
        */
    }
}

If you need to access the raw pixels contents of the NSImage, you will need to convert it to a NSBitmapImageRep using a technique like the one in this code snippet:

Listing 3: Accessing the raw pixels of an NSImage.

NSImage*          image;
NSSize            imageSize;
NSBitmapImageRep*  bitmapImage;
NSRect            imageRect;

image = [myRenderer valueForOutputKey:@"foo"];
if(image) {
    imageSize = [image size];
    imageRect = NSMakeRect(0, 0, imageSize.width, imageSize.height);
    [image lockFocus];
    bitmapImage = [[NSBitmapImageRep alloc] initWithFocusedViewRect:imageRect];
    [image unlockFocus];

    if(bitmapImage) {
        /*
            Do something with the raw pixels contents
            using [bitmapImage bitmapData] and [bitmapImage bytesPerRow]
        */
        [bitmapImage release];
    }
}

Back to Top 

Document Revision History

DateNotes
2005-10-04Fixed a typo in the NSImage to NSBitmapImageRep conversion code.
2005-06-24Describes how to efficiently pass images into and out of Quartz Composer

Posted: 2005-10-04


Did this document help you?
Yes: Tell us what works for you.
It’s good, but: Report typos, inaccuracies, and so forth.
It wasn’t helpful: Tell us what would have helped.