ADC Home > Reference Library > Technical Q&As > Legacy Documents > Hardware & Drivers >

Legacy Documentclose button

Important: This document is part of the Legacy section of the ADC Reference Library. This information should not be used for new development.

Current information on this Reference Library topic can be found here:

Secondary Interrupts on the Page Fault Path


Q: I'd like to use secondary interrupts in my SCSI Interface Module (SIM) but doing so causes strange deadlocks. Is it legal to use secondary interrupts in a SIM?

A: In general, using secondary interrupts on the page fault path is not legal. There are, however, circumstances under which it works. As with most of my Q&A, this takes some explanation.


Note:
The page fault path is the set system of system software and device drivers that are required to resolve a virtual memory page fault. The page fault path is an interesting concept in operating system design, because most operating systems require that all entities on the page fault path not cause a page fault. Double page faults will commonly cause an operating system to "panic".

On Mac OS, double page faults are always fatal. Mac OS prevents double page faults by disabling "user code" while page fault path entities are executing. In addition, all entities on the page fault path must be capable of performing an I/O request with interrupts disabled.

For more information about the virtual memory implementation on Mac OS, see Technote 1094 Virtual Memory Application Compatibility.


IMPORTANT:
While this Q&A is couched in terms of SCSI Manager SIMs, the same logic applies to any entity on the page fault path. Third-party developer opportunities on the page fault path currently include SIMs, AIMs (ATA Interface Modules), and disk drivers. This list will grow as new technologies, like FireWire, evolve to support disk devices.

The two common reasons for using secondary interrupts in a device driver are:

  1. Reducing interrupt latency -- By deferring complex work to a secondary interrupt, you can reduce the amount of time your driver spends at hardware interrupt time, and thereby reduce the overall interrupt latency of the system.

  2. Concurrency guarantee -- Secondary interrupts are guaranteed to be serialized; only one secondary interrupt handler can be running at any point in time. This is a very useful concurrency control mechanism. If you always access global data structures from your secondary interrupt handler, you can be assured that only one thread of execution is reading or writing those global data structures at a time.

However, secondary interrupts can cause problems for devices on the page fault path. Imagine the following scenario:

  1. Your SIM takes a hardware interrupt, schedules a secondary interrupt (SIH A), and then returns from its hardware interrupt handler. For your SIM to make forward progress, its secondary interrupt handler must execute to completion.

  2. The interrupt systems notices that it is returning to interrupt level 0 and begins processing secondary interrupts. It enables interrupts and starts running secondary interrupt handlers.

  3. While SIH A is running, a Time Manager interrupt occurs. The timer task executes, schedules a deferred task, and then returns.

  4. The interrupt system notices that it is returning from the Time Manager interrupt to interrupt level 0. There are deferred tasks to run, so it enables interrupts and starts running deferred tasks.

  5. One of the deferred tasks causes a page fault.

  6. The Virtual Memory Manager intercepts the page fault, and makes a synchronous request to the disk device driver to read the page contents from the backing store.

  7. The disk device driver calls SCSI Manager to execute a SCSI command, which in turn calls your SIM. Your SIM starts the command and returns, expecting a hardware interrupt to complete the command.

  8. Your SIM's hardware interrupt fires and schedules a secondary interrupt (SIH B) to complete the command.

The system is now deadlocked. The SIH B cannot run because secondary interrupt handlers must be single-threaded and SIH A is already running. But the SIH A cannot run because it has been interrupted by a page fault, which is sitting in a synchronous wait loop waiting for SIH B to run.

The solution is to not use secondary interrupt handlers in your SIM, or any software entity on the page fault path.

Newer versions of Mac OS provide another workaround for this problem. On such systems, the OS detects these potential deadlock cases (waiting for a synchronous device request with an interrupt mask of 0 and with secondary interrupt handlers queued) and calls your SIMInterruptPoll routine. Your interrupt poll routine can detect that it has queued a secondary interrupt that has not yet run, and perform the appropriate action directly. Typically this involves calling your secondary interrupt handler routine directly.

This approach has a number of important caveats:

  1. It does not work on older systems. You can check for the presence of secondary interrupt polling using the code in Listing 1. You should find it available on Mac OS 8.5 and later.

  2. Secondary interrupt polling may undermine the concurrency guarantees provided by secondary interrupts. In the above example, look carefully at what happens during the deadlock recovery process. The system calls your interrupt poll routine and your interrupt poll routine calls SIH B directly. At this point, you have two threads of execution inside secondary interrupt handlers, SIH A and SIH B. This could cause problems if you designed your SIM to rely on the concurrency guarantees provided by secondary interrupts. If you want to use secondary interrupts in your SIM, you must take this potential reentrancy into account.

  3. Secondary interrupt polling is only useful for devices with an interrupt poll routine, such as SIMs and AIMs. Other native drivers, such as disk drivers, do not have an interrupt poll routine and cannot use secondary interrupts if they are on the page fault path.

Because of these caveats, it may just be easier to do everything in your SIMs hardware interrupt handler. SIMs are mostly I/O bound, so the interrupt latency increase of doing everything in your hardware interrupt handler is minimal.

You can detect whether secondary interrupt polling is implemented using Gestalt, as shown in Listing 1.

Listing 1. Determining whether secondary interrupt polling is available


static Boolean HasSecondaryInterruptPolling(void)
{
    UInt32 response;

    return (Gestalt(gestaltSCSI, (SInt32 *) &response) == noErr) &&
            ((response & (1 << gestaltSCSIPollSIH)) != 0);
}

In summary:

  • Prior to Mac OS 8.5, you must never use secondary interrupts on the page fault path.

  • With Mac OS 8.5 and beyond, SIMs and AIMs may use secondary interrupts, but they must take special action in their interrupt poll routine.

  • Despite the difficulties using secondary interrupts in a SIM, DTS recommends that SIM developers seriously consider adopting this technique. The primary advantage, lower system interrupt latency, is a significant benefit to real-time applications such as QuickTime.

For more information about this mind-manglingly complex part of the Mac OS I/O subsystem, see:

[Dec 21 1998]


Did this document help you?
Yes: Tell us what works for you.
It’s good, but: Report typos, inaccuracies, and so forth.
It wasn’t helpful: Tell us what would have helped.