From: narad@nonsequitur.Eng.Sun.COM (Chuck Narad)
Newsgroups: comp.arch
Subject: Re: MP interrupt handling
Date: 25 Jul 1992 00:21:45 GMT
Organization: Sun Microsystems
Distribution: world
Reply-To: narad@nonsequitur.Eng.Sun.COM
NNTP-Posting-Host: nonsequitur

In article 92Jul24155432@hunter.cs.strath.ac.uk, jim@cs.strath.ac.uk (Jim Reid) writes:
>In article <l6tucvINNbqt@appserv.Eng.Sun.COM> limes@ouroborous.Eng.Sun.COM (Greg Limes) writes:
>
>   | So, how many processors can take and service interrupts?  
>
>   Two answers, as usual ;-)
>
>   First answer: All CPUs can accept and service interrupts,
>   without regard to which CPU initiated the action that
>   eventually triggered the interrupt. The same chip that handles
>   MBus arbitration handles distribution of the interrupts; an
>   internal register tells it which CPU gets them.
>
>   Second answer: Only one CPU at a time can receive interrupt
>   requests from devices in the system, and the software design of
>   the 4.1.2mp kernel requires that device interrupt service
>   routines execute within the single lock, so only one CPU at a
>   time will be inside interrupt service routines.
>
>So does this mean that this "symmetrical multi-processor SunOS" isn't
>really what it claims to be? If only one CPU at a time can be executing 
>interrupt service code, what we've got is an OS with one dedicated
>(but not fixed) I/O processor.

The fact that only one processor at a time can recieve interrupts from
devices does not imply (as you believe) that only one processor can
service interrupts concurrently.

Interrupts are divided into three classes; directed, undirected, and
broadcast.  Directed interrupts include a mechanism where any processor
can send any other processor interrupts at each level, which can be
used (along with mailboxes in memory) to dispatch cross-calls.
Undirected interrupts include all interrupts from I/O devices; at any
time, all undirected interrupts are steered to a selected processor.

In order to make a system like this truly symmetric (including service
of I/O interrupts) there are several choices; one is that a processor
can take an interrupt, relinquish the "interrupt servicer" status to
another processor, then proceed with the service routine.  Another is
to have the current holder of that status run a very lightweight
dispatch routine, where it determines the source of the interrupt,
selects the best candidate servicer by some algorithm (e.g. least busy,
interrupt thread affinity to warmer cache, etc) and dispatches the
interrupt to a selected servicer via a cross-call interrupt.  Call that
lightweight dispatch routine "microcode" if you prefer; I like to call
it that just to irritate our OS programmers :-)

So the fact that the interrupts are steered to a single processor at
any time does not preclude the entire *system* (including "microcode")
from being fully symmetric in the hardware and software.

This approach was found to be much more flexible, as well as being far
simpler to implement/ debug/prove than any hardware dispatch mechanism,
and does not lock the OS to dealing with any particular algorithm that
was carved in silicon.

That's about all I can say without stepping over the lines of what has
been stated publically.

chuck/

Disclaimer:  speaking for myself; the above does not neccessarily
reflect the opinions of my employer.