# Basic Physics of Nuclear Medicine/PACS and Advanced Image Processing

## Picture Archival & Communication Systems (PACS)

With the phenomenal development of computer technology in recent times
has come the possibility of storing and communicating medical images in
digital format. PACS
systems are
generally based on a dedicated computer which can access data stored in
the digital image processors of different imaging modalities and
transfer this data at high speeds to remote viewing consoles, to
archival storage media and to other computer systems either within the
hospital or at external locations -- see the figure below:

!Basic elements of a generic PACS solution for a teaching hospital.
Images from the modalities are sent to short term RAID storage and
archived simultaneously. Archived images may also be retrieved as
required. Access to images is enabled through the high speed local area
network which distributes the images
to clinical and diagnostic workstations, to a Web server and to a
teleradiology server for dispatch to remote sites via a wide area
network (WAN).\
_Glossary_ -- **HIS**: Hospital Information
System; **RIS**: Radiology Information System; **LAN**: Local Area
Network; **RAID**: Redundant Array of Independent
Disks.. Glossary – HIS: Hospital Information System; RIS: Radiology Information System; LAN: Local Area Network; RAID: Redundant Array of Independent Disks.")

Successful implementation of PACS is critically dependent on several
factors which include image format standardization,
HIS and
RIS integration, image
display devices, image transfer rates and storage capacity. These
features are discussed below.

Standardization of the manner in which the image data is interchanged
between different medical imaging devices. The Digital Imaging &
Communications in Medicine (DICOM) standard has
been embraced by most equipment manufacturers to facilitate this.
Another file format which has been developed specifically for nuclear
medicine images is called
Interfile. Besides specifying
the format of digital image data, these information interchange formats
also cover patient and examination details which are embedded within the
image file. This latter feature is particularly important in medical
imaging so that patient studies do not get mixed up, for example, and
can be regarded as generating a *birth certificate* for each acquired
image, or set of images. An example of such DICOM header information is
shown in the following four figures (the header, which is typically one
continuous document, has been broken into four parts here to assist with
our discussion):

!The first part of a DICOM file
header.

Notice that the data provide patient details as well as the image type,
the date and time of the study, the modality, the scanner manufacturer
and image processing workstation used. The second part of this header is
shown below:

!The second part of a DICOM file
header.

Notice that this data covers the slice thickness and spacing used in
this SPECT study, image sampling and quantization information, the
number of images and the photon energy window used by the scanner. The
third part of this header is shown next:

!The third part of a DICOM file
header.

Notice that this data covers details of the scanner movement used to
acquire the study. The fourth and final part of this header is shown
below:

!The fourth and final part of a DICOM file
header.

Notice that this final part details the patient and scanner orientation
as well as the actual image data.

Other image formats are in common use in medicine, for purposes other
than primary diagnosis. These formats can be useful for teaching,
multimedia and publication purposes. Examples of these formats are
included in the following table.

+--------+----------------------+----------------------+
| Format | ```{=html}           | ```{=html}           |
|        |
 |
 |
|        | ```                  | ```                  |
|        | **Compression**      | **Comment**          |
|        |                      |                      |
|        | ```{=html}           | ```{=html}           |
|        |
               |
               |
|        | ```                  | ```                  |
+========+======================+======================+
| JPEG   | ```{=html}           | ```{=html}           |
|        |
 |
 |
|        | ```                  | ```                  |
|        | Lossy                | Small file sizes     |
|        |                      |                      |
|        | ```{=html}           | ```{=html}           |
|        |
               |
               |
|        | ```                  | ```                  |
+--------+----------------------+----------------------+
| PNG    | ```{=html}           | ```{=html}           |
|        |
 |
 |
|        | ```                  | ```                  |
|        | Lossless             | Header information   |
|        |                      |                      |
|        | ```{=html}           | ```{=html}           |
|        |
               |
               |
|        | ```                  | ```                  |
+--------+----------------------+----------------------+
| TIFF   | ```{=html}           | ```{=html}           |
|        |
 |
 |
|        | ```                  | ```                  |
|        | Lossless/Lossy       | Large file sizes     |
|        |                      |                      |
|        | ```{=html}           | ```{=html}           |
|        |
               |
               |
|        | ```                  | ```                  |
+--------+----------------------+----------------------+
| GIF    | ```{=html}           | ```{=html}           |
|        |
 |
 |
|        | ```                  | ```                  |
|        | Lossy                | Graphical data       |
|        |                      |                      |
|        | ```{=html}           | ```{=html}           |
|        |
               |
               |
|        | ```                  | ```                  |
+--------+----------------------+----------------------+

The Joint Photographic Expert Group (JPEG) format
is widely used for image transfer using the World Wide
Web because the image data can be reduced
in size using image compression
techniques and hence can be transferred relatively quickly. The
compression technique used by this format typically results in the loss
of image data which cannot be exactly recovered. Hence the reference to
**Lossy** compression in the Table. The format, as you might appreciate,
is not used for primary diagnosis but is nevertheless useful for
teaching and related applications.

The Portable Network Graphics (PNG) format is the
most recent of these formats and has advantages in terms of **Lossless**
compression, platform independent image display and compression features
and the ability to embed patient and study identification information.

The Tagged Image File Format (TIFF) is widely used
in the publication industry and provides the capability for both
lossless and lossy compression. Its lossless compression however results
in large image file sizes.

Finally the Graphics Interchange Format (GIF) is
widely used for transferring graphical images (e.g. graphs, diagrams,
flowcharts etc.) via the World Wide Web, and can also be used for
animated graphics.

High quality display devices are generally needed for medical images.
Cathode ray tube (CRT) and liquid crystal
display (LCD) monitors are widely used with
their visual characteristics matched to the display task. Monitors for
digital mammograms and chest radiographs, for instance, require a
relatively high luminance and spatial resolution, while those used for
nuclear medicine and ultrasound images don\'t have such stringent
specifications but do require colour and dynamic imaging functionality
-- and those used for CT and MRI are the same, except maybe for colour
processing, but also require a high contrast resolution. In addition,
monitors used for clinical review generally have lower spatial
resolution requirements than those used for primary diagnosis.

The CRT technology has disadvantages which include a nonuniform
luminance and a high veiling glare. The
luminance of a CRT monitor is generally highest in the centre and falls
off, as does its spatial resolution, towards the periphery of the
screen. Veiling glare results from light reflections inside the tube and
can have a substantial negative influence on both spatial resolution and
contrast. LCD monitors, on the other hand, are characterized by
increased luminance, luminance uniformity, spatial resolution and
contrast, as well as a lower electrical power consumption and desktop
footprint. Visualization of displayed images is affected however when
the viewer is not directly in front of the screen, because of optical
polarization effects, but this is about the only disadvantage relative
to CRTs.

Luminance is an important feature because of its negative affect on
diagnostic accuracy and a number of medical display standards therefore
specify minimum values, e.g. the American College of
Radiology specifies a
minimum of 160 cd/m^2^. Viewboxes, which have been used traditionally in
medical imaging, have a considerably higher luminance than computer
display devices, whether they be CRTs or LCDs. Luminance values for
various display devices are compared in the following table:

+-------------+-------------+-------------+-------------+-------------+
| Display     | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
| Device      | <div clas   | <div clas   |  | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | **Size      | *           | **Luminance | **Contrast  |
|             | (cm)**      | *Resolution | (cd/m^2^)** | Ratio**     |
|             |             | (Pixels)**  |             |             |
|             | ```{=html}  |             | ```{=html}  | ```{=html}  |
|             | </div>      | ```{=html}  |
      |
      |
|             | ```         | </div>      | ```         | ```         |
|             |             | ```         |             |             |
+=============+=============+=============+=============+=============+
| Mammography | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
| Viewbox     | <div clas   | <div clas   |  | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 57          | \-          | 3,500 --    | \-          |
|             |             |             | 5,000       |             |
|             | ```{=html}  | ```{=html}  |             | ```{=html}  |
|             | </div>      | </div>      | ```{=html}  |
      |
|             | ```         | ```         |
      | ```         |
|             |             |             | ```         |             |
+-------------+-------------+-------------+-------------+-------------+
| C           | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
| onventional | <div clas   | <div clas   |  | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 57          | \-          | 1,000 --    | \-          |
|             |             |             | 3,000       |             |
|             | ```{=html}  | ```{=html}  |             | ```{=html}  |
|             | </div>      | </div>      | ```{=html}  |
      |
|             | ```         | ```         |
      | ```         |
|             |             |             | ```         |             |
+-------------+-------------+-------------+-------------+-------------+
| Greyscale   | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
| -- 3        | <div clas   | <div clas   |  | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 53          | 2048 x 1536 | 600         | 600:1       |
|             |             |             |             |             |
|             | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             |
      |
      |
      |
      |
|             | ```         | ```         | ```         | ```         |
+-------------+-------------+-------------+-------------+-------------+
| Greyscale   | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
| -- 2        | <div clas   | <div clas   |  | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 48          | 1200 x 1600 | 800         | 700:1       |
|             |             |             |             |             |
|             | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             |
      |
      |
      |
      |
|             | ```         | ```         | ```         | ```         |
+-------------+-------------+-------------+-------------+-------------+
| Colour      | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             | <div clas   | <div clas   |  | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 76          | 1280 x 768  | 450         | 350:1       |
|             |             |             |             |             |
|             | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             |
      |
      |
      |
      |
|             | ```         | ```         | ```         | ```         |
+-------------+-------------+-------------+-------------+-------------+
| Colour      | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             | <div clas   | <div clas   |  | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 51          | 1200 x 1600 | 350         | 350:1       |
|             |             |             |             |             |
|             | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             |
      |
      |
      |
      |
|             | ```         | ```         | ```         | ```         |
+-------------+-------------+-------------+-------------+-------------+
| Colour      | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             | <div clas   | <div clas   |  | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 46          | 1280 x 1024 | 240         | 350:1       |
|             |             |             |             |             |
|             | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             |
      |
      |
      |
      |
|             | ```         | ```         | ```         | ```         |
+-------------+-------------+-------------+-------------+-------------+

Notice in the table that viewboxes have a luminance five times or more
greater than LCD monitors. As a result,
windowing
techniques are widely used to compensate. Notice also in the table that
greyscale monitors tend to have a higher luminance than their colour
counterparts.

In addition, the image display workstation interface must be
*user-friendly*. That is, interfaces which control the display,
manipulation, analysis, storage and distribution of images need to be
intuitive, efficient and specific within a medical context.

+-----------------+----------------------+------------------------+
| Connection      | ```{=html}           | ```{=html}             |
|                 |
 |
   |
|                 | ```                  | ```                    |
|                 | **Speed**            | **Time to Transfer\    |
|                 |                      | a 5 Mbyte Image File** |
|                 | ```{=html}           |                        |
|                 | </div>               | ```{=html}             |
|                 | ```                  | </div>                 |
|                 |                      | ```                    |
+=================+======================+========================+
| Telephone Modem | ```{=html}           | ```{=html}             |
|                 |
 |
   |
|                 | ```                  | ```                    |
|                 | 56 kbit/s            | about 12 minutes       |
|                 |                      |                        |
|                 | ```{=html}           | ```{=html}             |
|                 |
               |
                 |
|                 | ```                  | ```                    |
+-----------------+----------------------+------------------------+
| ISDN            | ```{=html}           | ```{=html}             |
|                 |
 |
   |
|                 | ```                  | ```                    |
|                 | 128 kbit/s           | about 5 minutes        |
|                 |                      |                        |
|                 | ```{=html}           | ```{=html}             |
|                 |
               |
                 |
|                 | ```                  | ```                    |
+-----------------+----------------------+------------------------+
| DSL             | ```{=html}           | ```{=html}             |
|                 |
 |
   |
|                 | ```                  | ```                    |
|                 | 384 kbit/s           | 1.8 minutes            |
|                 |                      |                        |
|                 | ```{=html}           | ```{=html}             |
|                 |
               |
                 |
|                 | ```                  | ```                    |
+-----------------+----------------------+------------------------+
| Ethernet        | ```{=html}           | ```{=html}             |
|                 |
 |
   |
|                 | ```                  | ```                    |
|                 | 10 Mbit/s            | 4 seconds              |
|                 |                      |                        |
|                 | ```{=html}           | ```{=html}             |
|                 |
               |
                 |
|                 | ```                  | ```                    |
+-----------------+----------------------+------------------------+
| Fast Ethernet   | ```{=html}           | ```{=html}             |
|                 |
 |
   |
|                 | ```                  | ```                    |
|                 | 100 Mbit/s           | 0.4 seconds            |
|                 |                      |                        |
|                 | ```{=html}           | ```{=html}             |
|                 |
               |
                 |
|                 | ```                  | ```                    |
+-----------------+----------------------+------------------------+

Further, the efficient distribution of images throughout large hospital
campuses and associated clinics has been enhanced by the availability of
free web browsing software and has meant that distribution of images can
be achieved at a fraction of what the cost was prior to their
introduction.

Image transfer times should be short in any PACS system for obvious
reasons. Ideally an image should appear on the monitor within 2 seconds
of the request for the image. The increasing availability of high speed
networks are allowing this requirement to be met more readily. A
comparison of transfer speeds for a number of common network connections
are shown in the following table.

Finally, PACS environments should have access to relatively cheap
archival storage up to a few Tbytes (i.e. a few million Mbytes) of image
data and must provide retrieval of non-current image files in a
reasonable time -- say, less than a minute or two. Current solutions
include robotic digital tape archives and optical disk *Juke Boxes*.

## The Internet & The World Wide Web

The Internet is a global assemblage of computer
networks and an explosion in its use has occurred in recent years. Its
origins can be traced to activities associated with connecting US
university, military and research networks about 30 years ago by the
Advanced Research Projects
Agency and the
Inter-Networking Working Group,
to the US National Science
Foundation network in 1986,
through the release of public-domain software by groups at the European
Organization for Nuclear Research (CERN) in 1991
and at the University of Illinois National Center for Supercomputing
Applications
(NCSA)
in 1993 to the recent generation of substantial global interest.

The system facilitates the transfer of data, computer programs and
electronic mail and allows for discussion of
specialised topics among newsgroups as well as
other features such as telnet, internet relay
chat and file
sharing. Irrespective of the application
however, the system essentially allows for the convenient exchange of
information between computers on a global basis. This section gives a
very brief overview of the Internet from a general perspective of
electronic communication
protocols and the World Wide Web.

All forms of communication, be they based on electronic or other means,
are reliant on some form of protocol. A common protocol when someone
answers a telephone, for instance, is to say *hello*, to give a greeting
or to announce the location/telephone number of the receiver.
Communication between computers connected to the Internet uses a
protocol called the Transmission Control Protocol/Internet Protocol
(TCP/IP). This approach is an amalgam of two
protocols, the details of which are of no great relevance to our
discussion here, other than to note that they together provide an
electronic communication protocol which allows two computers to connect
via the Internet. One feature of TCP/IP to note however is that it can
be used to communicate between different types of computers, i.e. it is
platform independent. An
IBM-compatible personal computer can
therefore communicate with, for example, an Apple
computer or a UNIX
workstation. Related protocols which are
used when computers communicate over a telephone line are the Serial
Line Internet Protocol
(SLIP) and the
Point-to-Point Protocol (PPP).
Once communication has been established between two computers, an
additional protocol is needed to exchange computer files. A common
protocol used for this purpose is called the File Transfer Protocol
(FTP). The types of files which can be transferred
are typically computer programs as well as data such as word processed
documents, spreadsheets, database files and images.

A refinement to FTP is the Hypertext Transfer Protocol
(HTTP) which allows the transfer of documents which
contain data in the form of different media-types, and is widely used
for webpage display. Examples of media-types are text, images and sound.
Finally, two protocols relevant to electronic mail are the Post Office
Protocol (POP) and the Simple Mail
Transfer Protocol (SMTP), and a protocol in use for
newsgroups is the Network News Transfer Protocol
(NNTP).

!Illustration of a client-server connection on the
WWW.

The World Wide Web (WWW) is a conceptual
interpretation of the Internet when it is used to transfer documents
using the HTTP. These documents are generally called web pages and are
written using an editing language called Hypertext Markup Language
(HTML). This format provides control over, for
instance, the size and colour of text, the use of tables and, possibly
most importantly, the facility to link the document to documents which
exist elsewhere on the WWW. HTML also allows the insertion of various
media-types into documents. Images can be inserted, for instance, in
formats such as the Graphical Interchange Format (GIF), the Joint
Photographical Experts Group (JPEG) format or the Portable Network
Graphics (PNG) format, as discussed earlier, and image sequences can be
displayed using one of the Moving Picture Experts Group
(MPEG) formats. This latter functionality is useful
for instance, for display of dynamic nuclear medicine studies.

The transfer of HTML documents is illustrated in the following figure.
The user\'s computer (referred to as the
client) is equipped with software
(called a web browser) which allows it to
interpret HTML documents and to communicate via the Internet using
TCP/IP. The computer is also equipped with hardware which allows it to
physically connect to the Internet, for example:

:\* an ADSL modem for connecting via a
telephone line to an Internet Service
Provider, and

:\* Local Area Network (LAN) hardware
for connecting via an institutional network, such as an
Ethernet connection. At the other end of the
connection is a computer containing a document or set of documents of
interest to the user. This second computer is called a
server and contains the documents in
HTML format. An example of a software package which is used within the
server computer is Apache. The
sequence of events is typically as follows:

-   The user establishes contact between the client and server computers
    by directing the browser at the Uniform Resource Location
    (URL) of the server and requests a given HTML
    document. The direction is typically of the form:\
    \

```{=html}
<div class="center">
```
`<span style="color:red;">`{=html}`<big>`{=html}http:
//www.server.type.code/doc.html`</big>`{=html}`</span>`{=html}

```{=html}
</div>
```
      where:

  --------------- ---------------------------------------------------------------------------------------------------------------
  **<http://>**   the transfer protocol to be used
  **server**      the name of the server computer
  **type**        shorthand for the environment where that computer resides, e.g. com: company and edu: educational institution
  **code**        shorthand for the country where the server is located, e.g. au: Australia and i.e.: Ireland
  **doc**         the name of the document
  **.html**       identifies the format of the document
  --------------- ---------------------------------------------------------------------------------------------------------------

-   The server receives the request, gets the requested document from
    its storage device and sends the document using HTTP to the client.

```{=html}
<!-- -->
```
-   The client receives the document and the browser interprets the HTML
    so that text, links and media-types are presented appropriately on
    the display device.

Many WWW browsers also provide the ability for the user to download
files using the file transfer protocol (**FTP**), to send and receive
e-mail messages and to contribute to newsgroups. For example, the
process for downloading files using FTP is similar to that illustrated
in the last figure except that the user directs the browser at a URL of
the form:\

```{=html}
<div class="center">
```
`<span style="color:red;">`{=html}`<big>`{=html}ftp:
//ftp.server.type.code/doc.xxx`</big>`{=html}`</span>`{=html}

```{=html}
</div>
```
Sophisticated WWW browsers, such as Netscape
Navigator and Internet
Explorer, also provide the ability to
generate more than basic web-pages at the client\'s computer. One
implementation is the ability to interpret client-side
scripts. These are small programmes
which are downloaded as part of the HTML document and are executed using
the client computer\'s resources. By this means, for instance, the
script can read the date and time from the client computer or use its
arithmetic functions to make calculations, and embed this information in
downloaded webpages. Client-side scripts can be written using languages
such as JavaScript.

Another implementation is the ability to execute small applications
(called applets) which are downloaded with the
HTML document and run on the client computer. Such applets can be
generated using languages such as Java (not
to be confused with JavaScript!). Applets are well developed for
graphics applications, such as animations and scrolling banners. One
exciting development in this field is the ability to download image
processing software along with an image, so that the user can manipulate
the image without the need for a special image processing program.

Finally, a refinement to HTTP server software allows interaction from
the client so that information can be returned to the server for
executing specific tasks, such as searching a database, entering
information into a database or automatically correcting and giving
feedback on multiple choice exam questions. Additional software is
required for this server-side processing -- a common form of which uses
the Common Gateway Interface
(CGI) protocol. Small CGI
programs are generally referred to as scripts and are written in a
language such as Perl.

Online databases can also be accommodated within the client/server
model. For example, mySql is a package which has
been widely adopted for this purpose. Scripts for administering this
server software can be written in languages such as
PHP.

As might be anticipated, the field of electronic communication
introduces a vast range of additional concepts to those discussed above
-- and, like PACS, are just as unrelated to medical concepts! However,
further treatment of the subject is beyond our scope here since our
interest is confined in the main to the distribution of medical images.

## Spatial Registration of Images

Correlative imaging is widely used in medical diagnosis so that
information gleaned from a number of imaging modalities can be merged to
form a *bigger picture* about a patient\'s condition. However, the
actual merging of image data on a routine basis in hospitals and clinics
has had to await the development of relatively cheap and powerful
computers and such **image fusion** is now commonplace. It is generally
necessary to spatially align image data prior to the fusion process so
as to address differences in orientation, magnification and other
acquisition factors. This alignment process is generally referred to as
image registration.

Suppose we have two images to be registered -- a planar nuclear medicine
scan and a radiograph, for instance:

!A nuclear medicine bone scan of a patient\'s hands on the left and a
radiograph of their right hand on the right. The arrowed curves indicate
examples of correspondence between these images on the basis of our
knowledge of
anatomy.

The registration process generally assumes that a correspondence exists
between spatial locations in the two images so that a co-ordinate
transfer function (**CTF**) can be established which can be used to map
locations in one image to those of the other. In the above example, as
in many clinical situations, a number of compatibility issues need to be
addressed first. The obvious one arises from the different protocols
used for image acquisitions, i.e. a palmar view in the bone scan and a
posterior-anterior projection radiograph. We can handle this issue in
our example case by extracting the right hand data from the bone scan
and then flipping it around the horizontal axis. A related issue arises
when different digital resolutions are used -- in this case, the nuclear
medicine image was acquired using a 256 x 256 x 8-bit resolution, while
the radiograph was acquired using a 2920 x 2920 pixel matrix with a
12-bit contrast resolution. Since we may be interested in maintaining
the fine spatial resolution of the radiograph, we can magnify the bone
scan to the radiographic resolution using an interpolated
zoom
process. The outcome of these steps is illustrated below:

!A nuclear medicine bone scan of a patient\'s hand on the left
extracted from the previous image and mirrored horizontally, and the
radiograph on the right. The arrowed lines indicate examples of
correspondence between these
images.

When we assume minimal spatial distortion and identical positioning in
the projection radiograph, we can infer a spatially uniform CTF, i.e.
the transform applied to one pixel can also be applied to each and every
other pixel. Let\'s call the two images to be registered
`<span style="color:red;">`{=html}A`</span>`{=html} and
`<span style="color:red;">`{=html}B`</span>`{=html}, with image
`<span style="color:red;">`{=html}A`</span>`{=html} being the one to be
processed geometrically to correspond as exactly as possible with image
B. The CTF can then be represented by the following equations:

```{=html}
<div class="center">
```
`<span style="color:red;">`{=html}`<big>`{=html} u =
f(x,y)`</big>`{=html}`</span>`{=html}

and

`<span style="color:red;">`{=html}`<big>`{=html}v =
g(x,y)`</big>`{=html}`</span>`{=html}

```{=html}
</div>
```
where:

:\*`<span style="color:red;">`{=html}f`</span>`{=html} and
`<span style="color:red;">`{=html}g`</span>`{=html} define the transform
in the horizontal and vertical image dimensions;

:\*(`<span style="color:red;">`{=html}u,w`</span>`{=html}) are the
spatial co-ordinates in image
`<span style="color:red;">`{=html}A`</span>`{=html}; and

:\*(`<span style="color:red;">`{=html}x,y`</span>`{=html}) those in
image `<span style="color:red;">`{=html}B`</span>`{=html}. The first
computing step is to generate an initially empty image
`<span style="color:red;">`{=html}C`</span>`{=html} in the co-ordinate
frame (`<span style="color:red;">`{=html}x,y`</span>`{=html}) and fill
it with pixel values derived from applying the CTF to image
`<span style="color:red;">`{=html}A`</span>`{=html}. The resultant image
we can say is a version of image
`<span style="color:red;">`{=html}A`</span>`{=html} registered to image
`<span style="color:red;">`{=html}B`</span>`{=html}.

The question, of course, is how to determine the CTF. For situations
where simple geometric translations and rotations in the x- and
y-dimensions are required, the functions
`<span style="color:red;">`{=html}f`</span>`{=html} and
`<span style="color:red;">`{=html}g`</span>`{=html} can involve
relatively straight-forward bilinear interpolations. Such
transformations can also compensate for image magnification effects, and
the resultant processes are referred to as **rigid** transforms. When
spatial non-uniformities are encountered, **non-rigid** transforms can
be used to apply different magnification factors in both x- and
y-dimensions, as well as other geometric translations -- in which case
higher order interpolants can be applied.

Determination of the parameters of the CTF is obviously needed and there
are numerous methods we can use, for example:

:\* **Landmarks** -- where corresponding locations of prominent
anatomical features in both images can be identified and two sets of
co-ordinates can be derived on this basis to define the CTF. Note that
artificial landmarks can be created using external markers during image
acquisition, where, for instance, a set of markers which are both
radioactive and NMR sensitive can be fixed to the surface of the patient
during image acquisition for subsequent registration of SPECT and MRI
scans.

:\* **Function Minimization/Maximization** -- where an indicator of the
quality of registration is monitored as various geometric
transformations are applied to the image in an iterative fashion to
search for a set of parameters which minimize (or maximise) this
indicator. When both images are SPECT scans, acquired some months apart
for instance, a quality measure such as the sum of the absolute pixel
value differences can be applied. A more complex quality measure is
typically required when the two images are from different modalities. We
will concentrate our discussion below on this latter scenario.

The concept of the **joint histogram** is important in this regard.
Individual statistical histograms for our two images are shown below:

!Histograms of pixel values displayed in black (with the logarithms of
the frequencies displayed in grey so that lower frequencies can be
discerned) for the bone scan on the left and the radiograph on the
right. for the bone scan on the left and the radiograph on the right.")

The bone scan\'s histogram indicates that a substantial proportion of
the image is composed of dark pixels, with a much smaller number of
brighter pixels arising from the hot spot. The histogram for the
radiograph indicates that the image is composed of a substantial number
of bright pixels with a broad spread of grey shades. Note that the term
frequency used in this context refers to the **frequency of occurrence**
of pixel values, i.e. the number of times individual pixel values occur
in an image, and not to the terms **temporal frequency** and **spatial
frequency** which we\'ve encountered elsewhere in this
wikibook.

The joint histogram is a related concept to the individual image
histograms where the pixel values for pairs of pixels in the two images
are plotted against each other on the one graph. In other words, the
value of a pixel in one image is plotted against the value for the same
pixel location in the second image. A good way to introduce this concept
is by first of all comparing an image with a duplicate of itself, and
then with shifted versions of this duplicated image, as illustrated in
the following figure. We can use colour processing to assist in our
visual comparisons, where the bone scan (the **reference** image) can be
displayed using a red
CLUT,
for instance, and its shifted version using a green CLUT. As a result,
where the two images overlay each other, the layered image data is
displayed in shades of yellow, i.e. red plus green on the colour
spectrum, in overlapping regions.

The top row in our figure shows the situation when the two images are in
perfect alignment. Note the resultant yellow-only colour scale. The
joint histogram for this case consists of a diagonal straight line
because the values of all pixel pairs in the two images are identical.
The next row of our figure illustrates the effect of a horizontal shift
of eight pixels between the two images. Notice that regions of mismatch
are seen in shades of red and shades of green, as the case may be, and
overlapping regions in shades of yellow. The joint histogram now has the
appearance of a scatter plot because pixel
values in the two images no longer correspond spatially. A bright pixel
value in one image might now overlay with a dark region of the other,
for instance, and vice-versa.

!Joint histograms of an image against itself on the top row, and
against spatially shifted versions of itself on the other three rows.
See text for
details.

The third row in our figure illustrates the effect of a rotation of 15
degrees and the bottom row shows the combined effect of shifting and
rotation. The **basic lesson** to learn from this is that perfect
alignment between two identical images is indicated by a straight
diagonal line in the joint histogram and lack of alignment results in a
form of scatter plot. The **major lesson** to learn is that when two
images are out of alignment, statistical techniques can be applied which
endeavour to minimize the scatter in the joint histogram -- and hence
effect a spatial registration of the two images.

The situation is a bit more complicated when the two images to be
registered are acquired using different imaging modalities, e.g. a
nuclear medicine scan and a radiograph, because their individual
histograms are likely to be substantially different from each other --
as we\'ve seen in our earlier figure. Nevertheless, methods to minimize
the scatter plot in the resultant joint histogram can be used to
register the two images, as illustrated in the following figure:

!Joint histograms, on the top row for a non-registered bone scan and a
radiograph and, on the bottom row, following function
minimization.

Compatibility issues between the two images for this registration
process were addressed by first converting the radiograph from 2920 x
2920 x 12-bit resolution to 256 x 256 x 8-bits. The top row of the
figure illustrates the situation and the joint histogram shows
significant scattering in this data, as might be expected. The bottom
row illustrates the outcome of a Mutual
Information (**MI**) maximization
process where a solution was found that involved shifting, rotating and
magnifying the bone scan. While the joint histogram still depicts
substantial scattering, the MI index can be seen to increase from 0.17
in the non-registered situation to 0.63 following registration, and the
overlay image depicts the lesion in the bone scan registered to (or
colocalized with, as is sometimes said) the radiograph.

This type of image registration can ideally be generated automatically
using a computer. An iterative process is generally followed, where the
MI indicator is maximized initially for low resolution versions of the
two images and then progressively for increasingly higher resolutions.
Note however that reducing the resolution of the radiograph can
substantially effect its spatial quality and that, while registration
may be effected at this lower resolution, the resultant CTF can be used
with appropriate magnification to register the bone scan with the full
resolution radiograph -- as illustrated below:

!The bone scan registered with the radiograph, where a yellow CLUT has
been used for the radiograph and a red/white CLUT for the bone scan
data.{width="512"}

Other image similarity measures besides Mutual Information can also be
applied depending on the nature of the data in the two images. These
include:

:\* **Count Difference Minimization**: where the sum of the absolute
count differences between all pixels is minimized.

:\* **Shape Difference Minimization**: where segmentation techniques are
used to define the borders of the object to be registered in the two
images and a similarity measure based on the distance between these
borders is minimized.

:\* **Sign Change Maximization**: which maximizes the number of
positive/negative sign changes following subtraction of shifted versions
of one image relative to the reference image.

:\* **Image Variance Minimization**: which minimizes the statistical
variance between two images.

:\* **Square Root Minimization**: which minimizes the square root of the
absolute count differences between all pixels in the two images.

:\* **Gradient Matching**: which is based on comparing edges in the two
images.

The technique chosen for an individual pair of images is primarily
dependent on the nature of the image data. Some techniques may be
readily applied to **intra-patient**, **intra-modality** studies where,
for instance, bone scans of the same region of the same patient acquired
some time apart are compared -- in a follow-up study for example. Other
techniques may be required for **inter-modality** comparisons, e.g. a
nuclear medicine and an MRI scan, while others still for
**inter-patient**, **intra-modality** comparisons -- where a patient\'s
images are compared with those in an atlas of normal and diseased
conditions, for instance.

Finally, its of relevance to note that while spatial registration
techniques were introduced here using 2D images, the approach can also
be readily extended to the 3D case through comparisons of statistical
features in voxel data, e.g. registering SPECT and CT scans, or PET and
MRI scans, for example. Here registration of individual slices can be
applied sequentially through the volume of interest.

## Image Segmentation

Many forms of image analysis require the identification of structures
and objects within an image. Image segmentation is the process of
partitioning an image into distinct regions by grouping together pixels
which belong to the same object. Two general approaches have been
developed:

:\* **Threshold Definition**: where some property of an image is
compared to a fixed or variable threshold on a pixel-by-pixel basis. A
simple example is a grey-level threshold where a look-up
table
(**LUT**) of the form illustrated in the left hand panel of the figure
below is applied, and where the value of the threshold,
`<span style="color:red;">`{=html}`<big>`{=html}T`</big>`{=html}`</span>`{=html},
can be adjusted interactively.

!A threshold look-up table on the left and thresholding a bimodal
distribution on the
right.{width="512"}

:

    :   This is a useful technique when the image contains a single,
        well-defined object or group of objects of similar pixel value
        superimposed on a background with a substantially different
        pixel value. However, difficulties can arise with grey-level
        thresholding when objects are close together, e.g. cardiac
        chambers. Histogram analysis can be used as an alternative where
        pixel values are thresholded on the basis of their frequency of
        occurrence, as illustrated in the right hand panel of the figure
        above. Other alternatives include thresholding colours when a
        CLUT is applied, monitoring the time of arrival of a tracer or
        contrast medium in a region of an image and analysis of the
        variation in pixel values in the neighbourhood of a pixel within
        an object of interest.

:\* **Region Growth**: which exploits two characteristics of imaged
objects:

:::# pixels for an object tend to have similar pixel values, and

:::# pixels for the same object are
contiguous.

:

    :   A common technique is based on firstly defining a starting pixel
        in the object and then testing neighbouring pixels on the basis
        of a specific criterion for addition to a growing region. This
        criterion could be based on pixel value considerations, as in
        the following figure for instance, or on the anticipated size or
        shape of the object.

!**Region Growth**: The object of interest in a CT scan is identified
in the top left panel. A pixel value range for this object is specified
to highlight the resultant region in the top right panel, to identify
the borders of the region in the bottom left panel or to extract the
object into another image in the bottom right
panel.

:

    :   Note that this approach can be extended to grow regions in
        three-dimensions when the image data consists of a set of
        contiguous tomographic slices.

## Image Fusion

A method of combining image data to form a fusion display is required
once images have been registered. A simple approach is to add the two
images. Multiplying them is also an option. However, this form of image
fusion tends to obscure the underlying anatomy when hot spots exist in
the nuclear medicine data, as illustrated in the following figure:

!A bone scan is combined with a radiograph through addition on the
left, and multiplication on the
right.

A second approach is to interlace the two images so that a fusion
display is built up using alternate pixels, alternate pixel groups or
alternate lines of data from each image, as illustrated in the following
figure:

!A bone scan is combined with a radiograph using interlaced vertical
lines on the left, interlaced alternate pixels in the middle and tiling
on the
right.

However, such interlacing can highlight features associated with the
interlacing process itself, as illustrated by the vertical lines in the
left panel of the figure above. The so-called *checkerboard* display, as
illustrated in the right panel above, is a related technique where
alternate groups of pixels are displayed in a tiled fashion.

A third approach is to use an image compositing technique called Alpha
Blending, which uses a transparency
value, `<span style="color:red;">`{=html}α`</span>`{=html}, to determine
a proportional mixing of the two images, as illustrated in the following
figure:

!A bone scan is combined with a radiograph using alpha blending, with a
transparency value of 0.5 on the left, and 0.2 on the
right.

This type of approach is highly developed in the publishing industry and
a wide range of fusion options are available. A common one, which was
used for the images above, is to apply an equation of the form:

```{=html}
<div class="center">
```
`<span style="color:red;">`{=html}Fused Image = (α) Image1 + (1-α)
Image2`</span>`{=html}.

```{=html}
</div>
```
A transparency value of 0.5 was used to generate the image in the left
panel above, for instance, with the result that the underlying anatomy
can be discerned through the hot spot. A powerful feature of this
approach is that the fusion transparency can be varied interactively so
as to optimize the data presentation, for instance, or to confirm the
quality of the registration process (as illustrated in the right hand
panel above).

This blending approach can be extended to incorporate a variable opacity
function, where different transparency values are applied to different
parts of the grey scale of one image. Note that the terms
**transparency** and **opacity** have a reciprocal relationship in this
context. Example blends are shown in the following figure:

!Image blending using four different opacity functions -- **Linear**: a
linear opacity function; **High-Low-High**: a high opacity is used for
both small and large pixels values and a low opacity for intermediate
pixel values; **Low-High-Low**: low opacity used for both small and
large pixel values and high opacity for intermediate pixel values;
**Flat**: a constant opacity is
applied.

The **High-Low-High** opacity function, for instance, applies a high
level of opacity to pixel values at the top and bottom ends of the
contrast scale of one of the images and a low opacity to intermediate
pixel values. The result is improved visualization of fused data outside
of hot spot regions -- as illustrated in the top right panel above. The
**Low-High-Low** function has the opposite effect and generates the
capability to visualize the relevant anatomical detail with a
highlighted region around it -- as shown in the bottom left panel above.
Logarithmic, exponential and other opacity functions can also be
applied, depending on the nature of the two images to be fused.

The choice of opacity function and CLUT for use in image fusion
applications would appear to be an artistic, in contrast to a scientific
or infotech, endeavour in that the final result is often achieved using
aethestic impressions which convey the medical information of relevance.
Each study can require a reasonably unique combination of image
processing steps depending upon whether hot or cold spots exist in the
nuclear medicine study and upon the nature of the image data in the
anatomical study, be it from radiography, X-ray CT,
sonography or from one of the various forms
of magnetic resonance imaging. It is for this reason
that computers used for this type of application tend to feature highly
intuitive and tacit user interfaces with powerful visualization
capabilities. OsiriX, for example, runs only on
the Macintosh platform for this reason. A
second example is provided by a major medical equipment manufacturer
which gave the name **Leonardo** to a product line!

A final method of image fusion that we\'ll mention briefly is referred
to as **Selective Integration**, where segmentation techniques, for
instance, can be used to extract structures from one image so that they
can be pasted to relevant regions of a second, spatially-registered
image.

We\'ll conclude this chapter with an example illustrating the 3D
alignment and misalignment of the fused study shown in the following
image:

!Fused SPECT and CT multi-planar reconstructions with an interface
which can be used to control their relative 3D
alignment.


# Basic Physics of Nuclear Medicine/Three-Dimensional Visualization Techniques

## Introduction

This is a developing chapter of a wikibook entitled Basic Physics of
Nuclear Medicine.

Three-dimensional visualization techniques are used to integrate a
series of tomographic slices into a form which is often easier to
interpret than the slices on their own. An overview of the major
visualization techniques used in medical imaging is presented in this
chapter. We\'ll start with the integration of two-dimensional (2D) axial
images before considering various three-dimensional (3D) image
presentation methods.

The following set of images will be used to help us with our task. They
consist of a SPECT lung-ventilation study, illustrated by the animated
sequence on the left, a SPECT lung-perfusion study, in the middle, and a
CT pulmonary angiogram (CTPA) from the same patient on the right:

  -------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- -- -- -------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- -- -- ----------------------------------------------------------------------------------------------------------------------------------------------
  !Animated sequence from a patient\'s SPECT lung-ventilation study.{width="256"}            !Animated sequence from the patient\'s SPECT lung-perfusion study.{width="256"}            !Animated sequence from the patient\'s X-ray CTPA study.{width="256"}

  -------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- -- -- -------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- -- -- ----------------------------------------------------------------------------------------------------------------------------------------------

We\'ll use these image sets at various stages in this chapter so as to
provide examples which will help with demonstrating the essence of the
techniques we\'ll consider. Acquisition factors for the image sets
include 128 x 128 pixel slices each of thickness 4.8 mm for the SPECT
studies and 512 x 512 pixel contiguous slices of thickness 1.5 mm for
the axial images reconstructed from the CTPA helical scan. In addition,
the lung-perfusion scan was acquired immediately following the
ventilation scan, so that residual activity from the latter is present
in the former at a level of about 20%.

Its helpful, before proceeding, to consider the axial slices, be they
SPECT or CT, stacked one behind the other, as illustrated below:

!A stack of SPECT tomograms from a perfusion study of a patient\'s
lungs.

Notice that the figure illustrates each image as a thin slice when the
data represents in reality a slice wide enough to fill the gap between
it and the next slice, so that the image data can be considered to be a
matrix of volume elements -- called voxels for
short.

A convention applied in medical imaging is to display axial image stacks
with the axes oriented as shown in the following figure:

!Labels for the axial imaging
space.

So the left side of an axial scan represents the patient\'s right side
viewed from below, with their anterior surface at the top of the image.
A modification of a little ditty by Spike
Milligan might help you remember this
perspective:

  -----------------------------------------------------------------
  `<i>`{=html}I\'d love to be a fish,`</i>`{=html}
  `<i>`{=html}Who could swim beneath the ice,`</i>`{=html}
  `<i>`{=html}And look up at all the people skating,`</i>`{=html}
  `<i>`{=html}Oh wouldn\'t it be nice!`</i>`{=html}
  -----------------------------------------------------------------

You\'re likely to find that its a good idea to experience the real thing
when reading this chapter so as to help overcome the abstract nature of
many of the topics we\'ll encounter, as well as the limitations of using
a 2D medium (i.e. this webpage) to communicate 3D visualization
concepts. Numerous open source 3D programs and image libraries are
available via the World Wide Web. The
software used to generate images in this chapter, for instance, includes
OsiriX (MacOS X only),
Madena and
ImageJ (multi-platform).

## Axial Projection

The first technique we\'ll consider is a relatively simple one called
**Axial Projection**. It involves integrating a number of axial images
to display a composite which presents a three-dimensional impression of
that volume of image data. The technique is sometimes referred to as
**Thick Slab** or **Z-Projection**.

The figure below illustrates the outcome of a range of z-projection
methods, with a single slice shown in the bottom right hand corner for
reference purposes. The first image in the top left shows the result of
summing 16 slices, and the other two images on that row show the results
of computing the mean and median of these slices.

!A range of z-projections of 16 axial slices from the CT scan, with a
reference, single slice shown in the bottom right
corner.

The first two images in the second row show the result of what are
called a **Maximum Intensity Projection**
(MIP) and a **Minimum
Intensity Projection** (MinIP), respectively. A MIP evaluates each voxel
along each line of voxels through the volume to determine the maximum
voxel value and forms an image using the values so determined for each
line. A MinIP uses the minimum voxel values, as illustrated in the
following figure:

!A single line of voxels through eight axial slices illustrating the
determination of the maximum voxel value for MIPs on the left, and the
minimum value for MinIPs on the
right.

Volume rendered projections are shown in the first two images along the
bottom row of our collection of example axial projections. This image
compositing method involves applying an opacity function to the voxel
data as well as a recursive addition of the resulting data. An equation
of the form:

```{=html}
<div class="center">
```
`<span style="color:red;">`{=html}`<big>`{=html}C~n~ = A~n~ +
B~n~`</big>`{=html}`</span>`{=html}

```{=html}
</div>
```
where

:

    :   `<span style="color:red;">`{=html}A~n~ = (α).(Voxel Value of
        voxel, n)`</span>`{=html},
    :
    :   `<span style="color:red;">`{=html}B~n~ = (1-α).(Voxel Value of
        voxel, n-1)`</span>`{=html}, and
    :
    :   `<span style="color:red;">`{=html} α = opacity`</span>`{=html},
        in the range `<span style="color:red;">`{=html}0`</span>`{=html}
        (i.e. fully transparent) to
        `<span style="color:red;">`{=html}1`</span>`{=html} (i.e. fully
        opaque),
    :

is applied to each line of voxels as illustrated in the following
figure:

!Illustration of the volume rendering
technique.

The figure shows the line of voxels we\'ve used previously with an
opacity table in the top right corner. The opacity function shown is one
where a zero opacity is applied to voxel values below a threshold level,
a linear increase in opacity is applied to an intermediate range of
voxel values and maximum opacity applied to high voxel values. The
opacity table is somewhat like the look-up table used for greyscale
windowing which we\'ve described
earlier,
with the function applied to the opacity of voxel values instead of to
their grey levels. Note that more complex opacity tables to the one used
in our figure above can be also applied, e.g. logarithmic and
exponential functions.

The bottom half of the figure shows the steps involved in calculating
the volume rendered value of the composited voxel. Voxel values are
shown on the top row with opacity values, derived from a crude opacity
table, for each voxel shown on the second row. The third, fourth and
fifth rows detail the values of
`<span style="color:red;">`{=html}A`</span>`{=html},
`<span style="color:red;">`{=html}B`</span>`{=html} and
`<span style="color:red;">`{=html}C`</span>`{=html}, calculated using
our volume rendering equation above. The final voxel value is obtained
by summing the bottom row, and normalizing the result to, say, a 256
level grey scale.

The outcome of this form of processing is the generation of an image
which includes visual depth cues on the basis that similar voxel values
will be displayed with a similar transparency and those closest to the
reference slice having a stronger contribution than those from more
distal slices. Further, note that all voxel values in each line
contribute to the rendered image, in contrast to the limited number of
voxels that contribute to a MIP or a MinIP image. A 3D effect results
from volume rendering, as illustrated in the images above.

Notice that volume rendering can be applied from distal to proximal
slices, as illustrated in our figure, as well as in the opposite
direction, i.e. from proximal to distal slices. Hence the terms **Volume
Rendering Up** and **Volume Rendering Down** used in our set of nine
example images above.

The type of axial projection method appropriate to an individual patient
study is dependent on the anatomical and/or functional information of
relevance to the diagnostic process. Let\'s take the case of imaging
contrast-filled blood vessels, for example, in our nine example images
above. Note that a MIP can be used to give a visually-stunning
impression of the vessel bed in the patient\'s lungs. There\'s little
depth information in this projection, however, so that overlapping and
underlying vessels can obscure lesions that might be present in blood
vessels of interest. The application of this form of axial projection to
angiography is therefore limited to studies where vessel overlap isn\'t
an issue. The inclusion of voxel transparency and depth weighting in
volume rendered images addresses this limitation of MIP processing.

A final point to note is that this form of image projection can also be
applied to multi-planar reconstructions of axial slices, which we\'ll
consider next\.....

## Multi-Planar Reconstruction

In the simplest case, multi-planar reconstruction (MPR) involves
generating perspectives at right
angles to a stack of axial
slices so that coronal and sagittal images can be generated. We\'ll
start this section by describing these orthogonal projections before
considering their synchronized composite display. We\'ll also describe
three variants on this theme: oblique reconstruction, curved
reconstruction and integrated three-dimensional presentation.

-   `<big>`{=html}Coronal Reconstruction`</big>`{=html}

:

    :   Here the image stack is rotated so that the z-axis becomes the
        vertical and a stack of images is reconstructed using parallel
        planes of voxels arranged from the patient\'s anterior to
        posterior surface, as illustrated in the following figure:

!Labels for the coronal image
space.

:

    :   Example coronal reconstructions from our image sets are shown
        below:

  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- -- -- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  !Animated sequence of reconstructed coronal slices from a SPECT lung-perfusion study.            !Animated sequence of reconstructed coronal slices from a CT pulmonary angiography (CTPA) study. study.")

  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- -- -- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

:

    :   Here the reconstructed slices are typically displayed from the
        patient\'s anterior to their posterior surface with the
        patient\'s head towards the top of the slices and their right
        hand side on the left of the slices.

-   `<big>`{=html}Sagittal Reconstruction`</big>`{=html}

:

    :   Sagittal reconstructions are possible through additional
        rotations of the image stack so that a patient\'s left-to-right
        slice sequence can be generated as illustrated in the following
        figure:

!Labels for the sagittal image
space.

:

    :   Example sagittal reconstructions from our image sets are shown
        below:

  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- -- -- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  !Animated sequence of reconstructed (left-to-right) sagittal slices from a SPECT lung-perfusion study. sagittal slices from a SPECT lung-perfusion study.")            !Animated sequence of reconstructed (left-to-right) sagittal slices from a CT pulmonary angiography (CTPA) study. sagittal slices from a CT pulmonary angiography (CTPA) study.")

  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -- -- -- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

:

    :   Here the reconstructed slices are typically displayed from the
        patient\'s left to right side, with their head towards the top
        and their anterior surface towards the left of the slices. Note
        that a right-to-left sagittal stack can also be generated using
        additional geometric transformation of the data.

-   `<big>`{=html}Composite MPR Display`</big>`{=html}

:

    :   Coronal and sagittal reconstructions are referred to as
        Orthogonal MPRs because the perspectives generated are from
        planes of image data which are at right angles to each other.
        Composite MPR displays can be generated so that linked cursors
        or crosshairs can be used to locate a point of interest from all
        three perspectives, as illustrated in these images:

  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  !A composite orthogonal MPR display with linked cursors on the axial and sagittal images.   !A composite orthogonal MPR display with linked crosshairs on all three images.

  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

:

    :   This form of image presentation is sometimes referred to as a
        **TCS** display -- implying the viewing of
        _T_ransaxial,
        _C_oronal and
        _S_agittal slices. It can be combined
        with the slice projection methods we discussed earlier, as
        illustrated in the two sets of images below, where the blue
        lines highlight the limits of the coronal projections:

!Axial and sagittal reconstructions from the SPECT lung-perfusion study
with various coronal
projections.

!Axial and sagittal reconstructions from the CT study with a coronal
MIP.

-   `<big>`{=html}Oblique Reconstruction`</big>`{=html}

:

    :   Oblique MPRs are possible by defining angled planes through the
        voxel data , as illustrated in the following figure:

!CT MPR incorporating an oblique
MIP.

:

    :   Here the plane can be defined in, say, the axial images (red
        line, top left) and a maximum intensity projection (the limits
        used are highlighted by the blue lines), for example, can be
        displayed for the reconstructed plane (right). This technique is
        useful when attempting to generate perspectives in cases where
        the visualization of three-dimensional structures is complicated
        by overlapping anatomical detail.

-   `<big>`{=html}Curved Reconstruction`</big>`{=html}

:

    :   Curved MPRs can be used for the reconstruction of more complex
        perspectives, as illustrated in the next figure:

!An axial slice from the CT scan on the left with a curve (highlighted
in green) which is used to define the reconstruction on the
right. which is used to define the reconstruction on the right.")

:

    :   Here a curve (highlighted in green) can been positioned in the
        axial images (left panel) to define a curved surface which
        extends through the voxel data in the z-direction, and voxels
        from this data can be reconstructed into a two-dimensional image
        (right panel). Note that more complex curves than the one
        illustrated can be generated so that, for instance, the
        three-dimensional course of a major blood vessel can be
        isolated, or CT head scans can be *planarized* for orthodontic
        applications.

-   `<big>`{=html}3D Multi-Planar Reconstruction`</big>`{=html}

:

    :   A final variant on the MPR theme is the generation of a
        three-dimensional display showing all three orthogonal
        projections combined so that a defined point of interest locates
        the intersection of the planes, as illustrated in the following
        figure:

!3D Orthogonal MPR rotating
sequence.

:

    :   The point of intersection is located for illustrative purposes
        at the *centre* of the voxel data in the figure above. It can
        typically be placed at any point in the 3D data using
        interactive controls. In addition, the perspective used for the
        rotating sequence can typically be manipulated interactively to
        improve the visualization of a region of interest. Note that the
        image sequence illustrated above is one from a myriad of
        perspectives that can thus be generated. Note also that slice
        projections (e.g. MIPs) can be combined with this form of
        display to provide additional perspectives on a feature of
        interest.

## Maximum Intensity Projection

We\'ve described the maximum intensity projection (**MIP**) earlier in
the context of axial projection, where the maximum voxel value is
determined for lines running in parallel through the projected slice
thickness. A sequence of such images can be generated when this
computation is applied at successive angles around the voxel data. One
simple sequence is a rotating one for 360 degrees around the horizontal
plane, as illustrated in the left panel of the figure below, where the
maximum intensity is projected for every 9 degrees around the patient
and the resultant 40 images compiled into a repeating, temporal (e.g.
movie) sequence:

!3D MIPs of a CT scan: Horizontal rotating sequences using parallel
projections (left) and perspective projections
(right). and perspective projections (right).")

Notice that the 3D MIP derives its information from the most attenuating
regions of the CT scan (given that the
CT-number
is directly dependent on the linear attenuation coefficient) and hence
portrays bone, contrast media and metal with little information from
surrounding, lower attenuating tissues. Notice also that continued
viewing of the rotating MIP sequence can generate a disturbing effect
where the direction of rotation appears to periodically reverse -- which
may be an aspect of perceptual
oscillation.
The perspective MIP, illustrated in the panel on the right in the above
figure, can reduce this limitation by providing spatial cues which can
be used to guide continued visual inspection.

Perspective projections can be generated by changing from the parallel
lines used to generate the parallel projections to lines of voxels which
diverge from an apparent point behind the volume at a distance such that
the viewer of the display can visualize closer features of the image
data as relatively larger than deeper features -- see the following
figure:

!Illustration of parallel (left) and perspective (right) projections
using conceptualized lateral views of the voxel data and the eye of the
viewer of the projected
image. and perspective (right) projections using conceptualized lateral views of the voxel data and the eye of the viewer of the projected image.")

## Volume Rendering

Volume rendering can be applied to the voxel data in the successive
rotation manner described for MIPs above, as illustrated by the results
in the following figure:

!3D VR: Parallel projection (left) and perspective projection
(right). and perspective projection (right).")

Note that the volume rendering can be contrast enhanced so as to
threshold, for instance, through the voxel values to eliminate low
attenuating surfaces, as illustrated in the following figure:

!3D VR contrast enhancement progressively applied, from top left to
bottom right panels, through the voxel value
range.

Note also that the colour look-up table (CLUT) can be varied to
highlight features of particular interest, as shown in the set of images
below:

!3D volume renderings using four different
CLUTs.

The influence of the opacity table is illustrated in the following
example images:

!3D volume renderings using four different opacity
tables.

The influence on volume rendering of various shading settings is shown
below:

!3D VR shading comparison.

The shading settings used for the above images are as follows:

+-------------+-------------+-------------+-------------+-------------+
| Image       | Ambient     | Diffuse     | Specular    | Specular    |
|             | Coefficient | Coefficient | Coefficient | Power       |
+=============+=============+=============+=============+=============+
| Top Left    | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             | <div clas   | <div clas   | <div clas   | <div clas   |
|             | s="center"> | s="center"> | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 0.15        | 0.9         | 0.3         | 15.0        |
|             |             |             |             |             |
|             | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             | </div>      | </div>      | </div>      | </div>      |
|             | ```         | ```         | ```         | ```         |
+-------------+-------------+-------------+-------------+-------------+
| Top Middle  | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             | <div clas   | <div clas   | <div clas   | <div clas   |
|             | s="center"> | s="center"> | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | `<span sty  | 0.9         | 0.3         | 15.0        |
|             | le="color:r |             |             |             |
|             | ed;">`{=htm | ```{=html}  | ```{=html}  | ```{=html}  |
|             | l}0.75`</sp | </div>      | </div>      | </div>      |
|             | an>`{=html} | ```         | ```         | ```         |
|             |             |             |             |             |
|             | ```{=html}  |             |             |             |
|             | </div>      |             |             |             |
|             | ```         |             |             |             |
+-------------+-------------+-------------+-------------+-------------+
| Top Right   | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             | <div clas   | <div clas   | <div clas   | <div clas   |
|             | s="center"> | s="center"> | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 0.15        | `<span st   | 0.3         | 15.0        |
|             |             | yle="color: |             |             |
|             | ```{=html}  | red;">`{=ht | ```{=html}  | ```{=html}  |
|             | </div>      | ml}0.1`</sp | </div>      | </div>      |
|             | ```         | an>`{=html} | ```         | ```         |
|             |             |             |             |             |
|             |             | ```{=html}  |             |             |
|             |             | </div>      |             |             |
|             |             | ```         |             |             |
+-------------+-------------+-------------+-------------+-------------+
| Bottom Left | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
|             | <div clas   | <div clas   | <div clas   | <div clas   |
|             | s="center"> | s="center"> | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 0.15        | 0.9         | `<span st   | 15.0        |
|             |             |             | yle="color: |             |
|             | ```{=html}  | ```{=html}  | red;">`{=ht | ```{=html}  |
|             | </div>      | </div>      | ml}1.2`</sp | </div>      |
|             | ```         | ```         | an>`{=html} | ```         |
|             |             |             |             |             |
|             |             |             | ```{=html}  |             |
|             |             |             | </div>      |             |
|             |             |             | ```         |             |
+-------------+-------------+-------------+-------------+-------------+
| Bottom      | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
| Middle      | <div clas   | <div clas   | <div clas   | <div clas   |
|             | s="center"> | s="center"> | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 0.15        | 0.9         | 0.1         | `<span st   |
|             |             |             |             | yle="color: |
|             | ```{=html}  | ```{=html}  | ```{=html}  | red;">`{=ht |
|             | </div>      | </div>      | </div>      | ml}1.0`</sp |
|             | ```         | ```         | ```         | an>`{=html} |
|             |             |             |             |             |
|             |             |             |             | ```{=html}  |
|             |             |             |             | </div>      |
|             |             |             |             | ```         |
+-------------+-------------+-------------+-------------+-------------+
| Bottom      | ```{=html}  | ```{=html}  | ```{=html}  | ```{=html}  |
| Right       | <div clas   | <div clas   | <div clas   | <div clas   |
|             | s="center"> | s="center"> | s="center"> | s="center"> |
|             | ```         | ```         | ```         | ```         |
|             | 0.15        | 0.9         | `<span st   | `<span st   |
|             |             |             | yle="color: | yle="color: |
|             | ```{=html}  | ```{=html}  | red;">`{=ht | red;">`{=ht |
|             | </div>      | </div>      | ml}0.6`</sp | ml}1.0`</sp |
|             | ```         | ```         | an>`{=html} | an>`{=html} |
|             |             |             |             |             |
|             |             |             | ```{=html}  | ```{=html}  |
|             |             |             | </div>      | </div>      |
|             |             |             | ```         | ```         |
+-------------+-------------+-------------+-------------+-------------+
|             |             |             |             |             |
+-------------+-------------+-------------+-------------+-------------+

A final feature to note about volume rendering is that 3D editing
techniques can be applied so as to exclude unwanted features from the
computations and to expose internal structure. This is illustrated in
the following figure, where planes of an orthogonal frame can be moved
to crop the voxel data from six directions.

!3D volume rendering with cropping frame (left) and cropped, magnified
projection
(right). and cropped, magnified projection (right).")

## Surface Rendering

Surface rendering is also referred to as Shaded Surface Display
(**SSD**) and involves generating
surfaces
from regions with similar voxel values in the 3D data as illustrated by
the SPECT lung-perfusion scan shown in the left panel below:

!3D surface rendering: shaded surface and wireframe
display.

The process involves the display of surfaces which might potentially
exist within the 3D voxel data on the basis that the edges of objects
can be expected to have similar voxel values. One approach is to use a
grey-level thresholding technique where voxels are extracted once a
threshold value is encountered in the line of the projection -- see the
following diagram. Triangles are then used to tesselate the extracted
voxels, as shown in the right panel of the figure above -- and the
triangles are filled using a constant value with shading applied on the
basis of simulating the effects of a fixed virtual light source -- as
shown in the left panel above.

!Illustration of surface
rendering.

An opacity table can be applied to the results so that surfaces from
internal features can also be visualized. As an example, two surfaces
have been identified in the following image from the CT scan where voxel
values from bone surfaces are coded in an opaque yellow colour and
tissue surfaces in a transparent shade of red.

!SSD of two surfaces.

A second example of using an opacity table is shown in the following
figure. Here, axial CT data from the patient\'s airways have been
segmented using a region
growth
technique and the result processed using surface rendering, with full
opacity as shown in the left panel and with a reduced opacity (30%) as
shown in the right panel:

!3D SSD: opaque and transparent
display.

Notice that internal features of each lung can be discerned when the
opacity is reduced. Notice also that continued viewing of this type of
transparency display can generate apparent reversal of the image
rotation, similar to that noted for the 3D MIPs above. One method of
overcoming this type of problem is to segment each lung, for instance.
and to blend the results, as illustrated in the following figure:

!3D SSD: blending of each lung following
segmentation.

## subFusion Processing

We\'ll conclude this chapter by considering an application of 3D
visualization which integrates many of the image processing techniques
we\'ve described in this wikibook. We\'ll use the two SPECT scans, from
a patient\'s lung-ventilation (V) and lung-perfusion (Q) studies, in an
attempt to visualize any mismatch(es) characteristic of pulmonary
embolism (PE). The application we\'ll consider is called **subFusion
Processing** because it involves both image subtraction and image fusion
techniques.

Note again that the SPECT studies were generated using a *swamping*
technique where the perfusion scan was acquired immediately following
the ventilation scan using an administered activity which generated a
relative count rate of about 5:1 between scans. The first image
processing step therefore is to correct the perfusion scan for the
*background* ventilation activity.

Since the ventilation tracer in this case was administered using a
aerosol, we can assume for simplicity that its biodistribution is
reasonably identical in the two scans. Further, since the scans were
acquired about 15 minutes apart using the ^99m^Tc radioisotope, we can
assume a negligible effect from radioactive decay. We can simply
therefore subtract the ventilation stack from the perfusion stack, on
the basis of these assumptions, to isolate what we\'ll call the \"pure
perfusion\" scan.

The second step is to normalize the two scans by multiplying the
ventilation stack by a factor such that the mean counts (for example) in
the two stacks are similar.

:   `<span style="color:green;">`{=html}*We can now compare apples with
    apples!*`</span>`{=html}

Since a PE mismatch is likely to arise from regions of the lungs which
contain counts in the ventilation scan and are relatively bereft of such
counts in the perfusion scan, we can subtract the \"pure perfusion\"
stack from the ventilation stack, as a third image processing step, to
isolate any such differences as positively-valued features.

The final image processing step is to volume render this difference
stack along with the \"pure perfusion\" scan and to blend the results,
as illustrated in the following image:

frame\|center\|subFusion processing applied to a SPECT lung
ventilation-perfusion scan: Both lungs of the patient displayed on the
left, with their right lung displayed in the top right and their left
lung below it. The \"pure perfusion\" scan is displayed using a
grey-scale and the difference data using a spectrum CLUT, where large
differences are coded in red with intermediate differences in yellow and
smaller differences in green.

The steps involved are outlined in the following diagram. Note that
minor processes, such as CLUT selection, relative opacity adjustment and
contrast enhancement are omitted from this diagram for the sake of
simplicity. Note also that an image
registration
step may need inclusion at the beginning of the procedure in cases where
patient movement occurs between the two SPECT acquisitions.

!Block diagram of the 3D subFusion
process.

A final point to note is the larger appearance of the patient\'s lungs
in the segmented CTPA images relative to the SPECT images. This arises
because the CTPA study was acquired using a single breath-hold and the
SPECT studies with the patient breathing quietly over the period of
gamma camera rotation. The spatial registration of the three sets of
images is therefore not possible directly, and would require the
application of spatial warping and
other techniques which, unfortunately, are beyond the scope of our
treatment here.