From: tirone@neurocog.lrdc.pitt.edu (Steve Tirone)
Newsgroups: sci.med.physics,sci.image.processing
Subject: SUMMARY:  MRI Image Registration
Date: 10 Sep 92 15:44:25 GMT
Followup-To: sci.med.physics

Hello group...

	This is a summary on the image registration problem I asked
about a few weeks ago.  Sorry for the delay...

	Thanks very much to those of you who responded.  I was
particularly grateful to have my impression -- that automatic image
registration without fiducials is still an active research area --
confirmed.   

	I ended up implementing a centroid superposition algorithm,
which seems sufficient for our needs now.  Time constraints are really
what limit our ability to explore the other techniques available -- we
aren't in the registration research arena, and we must move on.  

	Being a nontrivial issue however, I wrote a small paper on
what I implemented and what other methods are available.  Complete
references to the articles I used are listed, as is the source code to
our registration program.  Email me if you want a copy of the paper,
and I'll send you the PostScript version of it.  I should mention that
1) the references are listed below anyhow, and 2) it appears that some
other source code (which handles rotations as well) may be available,
as indicated by the following recent post:

|| Article: 1063 of comp.graphics.visualization
|| Subject: Re: 3D reconstruction for histology
|| Message-ID: <andre.715944022@galaxy>
|| Date: 8 Sep 92 09:20:22 GMT
|| Sender: news@dutrun2.tudelft.nl (UseNet News System)
|| Organization: Delft University of Technology
|| 
|| lee.weller@anat.utas.edu.au (W. L. Weller) writes:
|| 
|| >I'm looking for information on Mac/DOS software for making 
|| >3D reconstructions
|| >from serial histological sections.  I'd like to know:
|| 
|| I have written my own reconstruction (or better alingment software).
|| It runs on a PC, but the actual matching is done on a Sun. However
|| when you want to putsome time in it, it must be easy to convert it to
|| Turbo or Microsoft C.
|| 
|| The idea is to find the optimal fit that minimizes the difference
|| between the greyvalues of both sections. I only correct for rotation
|| and translation, what limits the reconstruction to material with
|| limited deformation. (plastic sections).
|| 
|| >     1.  the name
|| No official name
|| >     2.  its source or contact
|| Its me: Andre Verweij : andre@duteinh.et.tudelft.nl
|| >     3.  its availability
|| Source is available on request.
|| >     4.  the platform it runs on
|| See above
|| >     5.  any criticisms or backpats (both firsthand and hearsay)
|| For other section types : parafin, etc, Rotation and Translation only is not
|| enough, so you should not use this program in that case.
|| >     6.  whether it's free, share or commercial
|| It is free as long as you don't use it to make it commercial
|| 
|| Andre Verweij
|| andre@duteinh.et.tudelft.nl
|| University of Technology Delft,
|| The Netherlands.
|| 

 =============== REFERENCES ==================

Here are the references I used to gain insight into the registration
problem:

\bibitem{rosenfeldkak}
Azriel Rosenfeld and Avinash~C. Kak.
\newblock {\em Digital Picture Processing}, volume~2.
\newblock Academic Press, Inc., San Diego, CA, 2nd edition, 1982.

\bibitem{foley}
J.~D. Foley, A.~van Dam, S.~K. Feiner, and J.~F. Hughes.
\newblock {\em Computer Graphics: Principles and Practice}.
\newblock Addison--Wesley, Reading, Mass., 2nd edition, 1990.

\bibitem{warping}
George Wolberg.
\newblock {\em Digital Image Warping}.
\newblock IEEE Computer Society Press, Los Alamitos, CA, 1990.

\bibitem{pelizzari}
Charles~A. Pelizzari, George T.~T. Chen, Danny~R. Spelbring, Ralph~R.
  Weichselbaum, and Chin-Tu Chen.
\newblock Accurate three--dimensional registration of {CT}, {PET}, and/or {MR}
  images of the brain.
\newblock {\em Journal of Computer Assisted Tomography}, 13(1):20--26, 1989.

\bibitem{fouriercorr1}
Lyndon~S. Hibbard, Tamara~L. Arnica-Sulze, Barbara~J. Dovey-Hartman, and
  Robert~B. Page.
\newblock Computed alignment of dissimilar images for three--dimensional
  reconstructions.
\newblock {\em Journal of Neuroscience Methods}, 41(2):133--152, 1992.

\bibitem{hough1}
Guido Gerig, Ron Kikinis, Walter Kuoni, Gustav~K. von Schulthess, and {Olaf
  K\"{u}bler}.
\newblock Semiautomated {ROI} analysis in dynamic {MR} studies. {P}art {I}:
  Image, analysis tools for automatic correction of organ displacements.
\newblock {\em Journal of Computer Assisted Tomography}, 15(5):725--732, 1991.


 ============ SUMMARY ====================

	And here is what I received by email. Requests for a summary
deleted. 

 ============ ORIGINAL POST ===============

        I've got several MRI images of the brain, taken over period of
time (minutes), and there's some slight movement of the head between
images.  I'd like to "register" these images, that is, align them so
the images all overlap exactly.  I do not have any external markers to
use as reference points (fiducials), BTW.

        What is a good way to go about this?  It appears that without
fiducials, automatic image registration has been attempted in various
ways, from superposition of centroids and principal axes (?) to
correlation using FFTs to contour matching via Hough
transforms...anyone care to comment on what might be the "best"
solution for my purposes?

        Also, I'm wondering what kind of error (in the pixel values) I
might introduce by registration and how I might quantify it.  Is it
possible to align images with subpixel accuracy?  How does resampling
the image affect my interpretation of the pixel value? (Like "its an
average signal value, plus x percent error introduced by resampling
after rotation" ??)

        I'm a newcomer to image processing.  Any pointers, solutions,
etc.  very much appreciated.

Steve
tirone@neurocog.lrdc.pitt.edu

 ========== RESPONSES ================

From: Regis Houde <regie@ireq-robot.hydro.qc.ca>
Message-Id: <9208241926.AA13851@ireq-robot.hydro.qc.ca>
To: tirone@neurocog.lrdc.pitt.edu
Subject: Image Registration

Hi Steve,

	A friend of mine is doing a PhD on this kind of stuff. He got some
pretty good results so far. He's on a business trip now but I'm sure
he'd be interested in helping you when he is back. His name is Marc Soucy
and his email is sushy@gel.ulaval.ca .

Hope it helps

--------------------------------------------------------------------
Regis Houde                             regie@ireq-robot.hydro.qc.ca
Institut de recherche d'Hydro-Quebec    regie@ireq-robot.uucp
1800, montee Sainte-Julie		(514) 652-8107
Varennes, Que., Canada   J3X 1S1	Telecopieur : (514) 652-8435
--------------------------------------------------------------------

From: jmm@imatron.COM (Jon Meyers)
Message-Id: <9208212221.AA02264@imatron.COM>
To: tirone@neurocog.lrdc.pitt.edu
Subject: Question on image registration and accuracy

Steve :

Howdy.

I'm a software engineer with a bit of experience in medical image
processing. You haven't mentioned in your article what type of
system you're using, and of course methodology in computers can
depend heavily on how much MIPS you've got.

There are a couple of interesting registration methods I've come
across.   Here I'll mention 2, one automated one requiring user
input.

Automated :

I've seen it done where one can do a 2-D convolution of two
images, with the resultant showing what the optimal registration
position is.  Actually, this is probably mathematically equivalent
to correlation with FFTs.

Non-automated :

If you're willing to accept some user input, you can use different
bit planes of your machine for displaying different images, then have
the user manipulate the "overlay" image(s) until they visually align
with some static image.

I'd be interested to hear what you come up with.  Good luck.

Jonathan Meyers
Senior Software Engineer
Imatron, Inc.

jmm@imatron.inc

From: Himanshu Gohel <gohel@zeus.csee.usf.edu>
Message-Id: <9208212048.AA05897@zeus.csee.usf.edu>
To: tirone@neurocog.lrdc.pitt.edu (Steve Tirone)
Subject: Re: Question on image registration and accuracy

Steve,

Image registration is not a very easy problem to solve, especially
without landmarks.  There have been some papers in Transactions
on Medical Imaging (IEEE publication), and one last spring in
IEEE Computer Graphics and Applications.  Sorry I can't recall
any other details.

Good luck

Himanshu

From: "John E. Moulder - Radiation Biology" <JMOULDER@mis.mcw.edu>
Subject: Question on image registration and accuracy
To: tirone@neurocog.lrdc.pitt.edu
Organization: Medical College of Wisconsin (Milwaukee, WI)

>From: tirone@neurocog.lrdc.pitt.edu (Steve Tirone)
>Subject:Question on image registration and accuracy
>Date: 20 Aug 92 17:26:56 GMT
>Message-ID:<4990@blue.cis.pitt.edu.UUCP>

>
>	I've got several MRI images of the brain, taken over period of
>time (minutes), and there's some slight movement of the head between
>images.  I'd like to "register" these images, that is, align them so
>the images all overlap exactly.  I do not have any external markers to
>use as reference points (fiducials), BTW.
>
>What is a good way to go about this?   It appears that without fiducials,
>automatic image registration has been attempted in various ways, from
>superposition of centroids and principal axes (?) to correlation using FFTs to
>contour matching via Hough transforms...anyone care to comment on what might 
be
>the "best" solution for my purposes?
>
>	Also, I'm wondering what kind of error (in the pixel values) I might
>introduce by registration and how I might quantify it.  Is it possible to 
align
>images with subpixel accuracy?   How does resampling the image affect my
>interpretation of the pixel value? (Like "its an average signal value, plus x
>percent error introduced by resampling after rotation" ??)
>
>	I'm a newcomer to image processing.  Any pointers, solutions, etc.
>very much appreciated.
>
>Steve
>tirone@neurocog.lrdc.pitt.edu


Our MRI group is very interested in this issue also, and has done a
considerable amount of what they call "difference imaging".  All the work to
date is in animals, and none is published.

The lead investigator is Dr Shi_jiang Li.  He is not on the net, but you can
reach him by phone at (414) 266-4029.

John Moulder (jmoulder@mis.mcw.edu)
Radiation Biology
Medical College of Wisconsin


From: jjc@mayo.EDU (Jon Camp)
Message-Id: <9208202025.AA26624@autobahn.mayo.EDU>
To: tirone@neurocog.lrdc.pitt.edu
Subject: Re: Question on image registration and accuracy

This seems to be a classic problem in medical imaging.  

Hongjian Jiang, one of our grad students (pardon me, he's now Dr.
Hongjian Jiang, Ph.D.) has developed an algorithm which has proven
robust for registering complete 3-D images (time sequence or
multimodality) from surfaces.  You might want to read:

"Image registration of of multimodality 3-D medical images by chamfer
matching", H. Jiang, K. Holton, and R. Robb - Proceedings of Vision
and Visualization, SPIE, San Jose CA, Feb. 1992

For the next few days, at least, you can contact Dr. Jiang at hjj@mayo.edu


-------------------------------------------------------------
Jon J. Camp                   Internet:   jjc@mayo.edu
Mayo Foundation                
Rochester, MN 55905           Phone: 	  (507) 284-3870                 
-------------------------------------------------------------

From: jjc@mayo.EDU (Jon Camp)
Message-Id: <9208281553.AA02237@autobahn.mayo.EDU>
To: tirone@neurocog.lrdc.pitt.edu
Subject: Re: Question on image registration and accuracy


1) Sorry - poor reference :

SPIE Proceedings : Biomedical Image Processing mand Three Dimensional 
Microscopy, Volume 1660  p. 356  


2) Dr. Jiang can be contacted at

Department of Radiology
Massachusets General Hospital
149 13th St.
Charlestown, Mass 02129-2060


Good Luck
-JJC
-------------------------------------------------------------
Jon J. Camp                   Internet:   jjc@mayo.edu
Mayo Foundation                
Rochester, MN 55905           Phone: 	  (507) 284-3870                 
-------------------------------------------------------------

From: zhenyu@magicave.mipg.upenn.edu (Zhenyu Wu (03-09-92))
Message-Id: <9208202049.AA02971@magicave.mipgyp>
To: tirone@neurocog.lrdc.pitt.edu
Subject: Re: Question on image registration and accuracy

Hi, Steve:

Early this year, I have worked on the registration between MR and PET
images, which is similar to your problem. The approach used was to
match the scalp surface (w/o fiducial markers). For both sets of
image, the scalp countors are extracted in a slice-by-slice basis
(simple thresholding and morphological filtering will do the job for
MR images); a (subic-spline) surface is fitted using boundary pixels
in one of the data set (in our case the PET data are used); the
registration is achieved by minimizing the total distance between
boundary pixels in the second set (after applying the registration
transform). In our case the error was in order of pixel size of the
PET images. I think a sub-pixel size error is achievable for the
registration between MR images, which have much better quality than
PET images. This method is similar to Pelizzri's.

Ref: C. A. Pelizzari et al."Accurate Three-Dimensional Registration of CT,
     PET, and/or MR Images of the Brain", J. Computer Assisted Tomography",
     pp. 20-26, No. 13, Vol. 1, 1989.

As for errors caused by reslicing the MR images, I doubt that the
errors will be visually noticeable. By the way, I used simple weighted
sum of the intensities in neighboring pixel.

Hopefully the above information is of any help. Let me know if you
need more details. Good luck.

Zhenyu Wu

From: Frank Muennemann <frankm@resonex.com>
To: tirone@neurocog.lrdc.pitt.edu
Subject: Re: Question on image registration and accuracy

In article <4990@blue.cis.pitt.edu.UUCP> you write:
>
>	I've got several MRI images of the brain, taken over period of
>time (minutes), and there's some slight movement of the head between
>images.  I'd like to "register" these images, that is, align them so
>the images all overlap exactly.  I do not have any external markers to
>use as reference points (fiducials), BTW.

You haven't given enough details to choose an appropriate technique.
Things to consider:

1) Were the different images made using the same imaging sequences
   and the same imaging parameters (TR, TE, type of sequence, etc.)

2) Was the patient physically moved between scans?  If so, it's
   very unlikely that the different images hit exactly the same
   slice positions in the brain, therefore difficult to register
   and get precision to pixel level.  Thicker slices would make
   registration more likely.

3) Were transmit/receive gains retuned between scans?  Transmit
   gains usually tune consistently, and have relatively little
   effect on many types of images (spin echoes especially), but
   variations in receiver gain make it unlikely that you'll get
   the same pixel values in two scans

4) How were the images reconstructed from raw data? Was image
   pixel level scaling fixed, or not?

Using pixel values in MRI scans is a tricky business.  Many
groups have attempted "tissue characterization" by comparing
values of the "same" pixels with different imaging sequences.
Unfortunately, changing imaging parameters also tends to change
image distortions to some degree, making registration difficult.

If you're trying to register mulitple images made with different
imaging sequences or even with different parameters and the
same sequences, you'll probably need careful control of gradient
amplitudes, or an image-warping algorithm.  Talk to the folks
at JPL for fancy collage-making software.

Good Luck!

Frank Muennemann |  frankm@resonex.com | My opinions are mine, etc.
Resonex, Inc     |  (408)720-8600 x333 | Gaia Lives!!
720 Palomar Ave., Sunnyvale, CA 94086 (USA)

From: buurlm@viva1.crd.ge.com (Lars Mikael Buur)
To: tirone@neurocog.lrdc.pitt.edu
Subject: Re: Question on image registration and accuracy

Hi Steve,

First of al you need to figure out what kind of tranformation you need
to correct for the motion. Simple x-, y-translation is obviously the
easiest, but not likely to be the case in head motion, which is
probably caused by the patient rolling his head from side to side.

You didn't mention whether you need automatic registration or if
manual registration would be allright. In any case, it's a good idea
to build a manual system first to get a feel for the kind of
correction that can be achieved. Display the result of subtracting the
two images you want to register and program a way to shift one of the
images in x and y with subpixel accuracy. (at least 0.1 pixel, you can
even have a rough registration followed by progressively finer
resolutions.) When you are able to register the image manually you can
put in an automatic registration step, and you will already have most
of the software in place.

Let me know if you need an automatic registration algorithm. I have
several references. Be warned though that this is not a trvial problem
nor one that has been solved well.


Lars Buur (buurlm@viva1.crd.ge.com)


From: andys@mph.sm.ucl.ac.uk (Andrew Simmons)
Subject: Re: Question on image registration and accuracy

Dear Steve,
           I'd be very grateful if you could summarise to the list on this
or send me a copy of any replies.

My group tries to circumvent such problems by having our radiographers
align patients to a standard position using three orthogonal pilot scans.
They can normally get images to within 1 mm in each orthogonal plane. If
this is not acceptable, cross-correlation will normally work well and can
be achieved to sub-pixel accuracy.

Yours,
      Andy Simmons 

-- 
 ----------------------------------------------------------------------------
 Andrew Simmons                JANET   : andys@uk.ac.ucl.sm.mph
 Dept. of Medical Physics      BITNET  : andys%uk.ac.ucl.sm.mph@ukacrl.bitnet
 University College London     INTERNET: andys@mph.sm.ucl.ac.uk
 Capper St,LONDON WC1E 6JA,UK  Tel: +44 71-380-9700     Fax: +44 71-380-9577
 ----------------------------------------------------------------------------

From: cjk@gl.pitt.edu
Message-Id: <9208211148.AA07684@ra.gl.pitt.edu>
To: tirone@neurocog.lrdc.pitt.edu (Steve Tirone)
Subject: Re: Question on image registration and accuracy


In article <4990@blue.cis.pitt.edu.UUCP>, you write:
> 
> 	I've got several MRI images of the brain, taken over period of
> time (minutes), and there's some slight movement of the head between
> images.  I'd like to "register" these images, that is, align them so
> the images all overlap exactly.  I do not have any external markers to
> use as reference points (fiducials), BTW.
> 
> What is a good way to go about this?   It appears that without fiducials,
> automatic image registration has been attempted in various ways, from
> superposition of centroids and principal axes (?) to correlation using FFTs 
to
> contour matching via Hough transforms...anyone care to comment on what might 
be
> the "best" solution for my purposes?
> 
> 	Also, I'm wondering what kind of error (in the pixel values) I might
> introduce by registration and how I might quantify it.  Is it possible to 
align
> images with subpixel accuracy?   How does resampling the image affect my
> interpretation of the pixel value? (Like "its an average signal value, plus x
> percent error introduced by resampling after rotation" ??)
> 
> 	I'm a newcomer to image processing.  Any pointers, solutions, etc.
> very much appreciated.
> 
> Steve
> tirone@neurocog.lrdc.pitt.edu


We were able to use Spyglass Transform on a Macintosh to do exactly
what you want to do.  With Transform you can select a pixel in one
frame and align it to an exact pixel in another frame & if necessary
construct a 3-D model of the data sets.  Since your next door; give me
a call and I will demonstrate the process for you.  It is not as
difficult as one might imagine...


-- 
Carl Joel Kuzmich  (412) 624-1885     | Love is a better teacher than duty |
University of Pittsburgh              |                - Albert Einstein   |
110 Old Engineering Hall 
Pittsburgh, PA   15260	
Newsgroups: sci.med.physics