本田財団レポート No.160
第 36 回本田賞授与式 記念講演（2015 年 11 月 17 日）「医療ロボット工学とコンピュータ統合支援治療」
ジョンズ・ホプキンス大学 John C. Malone 冠教授

ラッセル・テイラー博士

Medical Robotics and Computer-Integrated Interventional
Medicine
Commemorative lecture at the 36th Honda PrizeAward Ceremony on the 17th November 2015

Dr. Russell H. Taylor
John C. Malone Professor at Johns Hopkins University, U.S.A.

ラッセル・テイラー博士
ジョンズ・ホプキンス大学 John C. Malone 冠教授
John C. Malone Professor at Johns Hopkins University

Dr. Russell H. Taylor■生まれ
1948年6月■BORN
米国バージニア州生（アメリカ国籍）■EDUCATION AND TRAINING

■学 歴
1970年

ジョンズ・ホプキンス大学卒業

1976年

スタンフォード大学 コンピュータサイ
エンス 博士課程修了（工学博士）

1968年～70年

ジョンズ・ホプキンス大学 研究助手

1970年～76年

スタンフォード大学 研究助手

1976年～95年

IBM トーマス・J・ワトソン研究所
研究スタッフ、研究マネージャー

1995年～現在

ジョンズ・ホプキンス大学 コンピュータ
科学、放射線学科、機械工学、および外科学兼任教授（2011年に最初の John C. Malone
冠教授就任）

2013年～現在

1976:

Stanford University, Ph.D. in Computer Science(Doctor of Engineering)1970:

Johns Hopkins University■EMPLOYMENT HISTORY■職 歴

1998年～現在

June 1948, Virginia, USA (USA citizenship)

同大学 コンピュータ統合外科手術用システム技術工学研究センター（CISST ERC）所長
同大学 計算センシング・ロボティクス
研究所（LCSR）所長

2013–current: Johns Hopkins University, Director of the Laboratory
for Computational Sensing and Robotics (LCSR)
1998–current: Johns Hopkins University, Director of theEngineering Research Center for
Computer-Integrated Surgical Systems andTechnology (CISST ERC)
1995–current: Johns Hopkins University, Professor of Computer
Science with joint appointments in Radiology,
Mechanical Engineering, and Surgery (Named first
John C. Malone Professor in 2011)
1976–95:

IBM T. J. Watson Research Center, Research Staff
Member and Research Manager

1970–76:

Stanford University, Research Assistant

1968–70:

Johns Hopkins University, Research Assistant■略 歴ラッセル・テイラー博士はコンピュータ科学、ロボット工学および、コンピュータ統合支援治療の専門分野において38年を超える実績を積んでいます。彼は1970年にジョンズ・ホプキンス大学にて理工学の学士号を取得し、1976年にはスタンフォード
大学にてコンピュータ科学の博士号を取得しました。1976年にIBM Research に入社し1995年にジョンズ・ホプキンス大学
へ拠点を移すまでに、テイラー博士は AML ロボット言語を開発し、自動化技術部と現コンピュータ支援外科手術グループのマネージメントを行いました。ジョンズ・ホプキンス大学では機械工学、放射線学、および外科学を兼任しながらコンピュータ科学の John C. Malone 冠教授、コンピュータ統合外科手術用システム技術工学研究センター（CISST ERC）と計算センシ
ング・ロボット工学研究所（LCSR）の所長を務めています。
テイラー博士の研究対象はロボット工学、マン・マシン協調シ
ステム、医用画像化およびモデリング、そしてコンピュータ統
合介入システムです。テイラー博士は400を超える査読付出版
物／書籍の著者であり、米国電気電子学会（IEEE）、米国医用
生体工学会（AIMBE）、コンピュータ医用画像処理ならびにコンピュータ支援治療（MICCAI）学会、そして東京大学工学系研究科のフェローです。テイラー博士は IEEE ロボット工学パ
イオニア賞、MICCAI 学会永続的影響賞、IEEE 医療・生理部会（EMBS）技術分野賞および、コンピュータ支援整形外科手術における功績を称えたモーリスミューラー賞をはじめとする
数多くの賞の受賞者でもあります。

■主な出版物■BIOGRAPHICAL SKETCH
Dr. Russell H. Taylor has over 38 years of professional experience inthe fields of computer science, robotics, and computer-integrated
interventional medicine. He received a Bachelor of Engineering
Science degree from Johns Hopkins University in 1970 and a Ph.D.
in Computer Science from Stanford University in 1976. He joined
IBM Research in 1976, where he developed the AML robot language
and managed the Automation Technology Department and (later)
the Computer-Assisted Surgery Group before moving in 1995 toJohns Hopkins University, where he is the John C. Malone
Professor of Computer Science with joint appointments in
Mechanical Engineering, Radiology, and Surgery and is also
Director of the Engineering Research Center for
Computer-Integrated Surgical Systems and Technology (CISST
ERC) and of the Laboratory for Computational Sensing andRobotics (LCSR). Dr. Taylor’s research interests include robotics,
human-machine cooperative systems, medical imaging & modeling,
and computer-integrated interventional systems. He is the author ofover 400 peer-reviewed publications and book chapters, a Fellow of
the IEEE, of the The American Institute for Medical and BiologicalEngineering (AIMBE), of the The Medical Image Computing andComputer Assisted Intervention (MICCAI) Society, and of School ofEngineering the University of Tokyo. He is also a recipient of
numerous awards, including the IEEE Robotics Pioneer Award, the
MICCAI Society Enduring Impact Award, the IEEE EMBS
Technical Field Award, and the Maurice Müller Award for
Excellence in Computer-Assisted Orthopaedic Surgery.
■MAJOR PUBLICATIONSTaylor, R.H., S. Lavallee, G. Burdea, and R. Mosges,
Editors, Computer-Integrated Surgery, 1996, MIT Press:
Cambridge, Mass.

Taylor, R.H., S. Lavallee, G. Burdea, and R. Mosges, Editors,
Computer-Integrated Surgery, 1996, MIT Press: Cambridge,
Mass.

Taylor, R. H. and L. Joskowicz, "Computer-Integrated
Surgery and Medical Robotics," in Standard Handbookof Biomedical Engineering and Design, M. Kutz, Editor,
2002, McGraw Hill.

Taylor, R. H. and L. Joskowicz, “Computer-Integrated Surgery
and Medical Robotics,” in Standard Handbook of Biomedical
Engineering and Design, M. Kutz, Editor, 2002, McGraw Hill.

R. H. Taylor and P. Kazanzides, "Medica Robotics andComputer-Integrated Interventional Medicine," inAdvances in Computers, vol. 73, M. Zelkowitz, Editor:
Elsevier, 2008, pp. 217-258．

R. H. Taylor and P. Kazanzides, “Medical Robotics andComputer-Integrated Interventional Medicine,” in Advances in
Computers, vol. 73, M. Zelkowitz, Editor: Elsevier, 2008, pp.
217-258.

このレポートは、2015 年 11 月 17 日 東京、帝国ホテルにおいて行なわれた第 36 回本田賞授与式記念講演の要旨をまとめたものです。
This report is the gist of the commemorative lecture at the 36th Honda Prize Award Ceremony at the Imperial Hotel, Tokyo on
17th November 2015.

Medical Robotics and Computer-Integrated
Interventional Medicine
Russell H. Taylor

To begin with, I should say that I feel both proud and humble to be receiving this award. To
join such a very distinguished list of past awardees is something that I would not haveimagined even three months ago. I am also very grateful to the Honda Foundation, to Mr.
Hiroto Ishida, the President of the Foundation, to Mr. Masataka Yamamoto, the Managing
Director, to all the other members of the Honda Foundation Directors and staff, and to theselection committee for their parts in making this award possible and for choosing me.

40

Fig 1〈Fig 1〉 Even more, I must express my deep admiration for Soichiro Honda and his brother
Benjiro for their vision in establishing the Honda Foundation. This statement from Soichiro
Honda truly resonates with me. The sentiment expressed has been a major motivation for myresearch over my career, both at IBM and at Johns Hopkins.

Fig 2〈Fig 2〉 Let me say at the outset that I will be talking about the work of many people. I
have been fortunate over the years to work with an extraordinary number of extremely
talented and dedicated colleagues and students, and also to profit from lessons learned byothers in this rapidly expanding field. I am very grateful to them and to the various
commercial, Governmental, and philanthropic institutions that have helped further the work.
I will try to include acknowledgments on individual slides, but this can only begin to reflectthat I owe to them and to the many agencies that have supported my work over the years.

41

Fig 3〈Fig 3〉 My work for the past 25 or so years has focused on developing systems in whichhuman physicians, technology, and information systems work cooperatively to make surgery
and other forms of interventional medicine more precise, safer, and more effective.

Fig 4〈Fig 4〉 Let me start with a couple examples that I will discuss more later. This is a system
to assist a surgeon in joint replacement surgery.

42Fig 5〈Fig 5〉 And this is a system to augment the information available to a surgeon using arobot to perform surgery to remove a tumor from the base of the tongue.
The key thing about both of these examples is that they are systems combining informationand technology to help perform difficult surgical tasks.

Fig 6〈Fig 6〉

The next two slides illustrate my view of the information flow in computer-

integrated interventional medicine.
The key is information: We start with all the information we have about an individual
patient. For surgery, much of this information is in the form of medical images, but we areseeing progress in including other clinical data. This information is combined with knowledge
about people in general to produce some sort of computer representation – which we call amodel – that can be used to support the rest of the process. We can use it to diagnose thepatient’s condition and formulate a treatment plan. Then all of this information can be43

registered to the physical patient in the operating room or intervention suite, and appropriatetechnology can be used to assist the surgeon to carry out the planned intervention and toverify that it was done.
To an engineer like me, this is a control loop.
Note that this picture is replicated in the upper right hand corner. This reminds us that thisprocess actually occurs at many time scales, down to every second in the operating room. This
process can be used to give each individual patient a better intervention – safer, less invasive,
more effective, or the like.

Fig 7〈Fig 7〉 Just as with computer-integrated manufacturing, computers are involved at allphases of this process. Computer-integrated interventions can be more consistent, and(crucially) the information generated can be saved, so that we know a lot more about how theintervention was performed. Since we can eventually know the patient outcomes, we can use
statistical methods to improve treatment processes for future patients.
I believe that the synergy between the blue loops and the red loop has the potential to makeprofound improvement in both the quality and cost-effectiveness of health care delivery.
I do not have time to talk about all of the work I have done over the past 25 or so years inpursuit of this vision, but I would like to give you a few examples. In preparing this talk, I was
struck by the continuity between some of our early work and more recent efforts.

44

Fig 8〈Fig 8〉 Let me first talk about information-driven interventions for individual patients. My
involvement with medical robotics began in the late 1980’s, when I was a department
manager at IBM Research. We had a small collaboration with two surgeons at The Universityof California at Davis to explore the possibility of using a robot to accurately prepare thefemur for cementless hip implants. I was looking for an opportunity to spend more time in thelab, and my bosses agreed to let me form a small group to see if we could build a system that
could actually perform surgery.
One of the surgeons (Hap Paul) was a veterinarian, and it seemed reasonable to target hispatients. In about a year, we were able to produce a complete working system, which became
known as “Robodoc”. This was donated to UC Davis, and Hap performed the first canine
surgery in 1990. With IBM help, Hap also founded a startup company to develop a humanclinical version, and they performed their first human case in 1992. Robodoc went throughvarious vicissitudes and it is now developed and marketed by Think Surgical. I have been
fortunate to continue to interact with the development team over the years.
Also, Peter Kazanzides, the postdoc who developed the first prototype with me at IBM, wasa co-founder of the company and joined our Center at Johns Hopkins in 2002, where he is now
a research professor.
Since Robodoc was my introduction to medical robotics, I would like to say a few words
about the system and about a few of our more recent research results relating to this problem.

45

Fig 9〈 Fig 9〉In hip and knee surgery, the surgeon replaces a failing joint with artificial

components. The bones must be prepared very accurately to ensure that the replacement
components fit accurately and are in the right position.

Fig 10〈Fig 10〉 In the traditional manual hip surgery, the surgeon selects the desired component
by holding acetate templates up to x-rays. In the operating room, the surgeon uses a manual
broach to make a corresponding hole in the thigh bone.

46

Fig 11〈Fig 11〉 The implant is placed where the broach makes the hole, which is not necessarily
where the surgeon intended, and the hole is very ragged, so that the fit can be poor.

Fig 12〈Fig 12〉 Robodoc planning uses a 3D CT scan of the patient. The surgeon interactively
places a CAD model of the selected implant in the desired place relative to the CT images. The
computer calculates a desired path for a cutting tool relative to CT coordinates, as well as
information to enable the robot to locate the bone in the operating room.

47

Fig 13〈Fig 13〉 In the operating room, surgery proceeds normally until it is time to make the holefor the implant. The bone is held firmly in place relative to the robot and its position relative
to the robot is determined. The robot then uses a cutter to machine the desired implant shape.
The robot is then moved away and the surgery proceeds manually.
The implant is placed exactly where it was planned to go, and the implant fits accurately,
especially when compared to manual broaching.
Over the years, we have explored a number of ways to determine the position of the bonerelative to the robot. We call this process “registration”.

Fig 14〈Fig 14〉 Here is an example of some recent work with my colleagues Emad Boctor andPeter Kazanzides and our students Seth Billings, Hyun Jae Kang, and Alexis Cheng, in which
we combine ultrasound images of the distal femur with points sampled with an opticallytracked pointer in order to find the bone accurately and less invasively than earlier methods
used with Robodoc. His example uses a new registration algorithm developed by Seth Billings
as part of his Ph.D. thesis research.

48

Fig 15〈Fig 15〉 A common theme from very early days (both in my own work and that of others)
has involved combining prior information from patient images or from statistical models of
populations of patients with new images of a patient in order to produce a new patient-specificmodel. The specifics of how this works depend on the kinds of images involved and on theintended application.

Fig 16〈Fig 16〉 One example for orthopedics involves using a small set of x-ray images of anindividual patient, together with a statistical model derived from CT images of many patients,
in order to estimate what a CT scan of that patient would look like if one had been available.

49Fig 17〈 Fig 17 〉

Here, my student Gouthami Chintalapani, used three x-ray images and astatistical model to produce a 3D model of the pelvis and hips that would be sufficiently
accurate for surgical planning.

Fig 18〈Fig 18〉

In this example, CT scans are needed to provide very accurate models hip

cartilage in order to plan a periacetabular osteotomy, but less accuracy is OK elsewhere. In
order to reduce total radiation to the patient, Gouthami used a CT scan of the acetabular (i.e.,
hip socket) region of the pelvis, two x-rays, and a statistical model to produce a model forplanning the procedure.

50

Fig 19〈Fig 19〉 More recently, my student, Robert Grupp, has developed a method using a CT
scan through the acetabulum with a few additional slices in order to achieve much the samething. This can greatly simplify the work flow before surgery.

Fig 20〈 Fig 20, 21 〉

Fig 21In many cases, we do not need an active robot to help the surgeon.

Traditionally, surgery has relied on a surgeon’s hand-eye coordination and the surgeon’s
mental picture of the surgical plan and progress of the procedure.
However, a computer can help provide the surgeon with additional information that can bevery useful during the procedure. I became interested in this idea very early, and it has been arecurrent theme in my work ever since.

51

Fig 22〈Fig 22〉 While working on Robodoc, I also began collaborating with Dr. Court Cutting, aplastic surgeon at New York University Medical Center, in order to develop a computerintegrated system for craniofacial osteotomies. In these procedures, the surgeon cuts thebones of the patient’s face apart and rearranges the fragments to improve the patient’s
appearance and also to help with functions such as chewing. In this case, our primary goal
was to help the surgeon align the bone fragments relative to each other.

Fig 23〈Fig 23〉 This is a sketch that we made summarizing our system, which draws heavily on
Dr. Cutting’s ideas, as well as my own and those of Fred Bookstein. A CT scan of the patient
was used to make a model of the patient’s skull. This was compared to a statistical model
made from CT scans of many different patients, which was used to decide where the bones
would be cut and how they would be rearranged. In the operating room, the models and plan
would be registered to the actual patient, and computer graphic displays and other humanmachine interfaces would be used to help the surgeon carry out the plan.

52

Fig 24〈Fig 24〉 This is what the system looked like in our IBM lab. I should point out the use ofan optical tracking system to track surgical tools and parts of the patient’s anatomy, as well
as a passive manipulation aid to help the surgeon align the fragments and hold them in place
while they were being attached to each other.

Fig 25〈Fig 25〉 Here is a photo from the operating room at the NYU Hospital. I will spare you therather bloody photos of the patient’s skull during the procedure.
However, I might point out that this application was one of the first outside of neurosurgery
to use what we now call “surgical navigation” to provide information support during surgery.

53Fig 26〈Fig 26〉 More recently, researchers have begun to incorporate surgical video and otherimaging modalities into surgical assistance systems. The system above was developed forskull base surgery, using endoscopic video and cone-beam CT imaging. I collaborated in thiswork, but it was led by my colleagues Jeff Siewerdsen and Greg Hager.

Fig 27〈Fig 27〉

Around 1991, I gave a talk to an endoscopic surgery meeting. This got meinterested in the prospect of developing a robotic assistant for endoscopic surgery. This
interest eventually led to a collaboration between IBM and Johns Hopkins University todevelop a system for precise manipulation of endoscopes and surgical tools in minimallyinvasive surgery.

54

Fig 28〈Fig 28〉 The system had many functional capabilities that were new at that time. For
instance, we took the track-point device from IBM laptops and made a simple mouse/joystickthat clipped on the surgeon’s instruments and could be used to control the motion of the robot
or interact with the system controller through a video display with graphic overlays. We also
used video images to help control the robot’s motion and developed what we now call “virtual
fixtures” to assist the surgeon.
The system had a simple voice command interface as well. Since the voice synthesizer
sounded vaguely Scandinavian, we called the system “LARS”.

Fig 29〈Fig 29〉 One kinematic design feature that later became ubiquitous was what I called a“remote center of motion” or “RCM”. In minimally-invasive surgery, the RCM provides a pivot
point where the surgical instrument enters the patient’s body.

55

Perhaps the most familiar example of this feature in surgical robots is its use in Intuitive
Surgical’s da Vinci Surgical System.

Fig 30〈Fig 30〉 There are, of course, many ways to implement an RCM. Here are a few that wehave used over the years.

Fig 31〈Fig 31〉 Another idea that emerged from the early IBM work was what I call “steady hand”
cooperative control, which works something like power steering. The surgeon and the robot
both hold the surgical tool. The robot senses forces exerted by the surgeon on the tool andmoves to comply. Since the robot is doing the actual motion, there is no hand tremor.
Also, the robot can assist the surgeon in various ways, such as enforcing safety barriers or
helping align a tool to an anatomic target or comply with other sensors. We refer to thesecapabilities as “virtual fixtures”.

56Fig 32〈Fig 32〉 These ideas have been the focus of a great deal of research, both at JHU andelsewhere. Here are some of the systems we have developed over the years.
Let me discuss a couple examples from our work in more detail, showing how steady handrobots fit into computer-integrated surgical systems.

Fig 33〈Fig 33〉 Retinal disease is a leading cause of blindness, and surgical interventions often
offer the best hope for restoring or preserving vision. However, retinal surgery is extremely
difficult, requiring surgeons to operate at the extreme limits of human sensor-motor capability.
A typical procedure requires the surgeon to insert sub-millimeter diameter instruments
through small trocars inserted into the sclera (the white part of the eye) and peel scar tissue

57

from the retina without damaging it. This has been described to me as trying to peel sticky
tape from tissue paper without tearing the tissue paper.
The surgeon’s hand tremor is often as large or larger than the anatomical structures being
manipulated, and the forces between the tool and the retina are an order of magnitude
smaller than a human can feel and also an order of magnitude smaller than the forces
between the tools and the trocars.
The surgeon observes the procedure through a surgical microscope and relies on memory to
relate what he or she is seeing to any pre-operative information about the patient. The
surgeon’s posture is very rigid, and neck and back pain are common occupational disabilities.

Fig 34〈 Fig 34 〉

We have been taking a system approach, in which individual technicalcomponents can be combined in various ways to address these limitations. Although many
people have contributed to this work, the system as a whole was the basis of my student
Marcin Balicki’s Ph.D. thesis. This is what our system looks like in our engineering lab．

58

Fig 35〈Fig 35〉 And this is what it looks like in our microsurgery lab at the hospital, where wehave been testing the system capabilities on rabbits.
Notice that we are using steady-hand guiding with one of our RCM “eye robots”.

Fig 36〈Fig 36〉 We have augmented the surgeon’s direct view by capturing stereo video from themicroscope. This can be displayed on a stereo video monitor.
We can process the video to build up a mosaic map of the retina and can fuse this map withinformation obtained from preoperative images or from various sources during surgery. This
information can then be overlaid on the surgeon’s live view from the microscope.

59

Fig 37〈Fig 37〉 We can also track surgical tools and use the results for annotations or (as here) towarn the surgeon when the tool is near the retina.

Fig 38〈Fig 38〉 In work led by my colleague, Iulian Iordachita, we have been using optical fibers
to build millinewton resolution force sensors into sub-millimetric surgical tool shafts and havebeen incorporating the results into control of surgical robots and into other human interfaces.

60Fig 39〈Fig 39〉 In addition to feeding back the forces into the robot control, we have various otherways to inform the surgeon of tool-tissue interactions that also work with freehand tools. We
have found that one of the most effective is auditory. The computer emits sounds based on
how much force is being exerted on the tissue.

Fig 40〈Fig 40〉 It is also important to be able to sense the forces between the tool and the sclera.
Consider this example where we are using two steady hand robots. The surgeon moves bothtools together in order to rotate the eye under the microscope. If the motions are notcoordinated, the eyeball can be stretched.

61

Fig 41〈Fig 41〉

By adding additional force sensors to the tool shaft, Dr. Iordachita’s student

Xingchi He is able both to sense the lateral forces exerted by the tool on the sclera and tocompute the point along the tool shaft where it enters the sclera, as well to sense as the force
on the retina. He is able to incorporate this information into the control of the robot.

Fig 42〈Fig 42〉 We can incorporate other forms of sensing into surgical tools. Here, my colleagues
Iulian Iordachita and Jin Kang have built optical coherence tomography (OCT) sensors intosub-millimeter surgical tools. With these tools, one can determine the distance of the tool to
the retina as well as sense structures in the retina and measure tissue properties such as
oxygenation levels.

62

Fig 43〈Fig 43〉 In this early demonstration, my student Marcin Balicki used the OCT sensor as
feedback to the robot to keep the probe a constant distance of 150 microns from a moving
surface, with an error of 10 microns. This capability can be useful in applications like OCT
scanning or laser ablation.

Fig 44〈Fig 44〉 For one more example, I’d like to tell you about some of our current work todevelop a steady-hand robot for head-and-neck surgery.

63Fig 45〈Fig 45〉 The robot was specifically designed for tremor-free operation of long tools reaching
into holes while keeping the mechanism as much as possible out of the surgeon’s line of sight.
Here, we show a typical setup for laryngeal or vocal cord surgery. The parallel structure
reduces the moving mass, thus making the robot more responsive.

Fig 46〈Fig 46〉 In this precision experiment, the surgeon must navigate a tool around a mazewithout touching the sides.
Here you see side-by-side comparisons of the performance of a surgeon attempting the task,
which is practically impossible without assistance.

64

Fig 47〈Fig 47〉 This video shows an experiment performed in our lab by our surgeon collaborator,
Dr. Lee Akst, doing a common vocal cord procedure on a cadaveric specimen.
Another feature Dr. Akst likes is the ability to position a tool and then have the robot hold it
stably until he moved it again.

Fig 48〈Fig 48〉 This somewhat longer experiment shows the ability of the robot to be integrated
with a surgical navigation system, together with its ability to provide virtual fixtures.
In this case, we are again using a cadaver, for which we have a CT scan.
After the CT scan is registered to the robot, the navigational display shows where the tool isrelative to the CT images.

65In sinus surgery surgical instruments must be inserted many times through a complicatedpath into the nose. Tool-tissue collisions can cause bleeding. So we have implemented avirtual fixture to hold the tool on the desired path while the surgeon advances the tool along it.
This virtual fixture can either be “hard”, in which case the tool will never leave the path, or“soft”, in which case the surgeon can deviate from the path but will feel a force nudging thetool back to the path.
Our surgeon collaborator, Dr. Masaru Ishii, believes that this capability will be especially
useful in training surgical residents.

Fig 49〈Fig 49〉 Another early use for the robot was precise placement of needles or other surgicaltools on targets identified in images. On the left we see the LARS robot positioning theworking channel of an endoscope to assist placing a surgical grasper on a target identified inendoscopic video. On the right are some images showing an early experimental system using
the robot to inject therapy seeds into the liver under biplane x-ray guidance.

66

Fig 50〈Fig 50〉 In the years since then, many more examples of image-guided needle placement
have been developed at our Center; I have been actively involved in some of them, and havebeen an interested spectator in others.

Fig 51〈Fig 51〉 This slide shows work from my student Wen Liu’s Ph.D. thesis, done jointly with
Intuitive Surgical and Johns Hopkins. In this animal-lab experiment done at Intuitive, we areusing intraoperative x-rays, cone-beam CT, and video graphic overlays to help a surgeon
remove a base-of-tongue tumor using the DaVinci robot.

67

Fig 52〈Fig 52〉 Here is another example, showing some work of my student Paul Thienphrapa
done in collaboration with Philips Research. The problem is to remove a foreign object frominside a beating heart. This normally requires open-heart surgery. Instead, we want to insert
a small snake-like robot through a small hole into a beating heart while monitoring theprocedure with real time 3D ultrasound.

Fig 53〈Fig 53〉 In this case, we used a 4 millimeter diameter snake-like flexible manipulator thatmy postdoc Nabil Simaan and I had developed for minimally-invasive surgery.

68Fig 54〈Fig 54〉 This is the experimental setup in our laboratory, using a realistic artificial heart,a Philips 3D ultrasound system, and our snake end-effector mounted on a LARS robot.
One challenge was that the robot is fairly slow and the particle’s motion within the heart isoften very fast.

Fig 55〈Fig 55〉 However, we also realized that there are eddies within the heart that the particle
can be trapped in for a few seconds. So if we watch the particle for a while, we can predict
where it will be again soon and go wait for it, sort of like a bass waiting for a minnow to comeby. Here we see the probability map being built up(right figure). The computer decides where
to wait for the particle. The robot goes there and retrieves the particle.

69

Fig 56〈Fig 56〉 In the past few years, I have also been collaborating with my colleague Mehran
Armand and others on another sort of snake-like robotic end effector. In this case, the snake isessentially a bendable metal tube, with a 6 millimeter outer diameter and 4 millimeter innerdiameter through which we can pass surgical tools.

Fig 57〈Fig 57〉 Our initial application is treatment of osteolytic lesions in revision hip surgery.
Hip implants typically have a polyethylene liner between the cup component fixed in thepelvis and the ball of the stem component fixed in the femur. Wear particles from the liner cansometimes work their way behind the cup, causing the bone in the pelvis to turn to mush. Left70

untreated, this can cause a fracture or can cause the implant to loosen. It is relatively
straightforward to do surgery to replace the liner, but the material in the osteolytic lesionmust be cleaned out and replaced with epoxy cement or some sort of bone graft material.
Unfortunately, the cup is still held firmly to the pelvis by bony ingrowth into the surface of thecup. Removing it to get to the lesion can cause a fracture.

Fig 58〈Fig 58〉 Our approach is to insert our snake device through holes in the cup or throughsmall holes drilled into the bone behind the cup and to deploy tools through the snake in orderto clean out the cavity from the inside.

Fig 59〈Fig 59〉 I could give many other examples, but I would like to turn briefly to the statisticalprocess control aspects of computer-integrated interventional medicine.

71

We are beginning to use statistical methods and data bases to improve the quality oftreatment processes, in this case in radiation therapy for head and neck cancer.
The Johns Hopkins Radiation Oncology Department has been building a data base ofpatients to facilitate both clinical care and research. As Ted DeWeese, the department chair,
explains: in the long term, when he sees a patient, he would like to know which patients in thedata base most resemble this patient, in rank order what are the complications that patientslike this may be likely to experience, and what treatment options have worked best forpatients like this.
Here is one example of some joint work with Todd McNutt and to exploit this data base toimprove therapy planning.

Fig 60〈Fig 60〉

The goal is to plan a pattern of radiation beams that delivers a prescribedradiation dose to the tumor while minimizing the dose to surrounding critical structures likethe spinal cord, eyes, and salivary glands.
This is a very difficult problem for the computer. So in conventional planning, a dosimetrist
sets up a simpler optimization problem that the computer solves to get an approximate
answer. The computer makes a very accurate simulation of this plan using the patient’s CT
images and displays the result. The dosimetrist then modifies the optimization criteria, andthe process continues until there is a plan that both the dosimetrist and the physician accept.
This can take many iterations, and the plan quality depends on human judgment of when tostop, based on experience to predict how good a plan will be feasible for this patient.

72

Fig 61〈Fig 61〉 To improve the planning process, we were able to develop an efficient way tocharacterize the geometric relationship between the tumor and surrounding structures and touse this descriptor to search our data base of previously treated patients.
As a quality control check, given a proposed treatment plan, we can search the data base forsimilar patients whose plans do a better job of sparing surrounding structures while treating
the tumor. If we apply the optimization criteria for the data base patient to the current
patient, we find that in many cases the resulting plan is better.
Similarly, if we use the data base to find the best plan for a previously treated similar
patient as a starting point for planning, we can produce a plan in a very small number ofpasses through the interactive process that is as good or better than the conventionallyproduced plan.

73

Fig 62〈Fig 62〉 Here is one example showing significantly reduced radiation dose to the patient’s
salivary glands.
This planning method is now in the clinical workflow at Johns Hopkins.

Fig 63〈Fig 63〉 In concluding, let me return to the larger themes of my work over the past 25 or so
years. Computer-integrated interventional medicine is not about any specific robot, imager, orother piece of technology.
I believe that the operating room of the future will be a highly modular, information-richenvironment in which many different devices and subsystems will work cooperatively with thesurgeon and will also be completely integrated into the broader information infrastructure of
the hospital.
In many ways, computer-integrated health care delivery will resemble computer-integrated
manufacturing, with the same emphasis on precision, quality, and continuous process
improvement.

74

Fig 64〈 Fig 64 〉

Here is a block diagram of how the information flow might look in thisenvironment. Here are the “blue loops” of my earlier diagram, showing treatment processes
for individual patients. And here is the “red loop” showing the use of information to improvetreatments for future patients.

Fig 65〈Fig 65〉 Note that this picture stays pretty much the same, whether we are dealing withminimally-invasive robotic surgery …

75

Fig 66

Fig 67〈Fig 66, 67〉 or microsurgery or, indeed, other forms of interventions.

Fig 68〈Fig 68〉 Working in an area like this has been extremely rewarding, and the journey isreally just beginning.
One of the key lessons that I have learned is the importance of working in teams withclinicians who understand the needs and problems, with engineers from many different fields,
and with industry partners who have unique expertise and can make research available to the
public.
I have found also that taking a systems approach motivated by real applications, as well as
continued feedback and interaction among the entire team is often a key to success. Working
in teams also makes working in this area fun.

76

Fig 69〈Fig 69〉 In addition to being fun and technically interesting, working in this area is truly
rewarding in other ways. The ultimate “end user” of computer-integrated interventional
medicine is the patient, and some of the most satisfying occasions for me have been when I
have gotten to meet someone whom our technology has helped.
The emerging three-way partnership between physicians, technology, and information canenable development of novel ways of treating patients that would not otherwise be feasible,
while at the same time improving safety and treatment quality and reducing patient
morbidity.

Further, by promoting better outcomes and more-cost effective treatment

processes, we can address a serious and growing need in our society.

Fig 70〈Fig 70〉 Finally, and most importantly, I wish to thank my wife, Beverley Pederson, andmy son, Sam Taylor, who make it all worthwhile.

■ This report can be viewed in the Honda Foundation’s website.
You may not use the proceedings for the purposes other than personal use without theexpress written consent of The Honda Foundation.
77発行責任者
Editor in chief

山 本 雅 貴Masataka Yamamoto

104-0028 東京都中央区八重洲 2-6-20 ホンダ八重洲ビル
Tel. 03-3274-5125

Fax. 03-3274-5103

6-20, Yaesu 2-chome, Chuo-ku, Tokyo 104-0028 Japan
Tel. +81 3 3274-5125
Fax. +81 3 3274-5103

http://www.hondafoundation.jp