Imaging device and method

An imaging device and method which can easily obtain a curve of time-varying changes in pixel value of a region of interest, even if the region of interest moves with a subject's body motion. A controller includes an image processor executing various types of image processing on fluorescence images and visible light images. The image processor includes a pixel value measurement unit which sequentially measures values of pixels at positions corresponding to a region of interest (ROI) in the fluorescence image, a change curve creation unit which creates a curve of time-varying changes in pixel value of the ROI by sampling, among the pixel values measured by the pixel value measurement unit, a minimum pixel value within a period equal to or longer than a cycle of the subject's body motion, and a smoothing unit which smooths the curve created by the change curve creation unit.

BACKGROUND

The present disclosure relates to an imaging device and method for irradiating a fluorescent dye administered in a body of a subject with excitation light, and taking an image of fluorescence emitted from the fluorescent dye.

A technique called “near-infrared fluorescence imaging” has been used for angiography in surgery. According to the near-infrared fluorescence imaging, indocyanine green (ICG), which is a fluorescent dye, is administered to an affected area using an injector or any other suitable means. Upon receipt of near-infrared light having a wavelength of about 600 to 850 nm as excitation light, indocyanine green emits near-infrared fluorescence having a wavelength of about 750 to 900 nm. An image of the fluorescence is captured by an image sensor capable of detecting the near-infrared light, and is shown on a display unit such as a liquid crystal display panel. According to the near-infrared fluorescence imaging, blood vessels and lymphatics at the depth of about 20 mm from the body surface can be observed.

Further, attention has recently been paid to a technique of fluorescence-labeling a tumor for the purpose of surgery navigation. As a fluorescent marker used for the fluorescence-labeling of the tumor, 5-aminolevulinic acid is used. When administered to a subject, 5-aminolevulinic acid (will be hereinafter abbreviated as “5-ALA”) is metabolized by protoporphyrin IX (PpIX), which is one of the fluorescent dyes. PpIX specifically accumulates in cancer cells. When visible light having a wavelength of about 410 nm is applied to PpIX, which is a metabolite of 5-ALA, PpIX emits red visible light having a wavelength of about 630 nm as fluorescence. Thus, the cancer cells can be identified through the observation of the fluorescence from PpIX.

International Patent Publication No. 2009/139466 discloses a data collection method. In this method, an intensity distribution image of near-infrared fluorescence obtained through excitation light irradiation of a subject organ of a living body administered with indocyanine green is compared with a cancer lesion distribution image obtained through X-ray irradiation, nuclear magnetic resonance, or ultrasonography performed on the subject organ before the administration of indocyanine green. Then, data of a region which is detected in the intensity distribution image of the near-infrared fluorescence, but not in the cancer lesion distribution image is collected as data of a sub-lesion region of cancer.

In the imaging device configured to take an image of the fluorescence from the fluorescent dye injected in the body, the fluorescence from the subject and images of the subject under visible light are simultaneously recorded as a video, which is reproduced by a video recorder. Thus, according to a conventional imaging device, images taken at a predetermined frame rate are recorded and reproduced as a video, so that the courses of the blood vessels and the lymphatics after the administration of the fluorescent dye such as ICG can be observed, and a region of a cancer lesion can be identified, in a bright external lighting environment.

Such recorded data can be used not only for reference purposes, but also for obtaining new findings through analyses. For example, in a time intensity curve (TIC) analysis in which a curve of time-varying changes in signal of a region of interest (ROI) is created, time taken until the pixel value of the ROI reaches the peak is obtained so that imaging time of the fluorescent dye such as indocyanine green can be quantitatively evaluated. For example, it is advantageous to obtain a curve of time-varying changes in pixel value of the ROI for the confirmation of blood flow after coronary artery bypass graft surgery.

If a region accompanied by a body motion of a subject, such as heart, is selected as the ROI to be analyzed, correction taking the subject's body motion into account is required. For example, if cardiac muscle is selected as the ROI while cardiac vessels are moving with the pulsation during the analysis, the cardiac vessels may enter the region to be analyzed. As a result, periodical components of a cardiovascular region associated with the subject's body motion may be superimposed on the TIC.

Japanese Unexamined Patent Publication No. 2010-51729 discloses an ultrasonic diagnostic apparatus which corrects movements associated with a subject's body motion by vector calculation.

SUMMARY

For example, if a myocardial region is selected as the ROI of the TIC analysis, the cardiac vessels need to be fixed (locked). However, it is impossible to fix the whole blood vessels in every region for the vector calculation disclosed by Japanese Unexamined Patent Publication No. 2010-51729. Specifically, in the vector calculation disclosed by Japanese Unexamined Patent Publication No. 2010-51729, it takes a long time to perform the correction calculation involving spatial movement, and thus, it is impossible to fix the blood vessels in every region.

In view of the foregoing, the present disclosure has been achieved to provide an imaging device and method which can easily obtain a curve of time-varying changes in pixel value of a region of interest, even if the region of interest moves along with a body motion of a subject.

A first aspect of the present disclosure is directed to an imaging device which includes: an excitation light source which irradiates a subject with excitation light for exciting a fluorescent dye administered to the subject; a shooting unit which shoots fluorescence emitted from the fluorescent dye irradiated with the excitation light to obtain a fluorescence image; and an image storage which sequentially stores the fluorescence image that changes with a body motion of a subject. The imaging device further includes: a pixel value measurement unit which sequentially measures values of pixels at positions corresponding to a region of interest in the fluorescence image; and a change curve creation unit which creates a curve of time-varying changes in pixel value of the region of interest by sampling, among the pixel values measured by the pixel value measurement unit, a maximum pixel value within a period equal to or longer than a cycle of the subject's body motion.

A second aspect of the present disclosure is an embodiment of the first aspect. In the second aspect, the imaging device further includes a smoothing unit which smooths the curve created by the change curve creation unit.

A third aspect of the present disclosure is directed to an imaging method for sequentially obtaining a fluorescence image that changes with a body motion of a subject through irradiation of the subject with excitation light to excite a fluorescent dye administered to the subject, and shooting of fluorescence emitted from the fluorescent dye irradiated with the excitation light. The method includes: sequentially measuring values of pixels at positions corresponding to a region of interest in the fluorescence image; and creating a curve of time-varying changes in pixel value of the region of interest by sampling, among the pixel values measured by the pixel value measurement unit, a minimum pixel value within a period equal to or longer than a cycle of the subject's body motion.

A fourth aspect of the present disclosure is an embodiment of the third aspect. In the fourth aspect, the imaging method further includes smoothing the curve created by the change curve creation unit.

According to the first and third aspects of the present disclosure, a curve of time-varying changes in pixel value of a region of interest in a fluorescence image can easily be obtained, even if the region of interest moves along with a subject's body motion.

According to the second and fourth aspects of the present disclosure, a smoothed curve of time-varying changes in pixel value of a region of interest in a fluorescence image can be obtained.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in detail with reference to the drawings.FIG. 1is a perspective view illustrating an imaging device of the present disclosure.FIG. 2is a side view illustrating the imaging device of the present disclosure.FIG. 3is a plan view illustrating the imaging device of the present disclosure.

The disclosed imaging device irradiates indocyanine green, which is a fluorescent dye injected into a body of a subject, with excitation light, and shoots fluorescence emitted from indocyanine green. The imaging device includes a wagon11with four wheels13, an arm mechanism30disposed on a portion of a top surface of the wagon11toward the front in a traveling direction of the wagon11(toward the left inFIGS. 2 and 3), a lighting/shooting unit12provided for the arm mechanism30via a sub-arm41, and a monitor15. The “front in the traveling direction” of the wagon11will be hereinafter simply referred to as the “front” of the wagon11. A handle14used to move the wagon11is attached to a rear side of the wagon11in the traveling direction. A recess16is formed at the top surface of the wagon11so that a remote control used to operate the imaging device from a distance can fit therein.

The arm mechanism30is disposed on the front portion of the wagon11. The arm mechanism30includes a first arm member31which is coupled via a hinge33to a support37arranged on a column36standing upright on the front portion of the wagon11. The first arm member31is able to swing with respect to the wagon11via the column36and the support37by the action of the hinge33. The monitor15is attached to the column36.

A second arm member32is coupled to an upper end of the first arm member31via a hinge34. The second arm member32is able to swing with respect to the first arm member31by the action of the hinge34. In this configuration, the first and second arm members31and32are able to take a shooting position as indicated by reference character C and phantom lines inFIG. 2, and a standby position as indicated by reference character A and solid lines inFIGS. 1 to 3. In the shooting position, the first and second arm members31and32form a predetermined angle around the hinge34coupling the first and second arm members31and32. In the standby position, the first and second arm members31and32are adjacent to each other.

A support43is coupled to a lower end of the second arm member32via a hinge35. The support43is able to swing with respect to the second arm member32by the action of the hinge35. The support43supports a rotation axis42. The sub-arm41supporting the lighting/shooting unit12rotates about the rotation axis42disposed at a tip end of the second arm member32. Thus, through the rotation of the sub-arm41, the lighting/shooting unit12moves between a front position and a rear position with respect to the arm mechanism30in the traveling direction of the wagon11. The front position, which corresponds to the shooting position or the standby position, is indicated by reference character A and solid lines inFIGS. 1 to 3, or reference character C and phantom lines inFIG. 2. The rear position, which is a position during the movement of the wagon11, is indicated by reference character B and phantom lines inFIGS. 2 and 3.

The lighting/shooting unit12includes a camera21including a plurality of image sensors capable of capturing visible light and near-infrared light, a visible light source22disposed on the outer periphery of the camera21, and an excitation light source23disposed on the outer periphery of the visible light source22. The visible light source22emits white light (visible light). The excitation light source23emits near-infrared light having a wavelength of 810 nm as excitation light for exciting indocyanine green which is a fluorescent dye. Indocyanine green emits, as fluorescence, near-infrared light having a peak around 845 nm when irradiated with the near-infrared light having a wavelength of 810 nm.

In this embodiment, the visible light source22, the excitation light source23, and the camera21are integrated into the lighting/shooting unit12. Alternatively, the visible light source22, the excitation light source23, and the camera21may be separately arranged.

FIG. 5is a schematic view of the camera21of the lighting/shooting unit12.

The camera21includes a moving lens54which reciprocates for focusing, a wavelength selection filter53, a visible light image sensor51, and a fluorescence image sensor52. The visible light image sensor51and the fluorescence image sensor52are comprised of CMOS or CCDs. Visible light and fluorescence coaxially entering the camera21along its optical axis L pass through the moving lens54as a component of a focusing mechanism, and reach the wavelength selection filter53. The visible light that has entered coaxially together with the fluorescence is reflected by the wavelength selection filter53, and enters the visible light image sensor51. The fluorescence that has entered coaxially together with the visible light passes through the wavelength selection filter53, and enters the fluorescence image sensor52. At this time, by the action of the focusing mechanism including the moving lens54, the visible light is focused on the visible light image sensor51, while the fluorescence is focused on the fluorescence image sensor52.

FIG. 6is a block diagram illustrating a major control system of the imaging device of the present disclosure.

The imaging device includes a controller60comprised of a CPU which executes logical operations, a ROM which stores programs necessary for controlling the device, and a RAM which temporarily stores data during control. The controller60entirely controls the imaging device. The controller60includes an image processor61which executes various types of image processing on fluorescence images and visible light images. The image processor61includes a pixel value measurement unit66, a change curve creation unit67, and a smoothing unit68, as will be described later. The pixel value measurement unit66sequentially measures values of pixels at positions corresponding to a region of interest (ROI) in a fluorescence image. The change curve creation unit67creates a curve of time-varying changes in pixel value of the ROI by sampling, among the pixel values measured by the pixel value measurement unit66, the maximum pixel value within a period equal to longer than the cycle of a body motion of a subject. The smoothing unit68smooths the curve created by the change curve creation unit67.

The controller60is connected to an input unit62through which an operator enters various information items. The controller60is connected to the monitor15. The input unit62may be provided for a remote control used to operate the imaging device from a distance. If the monitor15is a touch panel, the input unit62may be shown on a screen of the monitor15, or disposed on the wagon11.

Further, the controller60is connected to the lighting/shooting unit12including the camera21, the visible light source22, and the excitation light source23. The controller60is also connected to an image storage63which sequentially stores images taken by the camera21. The image storage63includes a fluorescence image storage64which sequentially stores fluorescence images, and a visible light image storage65which sequentially stores visible light images. The fluorescence image storage64and the visible light image storage65may be replaced with a synthetic image storage which sequentially stores images obtained by synthesis (fusion) of the visible light images and the fluorescence images.

It will be described below an imaging operation using the imaging device configured as described above. For example, it will be described below the case where a TIC analysis is performed to obtain a curve of time-varying changes in pixel value of the ROI in a fluorescence image for the purpose of confirmation of blood flow after coronary artery bypass graft surgery.FIG. 7is a schematic view illustrating an image of a region of a heart91shown on the monitor15.

In this embodiment, a portion of a myocardial region of the heart91adjacent to a cardiac vessel92is selected as the ROI, and time-varying changes in pixel value of the ROI in a fluorescence image are measured. When the myocardial region is the ROI, a TIC analysis performed through observation of the time-varying changes in pixel value of the ROI in a fluorescence image is effective in confirming the blood flow after coronary artery bypass graft surgery. On the other hand, if a portion of the myocardial region adjacent to the cardiac vessel92is selected as the ROI for the analysis, the cardiac vessel92moving with the pulsation enters a region of pixel value measurement. As a result, periodical components of an image of the cardiac muscle region associated with the subject's body motion are superimposed on the TIC. Therefore, according to the imaging device of the present disclosure, among values of pixels at positions corresponding to the ROI in a fluorescence image taken by the camera21, the minimum pixel value within a period equal to or longer than the cycle of the subject's body motion, i.e., his or her pulsation, is sampled.

FIG. 8is a graph of pixel values of a region corresponding to the ROI in an image of a subject taken by the camera21.FIG. 9is a graph obtained through sampling of the graph ofFIG. 8.FIG. 10is a graph obtained through smoothing of the graph ofFIG. 9. In these graphs, a vertical axis represents the pixel value, and a horizontal axis time.

In the case where the portion of the myocardial region adjacent to the cardiac vessel92is selected as the ROI and the time-varying changes in pixel value of the ROI in a fluorescence image are measured, the pixel value measurement unit66shown inFIG. 6measures pixel values of a region corresponding to the ROI in a fluorescence image captured by the fluorescence image sensor52of the camera21. In this context, the “region corresponding to the ROI” is a region corresponding to the position of the ROI when the subject makes no body motion. The TIC analysis requires sequential measurement of the pixel values of the ROI. Actually, the position of the ROI changes with the pulsation of the subject. Therefore, in this embodiment, a region where the ROI is present when the subject makes no body motion is regarded as the region corresponding to the ROI, and the pixel values of this region are measured.

The measurement of the pixel values of the region corresponding to the ROI are continuously performed for a certain period, and the pixel values of the region corresponding to the ROI obtained in this period in every image are displayed as a curve L1shown inFIG. 8. For example, if the pixel value measurement is performed for 30 seconds at a frame rate of 60 fps (frame per second), 1800 pixel values are measured.

In such a case, as shown inFIG. 8, the curve L1of the pixel values of the region corresponding to the ROI repeatedly rises and falls at intervals equivalent to the pulsation cycle. This is because the position of the ROI changes with the pulsation of the subject and the cardiac vessel92enters the region corresponding to the ROI.

Therefore, according to the imaging device of the present disclosure, the change curve creation unit67shown inFIG. 6samples, among the pixel values measured by the pixel value measurement unit66, the minimum pixel value within a period equal to or longer than the cycle of the subject's body motion caused by his or her pulsation. Specifically, among the pixel values varying in a single cycle of the subject's body motion caused by the pulsation, the lowest pixel value is extracted. Then, the changes in the extracted pixel value are plotted as a graph.FIG. 9shows a curve L2of the changes in pixel value obtained by sampling the minimum pixel value.

Since the curve L2is obtained by sampling the minimum pixel value within the period equal to or longer than the cycle of the subject's body motion caused by his or her pulsation, influence of the entry of the cardiac vessel92due to the pulsation of the subject can be eliminated. Thus, the curve L2corresponds to a curve of time-varying changes in pixel value of the ROI.

The curve L2shown inFIG. 9has two peaks P1and P2. Of the peaks P1and P2, the peak P1indicates, for example, a state where the subject's pulsation was irregular, and the cardiac vessel92was present at all times in the region corresponding to the ROI during the period equal to or longer than the cycle of the subject's body motion caused by his or her pulsation. The peak P2indicates, for example, a state where a tool such as a surgical knife entered the region corresponding to the ROI.

Then, the smoothing unit68shown inFIG. 6smooths the curve L2ofFIG. 9corresponding to the curve of time-varying changes in pixel value of the ROI. As a result, the peaks P1and P2are excluded as shown inFIG. 10, and a curve L3, which is a curve of time-varying changes in pixel value of the ROI, can be obtained.

The “period equal to or longer than the cycle of the subject's body motion caused by his or her pulsation” is, for example, a period slightly longer than the cycle of the subject's body motion. For example, if the heart of the subject beat every second (60 times per minute), the period would be equal to or longer than one second. Setting the period longer makes it possible to obtain a curve of time-varying changes in pixel value of the ROI, which is smoothed and has no unwanted peaks, even if the smoothing described above is omitted. Therefore, in such a case, the smoothing described above can be omitted. However, setting the period longer causes a time lag in the curve of time-varying changes in pixel value of the ROI. Therefore, in a preferred embodiment, the “period equal to or longer than the cycle of the subject's body motion caused by his or her pulsation” is longer than, and shorter than the twice of, the cycle of the subject's body motion. In other words, in a preferred embodiment, the “period equal to or longer than the cycle of the subject's body motion caused by his or her pulsation” is longer than a single cycle, and shorter than two cycles, of the subject's body motion.

It has been described in the foregoing embodiment that indocyanine green is used as the fluorescent dye, and irradiated with near-infrared light of about 600 to 850 nm as the excitation light so that fluorescence in a near-infrared region having a peak around 810 nm is emitted from indocyanine green. Alternatively, light other than the near-infrared light may be used.

Further, indocyanine green used as the fluorescent dye may be replaced with other fluorescent dye such as 5-ALA mentioned above.