Patent ID: 12238268

DETAILED DESCRIPTION

Embodiments described herein provide a goggle system that includes a goggle device in communication with a computing device. The goggle device enables a user to view a subject in a plurality of imaging modes in real-time. The imaging modes include a hybrid-imaging mode that simultaneously displays pixels of image data of a first imaging mode and pixels of image data of a second imaging mode. Accordingly, a user is able to quickly and easily visualize a subject during a surgical operation.

FIG.1is a schematic diagram of an exemplary goggle system100.FIG.2is a block diagram of a goggle device102that may be used with goggle system100. Goggle device102may be worn by a user104, such as a surgeon, and aids user104in viewing a subject106, as described in detail herein. In the exemplary embodiment, goggle device102transmits and/or receives data from a computing device108. Data may be transferred between computing device108and goggle device102over a wired or wireless network.

In the exemplary embodiment, goggle device102includes a left eye assembly110that displays an image for a left eye of user104, and a right eye assembly112that displays an image for a right eye of user104. Goggle device102further includes a fastening device114, such as a strap, for securing goggle device102to user104during use.

Goggle device102enables user104to view subject106in a plurality of image modes. In the exemplary embodiment, goggle device102enables user104to view a far- or near-infrared (NIR) image of subject106, a visible light image of subject106, and a hybrid near-infrared/visible light image of subject106. Alternatively, goggle device102may enable user104to view a subject in any image mode and/or combination of image modes, including, for example, photoacoustic images, interference images, optical coherence tomography images, diffusion optical tomography images, polarization images, ultrasound images, magnetic resonance imaging (MRI) images, nuclear images (e.g., positron emission tomography (PET) images, single-photon emission computed tomography (SPECT) images), computed tomography (CT) images, gamma-imaging, and X-ray images.

A switch116on goggle device102enables user104to switch between image modes. In the exemplary embodiment, switch116is a toggle switch. Alternatively, switch116may be any type of switching device that enables goggle device102to function as described herein. For example, in one embodiment, switch116is a foot pedal, and user104activates switch116by depressing the foot pedal. In another embodiment, switch116is voice-activated.

Switch116controls which image mode is displayed on left eye assembly110and right eye assembly112. In the exemplary embodiment, switch116controls whether left eye assembly110and/or right eye112assembly displays a visible light image, a near-infrared image, or a hybrid near-infrared/visible light image. In one embodiment, left eye assembly110and right eye assembly112each have a respective switch116. Accordingly, in some embodiments, user104may view a first image mode on left eye assembly110and a second image mode on right eye assembly112.

Goggle device102further includes sources for imaging. In one embodiment, the goggle device includes a white light (i.e., visible light) source120and a near-infrared light source122. White light source120and near-infrared light source122illuminate subject106with visible light and near-infrared light, respectively. In other embodiments, different light sources may be utilized for different specific imaging modes. For example, suitable light sources may be integrated with goggle device102to enable the capture of photoacoustic images, interference images, optical coherence tomography images, diffusion optical tomography images, polarization images, far infrared images, thermal images, ultrasound images, and nuclear images (e.g., PET, SPECT, CT, gamma-imaging, X-ray).

In the exemplary embodiment, white light source120includes a plurality of white light-emitting diodes (LEDs), and near-infrared light source122includes a plurality of near-infrared LEDs for fluorescence imaging. White light source120and near-infrared light source122each provide a field of view of approximately 0.3 m in diameter at 1.0 m from the respective source. Alternatively, white light source120and near-infrared light source122, including any components that enable goggle device102to function as described herein, such as, for example, laser, laser diodes, light bulbs, or any combination of the aforementioned components. Alternatively, near-infrared light source122may include light sources in other wavelengths to enable absorption or luminescence, or fluorescence imaging in other spectral windows.

Near-infrared light emitted from near-infrared light source122excites fluorescent molecular probes present in subject106. For example, for a tumor resection operation, a molecular probe capable of fluorescent excitation is injected into a subject. The molecular probe includes a peptide sequence and a near-infrared fluorescent dye, such as indocyanine green dye, having absorption and emission maxima around 780 nm and 830 nm, respectively. After injecting the molecular probe into the subject106, the molecular probe binds to tumors. Accordingly, when near-infrared light from near-infrared light source122strikes subject106, the fluorescent dye in the molecular probe is excited. With the fluorescent dye excited, lesions such as tumors can quickly and easily be visualized in the near-infrared imaging mode of goggle device102. A similar procedure is applicable in other spectral regions.

To transfer data between goggle device102and one or more remote devices, such as computing device108, the goggle device includes a communications module130. In the exemplary embodiment, for transfer of data over a wireless network, communications module130includes a radio-frequency (RF) transmitter/receiver. Data transfer can also be accomplished through other suitable platforms such as, for example, Bluetooth, WI-FI, infrared (IR) communication, internet, 3G, 4G network, satellite, etc. Communications module130may also be transfer data to computing device108over a wired connection, such as for example, a USB video capture cable. Communications module130enables image data collected by goggles to be displayed and/or stored on computing device108. Accordingly, image data acquired by goggle device102can be viewed not only on goggle device102but also on computing device108.

Goggle device102includes a power module140that supplies power to goggle device102. In the exemplary embodiment, power module140is a battery unit that stores and provides electrical energy to goggle device102. Alternatively, power module140is any device configured to supply power to goggle device102.

To acquire image data of subject106to display on left eye assembly110and right eye assembly112, goggle device102includes a detector module150. In the exemplary embodiment, detector module150is a hybrid detector array capable of detecting both near-infrared and visible light. Detector module150is mounted on the front of goggle device102to collect image data from a direction that user104is facing. Detector module150displays received image data on left eye assembly110and right eye assembly112such that left eye assembly110and right eye assembly112display the same regions of subject106that a left eye and right eye of user104would observe in the absence of goggle device102.

FIG.3is an image of an alternative goggle device300that may be used with goggle system100(shown inFIG.1). Goggle device300includes components substantially similar to goggle device102, and like reference numerals are used herein to identify like components. Unlike goggle device102, goggle device300includes only a single eye assembly302. As such, when using goggle device300, one eye of user104views subject106through single eye assembly302, and the other eye of user104views subject uninhibited (i.e., with the naked eye).

Single eye assembly302functions substantially similar to left eye assembly110and right eye assembly112(shown inFIGS.1and2). In the exemplary embodiment, single eye assembly302covers the left eye of user104. Alternatively, single eye assembly302may cover the right eye of user104.

FIG.4is a schematic diagram of a detector element400that may be used with detector module150to collect image data for goggle device102(both shown inFIGS.1and2). Detector element400includes a plurality of differential layers402(denoted by Dn) along a differential length in a continuum. In the exemplary embodiment, each differential layer402is a p-n junction404. In the exemplary embodiment, detector element400includes eleven differential layers402. Alternatively, detector element400may include any number of differential layers402that enables detector element400to function as described herein. The longer the wavelength of the incident light, the deeper the light will penetrate detector element400. Accordingly, each layer402is configured to detect a different frequency range of incident light. For example, in one embodiment, channels D5-D8 detect blue light (i.e., light with a wavelength of approximately 440-490 nm), and the total wavelength spectrum detectable by detector element400is less than 300 nm to greater than 1300 nm. By using a continuum of differential layers402, detector element400is capable of simultaneously detecting a broad range of wavelengths of the incident light, including visible light (i.e., red, green, and blue light), near-infrared light, and various other wavelengths.

Images can be generated from the image data acquired by a selection of differential layers402. For example, to generate a visible light image, image data from differential layers402corresponding to red, blue, and green light is used. Further, images may be generated using addition, subtraction, integration, differentiation, and/or thresholding of differential layers402. Accordingly, image data acquired using detector element400may be used to generate a variety of images.

FIG.5is a schematic diagram of a portion of a detector array500that may be used with detector module150(shown inFIG.2). Detector array500includes a plurality of detector elements502. In some embodiments, each detector element502corresponds to one pixel. In the exemplary embodiment, array500includes near-infrared detector elements504and visible light detector elements506arranged in a checkerboard pattern. That is, near-infrared detector elements504and visible light detector elements506alternate in both a horizontal and vertical direction. Notably, this concept is not restricted to near-infrared detector elements or visible pixel elements. That is, for different imaging applications, near-infrared detector elements504and/or visible pixel detector elements506may be replaced detector elements in other wavelength windows, such as, for example, 500 nm-550 nm, 600 nm-630 nm, 1100 nm-1150 nm, 1200 nm-1300 nm, etc.

In the exemplary embodiment, visible light detector elements506are continuum detector elements, such as detector element400(shown inFIG.4), and near-infrared detector elements504are single-channel detector elements. Alternatively, visible light detector elements506and near-infrared detector elements504may be any type of detector element that enables detector array500to function as described herein.

FIG.6is a schematic diagram of a portion of a pixel array600displayed by left eye assembly110and right eye assembly112. Similar to the arrangement of detector array500, the pixel array600includes a plurality of pixels602, including near-infrared pixels604and visible light pixels606arranged in a checkerboard pattern. In the exemplary embodiment, detector elements502correspond to pixels602. As such, in the exemplary embodiment, to display different imaging modes on left eye assembly110and/or right eye assembly112, different combinations of detector elements602are utilized such that different combinations of near-infrared pixels604and visible light pixels606and are displayed.

For example, when switch116(shown inFIG.1) is set to a visible light imaging mode, visible light pixels506are used, and near-infrared pixels504are not used for imaging, such that visible light pixels606create a visible-light image. When switch116is set to a near-infrared imaging mode, near-infrared pixels504are used, and visible light pixels506are not used for imaging, such that near-infrared pixels604create a near-infrared image. Finally, in a hybrid imaging mode, both near-infrared detectors504and visible light detectors506are used for imaging, creating an image including near-infrared pixels604and visible light pixels606. Filtering for the different imaging modes may be accomplished by a filter and/or polarizer (neither shown).

FIGS.7A-7Care schematic diagrams of a near-infrared image702, a visible light image704, and a hybrid image706, respectively, that may be displayed on goggle device102(shown inFIGS.1and2). In near-infrared image702, the molecular probe excited by near-infrared light source122(shown inFIG.1) clearly delineates a tumor708, but the rest of a subject710is not visible. In visible light image704, subject710is visible, but tumor708is not visible. In hybrid image706, however, both tumor708and subject710are visible. Accordingly, when user104views hybrid image706on goggle device102, both tumors708and subject710are visible, enabling user104to better perform a surgical operation.

FIGS.8A-8Care a near-infrared image802, a visible light image804, and a hybrid image806, respectively, of a mouse808with tumors810. In near-infrared image802, the tumors810are visible, but the rest of mouse808is not visible. In visible light image804, mouse808is visible, but tumors810are not visible. In hybrid image806, however, both tumors810and mouse808are visible. Notably, using communications module130(shown inFIG.2), images802,804, and/or806may also be viewed on computing device108(shown inFIG.1).

FIG.9is an image of an alternative goggle device900that may be used with goggle system100(shown inFIG.1).FIG.10is a block diagram of the goggle device900shown inFIG.9. In the exemplary embodiment, goggle device900is a head-mounted display (HMD) capable of fast temporal resolution and switching between an optical see-through mode and a video see-through mode. In the optical see-through mode, user104sees the real world through half-transparent mirrors, and the half-transparent mirrors are also used to reflect computer-generated images into the eyes of user104, combining real and virtual world views. In the video see-through mode, cameras (or other suitable detection devices) capture the real-world view, and computer-generated images are electronically combined with the video representation of the real world view. As both the real and virtual world images are digital in the video see-through mode, lag between the real and virtual world images can be reduced. In the optical see-through mode, user104can visualize surroundings with natural vision. In the video see-through mode, real-time NIR fluorescence video is presented to user104with relatively high contrast. Allowing user104to switch between optical and video see-through modes simplifies surgical operations and allows user104to visualize subject106with natural vision or enhanced vision as desired.

Goggle device900includes a complementary metal-oxide-semiconductor (CMOS) imaging sensor902integrated onto a custom printed circuit board (PCB) platform (not shown). Alternatively, goggle device900may include a charge-coupled device (CCD) imaging sensor. A long-pass filter904is mounted on an objective imaging lens906. In the exemplary embodiment, long-pass filter904is an 830 nanometer (nm) filter.

A control module908interfaces between CMOS imaging sensor902and a computing device910, such as computing device108. In the exemplary embodiment, control module908includes a field-programmable gate array (FPGA) integration model with a universal serial bus (USB) communication capabilities and a laptop computer. Data received by CMOS imaging sensor902is read out in multiple stages. In the exemplary embodiment, data from CMOS imaging sensor902is read out via a state machine implemented on the FPGA, and the data is stored in a first in first out (FIFO) process and transferred to a first synchronous dynamic random-access memory (SDRAM) chip in control module908. In the exemplary embodiment, control module908includes two SDRAM chips, such that a first SDRAM chip stores pixel data from CMOS imaging sensor902, while a second SDRAM chip transfers the data to an output FIFO on the FPGA for transferring the data to computing device910via a universal serial bus (USB). In some embodiments, control module908may include a data compression chip for compressing the data.

To display information to user104, goggle device900includes an HMD unit920that interfaces with computing device910via a high-definition multimedia interface (HDMI) link to display real-time images on HMD unit920. Goggle device900further includes and NIR light source922that emits NIR light through illumination optics924and a short-pass filter926to illuminate fluorescent molecular probes (such as indocyanine green dye) in a surgical field930. Surgical field930may be, for example, a portion of subject106(shown inFIG.1). In the exemplary embodiment, NIR light source922includes four NIR LEDs, and short-pass filter926is a 775 nm filter.

A sensitivity of goggle device900to detect a fluorescence signal from surgical field930is characterized using a signal-to-noise ratio (SNR), which compares a level of the desired signal relative to a noise level. Pixel binning and temporal averaging may be used to improve SNR of goggle device900. Pixel binning involves combining signals from a group of pixels in a spatial domain, which is analogous to increasing the number of photons that contribute to the detected signal Binning improves the SNR estimate by the square root of the number of pixels binned. For instance, binning a neighborhood of 2 by 2 pixels improves the SNR by a factor of 2. However, improvement in SNR due to binning occurs at the expense of reduced spatial resolution and loss of high spatial frequency components in a final image.

Temporal averaging involves combing signals from a group of pixels in a time domain, which, like binning, is also analogous to increasing the number of photons that contribute to the detected signal. Temporal averaging increases SNR by the square root of the number of averaged pixels in the time domain. Hence, temporal averaging of four consecutive frames will increase SNR by a factor of 2. However, temporal averaging may create image lag when a moving target is imaged.

Both temporal averaging and pixel binning may be combined together to further improve SNR. For example, averaging 4 frames as well as averaging a pixel neighborhood of 2 by 2 pixels will improve SNR by a factor of 4 while reducing spatial and temporal resolution by a factor of 4. It was determined experimentally that SNR increases linearly with exposure time at a rate that depends on the concentration of a fluorescent molecular probe (e.g., indocyanine green dye). As the exposure time increases, SNR increases at the cost of a slower frame rate. Using goggle device900experimentally, sentinel lymph node mapping was performed on rats using NIR quantum dots (QDs), and fluorescence-guided liver surgery and intraoperative imaging were performed on mice. Goggle device900is capable of real-time fluorescence imaging of up to 60 frames per second (fps). Experimentally, it was determined that goggle device900detects fluorescence signals as low as 300 picomolar (pM) of indocyanine green dye. Compared to a charge-coupled device (CCD) imaging sensor, which has 20% quantum efficiency at 830 nm, CMOS imaging sensor902has a quantum efficiency of greater than 30% at 830 nm. Further, in the exemplary embodiment, goggle device900includes one or more buttons and/or switches (neither shown) for user104to select automatic or manual gain and automatic or manual exposure time.

FIG.11is a schematic diagram of a display module1101for an alternative goggle device1100that may be used with goggle system100(shown inFIG.1). Similar to goggle device900, goggle device1100is implemented in a dual-mode visual and optical see-through HMD. However, unlike goggle device900, goggle device1100provides three-dimensional (3D) imaging and display, as described herein. In one embodiment, a field of view for illumination and imaging of goggle device1100is 300 mm×240 mm at a distance between goggle device1100and subject106of 0.3 m-1.2 m.

Display module1101of goggle device1100includes a first organic light-emitting diode (OLED)1102and a second OLED1104to display images to a right eye1106and left eye1108, respectively of user104. First OLED1102emits light through a first optical assembly (e.g., imaging and/or focusing lenses)1110, and second OLED1104emits light through a second optical assembly1112.

OLED display technology provides benefits over liquid crystal display (LCD) technology as it uses approximately 80% less power than LCDs, has a nominal viewing area of approximately 160° (approximately 265% larger than LCDs), a nominal contrast ration of 10,000:1 (as compared to 60:1 for LCDs), and a significantly faster refresh rate, reducing eye fatigue and headaches. The OLED microdisplay is also more compact than its LCD counterpart because no additional illumination is needed. The proposed field of view of the display is 45°×36° with a microdisplay resolution of 1280×1024 (SXGA). In the exemplary embodiment, the pupil size of the goggle device1100is 10 mm in diameter. Off-axis design with aspherical plastic elements may be used to reduce the size and weight of goggle device1100.

To switch between optical and video see-through modes, goggle device1100includes a fast liquid crystal shutter1120positioned in front of a combiner1122. In the exemplary embodiment, combiner1122is a plastic element with 50% reflection on an inner (i.e., eye-facing) surface such that user104can see through combiner1122in optical see-through mode, and information from OLEDs1102and1104can be directed to user104in both modes.

When an external voltage is applied, fast liquid crystal shutter1120is transparent and transmits light. Without an external voltage, fast liquid crystal shutter1120blocks light from the surgical field and environment. Therefore, the goggle device1110can be switched between optical and video see-through modes easily and rapidly. In some embodiments, a switch (not shown) may be controlled by a foot paddle to enable hands-free operation. Using the video-see-through mode of goggle device1100, 3D reflectance images and fluorescence images can be registered and presented precisely. The 3D fluorescence images can also be viewed with the optical-see-through mode, while user104views the surrounding environment naturally.

FIG.12is a schematic diagram of an imaging and illumination module1150for goggle device1100. For true stereoscopic vision, imaging and illumination module1150includes two separate and identical imaging systems and one illumination system between and above the imaging systems that provide uniform NIR illumination to excite fluorescent molecular probes and visible light illumination for reflectance imaging. Each of the two imaging systems includes a CMOS detector1152and an imaging lens1154in the exemplary embodiment. CMOS detectors1152provide higher resolution and faster frame rates than CCD detectors. In the exemplary embodiment, the distance between the imaging systems is 67 mm (the average inter-pupillary distance for adults).

As shown inFIG.12, to provide a well-defined and uniform illumination region, the illumination system includes a first LED array1160and a second LED array1162. In the exemplary embodiment, first LED array1160is a high power 780 nm LED array that includes 60 diode chips and optical output power of 4 Watts (W), and second LED array1162is a high power white light LED array that includes 16 diode chips to provide uniform illumination over an area of 300×240 mm. The power of NIR excitation is approximately 3 mW/cm2in the exemplary embodiment.

Light from first and second LED arrays1160and1162is combined using a dichroic mirror1170such that the illumination area from both first and second LED arrays1160and1162overlaps. The combined light is distributed using illumination optics1172. Illumination optics1172, in the exemplary embodiment, include freeform plastic lenses (not shown) that generate uniform light distribution and an excitation filter (not shown) that blocks excitation light over 800 nm.

Each CMOS detector1152and imaging lens1154capture white light reflectance and NIR fluorescence images simultaneously. In the exemplary embodiment, each CMOS detector1152includes a sensor of vertically-stacked photodiodes and pixelated NIR/visible spectrum filters. More specifically, in the exemplary embodiment, CMOS detector1152includes an array of 2268×1512 imaging pixels, and each pixel includes three vertically-stacked photodiodes that can separate spectra of blue, green, and red-NIR light for color imaging. As each pixel includes three vertically-stacked diodes, unlike at least some known imaging systems, there is no need to interpolate between neighboring pixels. Experimentation suggests that the quantum efficiency of the vertically-stacked photodiode sensor is approximately 35% at 800 nm, which is significantly better than at least some known CCD sensors and ideal for NIR imaging. The scanning rate of CMOS detector1152may be as fast as 40 fps, and a subset of the pixel array can be read out at higher frame rates (e.g., 550 fps for 128×128 pixels).

FIG.13is a schematic diagram of a filter array1300that may be used with CMOS detector1152(shown inFIG.12). Filter array1300alternates between pixels with a visible spectrum filter1302and pixels with a NIR filter1304. Filter array1300may be created by taking a NIR filter the size of the entire array1300and selectively removing portions using a focus ion beam (FIB). Visible filters1302, which are then selectively removed parts of the larger NIR filter, will allow passage of visible spectrum light, which will be subsequently absorbed by the vertically-stacked photodiodes1306for color separation.

Thus, white light reflectance imaging can be achieved with visible pixels. On the other hand, NIR filters1304will only allow NIR light of interest (λ>820 nm) to pass, and the NIR signal will be read out from the deepest of vertically-stacked photodiodes1306. Due to the net effect of both spectrum separation mechanisms (NIR filter1304and vertically-stacked photodiodes1306), the performance of NIR fluorescence detection will be further enhanced compared to the conventional method of using a NIR filter alone. The CMOS detector1152enables co-registration of color images and NIR images on-chip while reducing the number of sensors required for 3D reflectance/fluorescence imaging. This facilitates eliminating artifacts in co-registration due to motion and minimizes the delay due to exhausting off-chip computation.

Goggle device1100includes an autofocus feature without moving parts that optimizes a focus in the exemplary embodiment. A zoom lens with a compact piezo actuator or a liquid lens with a variable, voltage-dependent focal length may be used to implement the autofocus feature. In the exemplary embodiment, image processing for goggle device1100is performed by an FPGA coupled to CMOS detectors1152and OLEDs1102and1104.

The goggle devices described herein may be used with or without contrast agents. In the absence of extrinsic contrast agents, imaging signals for endogenous fluorophores or biomolecules may be used to provide imaging contrast. At least some embodiments utilize NIR fluorescent or luminescent molecules or materials that localize selectively in a tissue of interest. As noted above, indocyanine green dye may be used as a fluorescent molecular probe with the goggle devices described herein. Other fluorescent dyes such as NIR pyrrolopyrrole cyanine dyes or luminescent materials such as quantum dots or dye-loaded nanoparticles may be utilized. However, uptake of high-affinity probes in small tumor cells may be overwhelmed by background fluorescence from normal tissue, decreasing contrast between tumor cells and background tissue. Instead, the fluorescent molecules such as dyes, or materials such as luminescent nanoparticles, could be linked to another molecule or group of molecules that will improve selective uptake in the tissues or cells of interest. For example, fluorescent molecular probes that bind selectively to protein receptors or other biomarkers overexpressed in tumor cells or target tissue may also be utilized with the goggle devices described herein.

FIGS.14A-14Dare a plurality of exemplary fluorescent molecular probes.FIG.14Ashows indocyanine green dye. Cypate, shown inFIG.14B, is a NIR cyanine dye with similar spectral properties to indocyanine green dye. Cypate, LS-276, shown inFIG.14C, and LS-288, shown inFIG.14D, are models of hydrophobic, intermediate hydrophilic, and hydrophilic dyes, respectively.

An example of a tumor-targeted molecular probe is LS-301, which has the structure, Cypate-cyclo (D-Cys-Gly-Arg-Asp-Ser-Pro-Cys)-Lys-OH. The spectral properties of LS-301 are suitable for NIR imaging applications (e.g., excitation/emission 790/810 nm in 20% DMSO solution; fluorescence quantum yield (ψ) 0.1 referenced to ICG). The ABIR binding affinity for LS-301 is Ki=26.2±0.4 nM relative to reference cypate-labeled RGD peptide (Ki=1.2±0.7 nM). In addition to conferring tumor selectivity on LS-301, the unnatural D-cysteine on the peptide moiety confers higher stability because of its resistance to degradation by proteases. This biological stability in serum allows initial imaging and potential follow-up surgery to be conducted within 48 hours before subsequent hydrolysis of the probe through L-amino acid residues.

Experimentally, NIR fluorescence microscopy of LS-301 in diverse tumor cells showed punctate intracellular fluorescence typical of receptor-mediated endocytosis and barely visible uptake in non-tumor cells. This uptake was successfully inhibited with unlabeled cyclic (RGDFV) reference peptide in A549 tumor cells, demonstrating the versatility of the imaging probe in detecting tumors relative to non-tumor cells.

Hydrophobic dyes, such as cypate, bind to albumin and other proteins. The high binding constant decreases their bioavailability for the target tumors and prolongs the blood circulation time, thereby increasing background fluorescence at early imaging time points. In contrast, more hydrophilic dyes and their peptide conjugates rapidly extravasate into tissues, quickly removing the probe from circulation. Although the hydrophilic probes are suitable for image-guided surgery because of the fast clearance, the contrast between tumor and surrounding tissue also depends on having sufficient time for molecular interaction between the target tumor proteins and the molecular probe. Thus, the low background signal obtained may be offset by the low signal at the tumor site. Experimental data suggest that LS-276 dye (shown inFIG.14C) will bridge the gap between rapid and delayed blood clearance, which affects the bioavailability of the probes to a target tissue.

Due to a direct linkage of a carboxylic acid with a phenyl group in LS-276, LS-276 may have relatively low reactivity with peptides and proteins, resulting in multiple side products that are difficult to remove. Accordingly, in some embodiments, a fluorophore based on a benzyl instead of the current phenyl carboxylic acid used for LS-276 may be utilized. Since the pure compound is a solid, re-crystallization methods may be used where ethyl acetate and chloroform mixtures are used to precipitate the dye in >99% HPLC/HRMS purity.FIG.15is a schematic diagram of the synthesis of reactive benzylcarboxylic acid for solid-phase labeling of LS-301 peptide. Using this derivative of LS-276 may double the quantum yield of cypate used in LS-301. In some embodiments, the method may produce the desired compound in high yield (>70%) and purity (>99%). The method is also scalable, with the potential to produce up to 10 grams of compound.

In some embodiments, the LS-301 peptide may be slightly altered to assess further improvements in tumor selectivity (e.g., cyclo(DCys-Gly-Arg-Asp-Ser-Pro-DCys)-Lys-OH, cyclo(Cys-Gly-Arg-Asp-Ser-Pro-Cys)-Lys-OH, cyclo(Cys-Arg-Gly-Asp-Ser-Pro-Cys)-Lys-OH, cyclo(DCys-Arg-Gly-Asp-Ser-Pro-Cys)-Lys-OH, and cyclo(DCys-Arg-Gly-Asp-Ser-Pro-DCys)-Lys-OH). These peptides are labeled with dye 2 at the N-terminus.

The goggle devices and fluorescent probes described herein may be implemented in a plurality of surgical settings, including, but not limited to, detecting tumors related to breast cancer in mice, adenocarcinoma in canines, and hepatocellular carcinoma (HCC) in humans.

The goggle devices and fluorescent probes described herein assist in identifying tumor boundaries and performing biopsies. The goggle devices and fluorescent probes described herein are also applicable to other surgical interventions such as cardiac surgery and assessing wound healing.

In one example, goggle device102(shown inFIG.1) and the fluorescent probe LS-301 described herein were utilized in mapping positive lymph nodes (PLNs) in mice. Based on the information provided by non-invasive fluorescence imaging using goggle device102and LS-301, regions of interest that might contain PLNs were explored and examined PLNs were identified and resected under image guidance. The presence of cancerous tissue in the PLNs was confirmed by bioluminescence imaging. This verifies the feasibility of the non-invasive fluorescence imaging, allowing clinicians to rapidly stage cancer non-invasively. Using goggle device102and fluorescent probe LS-301 non-invasively can provide a first-line screening that provides general information in an operating room in real-time. Further, the fluorescence signals monitored in PLNs might be used as an indicator of the efficacy of treatment regimens such as radiotherapy, chemotherapy, and targeted therapy.

In another example, a multimodal detection technique was implemented in which goggle-aided fluorescence imaging (e.g., using goggle device102(shown inFIG.1) and indocyanine green dye) was combined with ultrasound imaging and standard histology. Specifically, fluorescence imaging was used to detect liver tumors in mice and to perform liver resections on the tumors. In addition to single tumors, scattered satellite lesions were also detected. Further, liver resection was performed on a rabbit using ultrasound, fluorescence imaging, and standard histology. The presence of tumors in the rabbit was confirmed by ultrasound and then observed in real-time using fluorescence goggle system100(shown inFIG.1). The excised tissues were later examined by histopathology, and it was confirmed that the tumors were cancerous.

In another example, goggle device102(shown inFIG.1) and indocyanine green dye were used to image hepatocellular carcinoma (HCC) in human patients. Both intravenous and transarterial hepatic (TAH) delivery of indocyanine green dye were used. Primary tumors and satellite tumors were both detected using fluorescence imaging, some of which were not identified in pre-operative MRI and CT images or by visual inspection and palpation. Histologic validation was used to confirm HCC in the patients. The HCC-to-liver fluorescence contrast detected by goggle device102was significantly higher in patients that received TAH delivery instead of intravenous delivery.

The systems and methods described herein provide a goggle device in communication with a computing device. The goggle device enables a user to view a subject in a plurality of imaging modes in real-time. The imaging modes include a hybrid-imaging mode that simultaneously captures and displays pixels of image data of a first imaging mode and pixels of image data of a second imaging mode. Accordingly, a user is able to quickly and easily visualize a subject during a surgical operation.

Notably, the goggle system and goggle device described herein may be utilized in a broad variety of medical applications, including small animal imaging, veterinary medicine, human clinical applications, endoscopic applications, laparoscopic applications, dental applications, cardiovascular imaging, imaging inflammations, wound healing, etc. Further, the goggle system and goggle device described herein may be used in other imaging applications outside of the medical field.

The order of execution or performance of the operations in the embodiments of the disclosure illustrated and described herein is not essential unless otherwise specified. That is, the operations may be performed in any order, unless otherwise specified, and embodiments of the disclosure may include additional or fewer operations than those disclosed herein. For example, it is contemplated that executing or performing a particular operation before, contemporaneously with, or after another operation is within the scope of aspects of the disclosure.

When introducing elements of aspects of the disclosure or embodiments thereof, the articles “a,” “an,” “the,” and “said” are intended to mean that there are one or more of the elements. The terms “comprising,” including,” and “having” are intended to be inclusive and mean that there may be additional elements other than the listed elements.

This written description uses examples to disclose the disclosure, including the best mode, and also to enable any person skilled in the art to practice the disclosure, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the disclosure is defined by the claims and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims or if they include equivalent structural elements with insubstantial differences from the literal language of the claims.