Source: https://patents.google.com/patent/US10139613B2/en
Timestamp: 2019-08-21 13:44:39
Document Index: 429295391

Matched Legal Cases: ['Application No. 61', 'art 730', 'Application No. 201480021837', 'Application No. 15154503', 'Application No. 14198636', 'Application No. 14784707', 'Application No. 2908058', 'Application No. 2808105', 'Application No. 2908058', 'Application No. 2016', 'Application No. 2014800218372', 'Application No. 15194968', 'Application No. 201180047558', 'Application No. 201453889', 'Application No. 2014800218372', 'Application No. 201180047558', 'Application No. 201080017649', 'Application No. 201410415253', 'Application No. 14784707', 'Application No. 17202516', 'Application No. 2013', 'Application No. 15154503', 'Application No. 2016']

US10139613B2 - Digital microscope and method of sensing an image of a tissue sample - Google Patents
Digital microscope and method of sensing an image of a tissue sample Download PDF
US10139613B2
US10139613B2 US13/212,955 US201113212955A US10139613B2 US 10139613 B2 US10139613 B2 US 10139613B2 US 201113212955 A US201113212955 A US 201113212955A US 10139613 B2 US10139613 B2 US 10139613B2
US13/212,955
US20120044342A1 (en
Sven Hensler
2010-08-20 Priority to US37570310P priority Critical
2011-08-18 Application filed by Sakura Finetek USA Inc filed Critical Sakura Finetek USA Inc
2011-08-18 Priority to US13/212,955 priority patent/US10139613B2/en
2011-08-25 Assigned to SAKURA FINETEK U.S.A., INC. reassignment SAKURA FINETEK U.S.A., INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ROMER, CHRISTIAN, HENSLER, SVEN, HING, PAUL
2012-02-23 Publication of US20120044342A1 publication Critical patent/US20120044342A1/en
2018-11-27 Publication of US10139613B2 publication Critical patent/US10139613B2/en
A method including sensing an image without magnification of a portion of a tissue sample; displaying the sensed image on a display; and performing at least one of the following: refreshing a sensed image at a predetermined rate, storing the sensed image, modifying the sensor to display pixel ratio, and sensing a magnified view of an area of the portion of the tissue sample. A digital microscope comprising: at least one image sensor; a first optic configured to project an image with a magnification of one or less; a second optic disposed between the at least one sensor and the stage, the second optic configured to project an image with a magnification greater than one; and a computer operable to direct an image capture by the at least one image sensor of a portion of a microslide on the stage projected through the first optic or the second optic.
The application is Non-Provisional Application and claims the benefit of the earlier filing date of U.S. Provisional Patent Application No. 61/375,703, filed Aug. 20, 2010 and incorporated herein by reference.
Digital microscope.
In various settings, examination of biological specimens is required for diagnostic purposes. Generally speaking, pathologists and other diagnosticians collect and study samples from patients, utilize microscopic examination, and other devices to assess the samples at cellular levels. Numerous steps typically are involved in pathology and other diagnostic processes, including the collection of biological samples such as blood and tissue, processing the samples, preparation of microscope slides, staining, examination, re-testing or re-staining, collecting additional samples, re-examination of samples, and ultimately the offering of diagnostic findings.
The examination of a biological sample generally involves magnification of the sample or region of interest of the sample and an assessment by a pathologist or diagnostician. Traditionally, this is done by placing a slide containing a sample on a microscope and examining a magnified view of the tissue sample or region of interest of the tissue sample through a microscope. Recently, digital microscopes have been developed wherein a sample, particularly a sample on a microscope slide is placed in an instrument and a magnified digital image of the sample or region of interest of the sample is captured and displayed on a monitor such as thin film liquid crystal display monitor. While being able to view a sample or origin of interest of a sample on a display rather than through a lens of a microscope may be beneficial to the pathologist or other diagnosticians, often the time it takes to scan a magnified image and display that image presents an inconvenient delay or a significant delay when multiple samples need to be processed (magnified).
FIG. 1 is a block diagram of a system including a digital microscope for examining a sample.
FIG. 2 is a top perspective view of an embodiment of a slide carrier for use in a digital microscope system.
FIG. 3 is a cross-sectional side view through line 3-3′ of FIG. 2.
FIG. 4 shows a top view of an embodiment of a slide having a sample and a label thereon and indicating a field of view of a sensor of an embodiment of a digital microscope.
FIG. 5 shows a representation of three adjacent images of a sample each captured by a sensor of a digital microscope.
FIG. 6 shows the three adjacent images of FIG. 5 following their stitching together.
FIG. 7 shows an image of a slide including the three adjacent images of FIG. 5 stitched together and a slide label.
FIG. 8 describes a flow chart of an embodiment of image capture by a system including a digital microscope.
FIG. 9 is an embodiment of a screen shot of a display showing an embodiment of different modes of use of a system including a digital microscope.
FIG. 10 is an embodiment of a screen shot of a display showing low-resolution images of four slides and their labels.
FIG. 11 is an embodiment of a screen shot of a display showing a portion of a sample in a live mode.
FIG. 12 is an embodiment of a screen shot of a display showing a portion of three different samples in a live mode.
FIG. 13 is an embodiment of a screen shot of a display showing a portion of a single sample during a high-resolution scan.
FIG. 14 is an embodiment of a screen shot of a display showing a portion of a single sample during a high-resolution scan.
FIG. 1 shows a block diagram of a system for examining a sample such as a tissue sample. Referring to FIG. 1, system 100 includes computer 110. Computer 110 includes, for example, a central processing unit that in one embodiment is an Intel Core 2 Quad or better having at least 4 gigabytes of random access memory and at least one terabyte hard drive memory. Computer 110 also includes a DVD writer and an operating system such as Windows 7.
Connected to computer 110 is display 120 configured to display information transmitted from computer 110. Display 120 is, for example, a thin film liquid crystal display monitor that is S-IPS or PVA technology based. A 24-inch or greater color monitor is a representative example. Alternatively, two, three or more displays can be connected to computer 110 to provide a user with more information in a more structured way. For example, one display may provide the image of a sample having a hematoxylin and eosin (H&E) stain, while another one shows images of the same case using a different type of staining method, and a third may show clinical data from other disciplines such as Clinical Chemistry, Hematology, Radiology. Also connected to computer 110 is keyboard 130, mouse 1400A and mouse 1400B. In one embodiment, mouse 1400A is a two-dimensional conventional mouse and mouse 1400B is a three-dimensional mouse such as the 3DConnexion Space Navigator™. Three-dimensional mouse 1400B may, for example, be used to position or navigate the environment and mouse 1400A may, for example, be used to select, create or edit.
Computer 110 may have internet and/or internet connection 145 to allow remote transmission operation of system 100 and/or to connect to a network operating system.
Connected to computer 110 in this embodiment of system 100 is digital microscope 150. Digital microscope 150 may include one or several imaging systems including sensor 160 and sensor 165, optical imaging sub-system 168 and optical imaging sub-system 170, autofocus optics and illuminations. Each imaging system may have a different optical resolution or range of resolution. At least one optical system may reach magnification, m≤1. The system may also provide high resolution with magnifications, m>1. The digital microscope also includes stage 180 translatable in x-, y- and z-directions and control electronics.
Digital microscope 150 may be operated as a bright field and/or fluorescence microscope. In the case of a bright field operation, a sensor or sensors sense(s) the absorption of the sample and captures an image of the sample on the stage 180 with a light source on the opposite side of sensor(s) with respect to the sample. As shown in FIG. 1, light source 195 and light source 196, each, for example, a light emitting diode (LED) light source, are positioned below stage 180. An opening in stage 180 allows light to be emitted through stage 180 to illuminate a sample, such as a slide on stage 180. In the case of a fluorescence microscope operation, the imaging system images fluorescence markers which have been excited by a fluorescence illumination light source. The fluorescence light is typically coupled into the optical system via a dichroic mirror between the infinity corrected microscope objective and a tube lens. In such case, both the sensor and the illumination light source are on the same side of the sample. Referring to optical imaging sub-system 168 and sensor 160, in one embodiment, sensor 160 comprises a commercially available digital camera with an area sensor, for example, a charge-coupled device (CCD). CCD sensors are subdivided in several million photo-sensitive square units (pixels) which describe the resolution of the sensor. A typical pixel size (sensor resolution) of such a sensor is about 5 microns (μm)×5 microns. The size which a pixel represents using the magnification of the optical system on the sample is commonly referred to as the pixel resolution. Using an optical system with a magnification 0.1<m<40, results in a pixel resolution of approximately 50 microns to 125 nanometers.
In one embodiment of an imaging system, sensor 160 is configured to sense and capture an image of a sample, such as an image of a slide or a portion of a slide on stage 180. Optical imaging sub-system 168 in digital microscope 150 includes lens or objective 1680 that focuses light from light source 195 of an illumination sub-system on sensor 60. Light from light source 195 is emitted through an opening in stage 180, through a slide on stage 180. Mirror 1682 of optical imaging sub-system 168 directs the light to lens or objective 1680. Sensor 160 may capture such image by sensing the image, without magnification (m=1) or a magnification less than one (m<1) through optical imaging sub-system 168. In one embodiment, a location of optical imaging sub-system 168 and sensor are fixed. Mirror 1682 may be moved in an x- and a y-direction by xy-stepper motor 172 and in a z-direction by z-stepper motor 174.
Computer 110 receives signals representative of a sensed image from sensor 160 and generates an image for display and displays such generated image on display 120.
In one embodiment, sensor 165 is similar to sensor 160. Sensor 165 is configured to capture an image of a sample, such as an image of a slide or a portion of a slide on stage 180. Sensor 165 captures such image through optical imaging system 170 (m>1). Optical imaging sub-system 170 in digital microscope 150 may include multiple objectives. Objective 1700A, objective 1700B, and objective 1700C are shown. Objective 1700A is, for example, an infinity corrected type from Carl Zeiss having a magnification of 2.5×. Objective 1700B is, for example, an infinity corrected type from Carl Zeiss, having a 20 times (20×) magnification. Objective 1700C is, for example, a Carl Zeiss A-plan objective, having a 40 times (40×) magnification. Interchanging these objectives yield different optical systems, where each system results in a different pixel resolution (e.g., two microns for magnification 2.5 and 250 nanometers for magnification 20). As needed, other objectives can be substituted or more objectives can be added. The individual objectives are movable in an x- and a y-direction by xy-stepper motor 172 allowing a particular objective to be associated with sensor 165 and a slide on stage 180 when desired. Representatively, the objectives and mirror 1682 may individually be connected to a track and are motorized to move along the track and actuated into position when desired.
Disposed between sensor 165 and optical imaging sub-system 170, in one embodiment, may be automatic focusing system 167 including a beam splitter, an autofocus detector and autofocus illumination. An infrared filter and tube lens may also be disposed between sensor 165 and optical imaging sub-system 170, such as between automatic focusing system 167 and sensor 165.
In one embodiment, when capturing images through optical imaging sub-system 170, microscope 150 uses light source 196 of illumination sub-system positioned beneath stage 180. Light source 196 may be similar to light source 195. Associated with light source 196 (i.e., disposed between light source 196 and stage 180) and included in the illumination sub-system are motorized apertures or diaphragms 197 providing Köhler illumination that improves the specimen illumination.
Computer 110 receives signals representative of a sensed image from sensor 165 and generates an image for display. The generated image is displayed on display 120.
In the above-described embodiment, multiple sensors (sensor 160, sensor 165) are described to capture an image of a sample on a slide. In another embodiment of an imaging system, system 100 includes a single sensor configured to capture an image of a slide or a portion of a slide without magnification (m=1) or with a magnification less than one (m<1) and to capture an image or portion of an image through magnifying optical imaging sub-system 170 (m>1). In this embodiment, a single sensor may be utilized in connection with interchangeable optics (e.g., optical imaging sub-systems 168 and 170). Similarly, in the above embodiment, instead of light source 195 for optical imaging system 168 and light source 196 for optical imaging system 170, a single light source may be used for each imaging system.
In one embodiment, digital microscope 150 includes control unit 175. Control unit 175 is connected to computer 110. Control unit 175 is connected to computer 110. Control unit 175 is also connected to the various components of digital microscope 150 to control an operation of digital microscope based on signals received from computer 110. Control unit representatively controls xy-stepper motor, z-stepper motor, light source 185, light source 196, motorized apertures or diaphragms 187, optical imaging sub-system 168, optical imaging sub-system 170, sensor 160 and sensor 165.
Referring to digital microscope 150 operated as a bright field microscope, in one embodiment, where a tissue sample is a slide having a label with patient identification information and/or other information, including, for example, type of stain or process to which the sample was subjected, printed on the label, digital microscope 150 can sense and capture such information. Using a light source beneath a label to illuminate the label, however, may not make the label information visible. Accordingly, in one embodiment, second light source 198 is utilized to illuminate the slide or a label portion of the slide so that the data on a slide label may be sensed via reflectance. An image of the label may be captured with, for example, sensor 160 and optical imaging sub-system 168.
Referring again to FIG. 1, stage 180 is manipulated in three directions by control unit 175: x-direction (side to side as viewed), z-direction (up and down as viewed) and y-direction (into and out of page as viewed). The z-direction may also be accomplished by moving the optics with respect to the sample. Stage 180 is manipulated in an x-direction and a y-direction by xy-stepper motor 172 and in a z-direction by z-stepper motor controlled by control unit 175.
Referring again to digital microscope 150 of system 100, the microscope includes stage 180. In one embodiment, stage 180 is sized to handle one or more slides. In one embodiment, a slide tray including four slides may be contained on stage 180. FIG. 1 shows slide carrier 210. In another embodiment, a slide loader can be attached to system 100, which allows for the automatic loading and unloading of up to about 240 slides. The slide loader allows system 100 to perform slide imaging automatically, whether a user is present or not in a reflex imaging mode. Users can select to either use the automatic slide loader or the slide tray.
FIG. 2 shows a representative example of slide carrier 210 on stage 180 within microscope 150. Slide carrier 210 is, for example, a molded polymer material that includes four slide cavities 220A, 220B, 220C and 220D each for retaining an individual slide (e.g., a 25 millimeter×76 millimeter slide). Three slide cavities (cavity 220A, 220B and 200C), in the illustration of FIG. 2, contain a slide while fourth cavity (cavity 220D) is empty. Slides may be placed in respective slide cavities by a user or automatically by, for example, a mechanism associated with a slide loader (e.g., a pick and place robotic instrument). In one embodiment, a slide is not mechanically restrained by slide carrier 210 but instead rest partially within a slide cavity.
FIG. 3 shows a side view of slide carrier 210 on stage 180 through line 3-3′ of FIG. 2. In this embodiment, slides 320A, 320B and 320C are shown in slide cavities 220A, 220B and 220C, respectively. No slide is shown in slide cavity 220D. Each slide cavity includes a cavity (cut out) portion and a plateau portion (330A, 330B, 330C and 330D). A slide rests horizontally on a plateau portion. Each plateau portion has a height dimension such that when a slide rests on an upper surface of the plateau portion (as viewed), a portion of a thickness of the slide extends a distance 315 above a height dimension of slide carrier 210 (as viewed).
Referring to FIG. 2 and FIG. 3, slide carrier 210 may be seen to be within a pocket or holder of stage 180. Stage 180, in one embodiment, is a molded plastic piece having a size to support slide carrier 210 within digital microscope 150. The pocket or holder of stage 180 is formed by opposing L-shaped brackets protruding from surface 181 (a top surface as viewed) of stage 180. FIG. 2 shows bracket 183 and bracket 184, each having an inverted L shape and opposing one another (with a base of the inverted L or cantilevered portion facing toward the opposing bracket). Bracket 183 and bracket 184 are separated by a distance at least greater than a width dimension of slide holder 210. (For example, if slide holder 210 has a width dimension on the order of 10 centimeters (cm) to 12 cm, bracket 183 and bracket 184 are separated by that distance plus 0.25 cm-0.5 cm greater.) Each bracket extends a height dimension from surface 181 that is greater than a thickness of slide holder plus a distance 315 to which a slide in a slide cavity of slide holder 210 protrudes beyond surface 181 of slide holder 210. For example, if slide holder has a thickness on the order of 1 cm, a base of the cantilevered or inverted L portion of each of bracket 183 and bracket 184 is at a distance from surface 181 of stage 180 of 1 cm plus more than a distance 315. For example, if distance 315 is 1 mm, the cantilevered portion of each of bracket 183 and bracket 184 is at a distance from surface 181 of stage 180 of 1.2 cm or more. An orientation and configuration of bracket 183 and bracket 184 of stage 180 allow slide holder 210 to be guided in and out of the pocket formed by the brackets.
Referring to FIG. 3, in one embodiment, stage 180 includes hump 188 on surface 181 at a distal end of the pocket formed by bracket 183 and bracket 184 (distal from a point where slide holder 210 enters the pocket). Hump 188 has a dimension sufficient to elevate stage holder 210 from surface 181 of stage 180 and bring any slides in slide cavities of slide holder 210 into contact with cantilevered portions of bracket 183 and bracket 184. In this manner, cantilevered portions of bracket 183 and bracket 184 act to secure or support a position of a slide when slide holder 210 is on stage 180. Representatively, hump has a thickness or height dimension on the order of a few millimeters and a length dimension of 0.5 cm, and a width that extends between bracket 183 and bracket 184. Alternatively, two or more smaller width dimensioned bumps may be used.
In operation, digital microscope 150 uses one of sensors 160 and 165 to sense and capture images of a sample or a region of interest of a sample on the slide. The sensor captures slide images of the sample and transmits those images in digital signals to computer 110 and such signals are displayed on display 120. In one embodiment, when capturing an image and displaying that image on display 120, it may not be desirable to store the image in the sense that it could be retrieved in the future. Instead, the image is transmitted from sensor 160 or sensor 165 to computer 110 and absent some instruction from a user or from the system to take another action, the images are refreshed representatively at a refresh rate on the order of several images per second. The refreshing rate may vary. If there is no action of the microscope, for example, there is no need to refresh the image.
In one embodiment, sensor 160 captures an image of a sample on a slide with a magnification of one or less (m≤1). In other words, where a magnification is less than one (m<1), optical imaging sub-system 168 project an unmagnified or a demagnified image of the sample on the sensor. Representatively, sensor 160 is smaller than a slide (e.g., a sensor is approximately 3 to 4 millimeters in diameter while a slide is approximately 25 millimeters by 76 millimeters). Optical imaging sub-system 168 includes an objective that projects a larger field of view on sensor 160.
In one embodiment, system 100 creates an overview image of an entire sample on a slide or a portion of an entire sample. The overview image is an image captured without magnification as described above (i.e., a magnification of one or less than one). An advantage to capturing a overview image is the speed at which it can be captured. For example, an image of a full slide can be captured on the order of one to two seconds while capturing a magnified image may take on the order of 20 seconds or more.
As noted above, a sensor is smaller than a slide and typically smaller than a sample or a portion of a sample on the slide. In order to obtain an acceptable resolution of an image, such as an overview image, an area that individual pixels of the sensor represent is reduced. In one embodiment, to obtain an acceptable overview image of a sample on the slide, sensor 160 will take multiple images and stitch those images together. For example, in one embodiment, a slide or an image sample such as an entire sample on a slide is divided into thirds with a sensor capturing light through a third of the desired area for the overview image (e.g., a third of the usable area of a slide). To coordinate the capture of light representative of a third of a sample, stage 180, in one embodiment, is moved to a desired portion within the field of view of sensor 160.
Referring to FIG. 2, stage 180 is translatable in an x- and y-direction. Sensor 160 remains stationary. In one embodiment, stage 180 is translated in response to signals from control unit 175 to xy-stepper motor 172.
In one embodiment, microscope 150 and system 100 is calibrated using a reference slide carrier such that a nominal position of slides are known within a defined tolerance. The defined tolerance is a result of the xz coordinate system of stage 180 (±p); the mechanical tolerances of slide carrier 210 and its position when inserted into microscope 150 (±q); and the mechanical tolerances of slide cavities in slide carrier 210 that accept slides (±r). The defined tolerance considering these factors is p+q+r. An overview image of a slide, in one embodiment, consists of three overlapping images with the field of view of each image and overlap selected to accommodate the defined tolerance and to collectively capture an image of the entire slide. In other words, since the images obtained by sensor 160 will be stitched, in one embodiment, stage 180 is translated from one field of view of sensor 160 to a different field of view such that, at the different field of view, there is an overlap of another field of view (e.g., a previously imaged field of view). In one embodiment, the overlap is at least 20 pixels more than a maximum tolerance of the stage, slide carrier and cavities within the slide carrier (e.g., 20 to 50 pixels).
FIG. 4 demonstrates an image capture of an image or a portion of an image. Referring to FIG. 4, stage 180 is positioned so that slide 220A is within the field of view of sensor 160. Slide 220A is positioned so that a first image captured by sensor 160 will be at one edge of a slide. For example, assuming a slide is approximately 76 millimeters long with a slide label occupying approximately 16 millimeters of that length at one end, the remaining approximately 60 millimeters of the slide will constitute the usable area of the slide (i.e., an area where a sample could be located). Sensor 160 will be positioned at an opposite end such that it captures the first 20 to 25 millimeters length of the slide from that end. An edge of a slide or an edge of a slide label on a slide may provide an x-coordinate and a y-coordinate. Stage 180 can use these coordinates to establish a position to capture the first 20 to 25 millimeters of the slide from the slide label. In stitching images together, the first image may be considered a fixed reference to which other images are stitched to form the overview image. At a first portion of an image, FIG. 4 shows sensor 160 capturing a portion of slide 220A designated in area 310 as a first or Time1 image. Stage 180 then moves in an x-direction approximately 20 millimeters and sensor 160 captures a second image at Time2 represented by area 320. Finally, stage 180 moves to a third position in area 330 within the field of view of sensor 160 and sensor captures an image of area 330 at Time3.
In the description of image capture of adjacent images, in one embodiment, the system requires overlap of the captured images. Overlap is shown in FIG. 5 where area 420 overlaps a portion of area 410 and area 430 overlaps a portion of area 420. The overlap is put to beneficial use when the images are stitched together. In one embodiment, the system strives for approximately 20 to 50 pixels overlap between adjacent images. Following the capture of adjacent images, the adjacent images are assembled or stitched together.
FIGS. 5 and 6 illustrate an embodiment of stitching. In one embodiment, the system uses landmarks of the sample and seeks to align common landmarks to form an overall image. FIGS. 5 and 6 show simple geometric landmarks or features to represent this concept. FIG. 5 shows image represented by area 410 and image represented by area 420 are put together using the geometric symbols as landmarks or features common to adjacent areas. In this example, the image represented by area 410 and the image represented by area 420 need to be moved along a single axis for alignment. System 100, however, also allows more alignment possibilities, such as along two axes and rotation. For example, the image represented by area 430 is shown offset in the y-direction relative to the image represented by area 420. Thus, the image represented by area 480 may be moved in two directions (x-direction, y-direction) to align identified common landmarks or features between the images.
As noted above, in one embodiment, the stitching of portions of an image to gather the overview image is performed as the images are captured. Although FIG. 5 shows three separate images, in one embodiment, in the example with the respect to, the image represented by area 410 and the image represented by area 420 are stitched together immediately upon capturing of each image. As the image represented by area 430 is captured, that image is aligned and stitched to the combined image represented by area 410 and area 420. Images of the individual areas are not saved. Instead, once a sufficiently large area of an image of the sample is stitched and assembled, the assembled area is broken into similarly sized fractions and saved in a file structure or may be compressed using a compression format (e.g., JPEG). The saved image area is deleted from random access memory (RAM) of computer 110.
System 100 can establish approximately where a sample is on a slide as well as a location of significant features based on the field of view images used to assemble an overview image. For example, each pixel of an overview image represents a specific area, e.g., 5.4 μm×5.4 μm. Further, the assembled image is represented by the pixel number of sensor 160 which can be described in terms of x- and y-coordinates, e.g., 2504 x-direction pixels by 3324 y-direction pixels. With this information, a selection by mouse 1400A or mouse 1400B of a position in the overview image is a selection of a pixel or pixels in that image. Since a size of each pixel is known, system 100 can determine the number of pixels in an x-direction and a y-direction that a selected position (e.g., a position of a feature of the image) is located relative to a starting position, e.g., an edge of the image. An approximate coordinate system for the image may thus be established such that system 100 can identify a location of an area including a feature represented by a particular pixel or pixels.
As noted above, generally a slide has a label fixed to a surface. In one embodiment, it is desirous to have an overview image including not only of the sample on the slide but also the slide label. Because a slide label will obstruct light introduced in digital microscope 150 from below the slide, digital microscope 150 includes sensor 198 that captures an image of the slide label in reflectance. FIG. 7 shows label 450 that may be sized and manipulated (e.g., rotated) so that when assembled with the overview image taken from the slide field sensor (sensor 160), label 450 may be adjacent to the image. FIG. 7 shows an overview image 440 of stitched together portions of the image (a portion represented by area 410, a portion represented by area 420 and a portion represented by area 430) to create a full stitched together image of the active portion of the slide. FIG. 7 also shows an image of label 450 adjacent the overview image of the sample. As noted above, individual portions of the sample are not saved, only the composite stitched slide image (e.g., an overview image). In one embodiment, the stitched slide image is saved separate from the label image. In another embodiment, the stitched slide image is saved with the label image stitched adjacent to it. In either situation, the composite image may be saved with conventional compression software (e.g., JPEG).
Where sensor 160 and optical imaging sub-system 168 are positioned over a sample on a slide to capture an image of that slide, a user of system 100 can electronically “zoom” in to increase the resolution. In one embodiment, system 100 initially captures an overview image of a sample on a slide (e.g., an overview image of the entire sample) with sensor 160 and an image of the slide label. The initial overview image may be stitched together as described above. In one embodiment, the initial image is displayed at a relatively larger sensor to pixel ratio (multiple pixels of sensor 160 mapped to a pixel of display 120). It is appreciated that pixels on display 120 are generally larger than on sensor 160. For example, pixels on sensor 160 have a size on the order of five microns while a pixel size of display 120 are on the order of 0.5 millimeters.
In one example, the initial overview image is displayed at a sensor to display pixel ratio of four to one or greater. A user then uses mouse 1400A to select a region of interest in a sample. The user can then zoom in to increase the image resolution at that particular point or that particular region of interest and/or increase the magnification. Representatively, to zoom in electronically at a selected region of interest (selected by mouse 1400A and located by system 100 as described above), a user directs mouse 1400B to zoom in and, in response, system 100 will modify a sensor to display pixel ratio from, for example, four to one toward one to one or more (i.e., map fewer sensor pixels to individual display pixels). It is appreciated that as the individual sensor pixels are mapped to more display pixels, the image will appear to the user as being magnified as the display area of the region of interest on display 120 increases. A user can accept and save the image at any desired ratio of sensor pixel to display pixel.
At some point, a threshold resolution will be reached (e.g., a one-to-one sensor to display pixel ratio). If the user wishes to continue to zoom in on the region of interest, in response, system 100 will automatically switch from an optical magnification of one or less to the next higher magnification in microscope 150. In an embodiment where a separate sensor is associated with magnifying optics, system 100 will automatically switch to sensor 165 and position magnifying optical imaging sub-system 170 over the region of interest. Representatively, when the threshold resolution of the image is reached with sensor 160 and optical imaging sub-system 168, system 100 will switch to magnifying the image through objective lens 1700A. Objective lens 1700A is, for example, a 2.5× magnification.
Upon switching to capturing an image through magnifying optical imaging sub-system 170, the sensor to display pixel ratio will once again begin at a pixel ratio greater than one to one (e.g., four to one). A user can accept that captured image and save that captured image or continue to zoom in and accept and save the image at any desired ratio. Continuing to zoom in again initially involves modifying a sensor to display pixel ratio from a sensor to display pixel ratio greater than one to one towards a ratio of one to one or more. Once the threshold resolution is reached, system 100 will change objectives from objective 1700A to objective 1700B with the next higher optical magnification. In one embodiment, objective 1700B is a 20× magnification. Continued zooming follows the same action.
The above discussion involved user interaction to direct system 100 to electronically zoom in and/or increase magnifications. In another embodiment, system 100 may do this automatically. For example, system 100 may be configured to perform the above operations to capture saved images at different resolutions and/or magnifications.
In one embodiment, when a slide is placed in digital microscope 150, system 100 immediately creates an overview image with a magnification of one or less than one. A user can zoom in where sensor 160 is capturing an image from a sample in the slide as described above, or a user can alternatively capture a greater magnification of a region of interest on the sample by telling the system to increase magnification. One way a user does this is by using mouse 1400A and selecting a region of interest on the overview image and indicating the desired magnification. In the latter case, it is appreciated that a user may make the selection on an overview image whether or not a particular sample/slide is currently the slide to which sensor 160 may be capturing an image. For example, where sensor 160 is currently capturing an image of slide 320A (see FIG. 3) and a user wants a magnified image of a region of interest of slide 320B, a user will navigate with mouse 1400A to a thumbnail image of slide 320B present, for example, with thumbnail images of other slides along a side of display 120. The thumbnail image may be a smaller presentation of the image or a portion of the image, for example, a representation of a portion of the image using a reduced number of pixels. A user may then select the thumbnail overview image of slide 320B with mouse 1400A (e.g., by clicking on it). System 100 will then display a larger image of the overview image of slide 320B on screen 120 and stage 180 may move slide 320B into position for image capture by sensor 160. If a magnified image is stored in a memory of computer 110, system 100 will retrieve it and display it on display 120. If, however, a magnified image does not exist, system 100 will generate one. It is appreciated that slide 320B must be in slide carrier 210 in digital microscope 150.
Using the example of a user wanting a magnified view of a portion of a sample on slide 320B, initially a saved overview image of slide 320B will be displayed on display 120. If, for example, a user wants a magnified image of a portion of the image (e.g., a region of interest), the user can drag mouse 1400A to a desired position of display 120 showing a sample on slide 320B and then indicate to system 100 the desired area of magnification by clicking on mouse 1400A. As noted before, a specific coordinate system of the overview image may not be saved. However, system 100 knows the approximate location selected by a user, because it knows where on display 120 a user indicated (e.g., clicked), it knows the size of individual pixels in the overview image (e.g., 50 μm×50 μm) and it knows the pixel size of the image (e.g., 3324×2504 pixels). Since the system previously identified slide 320B in slide carrier 210 and the approximate location of the sample on the slide, system 100 will approximately know the region of interest to capture a magnified view of the region of interest. Similarly, if a magnified image including a region of interest had previously been saved, system 100 can retrieve that image based on a user indicating the region of interest on the overview image. In other words, an ability to identify a region of interest by a pixel position in an overview image applies not only to the overview image of a sample but to any other image of that sample.
In one embodiment, system 100 allows a user to put an annotation on an image and save the location of the annotation. For example, a user may wish to identify a region of interest in a 20× image by an arrow (an annotation) pointing at the region of interest. In one embodiment, a user locates the region of interest in 20× magnification, then moves mouse 1400A to the point where an annotation is desired. The user indicates to system 100 that it desires to put an annotation or object at a location (e.g., by previously selecting (clicking on) an icon in the display browser) and clicks mouse 1400A to place the annotation. To place the annotation, system 100 must locate the point in the 20× image. Similar to locating points in an overview image, system 100 can find the point because it knows a pixel resolution at 20×, a pixel size of the sensor (e.g., 5.4 μm×5.4 μm) from which a pixel size may be determined (5.4/20×1000=270 nm), and it knows the pixel number (e.g., 3324 pixels×2504 pixels). Based on this information, system 100 can locate and store information about a location of a point (e.g., a pixel or pixels) in the 20× magnified view as well as in the overview image of the slide.
Where slide information including an annotation or object of a point or region of interest of a slide is saved in system 100 and the slide is removed from digital microscope 150 and slide carrier 210, system 100 can also find the point or region of interest when the slide is reinserted in microscope 150. As described earlier, there are error tolerances associated with stage 180, slide carrier 210 and cavities within slide carrier 210. These error tolerances could effect alignment of an annotation to a particular point or region of interest of a sample on a slide when the slide is removed and then reinserted in slide carrier 210 and microscope 150. To account for this potential alignment error, in one embodiment, system 100 captures a new image of the slide or a portion of the slide and compares the new image to the saved image with the annotation. For example, system 100 may take a new image of the slide label or a corner of the sample and overlay that image on the saved image of the label or the corner of the sample, respectively. If the images are not aligned, system 100 rotates and/or linearly positions the new image until it is aligned. In doing this adjustment, system 100 stores information about the adjustment and uses this stored information to find where an annotation is located in the new view of the image. In a simple example, a new image of a sample on a slide reinserted into slide carrier 210 and microscope 150 is determined to be three pixel lengths in an x-direction different than the original image of the slide including the annotation. When indicating the annotation in the new image, system 100 knows the x-direction position of the annotation in the old image and then moves the annotation three pixels to the right to locate the annotation in the new image.
With regard to saving images (e.g., an overview image, a magnified image), in one embodiment, a single assembled image of a sample is saved. In another embodiment, a hierarchy of images of the sample is saved. In one embodiment, a hierarchy of images of a sample is created based on a sensor to display pixel ratio. In this embodiment, a highest ranked image in the hierarchy is an image having a one-to-one pixel ratio (full resolution sample mapping each sensor pixel with each display pixel). One or more lower ranked images of increasing sensor to display pixel ratio (a sensed image is displayed on display 120 such that one sensor pixel is mapped to more than one display pixel, e.g., 2:1, 4:1, etc.) make up the remainder of the hierarchy of images. Each of the full resolution sample and the one or more lower ranked images may be stored together in a data set.
For objectives with high magnification, a depth of field (i.e., objects within the depth of field (z-direction range)) is relatively small. The z-direction range is so small (e.g., 1 μm) that the image captured may not capture all objects in a sample having a thickness, for example, on the order of 10 μm, in one image capture. To capture as many objects as possible, in one embodiment, system 100 may capture several images at different focal planes by a depth of field. In the example, system 100 may capture 10 images moving stage 180 one micron in a z-direction between each capture. Such an operation will result in 10 image planes representing a z-direction stack or z-stack of the sample.
In another embodiment, a coordinate system for each slide may be established using a label image. In one embodiment, a slide label may be printed with more than one perceivable dot or point. FIG. 7 shows an embodiment of a slide label having three points (point 4500A, point 4500B, point 4500C). Sensor 160 or sensor 165 can perceive these points and, having perceived them, system 100 can determine a pixel location of each point and the number of pixels between each point. By associating image 440 adjacent label 450, system 100 can locate any position on image 440 based on its x- and y-distance from one or more points.
In one embodiment, a data set including a saved image or hierarchy of images of a sample, a z-stack, a coordinate system for that image, if any, and separately saved label image is assembled in a memory of computer 110. Such a data set may also contain comments or annotations (including markings on an image made by a user) and content of label (e.g., interpretation of label).
Having described certain components of system 100, a brief description of operation is now presented. In one embodiment, use of system 100 is software driven. In other words, a machine or computer readable medium is provided in computer 110 containing program instructions that, when executed, will carry out the various methods of operation described.
In one embodiment, a method of operation is illustrated in FIG. 8. Method 500 will be described with reference to components of system 100 and various screen shots that, in one embodiment, are displayed on display 120.
As a starting point, slide carrier 210 may be loaded into digital microscope 150 and placed on stage 180. Sensors may be located on stage 180 to sense a slide carrier. Computer 110 is responsive to such sensors. When computer 110 senses slide carrier 210 on stage 180, in one embodiment, system 100 has three modes: a live mode; a scanning mode; and a viewing mode. The live mode and viewing mode are interactive modes in that they include user interaction. The scanning mode may be operated interactively or be fully automated with specific predefined parameters or configurations for scanning (saving) images. For example, in the scanning mode, slide(s) may be loaded onto a slide carrier and inserted into a digital microscope and the system will sense and save one or more images of samples on a slide.
FIG. 9 shows an example of a screen shot of display 120 of system 100. This is an example of an entry screen where a user can select a live mode, a scanning mode or a viewing mode. An entry screen may also include an opportunity to enter or modify the basic settings of the instrument by selecting “Settings” 640 as well as an opportunity to exit the system by selecting “Exit” 650.
In the example where a user chooses a live mode by selecting “Live” 610, computer 110 will direct digital microscope 150 to move slide carrier 210 to a loading position, such as by extending slide carrier 210 out of the instrument so it is accessible by a user. At this time, when slide carrier 210 is accessible, computer 110 may indicate that the slide carrier may be removed from digital microscope 150 and loaded with one or more slides. One way to indicate this is by an announcement on monitor 120 (block 505, FIG. 8).
In the embodiment where slide carrier 210 has four slide cavities (slide cavities 220A, 220B, 220C and 220D), a user may place up to four slides onto slide carrier 210. After placing one or more slides on slide carrier 210, the carrier is loaded into the instrument and the instrument pulls the carrier inside and senses its presence and location (block 510, FIG. 8).
Once slide carrier 210 is placed within digital microscope 150, system 100 determines the number and position of slides inserted into slide cavities 220A, 220B, 220C and/or 220D (block 515, FIG. 8). System 100 selects each slide for imaging (block 520, FIG. 8). System 100 then aligns a selected slide and sensor 160/optical imaging sub-system 168 (block 520, FIG. 8). If a selected slide has been reinserted into the instrument and information of a previous session needs to be correlated, system 100 determines the rotation and shift of the selected slide with respect to a previous session. In one embodiment, sensor 160 initially senses an image or images of a sample on each slide without magnification or with a magnification of less than one (block 530, FIG. 8). The sensed image along with a separately sensed image of a label on the slide may then be displayed (block 535, FIG. 8). A low-resolution overview image of all inserted slides and of their labels is then taken.
The screen shot in FIG. 10 is an embodiment of a slide selection graphical user interface (GUI). In GUI title bar 710, the steps for the workflow are illustrated and the current step (“slide selection”) highlighted. To the right of these tabs (as viewed), the title bar includes controls to move within of the workflow (buttons 715), to eject the slide carrier (button 720) or to get assistance (button 725).
Main part 730 of screen 700 is divided into slide selection section 740, profile selection section 750 and slide information section 760. Slide selection section 740 shows a sketch of slide carrier 180 (see FIG. 2). In the occupied cavities of slide carrier 180, overview images 770A, 770B, 770C and 770D of slides in slide cavities 220A, 220B, 220C and 220D, respectively (see FIG. 2), as well as the labels of the respective slides can be seen. The GUI allows a user to select and group correlated slides. In one embodiment, grouped slides are scanned and simultaneously displayed. This feature allows the user to, for example, compare structures of different slides of the same group. A group of slides may be referred to as a case. In the “live mode,” a user is able to add pre-scanned, stored slides to a case. For example, in addition to any slides present on slide carrier 180 within digital microscope 150, computer 110 of system 100 may store previously captured (saved) images (e.g., overview image, magnified image) referred to as scanned images of slides that are no longer present.
Referring to profile selection section 750 of the GUI, the section allows the user to select specific, predefined scanned profiles that are optimized and customized image acquisitions for a specific stain or, in an embodiment utilizing fluorescent image, a fluorescence number. For example, a specific profile for an H&E stain might be a 20× image. Section 750 allows a user to select the tool bar and be offered a choice of a 20× view and scan. Slide and label information is presented in slide information section 760 and provides, in one embodiment, identifying information about the patient as well as the process step(s) that a sample was subjected to in preparing it for microscopic analysis.
Referring to the screen shot in FIG. 10, options to proceed include starting a “live mode” of a single slide or a group of slides or ejecting the slide carrier (slide carrier 180). In a “live mode,” further images (e.g., magnified images) of a slide or a group of slides may be sensed and/or captured (saved).
In an instance where a user selects a “live mode,” a user may select a particular slide or slides based on the displayed overview images (block 540, FIG. 8). FIG. 11 and FIG. 12 show typical user interface screens. While FIG. 11 illustrates the screen with a single image or scan, FIG. 12 shows views of three different images or scans. The image in FIG. 11 is a live image (indicated video icon in right corner, as viewed). The screen is divided into different areas. The two upper images (as viewed) are live images (see video icon) and the lower image is a saved (scanned) image (see camera icon). System 100 is able to show saved tissue images (tissue scans) and live images of slides that are in digital microscope 150 as well as stored tissue scans of pre-scanned slides. In one embodiment, for “live” images of slides present in digital microscope 150, sensor 160 continuously acquires (refreshes) images at the current position of the microscope at a rate of several images/second (block 545, FIG. 8). For example, if a sensor last sensed a sample on a slide in slide cavity 220A and no further activity by a user or system 100 has been dictated, sensor 160 will remain over the slide in slide cavity 220A and refresh that image at a rate of, for example, six images/second. In one embodiment, only the sample of the image is refreshed, not the associated label. Although FIG. 11 shows a single scan and FIG. 12 shows three scans, up to 16 scans can be viewed simultaneously. In such case, the view is split to show all selected scans. In another embodiment, images from a case can be displayed on more than one screen.
Views can be added and removed from the screen via selection in a journal. The journal is a collection of all objects in a case and can be found on the right hand side of the screen. At the same time, only one view is active. All controls reflect the setting of the active view. Any operation will only affect this specific image. The images from the other scans are “frozen.” To switch focus to another slide, the user needs to click on one of the displayed images or a label in the journal.
In title bars 810 and 910 of the screen shots shown in FIG. 11 and FIG. 12, respectively, system 100 offers the following controls:
Start/stop acquisition Image acquisition gets stopped/continued.
815/915:
Snap 820/920: Capture current image from the selected slide.
The image can be saved or exported.
Color channel selection Switch on/off color channels.
830/930:
Distance/area Calibrated tools to measure actual distances and
measurement areas on the sample.
tools 840/940:
Annotations and Using these tools, the user is able to add
annotation comments to the image with different object
setting 845/945: (e.g., text, square, arrow).
HR scan regions: Defines regions which will be scanned using
the selectable high resolution (HR) optics. For
regions smaller than a camera tile, the software
immediately acquires and displays the image
(i.e., no stitching required). However, if the
selected region is larger than a camera tile, the
region will be stitched and scanned.
In each view, the magnification of the image, a scale indicator and a focus control may be displayed.
FIG. 11 shows a single image at 2.5× magnification. A user can indicate regions of interest or desired scans by maneuvering mouse 1400A or mouse 1400B over an area (see e.g., Scan 1, Region 1, Region 2). FIG. 12 shows these areas at 20×.
Using keyboard 130, mouse 1400A or 3D mouse 1400B, the user is able to navigate within the scanned image data (block 550, FIG. 8). The following actions are possible:
Zoom: Digitally zoom in/out of the displayed image. For slides,
which are physically in the instrument, a user may digitally
zoom in and out of the displayed image. The zooming in and
out initially causes system 100 to modify the sensor to display
pixel ratio (block 565, FIG. 8). As soon as a microscope
objective can be used to obtain the view (block 570, FIG. 8),
the system will automatically use the optimal magnification
for a specific zoom level. At this point, a user may again
zoom in and out by initially changing a sensor to display
pixel ratio and then an objective where a higher magnification
Pan: Pan around the sample to display neighboring areas. If a
user examines different slides in digital microscope 150,
system 100 will move the sample accordingly.
Focus: In the live image mode, a user is able to change the focus of
the instrument. It is possible to trigger an autofocus and to
manually adjust the focus with respect to a nominal position.
In the case of a pre-scanned z-stack scan (i.e., scans of
different depths into the tissue, a user can change the focal
plane within the z-stack.
Each view can be viewed in a full screen mode. In this mode, the total screen is covered by the view. Only the magnification of the image, scale indicator and the navigation control are displayed in the image. A user can leave this total screen view mode by selecting (pushing) the “esc” key on keyboard 130.
At any time during the navigation, a user may save the displayed image (block 560, FIG. 8).
On the right hand side of the GUI in FIG. 11 and FIG. 12, display 120 shows navigation control 850/950, journal 860/960 and image parameter control 870/970.
Navigation control 850/950 shows an overview image of the active slide. The control indicates the position of the active view (e.g., the live image that can be refreshed at a desired refresh rate). By clicking on the control, the position can be moved. The control also indicates the positions of the annotations and scans which have been added to the slide.
Below navigation control 850/950, journal 860/960 is located. Journal 860/960 represents the structure of a case. In one embodiment, journal 860/960 contains slide information, annotations and comments of all slides in a case. Slides can be added and removed from the case. The slides are grouped into two parts. In the upper part of the list, slides in the instrument can be found. A user can examine these slides live. In this case, the user can add scans to these slides by scanning selected areas of the slides. In the lower part of the list (as viewed), pre-scanned slides from the memory in computer are presented.
In the structure of journal 860/960, each slide has by default three objects: a label image, the slide information and an overview image. The user can add additional objects (annotations, comments, bookmarks and high-resolution scans) to the journal. A user can choose one of the entries in journal 860/960 to jump to a specific image position.
In one embodiment, journal 860/960 will be a starting point for evaluation of specific regions or for the creation of a report of the considered case. Journal 860/960 also contains the “save session” button. By selecting this button, system 100 will store the session (including all slides, annotations and setting) to memory in computer 110. The file contains the labels, the overview image and the high-resolution scans, which are defined in the journal. A user can restore the session at a later time.
Below journal 860/960 in FIG. 11 and FIG. 12, image parameter control 870/970, in one embodiment, is located. Image parameter control 870/970 allows the acquisition parameters of the camera (exposure time, camera gain, etc.) and the image settings (brightness, contrast, color balance, black/white balance, image filters (sharpness, etc.)) to be modified. A user can save settings, for example, a specific staining. Afterwards, the user can access these settings without unfolding the controls.
Following optional modification of image parameters, a user may save the session. A user may also abort the session. If a user decides to abort the session, the user exits the live mode. If the user has modified entries in journal 860/960, which are not saved yet, system 100 will ask the user, if the user wants to save the session. Subsequently, system 100 moves to the “slide (group) selection” screen. The user can continue with the next slide(s).
If a user has defined a high-resolution scan region in the live mode, the high-resolution scan function is accessed. Before the scan starts, the user selects the scan parameters (resolution, acquisition parameters and z-stack parameters). FIG. 13 shows an embodiment of a screen shot on display 120 where scan parameters are established. The user can name and comments about the scan. In area 1020, the user can choose the scan display mode. The scan display mode describes how system 100 will show individual camera fields of view or tiles during a scan. The user can either follow the scan tile by tile or view the entire scan while it builds up. In the latter case, the user is able to pan and zoom in the recorded image data during the scan.
After the high-resolution scan is started, system 100 moves to the high-resolution scan display. FIG. 14 shows a screen shot of an embodiment of a high-resolution scan display. Depending on the selected display mode, system 100 either shows the current tiles or the entire scan area 1110. In the latter case, a user is able to zoom and pan within the recorded data, while the screen builds up. The high-resolution scan display also indicates progress 1120 of the scan screen.
Following a high-resolution scan, system 100 automatically reverts to the “live mode” screen. The completed scan is inserted into the journal and is located below the corresponding slide.
In another embodiment, system 100 can be directed to perform scans of multiple slides, such as high-resolution scans of each slide in slide carrier 180. An embodiment where slide carrier 180 holds four slides, a user places up to four slides onto slide carrier 180 and inserts the carrier digital microscope 150. Digital microscope 150 pulls slide carrier 180 inside and determines the slide carrier type, the number of slide and their positions. Automatic scanning of more than four slides is possible in combination with, for example, an external slide loader. In that case, system 100 would communicate with the slide loader so that slides to be scanned are automatically loaded and retrieved. For example, a commercially available slide loader can be associated with digital microscope 150. Such slide loaders are configured to receive and transfer slides. Accordingly, a transfer mechanism of a slide loader can load a slide or slides onto slide carrier 210 and a transfer mechanism (e.g., pick and place robotic grabber) associated with or connected to digital microscope 150 can return slides to the slide loader after scanning. The control of transferring, scanning and imaging may be controlled by controller 110.
In an automatic or reflex imaging mode, system 100 may be instructed to acquire scans of either the complete slide or of selected areas of the slide can be accomplished immediately or at a later time as selected by the user. Scanning of the slides in the reflex imaging mode is facilitated by an automatic slide loader. In addition, to select complete or partial slide scans, a user may select the scan parameters such as, but not limited to, magnification and z-stack. In addition, a user can select more than one scan at different magnifications or using other user-defined parameters. Default scan parameters can be programmed into system 100 to reflect differences between clinicians and/or tissue types. Completion of the reflex imaging of a slide or group of slides, as in a case, may be automatically signaled to one or more devices, for example, via internet connection 145 and to the computer 110 to enable rapid sign-out of the cases (see FIG. 1).
System 100 presents a user with selections for the scanning resolution and the predefined image parameter set such as described above in the live mode. Moreover, the user can define z-stack parameters (number of planes, distance between the planes and the focus offset from the focal plane).
The user can modify the save parameters (file name and scan directory). The default parameters are defined in the system settings. In the case of the file name, the user can either use the label content or define a file name. If a name is defined, in one embodiment, system 100 will add a progressive number to distinguish between the slides.
After defining the scan parameters, system 100 starts to scan slide carrier 180 slide by slide. For each slide, a low-resolution overview image and the label image are taken. Following the capture of overview and label images for each slide, system 100 will capture high-resolution images. In one embodiment, the GUI on display 120 will change to the first high-resolution scan screen. System 100 may depict the overview and the label image of current slide. It automatically identifies the tissue regions on the slide and indicates the detected tissue regions in the overview image. Subsequently, the focal plane for the tissue regions on the slide is determined. The progress of these steps is indicated on the screen.
In one embodiment, during the actual scan of the detected tissue regions, system 100, depending on the system settings, in one embodiment, display 120 displays either the image of the current scan position or shows the entire current scan region, in which the image builds up.
The scanned high-resolution images for a slide are stored in memory of computer 110 and can be accessed later by a user using the viewing mode. After all tissue regions on the slide are scanned, system 100 proceeds with the next slide. After all slides have been scanned, system 100 ejects slide carrier 180 and moves to the entry screen.
In a viewing mode, a user can retrieve and view stored images of previously scanned samples (slides) including multiple slides that may make up a case. In this manner, a user can view the images from display 120 directly connected to computer 110 and optical microscope 150. Alternatively, through intranet/internet connection 145 (FIG. 1), a user can access a memory of system 100 and view images remotely. Intranet/internet connection 145 of system 100 also allows images to be sent from one location to another (e.g., via email).
sensing an image at a first magnification of one or less of a portion of a tissue sample on a substrate with a sensor;
without storing the sensed image,
displaying at least a portion of the sensed image on a display at a sensor to display pixel ratio greater than one to one;
in response to user input, performing each of the following:
electronically modifying the sensor to display pixel ratio at which the sensed image is displayed on the display at the first magnification, and
when the sensor to display ratio reaches a threshold at the first magnification, automatically sensing a view of an area of the portion of the tissue sample at a second magnification in response to an attempt to modify the sensor to display pixel ratio beyond the threshold ratio; and
refreshing the at least a portion of the sensed image that is displayed at a computer controlled rate, and
storing the at least a portion of the sensed image that is displayed.
2. The method of claim 1, wherein displaying the sensed image of a portion of a tissue sample comprises assembling a plurality of segments of the portion and displaying the assembled plurality of segments.
3. The method of claim 2, wherein a segment of the plurality of segments overlaps an adjacent segment and assembling the adjacent segments comprises aligning common image content in the adjacent segments.
4. The method of claim 2, wherein the displaying the sensed image comprises displaying an entire cross-sectional area of the image.
5. The method of claim 1, wherein storing the sensed image comprises storing in response to user input.
6. The method of claim 1, wherein electronically modifying a sensor to display pixel ratio comprises transitioning from a ratio greater than one to one to a ratio of one to one.
7. The method of claim 1, wherein the view of the area of the portion of the tissue sensed at the second magnification corresponds to an area selected by the user input.
8. The method of claim 1, further comprises sensing an image of a label on the substrate and displaying the image on the display.
9. The method of claim 8, wherein sensing an image of a tissue sample on a substrate comprises sensing a bright field image and sensing an image of a label comprises sensing an image using reflectance.
10. The method of claim 1, further comprising displaying on the display at least one of the modified sensor to display pixel ratio view and the view at the second magnification.
11. The method of claim 10, concurrently displaying on the display each of the sensed image on a display at a sensor to display pixel ratio greater than one to one and the at least one of the modified sensor to display pixel ratio view and the view at the second magnification.
12. The method of claim 1, further comprising providing a plurality of substrates wherein each of the plurality of substrates comprises a tissue sample and, prior to sensing an image, the method further comprises designating at least one of the plurality of substrates for sensing.
13. The method of claim 1, further comprising providing a plurality of substrates wherein each of the plurality of substrates comprises a tissue sample and, displaying on the display at least one image of one of the plurality of substrates and at least one image of another of the plurality of substrates.
14. The method of claim 1, further comprising simultaneously displaying on the display at least one image of at least one area of the portion of the tissue sample.
15. The method of claim 1, wherein displaying comprises displaying a first image and a second image on the display and refreshing the sensed image comprises refreshing only one of the first image and the second image.
16. The method of claim 15, wherein the other of the first image and the second image is an image retrieved from a computer memory storage.
17. The method of claim 1, wherein performing comprises storing at least a portion of the sensed image in a data structure and the method further comprises:
sensing an image of the sample at the second magnification that is a magnification greater than one; and
storing the sensed image at the second magnification in the data structure.
18. The method of claim 1, wherein performing comprises storing the at least a portion of the sensed image in a data structure, wherein the storing comprises storing a hierarchy of images based on a sensor to display pixel ratio.
19. The method of claim 1, further comprising locating an area in an image based on a pixel size of the sensed image, a magnification and the number of pixels.
20. The method of claim 19, associating the area of the sensed image with an annotation provided by the user and storing a location of the area and the annotation in a data structure.
21. The method of claim 1, wherein the tissue sample on a substrate is a sample on a slide, the slide having a label and positioned on a stage performing comprises storing at least a portion of the sensed image, the method further comprising in response to a removal from and subsequent return of the slide to the stage, orienting a position of the tissue sample relative to its position prior to removal.
a digital microscope comprising:
a stage configured to support at least one microscope slide;
a first optical imaging sub-system disposed between the at least one sensor and the stage, the first optical imaging sub-system configured to project an image with a magnification of one or less; and
a second optical imaging sub-system disposed between the at least one sensor and the stage, the second optical imaging sub-system configured to project an image with a magnification greater than one;
an illumination sub-system comprising at least one light source; and
a computer coupled to the digital microscope and operable to direct an image capture by the at least one image sensor of a portion of a microscope slide on the stage, a display coupled to the computer and operable to display an image transmitted from the computer, wherein the computer comprises machine-readable instructions that when executed, the computer is operable to direct an image capture through the second optical imaging sub-system in response to an attempt by a user to modify a sensor to display pixel ratio of an image beyond a threshold through the first optical imaging sub-system.
23. A non-transitory computer-readable medium containing instructions that when executed perform a method comprising:
electronically modifying the sensor to display pixel ratio of the sensed image at the first magnification, and
refreshing the at least a portion sensed image that is displayed at a computer controlled rate, and
storing at least a portion of the sensed image that is displayed.
24. The computer-readable medium of claim 23 wherein displaying the sensed image of a portion of a tissue sample comprises assembling a plurality of segments of the portion and displaying the assembled plurality of segments.
25. The computer-readable medium of claim 23, wherein modifying the sensor to display pixel ratio, comprises transitioning from a ratio greater than one to one to a ratio of one to one in response to user input.
26. The computer-readable medium of claim 23, wherein an area of the portion of the tissue sensed at the second magnification corresponds to an area selected by the user input.
27. The computer-readable medium of claim 23, wherein performing comprises storing at least a portion of the sensed image in a data structure and the method further comprises:
sensing an image of the sample at the second magnification; and
28. The computer-readable medium of claim 23, wherein performing comprises storing the at least a portion of the sensed image in a data structure, wherein the storing comprises storing a hierarchy of images based on a sensor to display pixel ratio.
29. The computer-readable medium of claim 23, further comprising locating an area in an image based on a pixel size of the sensed image, a magnification and the number of pixels.
30. The computer-readable medium of claim 29, associating the area of the sensed image with an annotation provided by the user and storing a location of the area and the annotation in a data structure.
31. The computer-readable medium of claim 23, wherein the tissue sample on a substrate is a sample on a slide, the slide having a label and positioned on a stage performing comprises storing at least a portion of the sensed image, the method further comprising in response to a removal from and subsequent return of the slide to the stage, orienting a position of the tissue sample relative to its position prior to removal.
displaying at least a portion of the sensed image on a display; and
performing each of the following:
electronically modifying a sensor to display pixel ratio at which the sensed image is displayed on the display at the first magnification; and
when the sensor to display ratio reaches a threshold at the first magnification, automatically displaying a magnified view of an area of the portion of the tissue sample at a second magnification in response to an attempt to modify the sensor to display pixel ratio beyond the threshold ratio.
33. The method of claim 32, further comprising storing the sensed image.
34. The method of claim 32, wherein modifying a sensor to display pixel ratio comprises transitioning from a ratio greater than one to one to a ratio of one to one.
35. The method of claim 32, wherein an area of the portion of the tissue sensed by the second magnification corresponds to an area selected by the user input.
36. The method of claim 32, further comprises sensing an image of a label on the substrate and displaying the image on the display.
37. The method of claim 36, wherein sensing an image of a tissue sample on a substrate comprises sensing a bright field image and sensing an image of the label comprises sensing an image using reflectance.
sensing an image of a portion of a tissue sample on a substrate at a first magnification with a sensor;
displaying at least a portion of the sensed image on a display at the first magnification at a sensor to display pixel ratio greater than one to one; and
in response to user input, performing each of the following in response to user input:
when the sensor to display ratio reaches a threshold sensor at the first magnification, automatically sensing of an area of the portion of the tissue sample at a second magnification that is greater than the first magnification in response to an attempt to modify the sensor to display pixel ratio beyond the threshold ratio; and
39. The method of claim 38, wherein storing the sensed image comprises storing in response to user input.
40. The method of claim 38, wherein modifying a sensor to display pixel ratio comprises transitioning from a ratio greater than one to one to a ratio of one to one in response to user input.
41. The method of claim 38, wherein an area of the portion of the tissue sensed by the magnified view corresponds to an area selected by the user input.
42. The method of claim 38, further comprising displaying on the display at least one of a modified sensor to display pixel ratio view and a second magnification view.
43. The method of claim 42, concurrently displaying on the display each of the sensed image on a display at a sensor to display pixel ratio greater than one to one and the at least one of a modified sensor to display pixel ratio view and a second magnification view.
44. The method of claim 38, further comprising providing a plurality of substrates wherein each of the plurality of substrates comprises a tissue sample and, prior to sensing an image, the method further comprises designating at least one of the plurality of substrates for sensing.
45. The method of claim 38, further comprising providing a plurality of substrates wherein each of the plurality of substrates comprises a tissue sample and, displaying on the display at least one image of one of the plurality of substrates and at least one image of another of the plurality of substrates.
46. The method of claim 38, wherein performing comprises storing at least a portion of the sensed image in a data structure and the method further comprises:
sensing an image of the sample at an optical magnification greater than one; and
storing the sensed image at the optical magnification greater than one in the data structure.
47. The method of claim 38, wherein performing comprises storing the at least a portion of the sensed image in a data structure, wherein the storing comprises storing a hierarchy of images based on a sensor to display pixel ratio.
an optical imaging sub-system disposed between the at least one sensor and the stage, the optical imaging sub-system comprising at least a first objective having a first magnification and a second objective having a second magnification that is greater than the first magnification;
an illumination sub-system comprising at least one light source;
a computer coupled to the digital microscope and operable to direct an image capture by the at least one image sensor of a portion of a microscope slide on the stage; and
a display coupled to the computer and operable to display an image transmitted from the computer,
wherein the computer comprises machine-readable instructions that when executed, will cause the computer to automatically direct an image capture through the second objective in response to an attempt to electronically modify a sensor to display pixel ratio of a sensed image beyond a threshold ratio.
49. The apparatus of claim 48, wherein the first objective and the second objective are associated with a first imaging sub-system, the microscope further comprising:
a second imaging sub-system comprising a third objective having a magnification less than the first magnification and the second magnification; and
wherein the light source is operable to emit light through an opening in the stage and the mirror is operable to direct light emitted through the opening in the stage to the third objective.
displaying at least a portion of the sensed image on a display at the first magnification; and
when the sensor to display ratio reaches a threshold at the first magnification, automatically displaying a view of an area of the portion of the tissue sample at a second magnification that is greater than the first magnification in response to an attempt to modify the sensor to display pixel ratio beyond the threshold ratio.
US13/212,955 2010-08-20 2011-08-18 Digital microscope and method of sensing an image of a tissue sample Active 2033-06-06 US10139613B2 (en)
US37570310P true 2010-08-20 2010-08-20
US13/212,955 US10139613B2 (en) 2010-08-20 2011-08-18 Digital microscope and method of sensing an image of a tissue sample
US20120044342A1 US20120044342A1 (en) 2012-02-23
US10139613B2 true US10139613B2 (en) 2018-11-27
ID=44533206
US13/212,955 Active 2033-06-06 US10139613B2 (en) 2010-08-20 2011-08-18 Digital microscope and method of sensing an image of a tissue sample
US (1) US10139613B2 (en)
EP (3) EP2910993A1 (en)
JP (2) JP6186634B2 (en)
CN (2) CN103140789B (en)
AU (1) AU2011291517B2 (en)
BR (1) BR112013004006A2 (en)
CA (1) CA2808105C (en)
DE (2) DE202011110631U1 (en)
DK (2) DK2606394T3 (en)
ES (2) ES2623029T3 (en)
WO (1) WO2012024627A1 (en)
JP5730696B2 (en) * 2011-07-12 2015-06-10 オリンパス株式会社 The image processing device and an image display system
CN102707425B (en) 2012-06-21 2014-04-16 爱威科技股份有限公司 Image processing method and device
JP6147079B2 (en) * 2013-05-14 2017-06-14 オリンパス株式会社 Microscope system, the method of determining the stitched area, and the program
JP6173032B2 (en) * 2013-05-16 2017-08-02 オリンパス株式会社 Microscope system
CA2916768A1 (en) 2013-06-26 2014-12-31 Huron Technologies International Inc. Preview station and method for taking preview images of microscope slides
US20150124072A1 (en) * 2013-11-01 2015-05-07 Datacolor, Inc. System and method for color correction of a microscope image
US20150213616A1 (en) * 2014-01-27 2015-07-30 Baxter International Inc. Visual inspection system for automated detection of particulate matter in flexible medical containers
JP2016035394A (en) * 2014-08-01 2016-03-17 パイオニア株式会社 Terahertz wave imaging device and terahertz wave imaging method
DE102014112285A1 (en) * 2014-08-27 2016-03-03 Carl Zeiss Microscopy Gmbh A method for correcting a color reproduction of a digital microscope, and digital microscope
JP2016177152A (en) * 2015-03-20 2016-10-06 オリンパス株式会社 Microscope system
WO2017037875A1 (en) * 2015-09-01 2017-03-09 オリンパス株式会社 Method for acquiring 3-dimensional image, method for displaying 3-dimensional image, and device for acquiring 3-dimensional image
KR20170039923A (en) * 2015-10-02 2017-04-12 삼성전자주식회사 Multi-layer image sensor
US20170234874A1 (en) * 2015-10-07 2017-08-17 Clearbridge Biophotonics Pte Ltd. Integrated visual morphology and cell protein expression using resonance-light scattering
TWI599793B (en) 2015-11-23 2017-09-21 財團法人金屬工業研究發展中心 Image scanning system for tissue slides
DE102016115971A1 (en) 2016-08-26 2018-03-01 Leica Microsystems Cms Gmbh A method for controlling a digital microscope and digital microscope
CN106770302A (en) * 2017-03-24 2017-05-31 董成功 Histopathological observation machine for use under multifunctional mirror
US3309262A (en) 1963-12-03 1967-03-14 Container Corp Fluidized bed oxidation of waste liquors resulting from the digestion of cellulosic materials for paper making
US3525803A (en) 1966-08-01 1970-08-25 Int Research & Dev Co Ltd Means for detecting malignant cells in human and animal tissue
US3762798A (en) 1971-07-01 1973-10-02 Hamilton Co Microscope stage
US3765851A (en) 1970-12-14 1973-10-16 Chervon Res Co Gas production
US3862909A (en) 1972-09-05 1975-01-28 Copeland Systems Inc Fluidized bed autogenous combustion and pyrolysis of aqueous effluents to prepare activated carbon
US4079248A (en) 1975-09-29 1978-03-14 Thomson-Brandt Optical focussing sensor
US4089989A (en) 1975-04-04 1978-05-16 White Ronald D Method for preparing microscope slides by rotating during coating
US4148752A (en) 1976-04-09 1979-04-10 Bayer Aktiengesellschaft Production of activated carbon in a reactor having a lower static layer and an upper fluidized layer
US4404683A (en) 1979-10-24 1983-09-13 Omron Tateisi Electronics Co. Blood cell identification and classification system
JPS5971018A (en) 1982-10-15 1984-04-21 Ikegami Tsushinki Co Ltd Automatic microscope device
JPS61248168A (en) 1985-04-22 1986-11-05 Du Pont Electronic mosaic image processing
US4684799A (en) 1983-09-19 1987-08-04 Ricoh Company, Ltd. Focus detection method involving cutting more than half of light beam reflected from disc
US4761075A (en) 1985-12-10 1988-08-02 Hitachi, Ltd. Cellular analysis system
JPS63206793A (en) 1987-02-19 1988-08-26 Vlsi Technology Inc Video memory interface circuit
FR2620537A1 (en) 1987-09-14 1989-03-17 Micro Controle Auto-focusing optical device and optical apparatus comprising such a device
US4962264A (en) 1989-10-23 1990-10-09 Betz Laboratories, Inc. Methods for retarding coke formation during pyrolytic hydrocarbon processing
US5297215A (en) 1990-06-13 1994-03-22 Kabushiki Kaisha Toshiba Apparatus and method of displaying medical images of sliced tissue
US5311426A (en) 1988-08-02 1994-05-10 Abbott Laboratories Apparatus and method for providing assay calibration data
US5532874A (en) 1992-12-18 1996-07-02 Morphometrix Inc. Multiscanning confocal microscopy
US5561556A (en) 1994-04-21 1996-10-01 Compucyte Corporation Slide analysis system with slide having self contained microscope analysis information
JPH0980138A (en) 1995-09-14 1997-03-28 Hitachi Ltd Squid sensor
JPH09133856A (en) 1995-11-07 1997-05-20 Nikon Corp Automatic focus detecting device for microscope
JPH09161068A (en) 1995-12-12 1997-06-20 Furukawa Electric Co Ltd:The Picture photographing method and picture edition device using the method
US5659174A (en) 1995-04-10 1997-08-19 Hitachi, Ltd. Scanning electron microscope
JPH09218354A (en) 1996-02-13 1997-08-19 Olympus Optical Co Ltd Microscope information system
US5675141A (en) 1995-02-03 1997-10-07 Hitachi Denshi Kabushiki Kaisha Method and apparatus for automatically detecting focusing point and image processing method and apparatus using the same
US5768033A (en) 1996-06-14 1998-06-16 Brock; Dennis Microscope assembly comprising a supported and movable specimen wheel and fine adjustment means
US5864138A (en) 1995-09-14 1999-01-26 Hitachi, Ltd. Electron Microscope
US6061176A (en) 1998-09-14 2000-05-09 Shih; Song Hsin Microscope system for observation and display of microcirculation at multiple body areas
US6248995B1 (en) 1997-05-30 2001-06-19 Yokogawa Electric Corporation Confocal microscopic equipment
WO2001054052A1 (en) 2000-01-21 2001-07-26 Bacus Research Laboratories, Inc. Method and apparatus for internet, intranet, and local viewing of virtual microscope slides
JP2001281553A (en) 2000-03-31 2001-10-10 Bunshi Biophotonics Kenkyusho:Kk Optical microscope system
US6309607B1 (en) 1997-11-05 2001-10-30 Robert John Johnston Slide staining system
US20010035752A1 (en) 1997-11-28 2001-11-01 Kormos Donald W. In-room MRI display terminal and remote control system
JP2002031513A (en) 2000-07-14 2002-01-31 Minolta Co Ltd Three-dimensional measuring device
JP2002084554A (en) 1995-01-04 2002-03-22 Visualabs Inc Method for processing information
US20020176160A1 (en) 2001-03-30 2002-11-28 Nikon Corporation Microscope system
US6529271B1 (en) 1998-12-18 2003-03-04 Leica Microsystems Heidelberg Gmbh Method of finding, recording and evaluating object structures
US20030048931A1 (en) 2001-03-23 2003-03-13 Peter Johnson Quantification and differentiation of tissue based upon quantitative image analysis
US20030112330A1 (en) 2001-12-19 2003-06-19 Olympus Optical Co., Ltd. Microscopic image capture apparatus
US20030133009A1 (en) 1999-04-09 2003-07-17 Carl S Brown System and method for detecting with high resolution a large, high content field
US20030142882A1 (en) 2002-01-28 2003-07-31 Gabriel Beged-Dov Alignment of images for stitching
US6678398B2 (en) 2000-09-18 2004-01-13 Sti Medical Systems, Inc. Dual mode real-time screening and rapid full-area, selective-spectral, remote imaging and analysis device and process
US20040027462A1 (en) 2000-09-25 2004-02-12 Hing Paul Anthony Image sensor device, apparatus and method for optical measurements
US20040080758A1 (en) 2002-10-23 2004-04-29 Fanuc Ltd. Three-dimensional visual sensor
US20040141660A1 (en) 2002-12-18 2004-07-22 Lfk-Lenkflugkorpersysteme Gmbh Method of enlarging the image field of a focal plane array camera
EP1447699A2 (en) 2003-02-05 2004-08-18 Fairfield Imaging Ltd. Microscope system and method
US6800853B2 (en) 2000-11-16 2004-10-05 Jeol Ltd. Electron microscope and method of photographing TEM images
WO2005015120A2 (en) 2003-08-08 2005-02-17 Wallac Oy Method and arrangement for focusing detection in an optical measurement and method and arrangement for migitating the effect of surface reflection
US20050073649A1 (en) 2003-07-04 2005-04-07 Spector Robert T. Method of and apparatus for diagnosing and treating amblyopic conditions in the human visual system
US20050090017A1 (en) 2003-10-24 2005-04-28 Morales Azorides R. Simplified tissue processing
US20050094262A1 (en) 2003-11-05 2005-05-05 Visx, Incorporated Microscope magnification sensor
US20050112537A1 (en) 2003-11-20 2005-05-26 Ladder Publishing Co., Ltd. Mobile teaching aid with audiovisual amusement device
US20050211874A1 (en) 2003-12-01 2005-09-29 Olympus Corporation Optical device and imaging method
US6959720B2 (en) 2002-04-05 2005-11-01 Nifco Inc. Over-fueling prevention valve
US20050248837A1 (en) * 2004-05-10 2005-11-10 Nikon Corporation Microscope optical system, microscope, and virtual slide forming system
JP2006003543A (en) 2004-06-16 2006-01-05 Olympus Corp Microscope system
US20060039583A1 (en) 2002-11-25 2006-02-23 Stefan Bickert Method for recording a charcteristic of at least one object
US7016109B2 (en) 2002-09-10 2006-03-21 Olympus Optical Co., Ltd. Microscopic image capture apparatus and microscopic image capturing method
US20060077536A1 (en) 2004-10-07 2006-04-13 Bromage Timothy G Portable automated confocal microscope
US20060098861A1 (en) 2002-07-18 2006-05-11 See Chung W Image analysis method, apparatus and software
US7071969B1 (en) 2001-09-27 2006-07-04 National Semiconductor Corporation Parameterized preview array for iterative image optimization in remote applications
US20060146283A1 (en) 2002-10-28 2006-07-06 Oliver Baumann Ophthalmologic apparatus and related positioning method
US20060164623A1 (en) 2005-01-21 2006-07-27 Wagner Randall M Method and system for online evaluation of gemstones
US20060171560A1 (en) 2005-01-12 2006-08-03 Johannes Manus Video tracking and registering
US20060179992A1 (en) 2003-01-04 2006-08-17 Omid Kermani Microtome
CA2504245A1 (en) 2005-04-11 2006-10-11 Meinan Machinery Works, Inc. Method of inspecting a broad article
JP2006343595A (en) 2005-06-09 2006-12-21 Sumitomo Osaka Cement Co Ltd Confocal inspection device
US7171030B2 (en) 2000-11-30 2007-01-30 University Of Medicine & Denistry Of New Jersey Systems for analyzing microtissue arrays
US20070025606A1 (en) 2005-07-27 2007-02-01 Bioimagene, Inc. Method and system for storing, indexing and searching medical images using anatomical structures of interest
US20070091324A1 (en) 2003-12-22 2007-04-26 Leica Mircosystems Cms Gmbh Microscope system and method for operating a microscope system
US7233340B2 (en) 2003-02-27 2007-06-19 Applied Imaging Corp. Linking of images to enable simultaneous viewing of multiple objects
US20070198001A1 (en) 2006-02-21 2007-08-23 Brainlab Ag Computer network system and method for operating a network system screen shot and source shot control
US20070207061A1 (en) 2004-06-04 2007-09-06 California Institute Of Technology Optofluidic microscope device
US20070224699A1 (en) 2006-03-23 2007-09-27 Gates Jackson L X-ray visualizer, laser-beam operated micro-dissector, automated tissue processor
US20070285768A1 (en) * 2006-05-24 2007-12-13 Olympus Corporation Microscope system and method for synthesizing microscopic images
US20080002252A1 (en) * 2006-06-09 2008-01-03 Wegu-Device Inc. Method and apparatus for the auto-focussing infinity corrected microscopes
US20080020128A1 (en) 2006-07-24 2008-01-24 Van Ryper William John Automated microscope for blood cell analysis
US7349482B2 (en) 2002-11-22 2008-03-25 Samsung Electronics Co., Ltd. Echo canceller of ADSL system and training method thereof
US20080095424A1 (en) 2004-09-22 2008-04-24 Nikon Corporation Microscope System And Image Processing Method
US20080095467A1 (en) 2001-03-19 2008-04-24 Dmetrix, Inc. Large-area imaging by concatenation with array microscope
US20080142708A1 (en) 2006-10-31 2008-06-19 Workman Richard K System for scanning probe microscope input device
US7394979B2 (en) 2002-09-04 2008-07-01 Carl Zeiss Meditec Ag Camera adapter for optical devices, in particular microscopes
US7394482B2 (en) 2002-10-28 2008-07-01 Leica Microsystems Cms Gmbh Microscope system and method for the analysis and evaluation of multiple colorings of a microscopic specimen
US7400342B2 (en) 2002-11-25 2008-07-15 Carl-Zeiss-Stiftung Optical observation apparatus with video device
US7406215B2 (en) 1999-12-15 2008-07-29 Thomas Clune Methods and systems for correcting image misalignment
US20080180794A1 (en) 2006-08-04 2008-07-31 Ikonisys, Inc. Microscope enclosure system
US7426345B2 (en) 2005-04-11 2008-09-16 Claro Inc. Slide photograph data creation system, and slide photograph data
US20080240613A1 (en) * 2007-03-23 2008-10-02 Bioimagene, Inc. Digital Microscope Slide Scanning System and Methods
JP2008262100A (en) 2007-04-13 2008-10-30 Olympus Corp Sample scanner device, and sample position detecting method using device
WO2008141009A1 (en) 2007-05-10 2008-11-20 Illumina, Inc. Microarray analytical data stitching system and method
US7482600B2 (en) 2004-09-13 2009-01-27 Leica Microsystems Cms Gmbh Method for investigating transport processes
US7486329B2 (en) * 2001-11-29 2009-02-03 Olympus Optical Co., Ltd. Focusing state signal output apparatus for a microscope representing a relative distance between a subject and an image sensing apparatus
US20090040322A1 (en) 2007-08-06 2009-02-12 Microsoft Corporation Enhancing digital images using secondary optical systems
JP2009036969A (en) 2007-08-01 2009-02-19 Nikon Corp Cover glass, slide glass, preparation, observation method, and microscopic device
US7502519B2 (en) 2003-02-28 2009-03-10 Aperio Technologies, Inc. Systems and methods for image pattern recognition
EP2051051A2 (en) 2007-10-16 2009-04-22 Cambridge Research &amp; Instrumentation, Inc. Spectral imaging system with dynamic optical correction
US20090140169A1 (en) 2007-10-22 2009-06-04 Mmi Gmbh Molecular Machines & Industries Method and device for three dimensional microdissection
JP2009192824A (en) 2008-02-14 2009-08-27 Olympus Corp Method of controlling observation apparatus, observation apparatus, and program for controlling observation apparatus
EP2110696A1 (en) 2008-04-15 2009-10-21 Sensovation AG Method and apparatus for autofocus
US7623697B1 (en) 2004-07-28 2009-11-24 Genetix Corp. Linking of images to enable simultaneous viewing of multiple objects
US20100000383A1 (en) 2007-08-07 2010-01-07 Koos David S Microscope coupled tissue sectioning system
US7653300B2 (en) 2005-03-18 2010-01-26 Olympus Corporation Image recording apparatus for microscopes
US20100020157A1 (en) 2008-07-25 2010-01-28 Jan Jelinek Optical fingerprint acquisition
US7663078B2 (en) 2004-09-02 2010-02-16 3Dhistech Kft. Focusing method for the high-speed digitalisation of microscope slides and slide displacing device, focusing optics, and optical rangefinder
US20100039507A1 (en) 2008-08-13 2010-02-18 Olympus Corporation Image acquisition apparatus and endoscope system
EP2169379A2 (en) 2008-09-24 2010-03-31 Sysmex Corporation Sample imaging apparatus
US20100093022A1 (en) 2006-11-28 2010-04-15 Kenneth Hayworth Methods and apparatus for providing and processing sliced thin tissue
US20100102571A1 (en) 2008-10-28 2010-04-29 Fu-Hung Yang Manpower Power Generator
US20100109725A1 (en) 2008-10-31 2010-05-06 Yun Won Joo Dll circuit having duty cycle correction and method of controlling the same
US20100118133A1 (en) 2008-08-08 2010-05-13 Leica Biosystems Nussloch Gmbh Method and device for producing thin sections of a sample by means of an image recognition system
US20100118393A1 (en) 2008-11-07 2010-05-13 Kuang-Yuan Lin Electronic device for biological microscopy
US20100134655A1 (en) 2008-11-28 2010-06-03 Nikon Corporation Image file generation device, camera and image file generation method
US20100141753A1 (en) 2000-05-03 2010-06-10 Aperio Technologies, Inc. Optimizing Virtual Slide Image Quality
US20100141752A1 (en) * 2008-12-04 2010-06-10 Tatsuki Yamada Microscope System, Specimen Observing Method, and Computer Program Product
US20100141751A1 (en) * 2008-12-08 2010-06-10 Olympus Corporation Imaging system for microscope and microscope system
US20100171809A1 (en) 2008-12-08 2010-07-08 Olympus Corporation Microscope system and method of operation thereof
US7756357B2 (en) 2003-07-01 2010-07-13 Olympus Corporation Microscope system for obtaining high and low magnification images
US20100188738A1 (en) 2008-09-30 2010-07-29 Carl Zeiss Smt Ag Optical imaging device and imaging method for microscopy
US20100194873A1 (en) 2007-07-19 2010-08-05 Viereck Friedrich W Method and device for microscopically examining a sample, computer program and computer program product
US20100201800A1 (en) 2009-02-09 2010-08-12 Olympus Corporation Microscopy system
US20100225668A1 (en) 2009-03-09 2010-09-09 Biolmagene, Inc. Modes and Interfaces for Observation, and Manipulation of Digital Images on Computer Screen in Support of Pathologist's Workflow
DE102009012293A1 (en) 2009-03-11 2010-09-16 Sensovation Ag Light auto-focusing method for use in microscope, involves arranging detector elements such that profiles of radiation property registered by detector elements are different and focus position is set based on profiles
WO2010105015A2 (en) 2009-03-11 2010-09-16 The University Of North Carolina At Chapel Hill Methods, systems, and computer readable media for microscopy tracking
US7822257B2 (en) 2005-09-12 2010-10-26 Olympus Corporation Observation apparatus and observation method
US20100279342A1 (en) 2009-05-01 2010-11-04 Yukimitsu Kijima Thin-section slides manufacturing apparatus and method for manufactring thin-section slides
US7840300B2 (en) 2006-05-31 2010-11-23 Robert Arthur Harker Full spectrum lapidary 3D image scanner and method
US20100295932A1 (en) 2009-05-25 2010-11-25 Olympus Corporation Image obtaining apparatus, image synthesis method and microscope system
US20100310139A1 (en) * 2009-05-29 2010-12-09 Olympus Corporation Biological observation apparatus
US7864414B2 (en) 2005-09-22 2011-01-04 Nikon Corporation Microscope and virtual slide forming system
US7873193B2 (en) 2005-06-21 2011-01-18 Carl Zeiss Microimaging Gmbh Serial section analysis for computer-controlled microscopic imaging
US20110038523A1 (en) 2009-08-12 2011-02-17 Genetix Corporation Image segmentation
US20110043663A1 (en) 2009-08-20 2011-02-24 Olympus Corporation Imaging terminal, display terminal, display method, and imaging system
US20110064296A1 (en) 2008-05-16 2011-03-17 Huron Technologies International Inc. Imaging system with dynamic range maximization
US7912267B2 (en) 2008-11-26 2011-03-22 Olympus Corporation Virtual-slide specimen image acquisition apparatus
US20110074817A1 (en) 2009-09-30 2011-03-31 Shinichi Shinoda Composite picture forming method and picture forming apparatus
US7920163B1 (en) * 1999-06-15 2011-04-05 Tessera International, Inc. Sealed, waterproof digital electronic camera system and method of fabricating same
US20110102571A1 (en) 2009-10-29 2011-05-05 Olympus Corporation Microscope Apparatus and Microscope Observation Method
US20110109735A1 (en) 2009-11-09 2011-05-12 Olympus Corporation Virtual microscope system
US7944608B2 (en) 2007-12-25 2011-05-17 Olympus Corporation Microscope
US7967057B2 (en) 2005-11-30 2011-06-28 Kobe Steel, Ltd. Induction melting apparatus employing halide type crucible, process for producing the crucible, method of induction melting, and process for producing ingot of ultrahigh-purity Fe-, Ni-, or Co-based alloy material
US8000562B2 (en) 2007-12-14 2011-08-16 Xerox Corporation Image downsampling for print job processing
US8000560B2 (en) 2007-02-05 2011-08-16 Olympus Corporation Virtual slide generation device, virtual slide generation method, virtual slide generation program product and virtual slide generation program transmission medium
US20110221881A1 (en) 2010-03-10 2011-09-15 Olympus Corporation Virtual-Slide Creating Device
US8077959B2 (en) 2009-09-30 2011-12-13 General Electric Company Stain-based optimized compression of digital pathology slides
US8074547B2 (en) 2006-01-25 2011-12-13 Seiko Instruments Inc. Automatic slicing apparatus
US8085296B2 (en) * 2005-09-06 2011-12-27 Kabushiki Kaisha Toshiba Method and apparatus for measuring an operating position in a remote inspection
US20110316999A1 (en) 2010-06-21 2011-12-29 Olympus Corporation Microscope apparatus and image acquisition method
US20110316993A1 (en) 2010-06-23 2011-12-29 Shih-Chi Chen Oscillating microtome with flexure drive
US20120002043A1 (en) 2009-03-27 2012-01-05 Sony Corporation Observation apparatus
US8098279B2 (en) 2006-11-30 2012-01-17 Nikon Corporation Imaging apparatus and microscope
WO2012024627A1 (en) 2010-08-20 2012-02-23 Sakura Finetek U.S.A., Inc. Digital microscope
US8125534B2 (en) 2008-01-24 2012-02-28 Keyence Corporation Image processing apparatus
US20120069344A1 (en) 2009-01-29 2012-03-22 The Regents Of The University Of California High resolution structured illumination microscopy
US20120069171A1 (en) 2010-09-17 2012-03-22 Olympus Corporation Imaging device for microscope
US20120081536A1 (en) 2010-09-30 2012-04-05 Stephan Kuppig Microscope System, Microscopy Method and Storage Medium
US8159547B2 (en) 2004-07-30 2012-04-17 Hamamatsu Photonics K.K. Imaging apparatus and microscope apparatus using the same
US8174763B2 (en) 2007-12-27 2012-05-08 Cytyc Corporation Methods and systems for controlably scanning a cytological specimen
US20120127297A1 (en) 2010-11-24 2012-05-24 Baxi Vipul A Digital microscopy with focus grading in zones distinguished for comparable image structures
US8187536B2 (en) 2003-09-12 2012-05-29 Leica Biosystems Nussloch Gmbh System and method for the unequivocal allocation of histological cassettes and specimen slides
US8203575B2 (en) 2007-06-11 2012-06-19 3Dhistech Kft. Method and system for accessing a slide from a remote workstation
US20120163680A1 (en) 2010-12-28 2012-06-28 Sakura Finetek U.S.A., Inc. Automated system and method of processing biological specimens
US20120208184A1 (en) 2010-11-15 2012-08-16 Timothy Ragan Systems and methods for imaging and processing tissue
US8283176B2 (en) 2004-06-29 2012-10-09 Dako Denmark A/S Method of pre-treatment and staining of a biological sample and device for support of a biological sample and methods of using such device
US8305434B2 (en) 2009-06-08 2012-11-06 Olympus Corporation Photographing apparatus and microscope system
CN102782557A (en) 2010-02-12 2012-11-14 徕卡显微系统复合显微镜有限公司 Scanning microscope and method for optically scanning one or more samples
US8339703B2 (en) 2005-09-29 2012-12-25 Leica Microsystems Cms Gmbh Scanning microscope and method for manipulating samples by means of a manipulating light beam in a scanning microscope
CN102841079A (en) 2011-06-22 2012-12-26 徕卡显微系统复合显微镜有限公司 Method and light-microscopic device to show a sample as an image
US8350904B2 (en) 2008-10-14 2013-01-08 Olympus Corporation Microscope for virtual-slide creating system
US8385686B2 (en) 2003-10-29 2013-02-26 Nikon Corporation Image processing method based on partitioning of image data, image processing device based on partitioning image data and program
US8396669B2 (en) 2003-02-21 2013-03-12 Leica Biosystems Melbourne Pty Lt Analysis system and procedures
US20130076886A1 (en) 2011-09-27 2013-03-28 Olympus Integrated Technologies America, Inc. Automatic Focus and Sample Detection
US20130140459A1 (en) 2011-12-01 2013-06-06 Gatan, Inc. System and method for sample analysis by three dimensional cathodoluminescence
US8476585B2 (en) 2011-03-02 2013-07-02 Gatan, Inc. Microtome utilizing a movable knife in a retardation field scanning electron microscope and a retardation field scanning electron microscope including the same
US20130182922A1 (en) 2002-09-13 2013-07-18 Life Technologies Corporation Interactive and automated tissue image analysis with global training database and variable-abstraction processing in cytological specimen classification and laser capture microdissection applications
US8501435B2 (en) 2008-10-09 2013-08-06 Sti Medical Systems, Llc Process for preserving three dimensional orientation to allow registering histopathological diagnoses of tissue to images of that tissue
US20130250090A1 (en) 2010-11-19 2013-09-26 Olympus Corporation Method of preparing biological specimen
US8565503B2 (en) 2010-06-04 2013-10-22 Leica Biosystems Imaging, Inc. System and method to determine slide quality of a digitized microscope slide
US8582489B2 (en) 2009-11-24 2013-11-12 Electronics And Telecommunications Research Institute Method for recovering a frame that failed to be transmitted in a mu-mimo based wireless communication system
US20140030757A1 (en) 2012-07-30 2014-01-30 Aspect Imaging Ltd Guided slicing system for obtaining histological samples and methods thereof
US20140051158A1 (en) 2011-04-26 2014-02-20 Daisuke Nakajima Sliced specimen preparing apparatus
US20140049632A1 (en) 2012-08-15 2014-02-20 Lucid, Inc. Systems and methods for imaging tissue
US20140087411A1 (en) 2012-09-27 2014-03-27 University Of Southern California System and method for determining tumor invasiveness
US20140085453A1 (en) 2012-09-25 2014-03-27 Sony Corporation Image display device and image display system
US20140086463A1 (en) 2011-05-13 2014-03-27 Koninklijke Philips N.V. Generating a slicing scheme for slicing a specimen
US20140098376A1 (en) 2011-06-02 2014-04-10 Dune Medical Devices Ltd. Tissue sampling for pathological study
US20140112560A1 (en) 2007-06-06 2014-04-24 Leica Biosystems Imaging, Inc. System and Method For Assessing Image Interpretability in Anatomic Pathology
US20140118528A1 (en) 2012-10-29 2014-05-01 Carl Zeiss Microscopy Gmbh Setting unit and method for setting a sequence for the automatic recording of images of an object by means of a recording device, and recording device having such a setting unit
US8725237B2 (en) 2012-03-19 2014-05-13 Genetic Innovations, Inc. Devices, systems, and methods for virtual staining
US20140130613A1 (en) 2012-11-13 2014-05-15 Ues, Inc. Automated high speed metallographic system
US20140137715A1 (en) 2011-06-14 2014-05-22 Vib Vzw Microtome chamber, sample holder and sample processing method
US8827760B2 (en) 2011-11-28 2014-09-09 Carrie Park Ushibo Peripheral apparatus for positioning and using a portable electronic device
US20150177504A1 (en) 2013-12-23 2015-06-25 Sakura Finetek U.S.A., Inc. Microscope with slide clamping assembly
CN200952935Y (en) * 2006-05-25 2007-09-26 北京四维远见信息技术有限公司 Digital stereo microscopic measuring system
CN100535704C (en) * 2006-10-10 2009-09-02 北京华旗资讯数码科技有限公司;北京华旗数码技术实验室有限责任公司 Method for controlling digit microscope output image by data-handling terminal
JP2009025362A (en) * 2007-07-17 2009-02-05 Nikon Corp Microscopic image pickup apparatus and microscope system
2011-08-18 US US13/212,955 patent/US10139613B2/en active Active
2011-08-19 JP JP2013525005A patent/JP6186634B2/en active Active
2011-08-19 CN CN201180047558.XA patent/CN103140789B/en active IP Right Grant
2011-08-19 EP EP15154503.5A patent/EP2910993A1/en active Pending
2011-08-19 EP EP11749675.2A patent/EP2606394B1/en active Active
2011-08-19 DE DE202011110631.8U patent/DE202011110631U1/en active Active
2011-08-19 ES ES11749675.2T patent/ES2623029T3/en active Active
2011-08-19 DK DK11749675.2T patent/DK2606394T3/en active
2011-08-19 CA CA2808105A patent/CA2808105C/en active Active
2011-08-19 BR BR112013004006A patent/BR112013004006A2/en active Search and Examination
2011-08-19 DK DK15194968.2T patent/DK3018520T3/en active
2011-08-19 WO PCT/US2011/048488 patent/WO2012024627A1/en active Application Filing
2011-08-19 ES ES15194968.2T patent/ES2692204T3/en active Active
2011-08-19 CN CN201710309215.5A patent/CN107255863A/en active Search and Examination
2011-08-19 EP EP15194968.2A patent/EP3018520B1/en active Active
2011-08-19 DE DE202011110651.2U patent/DE202011110651U1/en active Active
2011-08-19 AU AU2011291517A patent/AU2011291517B2/en active Active
2017-04-27 JP JP2017087763A patent/JP6462763B2/en active Active
US7856131B2 (en) 1996-08-23 2010-12-21 Olympus America Inc. Method and apparatus for internet, intranet, and local viewing of virtual microscope slides
US20110145755A1 (en) 1996-08-23 2011-06-16 Olympus America Inc. Method and apparatus for internet, intranet, and local viewing of virtual microscope slides
US8306298B2 (en) 1996-08-23 2012-11-06 Olympus America Inc. Method and apparatus for internet, intranet, and local viewing of virtual microscope slides
US7542596B2 (en) 1996-08-23 2009-06-02 Olympus America Inc. Method and apparatus for internet, intranet, and local viewing of virtual microscope slides
US20100260407A1 (en) 2000-05-03 2010-10-14 Aperio Technologies, Inc. Viewing Digital Slides
US8094902B2 (en) * 2000-05-03 2012-01-10 Aperio Technologies, Inc. Data management in a linear-array-based microscope slide scanner
US20110037847A1 (en) 2000-05-03 2011-02-17 Aperio Technologies, Inc. Fully Automatic Rapid Microscope Slide Scanner
US20120114204A1 (en) 2000-05-03 2012-05-10 Aperio Technologies, Inc. Optimizing Virtual Slide Image Quality
US20130162802A1 (en) 2000-05-03 2013-06-27 Aperio Technologies, Inc. Fully Automatic Rapid Microscope Slide Scanner
US20120002892A1 (en) 2000-05-03 2012-01-05 Aperio Technologies, Inc. Viewing Digital Slides
US7250963B2 (en) * 2001-12-19 2007-07-31 Olympus Corporation Microscopic image capture apparatus
US7248403B2 (en) 2002-09-10 2007-07-24 Olympus Corporation Microscopic image capture apparatus and microscopic image capturing method
US20120120225A1 (en) 2003-02-05 2012-05-17 Hamamatsu Photonics K.K. Microscope system and method
US20120281931A1 (en) 2003-02-28 2012-11-08 Aperio Technologies, Inc. Image Processing and Analysis Framework
US7141802B2 (en) 2003-12-01 2006-11-28 Olympus Corporation Optical device and imaging method
US20050239113A1 (en) 2004-04-06 2005-10-27 Affymetrix, Inc. Methods and devices for microarray image
US7630113B2 (en) 2004-05-10 2009-12-08 Nikon Corporation Microscope optical system, microscope, and virtual slide forming system
US7957057B2 (en) 2004-05-10 2011-06-07 Nikon Corporation Microscope optical system, microscope, and virtual slide forming system
US20100177166A1 (en) 2004-05-27 2010-07-15 Aperio Technologies, Inc. Creating and Viewing Three Dimensional Virtual Slides
US8366857B2 (en) 2004-07-08 2013-02-05 Kenneth Hayworth Methods, apparatus and systems for production, collection, handling, and imaging of tissue sections
US20130216451A1 (en) 2004-07-08 2013-08-22 President And Fellows Of Harvard College Methods, apparatus and systems for production, collection, handling, and imaging of tissue sections
US7778485B2 (en) 2004-08-31 2010-08-17 Carl Zeiss Microimaging Gmbh Systems and methods for stitching image blocks to create seamless magnified images of a microscope slide
US8094914B2 (en) 2004-09-22 2012-01-10 Nikon Corporation Microscope system and image processing method used for observation of a specimen
US7949161B2 (en) 2006-05-24 2011-05-24 Olympus Corporation Microscope system and method for synthesizing microscopic images
US20090116101A1 (en) 2006-08-04 2009-05-07 Ikonisys, Inc. Microscope enclosure system
US20120076436A1 (en) 2007-03-23 2012-03-29 Ventana Medical Systems, Inc. Digital microscope slide scanning system and methods
US20120076411A1 (en) 2007-03-23 2012-03-29 Ventana Medical Systems, Inc. Digital microscope slide scanning system and methods
US8098956B2 (en) 2007-03-23 2012-01-17 Vantana Medical Systems, Inc. Digital microscope slide scanning system and methods
US20120076391A1 (en) 2007-03-23 2012-03-29 Ventana Medical Systems, Inc. Digital microscope slide scanning system and methods
US8687858B2 (en) 2008-08-08 2014-04-01 Leica Biosystems Nussloch Gmbh Method and device for producing thin sections of a sample by means of an image recognition system
US8730315B2 (en) 2009-10-29 2014-05-20 Olympus Corporation Microscope apparatus and microscope observation method
US20130003172A1 (en) 2010-02-12 2013-01-03 Leica Microsystems Cms Gmbh Scanning microscope and method for optically scanning one or more samples
US8388891B2 (en) 2010-12-28 2013-03-05 Sakura Finetek U.S.A., Inc. Automated system and method of processing biological specimens
US20130164781A1 (en) 2010-12-28 2013-06-27 Sakura Finetek U.S.A., Inc. Automated system and method of processing biological specimens
US8747746B2 (en) 2010-12-28 2014-06-10 Sakura Finetek U.S.A., Inc. Automated system and method of processing biological specimens
US20140273086A1 (en) 2010-12-28 2014-09-18 Sakura Finetek U.S.A., Inc. Automated System and Method of Processing Biological Specimens
Chinese Third Office Action dated Apr. 8, 2018. CN Application No. 201480021837.2.
EP Application No. 15154503.5.
Haruhisa, S., et al., "Application of telepathology for improvement of therapeutic effects and economic efficiency, and development of new equipment for it", Science Links Japan; http://sciencelinks.jp/j-east/article/200516/000020051605A0431066.php, Journal Code: N20051113, (2005), 166-125.
Sakura Finetek U.S.A., "Extended Search Report", European Application No. 14198636, (dated Sep. 30, 2015).
Sakura Finetek U.S.A., Inc., "EP Supplementary Search Report", EP Application No. 14784707.3, (dated Oct. 4, 2016).
Sakura Finetek U.S.A., Inc., "Examination Report", CA Application No. 2908058, (dated Nov. 16, 2016).
Sakura Finetek U.S.A., Inc., "Examiner's Report", CA Application No. 2808105, dated Jun. 12, 2017.
Sakura Finetek U.S.A., Inc., "Examiner's Report", CA Application No. 2908058, dated Jul. 24, 2017.
Sakura Finetek U.S.A., Inc., "Final Office Action", JP Application No. 2016-507909 (dated Apr. 28, 2017).
Sakura Finetek U.S.A., Inc., "Final Office Action", U.S. Appl. No. 14/138,740, (dated Jan. 26, 2017).
Sakura Finetek U.S.A., Inc., "Final Office Action", U.S. Appl. No. 14/138,740, dated Jun. 20, 2017.
Sakura Finetek U.S.A., Inc., "Final Office Action", U.S. Appl. No. 14/779,550 (dated May 24, 2017).
Sakura Finetek U.S.A., Inc., "Final Rejection", JP Application No. P2013-525005, (dated Dec. 27, 2016).
Sakura Finetek U.S.A., Inc., "First Office Action with search report", CN Application No. 2014800218372, (dated Nov. 30, 2016).
Sakura Finetek U.S.A., Inc., "First Office Action", EP Application No. 15194968.2, (dated Mar. 10, 2017).
Sakura Finetek U.S.A., Inc., "Fourth Office Action", CN Application No. 201180047558X, (dated Oct. 24, 2016).
Sakura Finetek U.S.A., Inc., "International Preliminary Report on Patentability", International Application No. PCT/US2014/034477, (dated Oct. 29, 2015).
Sakura Finetek U.S.A., Inc., "International Search Report and Written Opinion", International Application No. PCT/US2014/034477, (dated Sep. 22, 2014).
Sakura Finetek U.S.A., Inc., "Non final office action", U.S. Appl. No. 14/138,740, (dated Jul. 1, 2016).
Sakura Finetek U.S.A., Inc., "Non-Final Office Action", U.S. Appl. No. 14/779,550, (dated Jan. 19, 2017).
Sakura Finetek U.S.A., Inc., "Patent Examination Report No. 1", AU Application No. 201453889, (dated May 18, 2016).
Sakura Finetek U.S.A., Inc., "Second Office Action", CN Application No. 2014800218372, dated Aug. 1, 2017.
Sakura Finetek U.S.A., Inc., "Third Office Action", TW Application No. 201180047558X, (dated Apr. 1, 2016).
Sakura Finetek U.S.A., Inc., Chinese Final Office Action dated Jul. 3, 2014 for CN Application No. 201080017649.4.
Sakura Finetek U.S.A., Inc., Chinese First Office Action dated Feb. 9, 2018, CN Application No. 201410415253.5.
Sakura Finetek U.S.A., Inc., Chinese second office action dated Dec. 27, 2013 for CN201080017649.4.
Sakura Finetek U.S.A., Inc., et al., Canadian Examiner's Report dated Dec. 7, 2012 for CA 2,755,164.
Sakura Finetek U.S.A., Inc., et al., International Preliminary Report on Patentability dated Mar. 7, 2013 for PCT/US2011/048488.
Sakura Finetek U.S.A., Inc., European First Office Action dated Mar. 5, 2018, EP Application No. 14784707.3.
Sakura Finetek U.S.A., Inc., European Office Action dated Feb. 22, 2018.
Sakura Finetek U.S.A., Inc., Examination Report dated Jun. 19, 2014 for Australian App No. 2011291517.
Sakura Finetek U.S.A., Inc., Extended European Search Report dated Apr. 24, 2018, EP Application No. 17202516.
Sakura Finetek U.S.A., Inc., Extended European Search Report, EP App No. 15194968.2 (dated Mar. 18, 2016).
Sakura Finetek U.S.A., Inc., Final office action dated Sep. 2, 2014 for Japanese App No. 2011-553548.
Sakura Finetek U.S.A., Inc., Final office action, U.S. Appl. No. 13/255,827, (dated Oct. 20, 2015).
Sakura Finetek U.S.A., Inc., Non-Final Office Action dated Dec. 22, 2017, U.S. Appl. No. 14/779,550.
Sakura Finetek U.S.A., Inc., Notice of Allowance dated Feb. 13, 2018, U.S. Appl. No. 14/138,740.
Sakura Finetek U.S.A., Inc., Notice of rejection for Japanese Application No. 2013-525005, (dated Feb. 9, 2016).
Sakura Finetek U.S.A., Inc., PCT Search Report and Written Opinion dated Sep. 22, 2014 for International Application No. PCT/US2014/034477, 12 pages.
Sakura Finetek U.S.A., Inc., Second office action dated Jul. 6, 2015 for Chinese Appln. No. 201180047558.
Sakura Finetek USA Inc., "Office Action", EP Application No. 15154503.5, (dated Feb. 28, 2017).
Sakura Finetek USA, Extended Search Report for EP15154503 dated Jun. 19, 2015.
Sakura Finetek USA, Inc., "Office Action", JP Application No. 2016-507909, (dated Sep. 15, 2016).
Sakura Finetek, Australian Examination Report dated Dec. 24, 2013 for AU 2011291517, 3 pages.
Sakura Finetek, Australian Office Action dated Nov. 26, 2013 for Australian App No. 2010222633, 3 pages.
Sakura Finetek, EPO Office Action dated Jul. 30, 2013 for EPO App No. 10719379.9, 8 pages.
Sakura Finetek, Japanese Office Action dated Dec. 10, 2013 for JP App No. P2011-553548, 9 pages.
Sakura Finetek, PCT Search Report and Written Opinion dated Oct. 13, 2011 for Int'l Application No. PCT/US2011/048488., 13 pages.
Sensovation AG, PCT International Preliminary Report on Patentability dated Sep. 20, 2011 for Int'l Application No. PCT/IB2010/000518., 7 pages.
AU2011291517B2 (en) 2015-09-24
EP2606394A1 (en) 2013-06-26
DE202011110631U1 (en) 2015-06-03
EP3018520A1 (en) 2016-05-11
CN103140789A (en) 2013-06-05
EP2910993A1 (en) 2015-08-26
BR112013004006A2 (en) 2016-06-28
JP2017134434A (en) 2017-08-03
AU2011291517A1 (en) 2013-02-28
WO2012024627A1 (en) 2012-02-23
ES2692204T3 (en) 2018-11-30
JP6462763B2 (en) 2019-01-30
JP2013536471A (en) 2013-09-19
EP3018520B1 (en) 2018-10-03
EP2606394B1 (en) 2017-03-01
JP6186634B2 (en) 2017-08-30
CN107255863A (en) 2017-10-17
DK2606394T3 (en) 2017-04-10
DE202011110651U1 (en) 2015-06-03
DK3018520T3 (en) 2018-12-10
CA2808105A1 (en) 2012-02-23
CA2808105C (en) 2018-12-04
CN103140789B (en) 2017-05-24
ES2623029T3 (en) 2017-07-10
US20120044342A1 (en) 2012-02-23
US7027133B2 (en) 2006-04-11 Automated laser capture microdissection
EP0155247B1 (en) 1991-06-05 A method for microphotometering microscope specimens
JP3962104B2 (en) 2007-08-22 Apparatus for acquiring method and infrared data operating an infrared microscope
EP0557871A2 (en) 1993-09-01 Method and apparatus for automated assay of biological specimens
US7050622B2 (en) 2006-05-23 Image comparison apparatus, image comparison method, and program for causing computer to execute image comparison
JP4937850B2 (en) 2012-05-23 Microscope system, the vs image generation method, a program
EP1353212B1 (en) 2006-06-07 Microscope apparatus
US7426345B2 (en) 2008-09-16 Slide photograph data creation system, and slide photograph data
EP1598688A2 (en) 2005-11-23 Fluorescence microscope
US20030214706A1 (en) 2003-11-20 Microscopy imaging system and method
US7978898B2 (en) 2011-07-12 Image acquiring apparatus, image acquiring method, and image acquiring program
JP5698398B2 (en) 2015-04-08 Slide the entire fluorescence scanner
JP5058444B2 (en) 2012-10-24 Micrographs device and photomicrographs apparatus control method
US6804049B2 (en) 2004-10-12 Microscope device and microscope system
US7602996B2 (en) 2009-10-13 Microscope system and method
EP1967885A1 (en) 2008-09-10 Apparatus for measuring luminescence dose and method of measuring luminescence
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HING, PAUL;ROMER, CHRISTIAN;HENSLER, SVEN;SIGNING DATES FROM 20110818 TO 20110824;REEL/FRAME:026807/0880