Patent ID: 12232809

The system10shown inFIG.1contains a surgical microscope16for visualizing the section18of the patient's eye14with magnification. By way of example, the surgical microscope16can be the OPMI® Lumera 660. Rescan surgical microscope by Carl Zeiss Meditec AG. The system10comprises an OCT device20which provides an OCT scanning beam21for scanning the section18of the patient's eye14with an A-, B- and C-scan, as described, e.g., in chapter 3, pages 45 to 82 in A. Ehnes, “Entwicklung eines Schichtsegmentierungsalgorithmus zur automatischen Analyse von individuellen Netzhautschichten in optischen Kohärenztomographie—B-Scans”, Dissertation, University of Giessen (2013).

The system10comprises a reference object24embodied as a surgical instrument, which, on the basis of a marker22′, is identifiable and localizable in the section18of the patient's eye14by means of the OCT device20. An additional marker22can be arranged on the reference object24outside of the patient's eye14.

The surgical microscope16comprises a stereoscopic observation beam path38,40, which facilitates the examination of the patient's eye14through a microscope main objective42in the section18of the patient's eye14. The surgical microscope16comprises a zoom system44and an eyepiece46. It comprises an illumination device48which can illuminate the section18with illumination light through the microscope main objective42for the purposes of stereoscopically visualizing the patient's eye14in the eyepiece46.

The OCT device20provides the OCT scanning beam21with short coherent light, which is guided through the microscope main objective42to the section18of the patient's eye14by way of adjustable scanning mirrors50,52and beam splitters54and56. The light of the OCT scanning beam21scattered in the section18returns at least in part to the OCT device20via the same beam path. Then, the optical path length of the scanning light is compared in the OCT device20to the optical path length of a reference path. Using this, it is possible to capture the precise location of scattering centers in the section18, in particular the position of optically effective areas, with an accuracy which corresponds to the coherence length lcof the short coherent light in the OCT scanning beam21.

On account of refraction and path length changes of the OCT scanning beam in the patient's eye, errors may arise when determining the location of the scattering centers in the section18of the patient's eye14, which are visible as distortions in the captured OCT scanning data. As a matter of principle, these aberrations are not time-invariant since the optical properties of a patient's eye14can change during surgery.

In the surgical microscope16, there is a device58for controlling the OCT scanning beam21and for setting the position of the section18of the patient's eye14scanned by the OCT scanning beam21. The device58contains a computer unit60. The computer unit60has an input interface61as a means for entering information and commands by a user and contains a computer program for controlling the OCT scanning beam21and adjusting the spatial extent and position, i.e. the location and orientation, of the section18of the patient's eye14scanned by the OCT scanning beam21. In this case, the device58for controlling the OCT scanning beam21is embodied for successive continuous scanning of the section18and of the region of the section18of the patient's eye14containing the reference object24by means of the OCT scanning beam21. In this case, the OCT scanning beam21has a frame rate of 10 ms to 20 ms in order to allow the surgeon to have fast hand-eye coordination.

The device58for controlling the OCT scanning beam21contains a display unit28which is connected to the computer unit60and which is in the form of a display with an interface29for displaying the captured distorted OCT scanning data30with the reference object24and the generated rectified visualization data32of the section18of the patient's eye14. Moreover, in the system10, the OCT scanning information for the OCT device20may be visualized for a surgeon in the eyepiece46of the surgical microscope16by means of a device for mirroring-in data34.

Further, the computer program in the program memory of the computer unit60contains a control routine which specifies the reference length for the OCT scanning beam21and the settings of the adjustable scanning mirrors50,52for scanning the section18of the patient's eye14. There is a control member62, actuatable by an operator, in the device58for setting the section18scanned by means of the OCT scanning beam21. Moreover, the control routine contains a scanning routine for scanning the reference object24by way of special scanning patterns. In the process, the section18of the patient's eye14is scanned at a lower rate in comparison with the reference object24in order to keep the amount of data as small as possible and hence the computing time as short as possible.

The computer program in the program memory of the computer unit60serves to process the OCT scanning data30into the visualization data32within the scope of an image rectification algorithm80, which is designed to output the visualization data32at the interface29. The computer program moreover contains a view generation algorithm78for calculating image data in relation to a view76of a reference object24arranged in the section18of the patient's eye14from geometry data74about the reference object24fed to the view generation algorithm78and from the OCT scanning data30obtained in relation to the reference object24. In this case, the computer unit60has an algorithm control routine which specifies the image rectification algorithm80and determines the image rectification algorithm80from the image data of the view76of the reference object24calculated in the view generation algorithm78and from OCT scanning data30obtained in relation to the reference object24by scanning the section18of the patient's eye14.

FIG.2Ashows a view of a patient's eye14with a cornea12and a retina15. As a result of pressure on the patient's eye14, the shape and the curvature of the cornea12changes inFIG.2B, and so there is also change in the optical properties of the patient's eye14. This leads to distortions in the captured OCT scanning data30, as can be seen inFIG.3andFIG.4.FIG.3shows distorted OCT scanning data30of an anterior chamber of eyeball with a cornea12and an iris13.FIG.4shows distorted OCT scanning data30of a posterior chamber of eyeball with a section of the retina15. Both OCT scanning data30of the anterior chamber of the eyeball and OCT scanning data30of the posterior chamber of the eyeball can be rectified by means of the image rectification algorithm80.

FIG.5shows a flowchart for one embodiment variant of the computer-implemented method. Here, OCT scanning data30of the section18of the patient's eye14with a reference object24arranged therein are captured in a data capture step82. These OCT scanning data30form the input for the view generation algorithm78. In an object recognition step84, the type of reference object24is recognized first within the view generation algorithm78. In the present case, the type of reference object is understood to mean, for example, the use of the surgical instrument or the type of an implant arranged in the section18of the patient's eye14. In this case, the object recognition step84can be carried out by means of image processing or machine learning on the basis of the OCT scanning data30. Then, geometry data74in relation to the reference object24are loaded from a database, for example in the form of a CAD data record, on the basis of the type of reference object24within the scope of a geometry data provision step86. Alternatively, the reference object24can also be moved outside of the eye while OCT scanning data of the reference object24are captured. Then, the geometry data74of the reference object24can be ascertained by means of a 3D reconstruction method. A view76of the reference object24is generated in a view generation step88, which view corresponds as accurately as possible to the image representation75of the reference object24in the OCT scanning data30. Image processing algorithms or, as an alternative or in addition thereto, machine learning algorithms, for example matching methods, are used for this purpose. If the position and location of the reference object24are known relative to the OCT device20, for example on account of measurement using a measuring device, by recognizing markers22,22′ or by way of a robotic guide of the reference object24, it is possible to directly ascertain a suitable view76of the reference object24from the geometry data74and the known position and location of the reference object24. The output of the view generation algorithm78consists of the view76of the reference object24generated from the geometry data74. Now, the image rectification algorithm80is ascertained from the generated view76and the captured OCT scanning data30, for example by determining a rectification mapping77in a rectification mapping determination step90. In this case, a segmentation of the reference object24can be ascertained in the OCT scanning data30within the scope of a first step. Then, the rectification mapping77is determined on the basis of an image processing or machine learning method, by virtue of ascertaining the corresponding points of the reference object24in the generated view76of the reference object for points of the reference object24in the OCT scanning data30. Preferably, prominent points can be chosen here, for example edges and corners, in order to achieve matching that is as accurate as possible. The point correspondences define a mapping between the image representation75of the reference object24in the captured OCT scanning data30and the reference object24in the generated view76of the reference object24. Then, the rectification mapping77restricted to the points of the reference object24is determined from the point correspondences, for example by parameter estimation or by solving an optimization problem or by extrapolating the individual point correspondences to the neighboring points within the reference object24. In order also to rectify the OCT scanning data30outside of the reference object24, the rectification mapping77is defined for points arranged in the vicinity of the reference object24by means of an extrapolation79. Points whose distance from the reference object24exceeds a threshold can likewise be rectified by the extrapolation79, or they remain unchanged. The image rectification algorithm80generated in this way applies the rectification mapping77to the captured OCT scanning data30in an image rectification step92and thereby generates rectified visualization data32, which are output on the interface29.

On the basis of the visualization data32, rectified OCT scanning data30can be displayed to a surgeon during the surgery. The visualization data32can also be used to measure distances between points in the section18of the patient's eye14with a greater accuracy. Finally, the visualization data32can also be used to determine a refractive index of a medium in the section18of the patient's eye14.

FIG.6shows the processing steps of the captured OCT scanning data30when determining the image rectification algorithm80and the visualization data32. The distorted OCT scanning data30contain an image representation75of the reference object24and further distorted structures33outside of the image representation75of the reference object24in the section18of the patient's eye14. The type of reference object24is ascertained by means of object recognition in an object recognition step84, and the geometry data74in relation to this reference object24are generated. Then, a view76of the reference object24, which corresponds as accurately as possible to the reference object24in the captured OCT scanning data30, is generated in a view generation step88from the captured OCT scanning data30of the reference object24and from the geometry data74in relation to the reference object24. No complete correspondence of the two views76is possible on account of the distortions in the OCT scanning data30. In this case, the generated view76of the reference object24can be three-dimensional or two-dimensional. By way of example, a projection of the geometry data74can be determined for a two-dimensional view76. Then, the image representation75of the reference object in the OCT scanning data30is ascertained first by means of segmentation or marking within the scope of a rectification mapping determination step90and then a rectification mapping77is determined, which maps points of the image representation75of the reference object24onto points of the previously generated view76of the reference object24. The rectification mapping77is also extrapolated to the points of the OCT scanning data30in the vicinity of the reference object24. This rectification mapping77forms the image rectification algorithm80, which is applied to the OCT scanning data30in an image rectification step92. As a result, both the image representation75of the reference object24and the further structures33in the OCT scanning data30are rectified. The result of the image rectification algorithm80are the visualization data32, which contain a rectified image representation75′ of the reference object24and rectified structures33′ of the OCT scanning data30in the vicinity of the reference object24.

FIG.7shows a further system10′ with a surgical microscope16, with an OCT device20for scanning a section18of a patient's eye14, with a reference object24in the form of a surgical instrument, and with a robotics unit68. To the extent that the components and elements of the further system10′ shown inFIG.7correspond to the components and elements of the first system10visible inFIG.1, these have been identified with the same numerals as reference signs.

The robotics unit68comprises a micro robot70with a control unit72. By way of example, the micro robot70can be embodied as a manipulator for surgical instruments with motor drives, as provided in the ophthalmic surgical operating system R1.1 by Preceyes B.V.

To ensure automation of surgery to the greatest possible extent, a movement of the reference object24embodied as a surgical instrument here is set by means of the micro robot70. The micro robot70of the robotics unit68is controlled in this case on the basis of the information items processed by the computer unit60.

On the basis of the control commands in the micro robot70, the position and location of the reference object24in the form of a surgical instrument is known. This simplifies generating the view76of the reference object24.

To sum up, the following preferred features of the invention should be noted: A system10,10′ comprises an interface29for providing visualization data32for visualizing at least one section18of a patient's eye14and comprises an OCT device20for capturing OCT scanning data30by scanning the section18of the patient's eye14by means of an OCT scanning beam21. In the system10,10′, there is a computer unit60for processing the OCT scanning data30into the visualization data32within the scope of an image rectification algorithm80, which is designed to output the visualization data32at the interface29. The computer unit60contains a view generation algorithm78for calculating image data in relation to a view76of a reference object24arranged in the section18of the patient's eye14from geometry data74about the reference object24fed to the view generation algorithm78and from the OCT scanning data30obtained in relation to the reference object24. In the computer unit60, there is an algorithm control routine which specifies the image rectification algorithm80and determines the image rectification algorithm80from the image data of the view76of the reference object24calculated in the view generation algorithm78and from OCT scanning data30obtained in relation to the reference object24by scanning the section18of the patient's eye14.

LIST OF REFERENCE SIGNS

10,10′ System12Cornea13Iris14Patient's eye15Retina16Surgical microscope18Section20OCT device21OCT scanning beam22,22′ Marker24Reference object28Display unit29Interface30OCT scanning data32Visualization data33Structures33′ Rectified structures34Mirroring-in data38,40Stereoscopic observation beam path42Microscope main objective44Zoom system46Eyepiece48Illumination device50,52Scanning mirror54,56Beam splitter58Device60Computer unit61Input interface62Control member68Robotics unit70Micro robot72Control unit74Geometry data75Image representation75′ Rectified image representation76View77Rectification mapping78View generation algorithm79Extrapolation80Image rectification algorithm82Data capture step84Object recognition step86Geometry data provision step88View generation step90Rectification mapping determination step92Image rectification step94Computer-implemented method