Patent ID: 12254652

FIG.1shows a schematic representation of an embodiment of the surgical visualization system1. The surgical visualization system1is configured to carry out the method described in this disclosure.

The surgical visualization system1comprises a medical microscope2, for example a surgical microscope, and a data processing device3.

The medical microscope2comprises a light source2-1and an imaging unit2-2. The light source2-1is configured to generate at least one light marking10in a target region21on a patient20. A pose31of the light source2-1in a reference coordinate system30and/or a light path32in the reference coordinate system30is known. The imaging unit2-2is configured to capture at least one image representation4containing the at least one light marking10generated in the target region21. A pose33of the imaging unit2-2in the reference coordinate system30is known.

The data processing device3comprises a computing device3-1and a memory3-2. To determine three-dimensional positions34of points in the target region21in the reference coordinate system30of the surgical visualization system1, the data processing device3is configured to determine a three-dimensional position34of the at least one generated light marking10in the reference coordinate system30proceeding from image coordinates41of the at least one light marking10in the captured at least one image representation4, with the known pose31of the light source2-1and/or the respectively associated light path32and the known pose33of the imaging unit2-2being taken into account. The image coordinates41are coordinates in a two-dimensional image coordinate system40which for example contains a x-coordinate axis and a y-coordinate axis of pixels in the captured image representation4.

Further, the data processing device3is configured to prompt multiple repetition of the generation and capture of the at least one light marking10in the target region21and determination of the three-dimensional position34, and, in this respect, to prompt a respective change in a position of the at least one light marking10within the target region21during each repetition. In particular, the repetitions are implemented for a given number of positions.

The data processing device3provides the three-dimensional positions34determined for the points of the target region21in the reference coordinate system30on the basis of the light markings10. By way of example, this is implemented as an analogue or digital signal, for example in the form of a data packet.

It is possible to estimate a surface contour of the target region21proceeding from the provided three-dimensional positions34. Three-dimensional measurement data from an imaging method (e.g., CT, MRI, etc.) of the target region21can then be registered with the determined positions34or estimated surface contour on the basis thereof. During an operation, the three-dimensional measurement data can then be overlaid on the captured image representation4of the target region21.

Provision can be made for the poses31,33of the light source2-1and the imaging unit2-2to be fixed relative to one another, at least during the implementation of the generation of the at least one light marking10and the capture of the image representation4, with the light source2-1being moved together with the imaging unit2-2for the purpose of changing the position of the at least one light marking10in order to repeat the measures. In particular, a respective light path21is also fixed relative to the at least one light source2-1and at least one imaging unit2-2. To this end, the surgical visualization system1in particular comprises a robotic actuator system5, which is designed in the embodiment shown as a multi-link and multi-joint robotic arm, on which the medical microscope2is arranged. The position of the at least one light marking10in the target region21is changed by controlling the robotic actuator system5. The reference coordinate system30is also moved in the process. This changes a position of the at least one light marking10in the target region21because the light source2-1and imaging unit2-2move relative to the patient20. As a result, different positions can be marked with a light marking10within the target region21such that a respectively associated three-dimensional position34on the patient20can be determined within the reference coordinate system30.

Provision can be made for the position of the at least one light marking10to be changed successively in accordance with a set of predetermined positions. By way of example, in this case provision can be made for more positions, for example, to be homed in on for partial regions within the target region21than in the remaining partial regions, for example because a greater number of sampling points, that is to say a higher resolution, is desired there.

Provision can be made for the position of the at least one light marking10upon repetition to be changed in such a way that at least one partial region of the target region21is scanned. In particular, scanning in this case comprises line-by-line or column-by-column scanning of at least the partial region.

Provision can be made for at least one quality value6to be determined proceeding from the captured image representations4and/or the three-dimensional points34determined for the points in the target region21, with the three-dimensional positions34of the target region21being at least partially determined anew if the determined at least one quality value6drops below an associated specified minimum quality value7. The quality value6may comprise the quantities already described in the general description, with the minimum quality value7comprising a quantity corresponding therewith.

Provision can be made for a laser2-3to be used as a light source2-1. In particular, this can be a laser diode.

Provision can be made for the laser2-3also to serve to ascertain a focal distance of the medical microscope2. In particular, provision can be made for two lasers2-3to be provided to this end, which lasers are imaged onto the target region21by an imaging optical unit8. This is illustrated schematically inFIG.2. The illustrated embodiment corresponds to the embodiment shown inFIG.1. In this case, identical reference signs designate identical features and terms. The imaging optical unit8of the medical microscope2and/or the lasers2-3are then controlled in such a way that the light markings10generated by means of the two lasers2-3are made to fully overlap. What this can achieve is that the target region21is arranged at the correct focal distance from the imaging optical unit8of the medical microscope2and/or that a focal distance is known or able to be determined.

In particular, provision can be made for the one (or more) lasers2-3and/or the imaging optical unit8to be controllable in terms of a position and/or alignment. In this case, a position of the at least one light marking10can be changed by controlling the laser or lasers2-3and/or imaging optical unit8.

Provision can be made for the change in the position of the at least one light marking10in measure d) to comprise defocussing. In particular, the two (or more) lasers2-3also used to determine a focal distance are used here to generate the at least one light marking10. By way of example, provision can be made for a work distance of the medical microscope2from the target region21to be changed in order to attain a defocussing The light markings10of the two (or more) lasers2-3in the target region21made to fully overlap in the focus migrate apart during the defocussing, with the result that the positions of the light markings10change. Provision can alternatively or additionally be made for the positions of the light markings10to be changed by active control of the two (or more) lasers2-3if a position and/or an alignment of the two (or more) lasers2-3can be changed.

Provision can be made for at least two imaging units2-2to be used, with the at least two imaging units2-3being cameras9-xof the surgical visualization system1.

In particular, provision can be made for the at least two cameras9-2,9-3to be part of a stereo camera of the surgical visualization system1. In particular, provision can be made for the method to be carried out exclusively by means of the two cameras9-2,9-3of the stereo camera.

However, provision can also be made for at least one of the cameras9-xto be a surround camera9-1of the surgical visualization system1. By way of example, provision can be made for the latter to be used in addition to the cameras9-2,9-3of the stereo camera. This can improve quality when determining the positions34of the points in the target region21.

Provision can be made for at least one marking22securely arranged on the patient20(“patient target”) to be captured by means of the at least one imaging unit2-2in addition to the at least one light marking10, with a pose of the at least one marking22in the reference coordinate system30being determined and provided. This is shown schematically inFIG.2. There, the marking22is fastened to the head of the patient20by means of a clamp23such that said marking does not move relative to the head when the patient20or the head of the patient20is repositioned. This can allow a repositioning of the patient20without the positions34within the target region21having to be determined anew since already determined positions34can be converted (transformed) for the new pose of the target region21by way of the known pose of the marking22within the reference coordinate system30.

FIG.3shows a schematic flowchart of an embodiment of the method for determining the three-dimensional position of points in a target region on a patient in a reference coordinate system of a surgical visualization system. The method is carried out by means of one of the above-described embodiments of the surgical visualization system.

In a method step100, at least one light marking is generated by means of at least one light source of the surgical visualization system in the target region, with a pose of the at least one light source in the reference coordinate system and/or a light path in the reference coordinate system being known.

In a method step101, at least one imaging unit of the surgical visualization system is used to capture at least one image representation containing the at least one light marking generated in the target region, with a pose of the at least one imaging unit in the reference coordinate system being known.

In a method step102, a three-dimensional position of the at least one light marking in the reference coordinate system is determined proceeding from image coordinates of the at least one light marking in the captured at least one image representation, with the known pose of the at least one light source and/or the respectively associated light path and the known pose of the at least one imaging unit being taken into account.

In a method step103, a check is carried out as to whether a specified number of positions have already been determined for the target region. By way of example, provision can be made here for the target region to be at least partially scanned and/or raster scanned. In particular, a number of positions within the target region is defined in such a way here that a sufficient number of sampling points is available for the purpose of determining a surface contour of the target region on the patient (cf. method step107).

If a given number of positions has not yet been captured, a position of the at least one light marking within the target region is changed in a method step104, for example to the next position within a predetermined grid. Subsequently, method steps100to103are repeated.

By contrast, if the check in method step103yields that all positions have been determined, the three-dimensional positions determined for the points in the target region in the reference coordinate system on the basis of the light markings are provided in a method step106. By way of example, this is implemented as an analogue or digital signal, for example in the form of a data packet.

In a method step107, provision can be made for the provided points to be used to determine a surface of the target region within the reference coordinate system and carry out a registration of three-dimensional measurement data of an imaging method (CT, MRI, etc.) with the determined surface. Following the registration, it is possible to arrange the three-dimensional measurement data within the reference coordinate system and represent these overlaid on the target region. The three-dimensional measurement data can then be displayed to a surgeon in a manner overlaid with captured image representations of the target region in order to support a workflow in this way.

In a method step105, provision can be made for at least one quality value to be determined proceeding from the captured image representations and/or the three-dimensional points determined for the points in the target region, with the three-dimensional positions of the target region being at least partially determined anew if the determined at least one quality value drops below an associated specified minimum quality value. The at least one quality value can also be determined on the basis of a registration, as described for method step107, for example by virtue of a deviation of the individual registered points from one another being determined. If the check in method step105yields that the minimum quality value was undershot, the method is once again continued with method step100, with the positions and/or changed positions of the target region being captured anew. By way of example, the light markings can be arranged on a different grid, with in particular a distance between the points of the grid being able to be changed. In this way, it is possible for example to increase the resolution during the capture and determination.

In method step104, provision can be made for the at least one light source to be moved jointly with the at least one imaging unit for the purpose of changing the position of the at least one light marking. In this case, provision is made for the poses of the at least one light source and the at least one imaging unit to be fixed relative to one another, at least while carrying out measures100,101,102and104. In particular, a light path is also fixed relative to the at least one light source and at least one imaging unit. The change in position is implemented, in particular, by means of a robotic actuator system of the surgical visualization system.

Alternatively, provision can be made in method step104for the change in the position of the at least one light marking in method step104to comprise a change in a pose of the at least one light source and/or a change in the light path of a light emanating from the at least one light source. By way of example, this can be implemented by means of actuator systems configured to this end, in particular under open-loop and/or closed-loop control by the data processing device and/or a controller of the surgical visualization system.

Further, provision can be made for method step104to additionally or alternatively provide for the change in the position of the at least one light marking to comprise (incrementally increasing) defocussing.

Provision can be made in method step101for at least one marking securely arranged on the patient to be captured by means of the at least one imaging unit in addition to the at least one light marking, with a pose of the at least one marking in the reference coordinate system being determined and provided.

LIST OF REFERENCE SIGNS

1Surgical visualization system2Medical microscope2-1Light source2-2Imaging unit2-3Laser3Data processing device3-1Computing device3-2Memory4Image representation5Robotic actuator system6Quality value7Minimum quality value8Imaging optical unit9-1Surround camera9-2Camera (in particular of the stereo camera system)9-3Camera (in particular of the stereo camera system)10Light marking20Patient21Target region22Marking23Clamp30Reference coordinate system31Pose (light source)32Light path33Pose (imaging unit)34Three-dimensional position40Image coordinate system41Image coordinates100-107Method steps