Patent Description:
In the field of microscopy, it is a crucial demand to find an operational setting of the microscope in order to achieve an optimal image quality. Usually, the operational setting comprises a plurality of setting parameters which have to be optimized simultaneously before the image acquisition can be started. Optimizing the setting parameters is a cumbersome and time-consuming task for a user, which has to be repeated for each image acquisition. Therefore, in order to achieve an optimal image quality in an easy and reproducible manner, it is highly desirable to provide the user with a tool to make it easier for him or her to select an optimal operational setting.

Recently, machine learning algorithms have been developed assisting the user to find a proper operational setting without having to adjust a large number of setting parameters each time the microscope is used. An example of a microscope system using a machine learning algorithm for providing an operational setting is disclosed in the document <CIT>. The machine learning algorithm disclosed therein allows the microscope system to learn rules according to which setting parameters are adjusted. This learning of the system happens during the manufacturing or development cycle of the microscopes.

An imaging system according to the preamble of claim <NUM> is disclosed in <CIT>.

It is an object of the present invention to provide an imaging system comprising an imaging device and a controller as well as a method for enabling a user to find a proper operational setting of the imaging device fast and with minimum effort.

In order to achieve the afore-mentioned object, an imaging system according to claim <NUM> is proposed, comprising an imaging device and a controller for providing an operational setting of the imaging device.

Before an image is acquired, the controller provides the user with a preferred operational setting. This preferred operational setting may directly be used for the subsequent image acquisition, i.e. without being changed by the user prior to the image acquisition. Alternatively, the preferred operational setting may be changed by the user before the image is acquired. In such a case, the preferred operational setting is overwritten with the changed operational setting, this changed operational setting subsequently being used for the image acquisition.

After the image has been acquired based on the operational setting, which may be the afore-mentioned preferred setting provided by the controller in the first place or the setting changed by the user and overwriting the preferred setting, the controller receives a user input based on which the controller can decide whether or not the image generated by the imaging device according to the selected operational setting is discarded by the user. Subsequently, based on this discard information derived from the user input and based on the operational setting used for acquiring the image, the controller updates the preferred operational setting applying a machine learning algorithm.

The discard information represents a simple decision criterion which can be used by the machine learning algorithm for updating the preferred operational setting of the imaging device efficiently. In particular, as the discard information is generated based on the user input during the imaging operation performed by the imaging device, the process for updating the operational setting using the machine learning algorithm can be performed while the user is actually working with the imaging device. Thus, it is not necessary to implement a separate learning process preceding the actual image acquisition during which the user is working with the imaging device. In other words, the user does not have to manually vote if the acquired image is a good one or not, as it is e.g. taught by the aforementioned document <CIT>. In contrast the controller of the imaging system according to embodiments of the present invention concludes based on the behavior of the user if the acquired image is a good one or not. Based on the conclusion the controller can feed the machine learning algorithm.

The proposed controller enables a reliable prediction of a preferred operational setting and an immediate application of the predicted setting. Thus, it is possible to improve the work flow, and the user is relieved of the need to adjust a plurality of setting parameters before the image acquisition can even be started. The prediction is based on preceding user inputs which indicate whether the current image acquired in the on-going imaging process is accepted by the user or discarded.

Preferably, the controller is further configured to conclude that the image is a good image when the discard information indicates that the image is not discarded by the user, to conclude that the image is not a good image when the discard information indicates that the image is discarded by the user, and to correspondingly feed the machine learning algorithm with an information indicating whether or not the image was concluded to be a good image. In the following, a good image may be understood as an image meeting the user's expectations in particular in terms of image quality so that the resulting user input subsequent to the image acquisition indicates that the image is accepted, i.e. not discarded by the user. Thus, a good image may be an image with which the user continues to work during the imaging process in one way or another. An image not accepted by the user to be a good image as defined above will be referred to as a bad image hereinafter.

The controller is configured to update the preferred operational setting further based on the operational setting used for acquiring the image. The latter setting, which is used for image acquisition, may be either the afore-mentioned preferred operational setting, which is provided by the controller in the first place, or a setting which is deliberately adjusted by the user to be used for the image acquisition. This deliberately adjusted setting may be derived from the preferred operational setting.

The update of the preferred operational setting, i.e. the prediction of a next setting to be used for image acquisition, is made on the basis of the previous setting. Thus, the controller feeds the machine learning algorithm with the current operational setting in order to make a decision on an operational setting which is likely to be required by the user at a later stage.

Preferably, the controller is configured to update the preferred operational setting so as to reinforce the operational setting used for acquiring the image when the image was concluded to be a good image, and/or to update the preferred operational setting so as to attenuate the operational setting used for acquiring the image when the image was concluded not to be a good image. In such an embodiment, a reinforcement machine learning may be performed in such a way that a good image causes a reinforcement of the selected operational setting, so to speak, the update of the preferred operational setting integrates the setting used for acquiring the image. In contrast, a bad image causes an attenuation of the current operational setting, i.e. the update of the preferred operational setting departs from the setting used for acquiring the image. For instance, it is assumed that a preferred setting parameter has value A. Then, the user overwrites A with a changed parameter value B, and the image is acquired based on value B. In case that the acquired image is a good image, then the subsequent preferred parameter may be B. On the other hand, in case that the acquired image is a bad image, the next preferred setting parameter may be again A.

The controller is configured to generate the discard information indicating that the image is discarded if after acquiring the image a setting parameter of a first group of parameters of the operational setting is changed by the user compared to the previous operational setting with which the image was acquired.

The first group of parameters includes at least one of exposure time setting, gain setting, contrast setting and illumination light setting.

The illumination light setting may comprise a setting for each light type including an ON/OFF setting for a ring light in combination with light segments, an ON/OFF setting for coaxial light, an ON/OFF setting for transmitted light.

The controller is configured to generate the discard information indicating that the image is not discarded if after acquiring the image only a setting parameter of a second group of parameters of the operational setting is changed by the user compared to the previous operational setting.

The second group of parameters includes at least one of a zoom setting and a stage setting.

The controller may be configured to control the imaging device automatically based on the updated operational setting.

The imaging system is a microscope system, for instance a confocal microscope system.

According to another aspect, a method according to claim <NUM> is provided.

According to another aspect, a computer program according to claim <NUM> is provided.

Hereinafter, preferred embodiments are described with reference to the drawings in which:.

An imaging system <NUM> according to an embodiment is shown in the block diagram of <FIG>. According to the invention, the imaging system <NUM> is formed by a microscope system.

The imaging system <NUM> comprises an imaging device <NUM>, which is a microscope, and a controller <NUM>. In the present invention, the controller <NUM> may include an input module <NUM> and a learning module <NUM>. Both modules <NUM>, <NUM> may be implemented in hardware and/or software. Further, in the present embodiment, the controller <NUM> is configured to control the overall operation of the imaging system <NUM>. In particular, the controller <NUM> provides a user with a preferred operational setting when starting the imaging device <NUM>. This preferred operational setting comprises a plurality of setting parameters which serve to control the imaging device <NUM> in order to acquire an image. The preferred operational setting may have been stored in advance in memory means included in the controller <NUM>. In order to inform the user on the preferred operational setting, the imaging system <NUM> may comprise e.g. a monitor not shown in <FIG>, on which the preferred operational setting is displayed.

After the preferred operational setting has been adjusted, the imaging device <NUM> is operated in order to generate the image. For this, the imaging device <NUM> is either controlled based on the preferred operational setting or based on an operational setting which has been changed by the user before the image is acquired. In the latter case, the preferred operational setting may be overwritten in the memory means of the controller <NUM> in response to a corresponding user input.

After the image has been captured by the imaging device, the input module <NUM> of the controller <NUM> receives a user input made by the user by means of a suitable input device included in the imaging system <NUM>. Such an input device may be a control panel, a keyboard etc. configured to transmit the user input to the input module <NUM>. The controller <NUM> is configured to generate a discard information based on the user input, i.e. to derive said discard information from the user input such that the discard information indicates whether or not the image generated by the imaging device <NUM> is discarded by the user.

Subsequently, based on the discard information derived from the user input, the controller <NUM> updates the preferred operational setting using the learning module <NUM>. For this, the learning module <NUM> is configured to apply a machine learning algorithm on input data including the discard information. This algorithm enables a prediction of an operational setting which is assumed to be preferred by the user when operating the imaging device for the next image acquisition. In this respect, it is worth noting that the controller <NUM> is configured to update the operational setting by means of the learning module <NUM> while the user is actually working with the imaging device <NUM>. In other words, the image acquisition and the process for updating the operational setting are performed simultaneously. Thus, the controller <NUM> is enabled to control the imaging device <NUM> automatically based on the updated operational setting.

In addition to the discard information indicating whether or not the image generated by the imaging device <NUM> has been discarded by the user, the learning module <NUM> of the controller <NUM> is further fed with the operational setting which is currently used for acquiring the image in order to predict the next update of the setting.

<FIG> is a flow chart illustrating an exemplary workflow for updating the operational setting of the imaging device <NUM> included in the imaging system <NUM> of <FIG>. For the sake of simplicity, <FIG> is a kind of hybrid diagram combining elements of a block diagram and a flow diagram. In particular, <FIG> illustrates the interaction between the learning module <NUM> and the remaining parts of the imaging system <NUM>.

In step S1 of the workflow shown in <FIG>, the controller <NUM> determines whether or not the image generated by the imaging device <NUM> is discarded by the user. For this, the controller <NUM> analyses the discard information derived from the user input which is entered by the user in response to the image presented to him or her e.g. by being displayed on a monitor. If the controller <NUM> determines in S1 that the image is discarded, the controller <NUM> concludes in step S2 that the image is a good image. On the other hand, if the controller determines in S1 that the image is discarded by the user, the controller concludes the image in step S3 to be a bad image.

The conclusion whether or not the image is discarded, i.e. whether the image is a good image or a bad image, is drawn depending on whether one or more specific setting parameters of the operational setting are changed by the user after the image has been acquired. The image is concluded to be a bad image, i.e. discarded (step S3) if after acquiring the image a setting parameter of a first group of parameters of the operational setting is changed by the user compared to the previous operational setting with which the image was acquired. The afore-mentioned first group of parameters includes at least one of exposure time setting, gain setting, contrast setting and illumination light setting.

On the other hand, the image is concluded to be a good image, i.e. not discarded (step S2), if after acquiring the image only a setting parameter of a second group of parameters of the operational setting is changed by the user compared to the previous operational setting with which the image was acquired. The afore-mentioned second group of parameters includes at least one of a zoom setting and a stage setting. In other words, the image is not discarded if only the zoom and/or stage setting has been changed.

In step S4 of the work flow shown in <FIG>, the learning module <NUM> of the controller <NUM> is fed with a discard information which indicates whether the current image is concluded to be a good image or a bad image. Further, in the present embodiment, the learning module <NUM> is further fed with the operational setting that was used for acquiring the current image. Applying the learning algorithm taking into account the afore-mentioned information, the learning module <NUM> makes a prediction based on which the preferred operational setting to be used in the next image acquisition is updated. As a result, the learning module <NUM> provides the imaging system <NUM> with the updated preferred operational setting to be used next.

When using the operational setting which was used for acquiring the current image, the learning module may update the preferred operational setting either in the affirmative or in the negative of the operational setting. Thus, the learning module <NUM> may be configured to apply a reinforcement machine learning in such a way that a good image causes a reinforcement of the selected operational setting whereas a bad image causes an attenuation of the current operational setting.

In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on the controller of the imaging system.

A further embodiment of the present invention is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when it is performed by the controller of the imaging system.

Claim 1:
An imaging system (<NUM>), comprising an imaging device (<NUM>) and a controller (<NUM>), wherein the controller is configured to:
provide a user with a preferred operational setting of the imaging device (<NUM>) for acquiring an image;
control the imaging device (<NUM>) to acquire the image based on an operational setting, wherein the preferred operational setting is used for the image acquisition with or without being changed by the user prior to the image acquisition,
receive a user input and to generate a discard information based on the user input, said discard information indicating whether or not the image generated by the imaging device (<NUM>) is discarded by the user; and
update the preferred operational setting using a machine learning algorithm based on the discard information,
characterized in that the controller (<NUM>) is configured to update the preferred operational setting further based on the operational setting used for acquiring the image,
wherein the controller (<NUM>) is configured to generate the discard information indicating that the image is discarded if after acquiring the image a setting parameter of a first group of parameters of the operational setting for a next image acquisition is changed by the user compared to the operational setting with which the image was acquired,
wherein the controller (<NUM>) is configured to generate the discard information indicating that the image is not discarded if after acquiring the image only a setting parameter of a second group of parameters of the operational setting is changed by the user compared to the operational setting with which the image was acquired,
wherein the first group of parameters includes at least of one of exposure time setting, gain setting, contrast setting, and illumination light setting,
wherein the second group of parameters includes at least one of a zoom setting and a stage setting, and
wherein the imaging system (<NUM>) is a microscope system.