Patent Description:
In digital holography, a light source irradiates an observation target with illumination light. Moreover, an interference fringe image is output from an imaging element by imaging, but the imaging element, an interference fringe between diffracted light which is illumination light diffracted by the observation target and reference light which is illumination light that does not pass through the observation target. The interference fringe image includes information on the observation target in a thickness direction along an irradiation direction of the illumination light. Therefore, by performing appropriate operation processing on the interference fringe image, it is possible to obtain a reconstructed image representing any tomographic plane of the observation target.

<CIT> discloses the technology of generating a super-resolution interference fringe image having a resolution exceeding a resolution of an imaging element and generating a reconstructed image from the super-resolution interference fringe image. Specifically, in <CIT>, illumination light is emitted from a plurality of irradiation positions having different irradiation angles, and an interference fringe image is output for each of the plurality of irradiation positions. Moreover, the super-resolution interference fringe images are generated based on the plurality of interference fringe images at the irradiation positions. <CIT> discloses a system for three-dimensional imaging of an object. The system comprises an image sensor, and an illumination source emitting partially coherent light. The illumination source is configured to illuminate the sample in a plurality of different angles. At each illumination angle several sub-pixel shifted images of the object are recorded. The sub-pixel images are then digitally processed to generate a single, high resolution (e.g., pixel super-resolution) hologram of each angular projection. <NPL>, refers to lensfree holographic microscopy to achieve high spatial resolution. By emitting partially coherent light from a large aperture lower resolution in-line holograms of the objects are created. A sub-pixel shifting based algorithm is used to recover much higher resolution digital holograms of the objects, permitting sub-micron spatial resolution to be achieved across the entire sensor chip active area. A device and method for detecting and sizing small particles (e.g. nanoparticles) is known from <CIT>. An image sensor faces towards an interior volume of a housing, where objects can be located. Vapor that resides in the interior volume can be formed. The objects are positioned within an optical path between an array of spatially separated light sources and the image sensor. The spatially separated light sources are individually turned ON/OFF to acquire raw, low resolution holographic images of the objects. These raw images are transferred to a separate computing device configured to generate time-resolved, super-resolution holograms from a plurality of low-resolution image frames. Peak phase values are extracted from phase image reconstructions obtained from the super-resolution holograms, wherein the one or more processors outputs a size of the objects based on the peak phase values. A calibration curve or look-up table that is empirically derived is used to translate peak phase values into object sizes.

In <CIT>, the illumination light is emitted from all of the plurality of irradiation positions, and the interference fringe image is output each time of the irradiation. However, depending on the position of the observation target, the interference fringe image, which has almost no contribution to a super-resolution and can be omitted, may be present in the plurality of interference fringe images corresponding to the plurality of irradiation positions. In this case, processing of capturing the interference fringe image having almost no contribution to the super-resolution is wasted.

The technology of the present disclosure is to provide a control device, an operation method of a control device, and an operation program of a control device in which a super-resolution interference fringe image having a resolution exceeding a resolution of an imaging element can be obtained without wasteful labor.

In order to achieve the above object, the present disclosure relates to a control device of an imaging apparatus including a light source and an imaging element, in which the light source is able to irradiate an observation target with illumination light from a plurality of irradiation positions having different irradiation angles, and the imaging element outputs an interference fringe image by imaging an interference fringe between diffracted light, which is the illumination light diffracted by the observation target, and reference light, which is the illumination light that does not pass through the observation target, the control device comprising an acquisition unit that acquires positional information indicating a position of the observation target, a setting unit that sets, from among the plurality of irradiation positions, required irradiation positions, which are irradiation positions corresponding to the position of the observation target indicated by the positional information and are irradiation positions required for obtaining a plurality of the interference fringe images that are sources of a super-resolution interference fringe image having a resolution exceeding a resolution of the imaging element, a light source control unit that emits the illumination light from the required irradiation position by controlling an operation of the light source, and an imaging control unit that outputs the interference fringe image from the imaging element at each required irradiation position.

It is preferable that the light source have a configuration in which a plurality of light emission units of the illumination light are arranged at the plurality of irradiation positions, and the light source control unit emit the illumination light from the light emission unit corresponding to the required irradiation position.

It is preferable that the light source include at least one light emission unit of the illumination light and a moving mechanism of the light emission unit, and the light source control unit emit the illumination light from the light emission unit while moving the light emission unit to the required irradiation position by the moving mechanism.

It is preferable that the acquisition unit acquire the positional information by detecting the position of the observation target from a standard interference fringe image, which is the interference fringe image obtained by emitting the illumination light from one preset standard irradiation position among the plurality of irradiation positions, or a standard reconstructed image, which is a reconstructed image representing any tomographic plane of the observation target and is a reconstructed image generated based on the standard interference fringe image.

It is preferable that the control device further comprise a display control unit that performs a control of displaying a display screen of a standard interference fringe image, which is the interference fringe image obtained by emitting the illumination light from one preset standard irradiation position among the plurality of irradiation positions, or a display screen of a standard reconstructed image, which is a reconstructed image representing any tomographic plane of the observation target and is a reconstructed image generated based on the standard interference fringe image, in which the acquisition unit acquires the positional information by receiving designation of the position of the observation target on the display screen.

It is preferable that the acquisition unit acquire size information indicating a size of the observation target, in addition to the positional information, and the setting unit change the number of the required irradiation positions in accordance with the size information.

It is preferable that, in a case in which a plurality of the observation targets are present and the required irradiation positions of the plurality of observation targets overlap, the light source control unit emit the illumination light only once from overlapping required irradiation positions.

It is preferable that the observation target be a cell in culture.

It is preferable that the illumination light be coherent light.

The present disclosure relates to an operation method of a control device of an imaging apparatus including a light source and an imaging element, in which the light source is able to irradiate an observation target with illumination light from a plurality of irradiation positions having different irradiation angles, and the imaging element outputs an interference fringe image by imaging an interference fringe between diffracted light, which is the illumination light diffracted by the observation target, and reference light, which is the illumination light that does not pass through the observation target, the operation method comprising an acquisition step of acquiring positional information indicating a position of the observation target, a setting step of setting, from among the plurality of irradiation positions, required irradiation positions, which are irradiation positions corresponding to the position of the observation target indicated by the positional information and are irradiation positions required for obtaining a plurality of the interference fringe images that are sources of a super-resolution interference fringe image having a resolution exceeding a resolution of the imaging element, a light source control step of emitting the illumination light from the required irradiation position by controlling an operation of the light source, and an imaging control step of outputting the interference fringe image from the imaging element at each required irradiation position.

The present disclosure relates to a storage device storing an operation program of a control device of an imaging apparatus including a light source and an imaging element, in which the light source is able to irradiate an observation target with illumination light from a plurality of irradiation positions having different irradiation angles, and the imaging element outputs an interference fringe image by imaging an interference fringe between diffracted light, which is the illumination light diffracted by the observation target, and reference light, which is the illumination light that does not pass through the observation target, the operation program causing a computer to function as an acquisition unit that acquires positional information indicating a position of the observation target, a setting unit that sets, from among the plurality of irradiation positions, required irradiation positions, which are irradiation positions corresponding to the position of the observation target indicated by the positional information and are irradiation positions required for obtaining a plurality of the interference fringe images that are sources of a super-resolution interference fringe image having a resolution exceeding a resolution of the imaging element, a light source control unit that emits the illumination light from the required irradiation position by controlling an operation of the light source, and an imaging control unit that outputs the interference fringe image from the imaging element at each required irradiation position.

According to the technology of the present disclosure, it is possible to provide the control device, the operation method of the control device, and the operation program of the control device in which the super-resolution interference fringe image having the resolution exceeding the resolution of the imaging element can be obtained without wasteful labor.

In <FIG>, a digital holography system <NUM> is composed of an imaging apparatus <NUM> and an information processing apparatus <NUM>. The imaging apparatus <NUM> and the information processing apparatus <NUM> are electrically connected to each other, and data can be exchanged with each other. A culture container <NUM> for a cell <NUM> is introduced into the imaging apparatus <NUM>. The cell <NUM> is an example of an "observation target" according to the technology of the present disclosure. The information processing apparatus <NUM> is, for example, a desktop-type personal computer.

In <FIG>, the imaging apparatus <NUM> comprises a light source <NUM>, a stage <NUM>, and an imaging element <NUM>. The light source <NUM> emits coherent light <NUM> to the culture container <NUM> placed on the stage <NUM>. The coherent light <NUM> is incident on the cell <NUM> and the culture container <NUM>. More specifically, as shown in <FIG>, the entire region of an observation region <NUM>, which is a partial region in the vicinity of the center of the culture container <NUM>, is irradiated with the coherent light <NUM>. The observation region <NUM> has a size of <NUM> × <NUM>, for example. The coherent light <NUM> is an example of "illumination light" according to the technology of the present disclosure. It should be noted that a Z direction is an irradiation direction of the coherent light <NUM>. An X direction and a Y direction are directions orthogonal to the Z direction and parallel to an imaging surface <NUM> (see <FIG>) of the imaging element <NUM>. In addition, the X direction and the Y direction are directions orthogonal to each other and are directions along an arrangement direction of pixels <NUM> (see <FIG>) of the imaging element <NUM>.

As shown in <FIG>, the coherent light <NUM> incident on the cell <NUM> and the culture container <NUM> is divided into diffracted light <NUM> diffracted by the cell <NUM> and the culture container <NUM> and transmitted light <NUM> transmitted without passing through the cell <NUM> and the culture container <NUM>. The diffracted light <NUM> and the transmitted light <NUM> interfere with each other on the imaging surface <NUM> of the imaging element <NUM> to generate an interference fringe <NUM>. The imaging element <NUM> images the interference fringe <NUM> and outputs an interference fringe image <NUM>. The transmitted light <NUM> is an example of "reference light" according to the technology of the present disclosure.

As shown in <FIG>, among lines representing the diffracted light <NUM> and the transmitted light <NUM>, a solid line indicates a wave surface having the maximum amplitude of the diffracted light <NUM> and the transmitted light <NUM>. On the other hand, a broken line indicates a wave surface having the minimum amplitude of the diffracted light <NUM> and the transmitted light <NUM>. A white spot <NUM> shown on the imaging surface <NUM> is a portion in which the wave surfaces of the diffracted light <NUM> and the transmitted light <NUM> are aligned and strengthened (see <FIG>). This portion of the white spot <NUM> appears as a bright portion <NUM> in the interference fringe <NUM>. On the other hand, a black spot <NUM> shown on the imaging surface <NUM> is a portion in which the wave surfaces of the diffracted light <NUM> and the transmitted light <NUM> deviate by half a wavelength and are weakened (see <FIG>). This portion of the black spot <NUM> appears as a dark portion <NUM> in the interference fringe <NUM>.

As shown in <FIG>, the light source <NUM> has a rectangular parallelepiped housing <NUM>. On a surface of the housing <NUM> facing the stage <NUM>, <NUM> × <NUM> = <NUM> light emission units <NUM> are arranged at equal intervals in the X direction and the Y direction. The light emission unit <NUM> individually emits the coherent light <NUM>. Examples of the light source <NUM> having such a configuration in which a plurality of light emission units <NUM> are arranged include a vertical cavity surface emitting laser (VCSEL) array element. It should be noted that the light emission unit <NUM> has a several µm-order size.

As shown in <FIG>, installation positions of the light emission units <NUM>, such as IP11, IP12,. , IP54, and IP55, are a plurality of irradiation positions of the coherent light <NUM> having different irradiation angles. By using the light source <NUM> having such a configuration in which the plurality of light emission units <NUM> are arranged at the plurality of irradiation positions IP <NUM> to IP55, a super-resolution interference fringe image <NUM> (see <FIG> and the like) having a resolution exceeding a resolution of the imaging element <NUM> can be generated. It should be noted that "different irradiation angles" means that incidence angles of the coherent light <NUM> on the imaging surface <NUM> of the imaging element <NUM> are different. In addition, <FIG> is a view of the light source <NUM> as viewed from a side of the imaging element <NUM>.

<FIG>, and <FIG> are views conceptually showing the generation principle of the super-resolution interference fringe image <NUM>. First, in <FIG> shows a case in which the coherent light <NUM> is emitted from certain light emission unit 41A, and <FIG> shows a case in which the coherent light <NUM> is emitted from a light emission unit 41B adjacent to the light emission unit 41A of <FIG> in the X direction. The incidence angle of the coherent light <NUM> from the light emission unit 41A on the cell <NUM> is different from the incidence angle of the coherent light <NUM> from the light emission unit 41B on the cell <NUM>. Therefore, pieces of information on the interference fringe <NUM> by the cell <NUM> obtained by the pixels <NUM> of the imaging element <NUM> are also different. Therefore, interference fringe images 34A and 34B having different pixel values are obtained in a case of <FIG> and a case of <FIG>, respectively. A circle mark represents a pixel value of the interference fringe image 34A, and a square mark represents a pixel value of the interference fringe image 34B. It should be noted that the pixel <NUM> of the imaging element <NUM> has a size of <NUM> × <NUM>, for example.

It is assumed that a sampling point of the cell <NUM> deviates by half of the pixel <NUM>, that is, by half a pixel between a case of <FIG> and a case of <FIG>. In this case, as shown in <FIG>, for example, the interference fringe image 34B obtained in a case of <FIG> deviate by half a pixel with the interference fringe image 34A obtained in a case of <FIG> as a standard to obtain an interference fringe image 34BB. Then, the interference fringe image 34A obtained in a case of <FIG> and the interference fringe image 34BB are integrated into an interference fringe image 34ABB. The interference fringe image 34ABB is an image having twice the number of pixels as the interference fringe images 34A and 34B. That is, the interference fringe image 34ABB is none other than the super-resolution interference fringe image <NUM> having the resolution exceeding the resolution of the imaging element <NUM>. It should be noted that the processing of causing the interference fringe image 34B to deviate by half a pixel to obtain the interference fringe image 34BB is called registration processing. In addition, the processing of integrating the interference fringe image 34A and the interference fringe image 34BB is called reconstruction processing.

In <FIG>, and <FIG>, the description is made in one dimension only in the X direction, but the basic idea of the generation principle of the super-resolution interference fringe image <NUM> is the same even in the two dimensions in which the Y direction is added. For example, a case is considered in which the coherent light <NUM> is emitted from <NUM> × <NUM> = <NUM> light emission units <NUM> adjacent to each other in the X direction and the Y direction, and the interference fringe image <NUM> is output from the imaging element <NUM> each time of the irradiation. In this case, it is assumed that the sampling point of the cell <NUM> deviates by half a pixel as described above, the super-resolution interference fringe image <NUM> having <NUM> × <NUM> = <NUM> times the number of pixels as the interference fringe image <NUM> output from the imaging element <NUM> is obtained.

In <FIG>, the imaging apparatus <NUM> comprises a storage device <NUM>, a memory <NUM>, and a central processing unit (CPU) <NUM>. The storage device <NUM> and the memory <NUM> are connected to the CPU <NUM>. The storage device <NUM>, the memory <NUM>, and the CPU <NUM> are examples of a "computer" according to the technology of the present disclosure.

The storage device <NUM> is a hard disk drive or a solid state drive. The memory <NUM> is a work memory in which the CPU <NUM> executes processing. The CPU <NUM> loads the program stored in the storage device <NUM> to the memory <NUM> and executes the processing in accordance with the program to comprehensively control the units of the computer.

An operation program <NUM> is stored in the storage device <NUM>. The operation program <NUM> is an application program for causing the computer composed of the storage device <NUM>, the memory <NUM>, and the CPU <NUM> function as a control device. That is, the operation program <NUM> is an example of an "operation program of a control device" according to the technology of the present disclosure. A required irradiation position table <NUM> is also stored in the storage device <NUM>.

In a case in which the operation program <NUM> is activated, the CPU <NUM> functions as a light source control unit <NUM>, an imaging control unit <NUM>, an acquisition unit <NUM>, a setting unit <NUM>, and a transmission control unit <NUM>, in cooperation with the memory <NUM> and the like. Among these units, the light source control unit <NUM>, the imaging control unit <NUM>, the acquisition unit <NUM>, and the setting unit <NUM> realize a control device <NUM> according to the present disclosure.

The light source control unit <NUM> controls an operation of the light source <NUM> and emits the coherent light <NUM> from the light emission unit <NUM>. The imaging control unit <NUM> controls an operation of the imaging element <NUM> and outputs the interference fringe image <NUM> from the imaging element <NUM>. The light source control unit <NUM> and the imaging control unit <NUM> synchronize an irradiation timing of the coherent light <NUM> from the light emission unit <NUM> with an imaging timing of the interference fringe image <NUM> by the imaging element <NUM>.

The acquisition unit <NUM> receives a standard interference fringe image 34R from the imaging element <NUM>. The acquisition unit <NUM> detects a position of the cell <NUM> from the standard interference fringe image 34R. As a result, the acquisition unit <NUM> acquires positional information <NUM> indicating the position of the cell <NUM>. The acquisition unit <NUM> outputs the positional information <NUM> to the setting unit <NUM>.

The setting unit <NUM> sets a required irradiation position, which is an irradiation position IP corresponding to the position of the cell <NUM> indicated by the positional information <NUM>, from among the plurality of irradiation positions IP11 to IP55, with reference to the required irradiation position table <NUM>. The required irradiation position is the irradiation position IP required for obtaining a plurality of interference fringe images <NUM> that are the sources of the super-resolution interference fringe image <NUM>. The setting unit <NUM> outputs setting information <NUM> indicating the required irradiation position to the light source control unit <NUM>.

The light source control unit <NUM> causes the light emission unit <NUM> to emit the coherent light <NUM> from the light emission unit <NUM> at the required irradiation position indicated by the setting information <NUM>. The imaging control unit <NUM> causes the imaging element <NUM> to output the interference fringe image <NUM> at each required irradiation position.

The transmission control unit <NUM> receives the interference fringe image <NUM> from the imaging element <NUM>. The transmission control unit <NUM> performs a control of transmitting the interference fringe image <NUM> to the information processing apparatus <NUM>. It should be noted that the interference fringe image <NUM> may be transitorily stored in the storage device <NUM> and then transmitted to the information processing apparatus <NUM> by the transmission control unit <NUM>.

As shown in <FIG>, the standard interference fringe image 34R is the interference fringe image <NUM> obtained by emitting the coherent light <NUM> from the light emission unit <NUM> at one irradiation position IP33 positioned at the center indicated by hatching among the plurality of irradiation positions IP11 to IP55. That is, the irradiation position IP33 is an example of a "standard irradiation position" according to the technology of the present disclosure.

As shown in <FIG>, the acquisition unit <NUM> performs image analysis on the standard interference fringe image 34R and detects a position of a center point C1 of the interference fringe <NUM> reflected in the standard interference fringe image 34R as the position of the cell <NUM>, for example. The acquisition unit <NUM> outputs a position coordinate (X_C1, Y_C1) of the center point C1 of the interference fringe <NUM> to the setting unit <NUM> as the positional information <NUM>.

<FIG> is a view showing a correspondence relationship between regions R11, R12,. , R54, and R55 obtained by dividing the interference fringe image <NUM> into <NUM> × <NUM> = <NUM> and the irradiation positions IP11 to IP55. For example, the region R11 in an upper left corner corresponds to the irradiation position IP15, and the region R15 in an upper right corner corresponds to the irradiation position IP11. In addition, the region R51 in a lower left corner corresponds to the irradiation position IP55, and the region R55 in a lower right corner corresponds to the irradiation position IP51.

In <FIG>, in the required irradiation position table <NUM>, the corresponding required irradiation position is registered for each of the regions R11 to R55 of the standard interference fringe image 34R. For example, in a case in which the region of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned is R11, the required irradiation positions are four of the irradiation position (hereinafter, referred to as a center point correspondence irradiation position) IP15 corresponding to the region R11, and the irradiation positions IP14, IP24, and IP25 adjacent to the center point correspondence irradiation position IP15. In a case in which the region of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned is R12, the required irradiation positions are six of the center point correspondence irradiation position IP14 corresponding to the region R12, and the irradiation positions IP13, IP15, IP23, IP24, and IP25 adjacent to the center point correspondence irradiation position IP14. In a case in which the region of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned is R42, the required irradiation positions are nine of the center point correspondence irradiation position IP44 corresponding to the region R42, and the irradiation positions IP33, IP34, IP35, IP43, IP45, IP53, IP54, and IP55 adjacent to the center point correspondence irradiation position IP44. As described above, the required irradiation position always includes the center point correspondence irradiation position corresponding to the region R of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned. Moreover, the required irradiation position is composed of the center point correspondence irradiation position and the irradiation position IP adjacent to the center point correspondence irradiation position. In addition, the number of required irradiation positions is four at the minimum and nine at the maximum.

<FIG> are views showing specific examples of the required irradiation position. First, <FIG> and <FIG> show a case in which the region of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned is R14. In this case, in accordance with the required irradiation position table <NUM>, the required irradiation positions are the irradiation positions IP11, IP12, IP13, IP21, IP22, and IP23. The light source control unit <NUM> causes the light emission unit <NUM> to emit the coherent light <NUM> from the light emission unit <NUM> in the order of the irradiation positions IP11, IP12, IP13, IP23, IP22, and IP21, for example.

First, <FIG> and <FIG> show a case in which the region of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned is R42. In this case, in accordance with the required irradiation position table <NUM>, the required irradiation positions are the irradiation positions IP33, IP34, IP35, IP43, IP44, IP45, IP53, IP54, and IP55. The light source control unit <NUM> causes the light emission unit <NUM> to emit the coherent light <NUM> from the light emission unit <NUM> in the order of the irradiation positions IP33, IP34, IP35, IP45, IP44, IP43, IP53, IP54, and IP55, for example.

<FIG> show a case in which there is one interference fringe <NUM> reflected in the standard interference fringe image 34R, whereas <FIG> and <FIG> show a case in which there are two interference fringes <NUM> reflected in the standard interference fringe image 34R. That is, <FIG> and <FIG> show a case in which the regions of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned are R21 and R42. In this case, in accordance with the required irradiation position table <NUM>, the required irradiation positions corresponding to the region R21 are the irradiation positions IP14, IP15, IP24, IP25, IP34, and IP35. In addition, the required irradiation positions corresponding to the region R42 are the irradiation positions IP33, IP34, IP35, IP43, IP44, IP45, IP53, IP54, and IP55. That is, the irradiation positions IP34 and IP35 overlap as the required irradiation position. The light source control unit <NUM> causes the light emission unit <NUM> to emit the coherent light <NUM> from the light emission unit <NUM> in the order of the irradiation positions IP33, IP43, IP53, IP54, IP44, IP34, IP24, IP14, IP15, IP25, IP35, IP45, and IP55, for example. That is, the light source control unit <NUM> causes the light emission unit <NUM> to emit the coherent light <NUM> only once from the irradiation positions IP34 and IP35, which are the overlapping required irradiation positions.

In <FIG>, the computer constituting the information processing apparatus <NUM> comprises a storage device <NUM>, a memory <NUM>, a central processing unit (CPU) <NUM>, a communication unit <NUM>, a display <NUM>, and an input device <NUM>. These components are connected to each other via a bus line <NUM>.

The storage device <NUM> is a hard disk drive that is built in the computer constituting the information processing apparatus <NUM> or is connected thereto via a cable or a network. Alternatively, the storage device <NUM> is a disk array in which a plurality of hard disk drives are mounted. In the storage device <NUM>, a control program, such as an operating system, various application programs, various data associated with such programs, and the like are stored. It should be noted that a solid state drive may be used instead of the hard disk drive.

The memory <NUM> is a work memory in which the CPU <NUM> executes processing. The CPU <NUM> loads the program stored in the storage device <NUM> to the memory <NUM> and executes the processing in accordance with the program to comprehensively control the units of the computer.

The communication unit <NUM> is a network interface that performs a transmission control of various information via a network, such as a local area network (LAN) or a wide area network (WAN). The display <NUM> displays various screens. The computer constituting the information processing apparatus <NUM> receives an input of an operation instruction from the input device <NUM> via the various screens. Examples of the input device <NUM> include a keyboard, a mouse, and a touch panel.

In <FIG>, an operation program <NUM> is stored in the storage device <NUM> of the information processing apparatus <NUM>. An interference fringe image group <NUM> and a reconstructed image <NUM> are also stored in the storage device <NUM>. The interference fringe image group <NUM> is a collection of the plurality of interference fringe images <NUM> that are the sources of the super-resolution interference fringe image <NUM> transmitted from the imaging apparatus <NUM>.

In a case in which the operation program <NUM> is activated, the CPU <NUM> of the computer constituting the information processing apparatus <NUM> functions as a read write (hereinafter, abbreviated as RW) control unit <NUM>, a super-resolution processing unit <NUM>, a reconstruction processing unit <NUM>, and a display control unit <NUM>, in cooperation with the memory <NUM> and the like.

The RW control unit <NUM> controls storing of various data in the storage device <NUM> and reading out of the various data in the storage device <NUM>. For example, the RW control unit <NUM> receives the interference fringe image <NUM> from the imaging apparatus <NUM> and stores the received interference fringe image <NUM> in the storage device <NUM> as the interference fringe image group <NUM>. In addition, the RW control unit <NUM> reads out the interference fringe image group <NUM> from the storage device <NUM> and outputs the interference fringe image group <NUM> to the super-resolution processing unit <NUM>.

The super-resolution processing unit <NUM> generates the super-resolution interference fringe image <NUM> from the interference fringe image group <NUM>. The super-resolution processing unit <NUM> outputs the super-resolution interference fringe image <NUM> to the reconstruction processing unit <NUM>.

The reconstruction processing unit <NUM> generates the reconstructed image <NUM> from the super-resolution interference fringe image <NUM>. The reconstruction processing unit <NUM> outputs the reconstructed image <NUM> to the RW control unit <NUM>. The RW control unit <NUM> stores the reconstructed image <NUM> in the storage device <NUM>. In addition, the RW control unit <NUM> reads out the reconstructed image <NUM> from the storage device <NUM> and outputs the reconstructed image <NUM> to the display control unit <NUM>.

The display control unit <NUM> controls display of the various screens on the display <NUM>. The various screens include a reconstructed image display screen <NUM> (see <FIG>), which is a screen on which the reconstructed image <NUM> is displayed.

In <FIG>, the super-resolution processing unit <NUM> includes a registration processing unit <NUM> and a generation unit <NUM>. The registration processing unit <NUM> performs the registration processing outlined in <FIG> on the plurality of interference fringe images <NUM> constituting the interference fringe image group <NUM>. The registration processing unit <NUM> outputs registration information <NUM>, which is a result of the registration processing, to the generation unit <NUM>.

The generation unit <NUM> performs the reconstruction processing outlined in <FIG> on the plurality of interference fringe images <NUM> constituting the interference fringe image group <NUM> with reference to the registration information <NUM>. As a result, the super-resolution interference fringe image <NUM> is generated.

<FIG> is a view showing details of the registration processing by the registration processing unit <NUM>. The registration processing unit <NUM> performs the registration processing by, for example, region-based matching. First, the registration processing unit <NUM> applies various deformation parameters, such as parallel translation, rotation, and enlargement/reduction, to a registration target image <NUM> and deforms the registration target image <NUM> to obtain a deformed registration target image 120D (step ST1). Then, the deformed registration target image 120D and a registration standard image <NUM> are compared, and a degree of similarity between the deformed registration target image 120D and the registration standard image <NUM> is calculated (step ST2). Moreover, the deformation parameters are updated such that the degree of similarity is increased (step ST3). The registration processing unit <NUM> repeats the processing of steps ST1 to ST3 until the degree of similarity between the deformed registration target image 120D and the registration standard image <NUM> is equal to or larger than a preset threshold value. The registration processing unit <NUM> outputs the deformation parameters in a case in which the degree of similarity between the deformed registration target image 120D and the registration standard image <NUM> is equal to or larger than the threshold value to the generation unit <NUM> as the registration information <NUM>.

The registration standard image <NUM> is one of the plurality of interference fringe images <NUM> constituting the interference fringe image group <NUM>, and the registration target image <NUM> is the interference fringe image <NUM> other than the registration standard image <NUM>. The registration standard image <NUM> is, for example, the interference fringe image <NUM> obtained in a case in which the coherent light <NUM> is emitted from the light emission unit <NUM> at the center point correspondence irradiation position. In the example of <FIG> and <FIG>, the region of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned is R14, and the center point correspondence irradiation position corresponding to the region R14 is IP12. Therefore, the interference fringe image <NUM> obtained in a case in which the coherent light <NUM> is emitted from the light emission unit <NUM> of the center point correspondence irradiation position IP12 is the registration standard image <NUM>. In a case in which there are a plurality of interference fringes <NUM> as in the examples of <FIG> and <FIG>, the registration standard image <NUM> is the interference fringe image <NUM> obtained in a case in which the coherent light <NUM> is emitted from the light emission unit <NUM> at the center point correspondence irradiation position corresponding to the region R of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned, which is the center point correspondence irradiation position closer to the irradiation position IP33 corresponding to the center region R33 (irradiation position IP44 corresponding to the region R42 in cases of <FIG> and <FIG>).

<FIG> is a view showing details of the reconstruction processing by the generation unit <NUM>. The generation unit <NUM> performs the reconstruction processing based on, for example, a maximum a posteriori (MAP) estimation. First, the generation unit <NUM> generates an appropriate assumed super-resolution interference fringe image 104AS, and generates an estimated interference fringe image group 91ES from the assumed super-resolution interference fringe image 104AS based on the point spread function (PSF) of the imaging element <NUM>, the registration information <NUM>, and the like (step ST5). Then, the estimated interference fringe image group 91ES and the actual interference fringe image group <NUM> are compared (step ST6). Moreover, the assumed super-resolution interference fringe image 104AS is updated such that a difference between the estimated interference fringe image group 91ES and the actual interference fringe image group <NUM> is reduced (step ST7). The generation unit <NUM> repeats the processing of steps ST5 to ST7 until the difference between the estimated interference fringe image group 91ES and the actual interference fringe image group <NUM> is less than a preset threshold value. The generation unit <NUM> outputs the assumed super-resolution interference fringe image 104AS in a case in which a difference between the estimated interference fringe image group 91ES and the actual interference fringe image group <NUM> is less than the threshold value, as the final super-resolution interference fringe image <NUM>.

<FIG> is a view showing an outline of operation processing by the reconstruction processing unit <NUM>. First, the reconstruction processing unit <NUM> reconstructs the super-resolution interference fringe image <NUM> to generate the reconstructed image <NUM>. A reconstructed image group <NUM> is a collection of a plurality of reconstructed images <NUM>. These plurality of reconstructed images <NUM> are images representing tomographic planes <NUM> arranged at equal intervals in a thickness direction of the cell <NUM> and the culture container <NUM> along the Z direction, respectively.

The reconstruction processing unit <NUM> selects one reconstructed image <NUM> that is most in focus from among the plurality of reconstructed images <NUM> of the reconstructed image group <NUM>. The reconstruction processing unit <NUM> outputs the selected reconstructed image <NUM> to the RW control unit <NUM>. It should be noted that, as a method of selecting the reconstructed image <NUM> that is most in focus, a method can be adopted in which a contrast value of each of the plurality of reconstructed images <NUM> is calculated, and the reconstructed image <NUM> having the highest contrast value is selected as the reconstructed image <NUM> that is most in focus.

As shown in <FIG>, the reconstructed image <NUM> is displayed on the reconstructed image display screen <NUM> together with type information <NUM>. The type information <NUM> includes the type of the cell <NUM>, the number of days for culturing the cell <NUM>, the type of the culture container <NUM>, the type of a culture solution, and a usage amount of the culture solution. The display of the reconstructed image display screen <NUM> disappears in a case in which a confirmation button <NUM> is selected.

Next, the actions of the configuration described above will be described with reference to the flowcharts of <FIG>. First, in a case in which the operation program <NUM> is activated by the imaging apparatus <NUM>, as shown in <FIG>, the CPU <NUM> of the imaging apparatus <NUM> functions as the light source control unit <NUM>, the imaging control unit <NUM>, the acquisition unit <NUM>, the setting unit <NUM>, and the transmission control unit <NUM>.

In <FIG>, first, under the control of the light source control unit <NUM>, the coherent light <NUM> is emitted from the light emission unit <NUM> at the irradiation position IP33, which is the standard irradiation position, as shown in <FIG> (step ST100). Moreover, the interference fringe <NUM> generated by the irradiation is imaged by the imaging element <NUM> under the control of the imaging control unit <NUM>, and the standard interference fringe image 34R is output from the imaging element <NUM> (step ST110). The standard interference fringe image 34R is input to the acquisition unit <NUM>.

As shown in <FIG>, the acquisition unit <NUM> detects the position of the center point C1 of the interference fringe <NUM> from the standard interference fringe image 34R as the position of the cell <NUM>. As a result, the positional information <NUM> is acquired by the acquisition unit <NUM> (step ST120). The positional information <NUM> is output to the setting unit <NUM> from the acquisition unit <NUM>. It should be noted that step ST120 is an example of an "acquisition step" according to the technology of the present disclosure.

In the setting unit <NUM>, as shown in <FIG>, the required irradiation position is set from among the plurality of irradiation positions IP11 to IP55 based on the required irradiation position table <NUM> and the positional information <NUM> (step ST130). Moreover, the setting information <NUM> indicating the required irradiation position is generated. The setting information <NUM> is output to the light source control unit <NUM> from the setting unit <NUM>. It should be noted that step ST130 is an example of a "setting step" according to the technology of the present disclosure.

Under the control of the light source control unit <NUM>, the coherent light <NUM> is emitted from the light emission unit <NUM> at the required irradiation position indicated by the setting information <NUM> (step ST <NUM>). Moreover, the interference fringe <NUM> generated by the irradiation is imaged by the imaging element <NUM> under the control of the imaging control unit <NUM>, and the interference fringe image <NUM> is output from the imaging element <NUM> (step ST <NUM>). In this case, as shown in <FIG> and <FIG>, in a case in which there are a plurality of cells <NUM> and the required irradiation positions of the plurality of cells <NUM> overlap, the coherent light <NUM> is emitted only once from the overlapping required irradiation positions. It should be noted that step ST140 is an example of a "light source control step" according to the technology of the present disclosure. In addition, step ST150 is an example of an "imaging control step" according to the technology of the present disclosure.

The interference fringe image <NUM> is input to the transmission control unit <NUM> and transmitted to the information processing apparatus <NUM> by the transmission control unit <NUM> (step ST160). These steps ST140 to ST160 are repeated while capturing of the interference fringe image <NUM> at all the required irradiation positions is not terminated (NO in step ST170). In a case in which capturing of the interference fringe image <NUM> at all the required irradiation positions is terminated (YES in step ST170), the repetitive processing of steps ST140 to ST160 is terminated.

First, in a case in which the operation program <NUM> is activated by the information processing apparatus <NUM>, as shown in <FIG>, the CPU <NUM> of the information processing apparatus <NUM> functions as the RW control unit <NUM>, the super-resolution processing unit <NUM>, the reconstruction processing unit <NUM>, and the display control unit <NUM>.

The information processing apparatus <NUM> receives the interference fringe image <NUM> from the imaging apparatus <NUM>. The interference fringe image <NUM> is stored in the storage device <NUM> by the RW control unit <NUM>. As a result, the interference fringe image group <NUM> is stored in the storage device <NUM>.

In <FIG>, the RW control unit <NUM> reads out the interference fringe image group <NUM> from the storage device <NUM> (step ST200). The interference fringe image group <NUM> is output to the super-resolution processing unit <NUM> from the RW control unit <NUM>.

As shown in <FIG>, the super-resolution processing unit <NUM> performs the registration processing and the reconstruction processing to generate the super-resolution interference fringe image <NUM> from the interference fringe image group <NUM> (step ST210). The super-resolution interference fringe image <NUM> is output to the reconstruction processing unit <NUM> from the super-resolution processing unit <NUM>.

As shown in <FIG>, the reconstruction processing unit <NUM> generates the reconstructed image <NUM> from the super-resolution interference fringe image <NUM> (step ST220). The reconstructed image <NUM> is output to the RW control unit <NUM> from the reconstruction processing unit <NUM>, and is stored in the storage device <NUM> by the RW control unit <NUM> (step ST230).

The RW control unit <NUM> reads out the reconstructed image <NUM> from the storage device <NUM>. The reconstructed image <NUM> is output to the display control unit <NUM> from the RW control unit <NUM>. Moreover, as shown in <FIG>, the display control unit <NUM> displays the reconstructed image display screen <NUM> on the display <NUM>, and the reconstructed image <NUM> is provided for viewing by a user (step ST240).

As described above, the control device <NUM> of the imaging apparatus <NUM> comprises the acquisition unit <NUM>, the setting unit <NUM>, the light source control unit <NUM>, and the imaging control unit <NUM>. The acquisition unit <NUM> acquires the positional information <NUM> indicating the position of the cell <NUM>, which is the observation target. The setting unit <NUM> sets the required irradiation position from among the plurality of irradiation positions IP11 to IP55. The required irradiation position is the irradiation position corresponding to the position of the cell <NUM> indicated by the positional information <NUM>, and is the irradiation position required for obtaining the plurality of interference fringe images <NUM> which are sources of the super-resolution interference fringe image <NUM> having the resolution exceeding the resolution of the imaging element <NUM>. The light source control unit <NUM> controls the operation of the light source <NUM> to emit the coherent light <NUM> from the required irradiation position. The imaging control unit <NUM> causes the imaging element <NUM> to output the interference fringe image <NUM> at each required irradiation position. Therefore, the interference fringe image <NUM>, which has almost no contribution to super-resolution, is not captured. Therefore, it is possible to obtain the super-resolution interference fringe image <NUM> without wasteful labor.

The acquisition unit <NUM> acquires the positional information <NUM> by detecting the position of the cell <NUM> from the standard interference fringe image 34R which is the interference fringe image <NUM> obtained by emitting the coherent light <NUM> from one preset standard irradiation position IP33 among the plurality of irradiation positions IP11 to IP55. Therefore, it is possible to acquire the positional information <NUM> without bothering the user.

In a case in which there are the plurality of cells <NUM> and the required irradiation positions of the plurality of cells <NUM> overlap, the light source control unit <NUM> causes the light emission unit <NUM> to emit the coherent light <NUM> only once from the overlapping required irradiation positions. Therefore, it is possible to save labor of emitting the coherent light <NUM> many times from the overlapping required irradiation positions and capturing the plurality of substantially the same interference fringe images <NUM>, so that the super-resolution interference fringe image <NUM> can be obtained in a shorter time.

The field of cell culture has recently been in the limelight with the advent of induced pluripotent stem (iPS) cells and the like. Therefore, there is a demand for the technology of analyzing the cell <NUM> in culture in detail without wasteful time. In the technology of the present disclosure, the observation target is the cell <NUM> in culture. Therefore, it can be said that the technology of the present disclosure is the technology that can meet recent demands.

It should be noted that the acquisition unit <NUM> acquires the positional information <NUM> by detecting the position of the cell <NUM> from the standard interference fringe image 34R, but the technology of the present disclosure is not limited to this. As shown in <FIG>, the acquisition unit <NUM> may acquire the positional information <NUM> by detecting the position of the cell <NUM> from a standard reconstructed image 92R, which is the reconstructed image <NUM> generated based on the standard interference fringe image 34R, instead of the standard interference fringe image 34R.

In this case, the CPU <NUM> of the imaging apparatus <NUM> also functions as a generation unit <NUM> in addition to the units <NUM> to <NUM> shown in <FIG> (only the acquisition unit <NUM> is shown in <FIG>). The generation unit <NUM> has the same function as the reconstruction processing unit <NUM> of the information processing apparatus <NUM>. The generation unit <NUM> generates the standard reconstructed image 92R from the standard interference fringe image 34R in the same way that the reconstruction processing unit <NUM> generates the reconstructed image <NUM> from the super-resolution interference fringe image <NUM>. The generation unit <NUM> outputs the standard reconstructed image 92R to the acquisition unit <NUM>.

The acquisition unit <NUM> performs the image analysis on the standard reconstructed image 92R and detects a position of a center point C2 of the cell <NUM> reflected in the standard reconstructed image 92R as the position of the cell <NUM>, for example. The acquisition unit <NUM> outputs a position coordinate (X_C2, Y_C2) of the center point C2 of the cell <NUM> to the setting unit <NUM> as the positional information <NUM>.

The interference fringe <NUM> is generated due to dust and the like in the culture solution in addition to the cell <NUM>. Therefore, in a case in which the position of the center point C1 of the interference fringe <NUM> reflected in the standard interference fringe image 34R is detected as the position of the cell <NUM>, there is a considerable possibility that dust or the like is erroneously recognized as the cell <NUM>. Therefore, as shown in <FIG>, it is preferable to detect the position of the cell <NUM> from the standard reconstructed image 92R generated based on the standard interference fringe image 34R, instead of the standard interference fringe image 34R. It should be noted that it takes labor to generate the standard reconstructed image 92R from the standard interference fringe image 34R. Therefore, in a case in which the reduction of such labor is considered first, it is better to adopt the method of detecting the position of the cell <NUM> from the standard interference fringe image 34R.

It should be noted that an aspect shown in <FIG> may be adopted. That is, the standard interference fringe image 34R is transmitted from the imaging apparatus <NUM> to the information processing apparatus <NUM>, and the reconstruction processing unit <NUM> of the information processing apparatus <NUM> generates the standard reconstructed image 92R from the standard interference fringe image 34R. Moreover, the standard reconstructed image 92R is transmitted from the information processing apparatus <NUM> to the imaging apparatus <NUM>. In this way, it is not necessary to provide the generation unit <NUM> in the imaging apparatus <NUM>.

The aspect in which the positional information <NUM> is acquired is not limited to the aspect in which the position of the cell <NUM> is detected from the standard interference fringe image 34R or the standard reconstructed image 92R, which has been described in the first embodiment. A second embodiment shown in <FIG> and <FIG> may be adopted.

In <FIG>, the imaging apparatus <NUM> transmits the standard interference fringe image 34R to the information processing apparatus <NUM>. The display control unit <NUM> of the information processing apparatus <NUM> performs a control of displaying a standard interference fringe image display screen <NUM> (see also <FIG>), which is the display screen of the standard interference fringe image 34R, on the display <NUM>. Moreover, on the standard interference fringe image display screen <NUM>, the user is made to designate the position of the cell <NUM> via the input device <NUM>. The information processing apparatus <NUM> generates the positional information <NUM> based on the designation of the position of the cell <NUM> on the standard interference fringe image display screen <NUM>, and transmits the generated positional information <NUM> to the imaging apparatus <NUM>. The acquisition unit <NUM> acquires the positional information <NUM> from the information processing apparatus <NUM>.

As shown in <FIG>, the standard interference fringe image 34R is displayed on the standard interference fringe image display screen <NUM>, and a designation release button <NUM> and a designation button <NUM> are provided below the standard interference fringe image 34R. The center point C1 of the interference fringe <NUM>, which is the position of the cell <NUM>, can be input by, for example, moving the mouse cursor of the input device <NUM> to a desired position on the standard interference fringe image 34R and double-clicking the mouse. In a case in which the designation release button <NUM> is selected, the designation of the center point C1 of the most recently designated interference fringe <NUM> is released. In a case in which the designation button <NUM> is selected, the position coordinate (X_C1, Y_C1) of the center point C1 of the interference fringe <NUM> designated in that case is acquired by the acquisition unit <NUM> as the positional information <NUM>.

<FIG> is an example in which the standard reconstructed image 92R is used instead of the standard interference fringe image 34R. In this case, the reconstruction processing unit <NUM> of the information processing apparatus <NUM> generates the standard reconstructed image 92R from the standard interference fringe image 34R. The reconstruction processing unit <NUM> outputs the standard reconstructed image 92R to the display control unit <NUM>. The display control unit <NUM> performs a control of displaying a standard reconstructed image display screen <NUM> (see also <FIG>), which is the display screen of the standard reconstructed image 92R, on the display <NUM>. Moreover, on the standard reconstructed image display screen <NUM>, the user is made to designate the position of the cell <NUM> via the input device <NUM>. The information processing apparatus <NUM> generates the positional information <NUM> based on the designation of the position of the cell <NUM> on the standard reconstructed image display screen <NUM>, and transmits the generated positional information <NUM> to the imaging apparatus <NUM>. The acquisition unit <NUM> acquires the positional information <NUM> from the information processing apparatus <NUM>.

As shown in <FIG>, the standard reconstructed image 92R is displayed on the standard reconstructed image display screen <NUM>, and a designation release button <NUM> and a designation button <NUM> are provided below the standard reconstructed image 92R. The center point C2 of the cell <NUM>, which is the position of the cell <NUM>, can be input by moving the mouse cursor of the input device <NUM> to a desired position on the standard reconstructed image 92R and double-clicking the mouse, as in a case of the standard interference fringe image display screen <NUM>. In a case in which the designation release button <NUM> is selected, the designation of the center point C2 of the most recently designated cell <NUM> is released as in a case of the designation release button <NUM>. In a case in which the designation button <NUM> is selected, the position coordinate (X_C2, Y_C2) of the center point C2 of the cell <NUM> designated in that case is acquired by the acquisition unit <NUM> as the positional information <NUM>, as in a case of the designation button <NUM>.

As described above, in the second embodiment, the display control unit <NUM> of the information processing apparatus <NUM> performs a control of displaying the standard interference fringe image display screen <NUM> shown in <FIG> and <FIG>, or the standard reconstructed image display screen <NUM> shown in <FIG> and <FIG>. The acquisition unit <NUM> acquires the positional information <NUM> by receiving the designation of the position of the cell <NUM> on the standard interference fringe image display screen <NUM> or the standard reconstructed image display screen <NUM>. Therefore, it is possible to acquire the more probable positional information <NUM> designated by the user himself/herself.

It should be noted that, in the second embodiment, the control device according to the present disclosure is realized by the light source control unit <NUM>, the imaging control unit <NUM>, the acquisition unit <NUM>, the setting unit <NUM>, and the display control unit <NUM> of the information processing apparatus <NUM>. As described above, the processing unit constituting the control device may be provided in the information processing apparatus <NUM>, in addition to the imaging apparatus <NUM>.

In a third embodiment shown in <FIG> and <FIG>, the number of required irradiation positions is changed in accordance with size information <NUM> indicating a size of the cell <NUM>.

As shown in <FIG>, the acquisition unit <NUM> performs the image analysis on the standard reconstructed image 92R and counts the number of pixels of the cell <NUM> reflected in the standard reconstructed image 92R, for example. Moreover, three sizes of large, medium, and small, are assigned to the cell <NUM> in accordance with the number of counted pixels. In a case in which there are the plurality of cells <NUM>, the number of pixels of each of the plurality of cells <NUM> is counted, and the size is assigned to the cell <NUM> having the smallest number of counted pixels as a representative. As a result, the acquisition unit <NUM> acquires the size information <NUM> indicating the size of the cell <NUM>. The acquisition unit <NUM> outputs the size information <NUM> to the setting unit <NUM>.

The setting unit <NUM> changes the number of required irradiation positions in accordance with the size information <NUM>. Specifically, as shown in a required irradiation position number table <NUM>, in a case in which the size of the cell <NUM> is large, the number of required irradiation positions is five, in a case in which the size of the cell <NUM> is medium, the number of required irradiation positions is nine, and in a case in which the size of the cell <NUM> is small, the number of required irradiation positions is <NUM>. As described above, the number of required irradiation positions is set to larger as the size of the cell <NUM> is smaller. The reason for increasing the number of required irradiation positions as the size of the cell <NUM> is smaller is that, as the size of the cell <NUM> is smaller, the limit of the resolution of the imaging element <NUM> is closer and it is more difficult to obtain the clear interference fringe image <NUM>.

<FIG> show the required irradiation positions in a case in which the region of the standard interference fringe image 34R in which the center point C1 of the interference fringe <NUM> is positioned is R33. <FIG> shows a case in which the size of the cell <NUM> is large, <FIG> shows a case in which the size of the cell <NUM> is medium, and <FIG> shows a case in which the size of the cell <NUM> is small. In a case in which the size of the cell <NUM> in <FIG> is large, the required irradiation positions are a total of five irradiation positions IP22, IP24, IP33, IP42, and IP44. In a case in which the size of the cell <NUM> in <FIG> is medium, the required irradiation positions are a total of nine irradiation positions IP22, IP23, IP24, IP32, IP33, IP34, IP42, IP43, and IP44. In a case in which the size of the cell <NUM> in <FIG> is small, the required irradiation positions are a total of <NUM> irradiation positions IP11, IP13, IP15, IP22, IP23, IP24, IP31, IP32, IP33, IP34, IP35, IP42, IP43, IP44, IP51, IP53, and IP55.

As described above, in the third embodiment, the acquisition unit <NUM> acquires the size information <NUM> indicating the size of the cell <NUM>, in addition to the positional information <NUM>. The setting unit <NUM> changes the number of required irradiation positions in accordance with the size information <NUM>. Therefore, it is possible to obtain the number of interference fringe images <NUM> adapted to the size of the cell <NUM>, and it is possible to generate the super-resolution interference fringe image <NUM> adapted to the size of the cell <NUM>.

It should be noted that the size of the cell <NUM> is not limited to the three stages of large, medium, and small. The size of the cell <NUM> may be two stages, small and other. Alternatively, the size of the cell <NUM> may be divided into three stages. In addition, the number of required irradiation positions to be changed is not limited to the five, nine, and <NUM> described above. In a case of the size of large, the required irradiation position may be set to only one not to generate the super-resolution interference fringe image <NUM> itself.

In each of the embodiments described above, the light source <NUM> having the configuration in which the plurality of light emission units <NUM> are arranged at the plurality of irradiation positions IP11 to IP55 has been described, but the technology of the present disclosure is not limited to this. A fourth embodiment shown in <FIG> and <FIG> may be adopted.

In <FIG>, a light source <NUM> includes one light emission unit <NUM>, a moving stage <NUM>, and a moving mechanism <NUM>. The light emission unit <NUM> is moved on the moving stage <NUM> by the moving mechanism <NUM>. The moving mechanism <NUM> includes, for example, a motor and a rack and pinion gear that converts the rotation of the motor into translational motion along the X direction and the Y direction. The moving mechanism <NUM> moves the light emission unit <NUM> in the X direction and the Y direction under the control of the light source control unit <NUM>, and guides the light emission unit <NUM> to the <NUM> × <NUM> = <NUM> irradiation positions IP11 to IP55.

Even with the light source <NUM> having such a configuration, it is possible to emit the coherent light <NUM> from the plurality of irradiation positions IP11 to IP55 having different irradiation angles. It should be noted that the configuration becomes complicated due to the moving stage <NUM> and the moving mechanism <NUM>. In addition, since the light emission unit <NUM> should be moved to each required irradiation position, an imaging interval of the plurality of interference fringe images <NUM> is longer than that of the light source <NUM> of each of the embodiments described above. In a case in which the imaging interval is longer, the cell <NUM> may be moved during imaging. Therefore, the light source <NUM> of each of the embodiments described above is more preferable.

It should be noted that the number of light emission units <NUM> that moves the moving stage <NUM> is not limited to one, and may be plurality. In addition, for example, as shown in <FIG>, the light source <NUM> having a configuration in which <NUM> × <NUM> = <NUM> light emission units <NUM> are arranged may be moved in the X direction and the Y direction by a moving mechanism <NUM>.

In the light sources <NUM>, <NUM>, and <NUM> shown in each of the embodiments described above, the light emission units <NUM>, <NUM>, and <NUM> are arranged in parallel with the imaging surface <NUM>, but the technology of the present disclosure is not limited to this. For example, a light source <NUM> shown in <FIG> may be used.

In <FIG>, in the light source <NUM>, a light emission unit 221B other than a center light emission unit 221A is disposed to be inclined with respect to the imaging surface <NUM>. An inclined angle of the light emission unit 221B is increased toward the end. In addition, the inclined angle of the light emission unit 221B at a symmetrical position with respect to the light emission unit 221A is the same. It should be noted that a configuration may be adopted in which the light emission unit 221B is movable and the inclined angle of the light emission unit 221B is changeable.

The standard interference fringe image 34R may be diverted to generate the super-resolution interference fringe image <NUM>.

The light emission unit may be, for example, a distal end of an optical fiber that is connected to a laser diode that emits the coherent light <NUM> and guides the coherent light <NUM>. In addition, the irradiation angle of the coherent light <NUM> may vary by swinging the light emission unit around an axis along the X direction or the Y direction. In this case, a swing position of the light emission unit corresponds to the irradiation position.

The irradiation position is not limited to the <NUM> × <NUM> = <NUM> positions described above. The irradiation position can be appropriately changed in accordance with the size of the pixel <NUM> of the imaging element <NUM>, the size of the cell <NUM>, and the like.

The observation target is not limited to the cell <NUM> described above. Bacteria, viruses and the like may be the observation target. In addition, the diffracted light is not limited to the diffracted light <NUM> transmitted through the observation target, and may be diffracted light reflected by the observation target. Further, the coherent light <NUM> from the light source <NUM> may be split into two beams for diffracted light and one for reference light to irradiate the observation target with each light. In addition, the illumination light does not have to be the coherent light <NUM>, and need only be any light that generates the interference fringe <NUM> that can withstand observation.

A hardware configuration of the computer constituting the control device can be modified in various ways. For example, the control device can be composed of a plurality of computers separated as hardware in order to improve processing capacity and reliability. Specifically, the functions of the light source control unit <NUM> and the imaging control unit <NUM> and the functions of the acquisition unit <NUM> and the setting unit <NUM> are distributed and assigned to two computers. In this case, the two computers constitutes the control device. It should be noted that the two computers may be the imaging apparatus <NUM> and the information processing apparatus <NUM>. For example, the functions of the light source control unit <NUM> and the imaging control unit <NUM> are assigned to the imaging apparatus <NUM>, and the functions of the acquisition unit <NUM> and the setting unit <NUM> are assigned to the information processing apparatus <NUM>, respectively. All the functions of the light source control unit <NUM>, the imaging control unit <NUM>, the acquisition unit <NUM>, and the setting unit <NUM> may be assigned to the information processing apparatus <NUM>.

As described above, the hardware configuration of the computer of the imaging control device can be appropriately changed in accordance with required performance, such as processing capacity, safety, and reliability. Further, in addition to the hardware, the application programs, such as the operation programs <NUM> and <NUM>, can be duplicated or distributed to a plurality of storage devices for the purpose of ensuring safety and reliability.

In each of the embodiments described above, for example, the following various processors can be used as a hardware structure of processing units that executes various pieces of processing, such as the light source control unit <NUM>, the imaging control unit <NUM>, the acquisition unit <NUM>, the setting unit <NUM>, the transmission control unit <NUM>, the RW control unit <NUM>, the super-resolution processing unit <NUM> (registration processing unit <NUM> and generation unit <NUM>), the reconstruction processing units <NUM> and <NUM>, and the display control unit <NUM>. As described above, the various processors includes, in addition to the CPUs <NUM> and <NUM>, which are general-purpose processors that execute software (operation programs <NUM> and <NUM>) to function as the various processing units, a programmable logic device (PLD), which is a processor of which a circuit configuration can be changed after the manufacturing, such as a field programmable gate array (FPGA), a dedicated electric circuit, which is a processor having a circuit configuration designed specially for executing specific processing, such as an application specific integrated circuit (ASIC), and the like.

One processing unit may be composed of one of various processors described above or may be composed of a combination of two or more processors (for example, a combination of a plurality of FPGAs and/or a combination of a CPU and an FPGA) of the same type or different types. In addition, a plurality of the processing units may be composed of one processor.

As an example in which the plurality of processing units are composed of one processor, first, as represented by a computer, such as a client and a server, there is a form in which one processor is composed of a combination of one or more CPUs and software and the processor functions as the plurality of processing units. Second, as represented by a system on chip (SoC) and the like, there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with a single integrated circuit (IC) chip is used. As described above, various processing units are composed of using one or more of the various processors as a hardware structure.

Further, as the hardware structure of these various processors, more specifically, it is possible to use an electrical circuit (circuitry) in which circuit elements, such as semiconductor elements, are combined.

In the technology of the present disclosure, it is possible to appropriately combine various embodiments and various modification examples described above. In addition, it is needless to say that the present disclosure is not limited to each of the embodiments described above, various configurations can be adopted as long as the configuration does not deviate from the claimed invention.

The contents described and shown above are the detailed description of the parts relating to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, the function, the action, and the effect are the description of examples of the configuration, the function, the action, and the effect of the parts relating to the technology of the present disclosure. Therefore, it is needless to say that unnecessary parts may be deleted, new elements may be added, or replacements may be made with respect to the contents described and shown above within a range that does not deviate from the claimed invention. In addition, in order to avoid complications and facilitate understanding of the parts relating to the technology of the present disclosure, in the contents described and shown above, the description of technical general knowledge and the like that do not particularly require description for enabling the implementation of the technology of the present disclosure are omitted.

Claim 1:
A control device (<NUM>) of an imaging apparatus (<NUM>) including a light source (<NUM>, <NUM>) and an imaging element (<NUM>), in which the light source (<NUM>, <NUM>) is able to irradiate an observation target with illumination light (<NUM>) from a plurality of irradiation positions having different irradiation angles, and the imaging element (<NUM>) outputs an interference fringe image (<NUM>) by imaging an interference fringe (<NUM>) between diffracted light (<NUM>), which is the illumination light (<NUM>) diffracted by the observation target, and reference light (<NUM>), which is the illumination light (<NUM>) that does not pass through the observation target, the control device comprising:
an acquisition unit (<NUM>) that acquires positional information indicating a position of the observation target;
characterized by:
a setting unit (<NUM>) that sets, from among the plurality of irradiation positions (IP11 to IP55), required irradiation positions, which are irradiation positions corresponding to the position of the observation target indicated by the positional information and are irradiation positions required for obtaining a plurality of the interference fringe images (<NUM>) that are sources of a super-resolution interference fringe image (<NUM>) having a resolution exceeding a resolution of the imaging element (<NUM>);
a light source control unit (<NUM>) that causes the light source (<NUM>, <NUM>) to emit the illumination light (<NUM>) from the required irradiation positions by controlling an operation of the light source (<NUM>); and
an imaging control unit (<NUM>) that causes the imaging element (<NUM>) to output the interference fringe image (<NUM>) at each required irradiation position.