Patent Description:
A distance to a subject, the size of a subject, and the like are acquired in an endoscope system that includes a light source device, an endoscope, and a processor device. For example, in <CIT>, a subject is irradiated with illumination light and measurement beam light and a beam irradiation region, such as a spot, appears on the subject due to irradiation with beam light. Then, a measurement marker used to measure the size of the subject is displayed in a subject image to correspond to the position of the spot. An endoscope system is described in <CIT>, which also discloses the preamble of claim <NUM>.

As a method of correctly detecting the position of a spot from a subject image, there is a color separation method using the color/brightness/chroma saturation and the like of an image, such as RGB/HSV. However, in a case where scattered reflection in a living body, residues, or altered (discolored) tissue is included in a subject, the color or the like of an image is changed due to scattered reflection in a living body, or the like. For this reason, it may be difficult to detect the position of the spot. Accordingly, it has been required to stably detect the irradiation region of measurement beam light, such as the position of the spot even in a case where there is scattered reflection in a living body, or the like.

An object of the present invention is to provide an endoscope system that can stably detect an irradiation region of measurement beam light and a method of operating the endoscope system.

An endoscope system according to an aspect of the present invention is provided as defined by claim <NUM>.

It is preferable that the processor recognizes the beam irradiation region by a learning model outputting the beam irradiation region in response to an input of the subject image. It is preferable that the processor recognizes the beam irradiation region by performing pattern matching processing on the subject image using a predetermined template image of the beam irradiation region.

It is preferable that the processor is capable of recognizing the beam irradiation region having a pattern of which the specific shape is deformed.

It is preferable that the specific shape is a circular shape. It is preferable that the specific shape is a linear shape. It is preferable that the processor receives the subject image including a plurality of color images and an amount of the measurement beam light is set to a specific amount of light and a pixel value of each of the color images is set to a maximum pixel value, so that a central region of the beam irradiation region is white.

It is preferable that the processor displays the measurement marker, which corresponds to a position of the beam irradiation region, in the measurement image. It is preferable that the processor displays the measurement marker, which corresponds to a pattern of the beam irradiation region, in the measurement image.

According to another aspect of the present invention, there is provided a method of operating an endoscope system as defined by claim <NUM>.

According to the present invention, it is possible to stably detect the irradiation region of measurement beam light by recognizing a beam irradiation region, which is formed on a subject by the measurement beam light, as a pattern having a specific shape that includes a white central region and a peripheral region.

As shown in <FIG>, an endoscope system <NUM> includes an endoscope <NUM>, a light source device <NUM>, a processor device <NUM>, a display <NUM>, and a user interface <NUM>. The endoscope <NUM> includes an insertion part 12a that is to be inserted into an object to be examined, an operation part 12b that is provided at the proximal end portion of the insertion part 12a, and a universal cable 12c. The universal cable 12c is a cable in which a light guide <NUM> (see <FIG>) for guiding illumination light emitted from the light source device <NUM>, a control line for transmitting control signals used to control the endoscope <NUM>, a signal line for transmitting image signals obtained from the image pickup of an object to be observed, a power line for supplying power to each part of the endoscope <NUM>, and the like are integrated. The distal end of the universal cable 12c is provided with a connector <NUM> to be connected to the light source device <NUM>.

The light source device <NUM> generates illumination light by, for example, a semiconductor light source, such as a light emitting diode (LED) or a laser diode (LD), or a halogen lamp, such as a xenon lamp. In a case where the connector <NUM> is connected to the light source device <NUM>, illumination light is incident on the light guide <NUM> (see <FIG>) of the connector <NUM> and the object to be observed is irradiated with the illumination light from the distal end of the insertion part 12a.

Further, the light source device <NUM> is electrically connected to the processor device <NUM> and the connector <NUM> of the endoscope <NUM> is connected to the processor device <NUM> through the light source device <NUM>. The transmission and reception of control signals, image signals, and the like between the light source device <NUM> and the connector <NUM> are performed by wireless communication. For this reason, the light source device <NUM> transmits the control signals and the like, which are wirelessly transmitted to and received from the connector <NUM>, to the processor device <NUM>. Furthermore, the light source device <NUM> supplies power, which is used to drive the endoscope <NUM>, to the connector <NUM>, and the supply of this power is also wirelessly performed.

The processor device <NUM> controls the amount and emission timing of illumination light, which is emitted from the light source device <NUM>, and each part of the endoscope <NUM> and generates an endoscopic image using image signals that are obtained from the image pickup of an object to be observed irradiated with the illumination light. Further, the processor device <NUM> is electrically connected to the display <NUM> and the user interface <NUM>. The display <NUM> displays the endoscopic image that is generated by the processor device <NUM>, information about the endoscopic image, and the like. The user interface <NUM> has a function to receive an input operation, such as function settings.

The endoscope <NUM> has a normal observation mode, a special observation mode, and a length measurement mode, and these three modes are switched by a mode changeover switch 13a that is provided on the operation part 12b of the endoscope <NUM>. The normal observation mode is a mode in which an object to be observed is illuminated with the illumination light. The special observation mode is a mode in which an object to be observed is illuminated with special light different from the illumination light. In the length measurement mode, an object to be observed is illuminated with the illumination light and measurement beam light and a measurement marker to be used for the measurement of the size and the like of an object to be observed is displayed in a subject image obtained from the image pickup of the object to be observed. The illumination light is light that is used to observe the entire object to be observed by applying brightness to the entire object to be observed. The special light is light that is used to emphasize a specific region of the object to be observed. The measurement beam light is light that is used for the display of the measurement marker.

Further, the operation part 12b of the endoscope <NUM> is provided with a freeze switch 13b that is used to give a static image-acquisition instruction to acquire the static image of a subject image. In a case where a user operates the freeze switch 13b, the screen of the display <NUM> is frozen and displayed and an alert sound (for example, "beep") informing the acquisition of a static image is generated together. Then, the static images of the subject image, which are obtained before and after the operation timing of the freeze switch 13b, are stored in a static image storage unit <NUM> (see <FIG>) provided in the processor device <NUM>. The static image storage unit <NUM> is a storage unit, such as a hard disk or a universal serial bus (USB) memory. In a case where the processor device <NUM> can be connected to a network, the static image of the subject image may be stored in a static image storage server (not shown), which is connected to the network, instead of or in addition to the static image storage unit <NUM>.

A static image-acquisition instruction may be given using an operation device other than the freeze switch 13b. For example, a foot pedal may be connected to the processor device <NUM>, and a static image-acquisition instruction may be given in a case where a user operates the foot pedal (not shown) with a foot. A static image-acquisition instruction may be given by a foot pedal that is used to switch a mode. Further, a gesture recognition unit (not shown), which recognizes the gestures of a user, may be connected to the processor device <NUM>, and a static image-acquisition instruction may be given in a case where the gesture recognition unit recognizes a specific gesture of a user. The gesture recognition unit may also be used to switch a mode.

Further, a sight line input unit (not shown), which is provided close to the display <NUM>, may be connected to the processor device <NUM>, and a static image-acquisition instruction may be given in a case where the sight line input unit recognizes that a user's sight line is in a predetermined region of the display <NUM> for a predetermined time or longer. Furthermore, a voice recognition unit (not shown) may be connected to the processor device <NUM>, and a static image-acquisition instruction may be given in a case where the voice recognition unit recognizes a specific voice generated by a user. The voice recognition unit may also be used to switch a mode. Moreover, an operation panel (not shown), such as a touch panel, may be connected to the processor device <NUM>, and a static image-acquisition instruction may be given in a case where a user performs a specific operation on the operation panel. The operation panel may also be used to switch a mode.

As shown in <FIG>, a distal end part 12d of the endoscope <NUM> has a substantially circular shape, and is provided with an image pickup optical system <NUM> that receives light from a subject, an illumination optical system <NUM> that is used to irradiate the subject with the illumination light, a beam light-emitting unit <NUM> that radiates the measurement beam light to the subject, an opening <NUM> that allows a treatment tool to protrude toward the subject, and an air/water supply nozzle <NUM> that is used to supply air and water.

An optical axis Ax of the image pickup optical system <NUM> extends in a direction perpendicular to the plane of paper. A vertical first direction D1 is orthogonal to the optical axis Ax, and a horizontal second direction D2 is orthogonal to the optical axis Ax and the first direction D1. The image pickup optical system <NUM> and the beam light-emitting unit <NUM> are provided at different positions in the distal end part 12d, and are arranged in the first direction D1.

As shown in <FIG>, the light source device <NUM> comprises a light source unit <NUM> and a light source controller <NUM>. The light source unit <NUM> generates the illumination light or the special light that is used to illuminate the subject. The illumination light or the special light, which is emitted from the light source unit <NUM>, is incident on the light guide <NUM>, and the subject is irradiated with the illumination light or the special light through an illumination lens 22a. A white light source emitting white light, a plurality of light sources, which include a white light source and a light source emitting another color light (for example, a blue light source emitting blue light), or the like is used as a light source of the illumination light in the light source unit <NUM>. Further, a light source, which emits broadband light including blue narrow-band light used to emphasize superficial information about superficial blood vessels and the like, is used as a light source of the special light in the light source unit <NUM>. The light source controller <NUM> is connected to a system controller <NUM> of the processor device <NUM>. Mixed white light, which is a combination of blue light, green light, and red light, may be used as the illumination light. In this case, it is preferable that the illumination optical system <NUM> is optically designed to allow the irradiation range of green light to be wider than the irradiation range of red light.

The light source controller <NUM> controls the light source unit <NUM> on the basis of an instruction given from the system controller <NUM>. The system controller <NUM> not only gives an instruction related to light source control to the light source controller <NUM> but also controls a light source 23a (see <FIG>) of the beam light-emitting unit <NUM>. In the case of the normal observation mode, the system controller <NUM> performs a control to turn on the illumination light and to turn off the measurement beam light. In the case of the special observation mode, the system controller <NUM> performs a control to turn on the special light and to turn off the measurement beam light. In the case of the length measurement mode, the system controller <NUM> performs a control to turn on the illumination light and to turn on the measurement beam light.

The illumination optical system <NUM> includes the illumination lens 22a, and the object to be observed is irradiated with light, which is emitted from the light guide <NUM>, through the illumination lens 22a. The image pickup optical system <NUM> includes an objective lens 21a, a zoom lens 21b, and an image pickup element <NUM>. Light reflected from the object to be observed is incident on the image pickup element <NUM> through the objective lens 21a and the zoom lens 21b. Accordingly, the reflected image of the object to be observed is formed on the image pickup element <NUM>.

The zoom lens 21b has an optical zoom function to enlarge or reduce the subject as a zoom function by moving between a telephoto end and a wide end. ON/OFF of the optical zoom function can be switched by a zoom operation part 13c (see <FIG>) provided on the operation part 12b of the endoscope, and the subject can be enlarged or reduced at a specific magnification ratio in a case where the zoom operation part 13c is further operated in a state where the optical zoom function is ON.

The image pickup element <NUM> is a color image pickup sensor, and picks up the reflected image of an object to be examined and outputs image signals. It is preferable that the image pickup element <NUM> is a charge coupled device (CCD) image pickup sensor, a complementary metal-oxide semiconductor (CMOS) image pickup sensor, or the like. The image pickup element <NUM> used in an embodiment of the present invention is a color image pickup sensor that is used to obtain red images, green images, and blue images corresponding to three colors of R (red), G (green), and B (blue). The red image is an image that is output from red pixels provided with red color filters in the image pickup element <NUM>. The green image is an image that is output from green pixels provided with green color filters in the image pickup element <NUM>. The blue image is an image that is output from blue pixels provided with blue color filters in the image pickup element <NUM>. The image pickup element <NUM> is controlled by an image pickup controller <NUM>.

Since a red color filter RF has high transmittance in a red-light wavelength range of <NUM> to <NUM> as shown in <FIG>, a red pixel has high sensitivity in the red-light wavelength range. Since a green color filter GF has high transmittance in a green-light wavelength range of <NUM> to <NUM>, a green pixel has high sensitivity in the green-light wavelength range. Since a blue color filter BF has high transmittance in a blue-light wavelength range of <NUM> to <NUM>, a blue pixel has high sensitivity in the blue-light wavelength range. A red pixel has sensitivity even in the green-light wavelength range or the blue-light wavelength range. Further, a green pixel has sensitivity even in the red-light wavelength range or the blue-light wavelength range. Furthermore, a blue pixel has sensitivity even in the red-light wavelength range or the green-light wavelength range.

Image signals output from the image pickup element <NUM> are transmitted to a CDS/AGC circuit <NUM>. The CDS/AGC circuit <NUM> performs correlated double sampling (CDS) or auto gain control (AGC) on the image signals that are analog signals. The image signals, which have been transmitted through the CDS/AGC circuit <NUM>, are converted into digital image signals by an analog/digital converter (A/D converter) <NUM>. The digital image signals, which have been subjected to A/D conversion, are input to a communication interface (I/F) <NUM> of the light source device <NUM> through a communication interface (I/F) <NUM>.

In the processor device <NUM>, programs related to various types of processing are incorporated into a program memory (not shown). The programs incorporated into the program memory are operated by the system controller <NUM> formed of a processor, so that the processor device <NUM> realizes the functions of a reception unit <NUM>, a signal processing unit <NUM>, and a display controller <NUM>.

The reception unit <NUM> is connected to the communication interface (I/F) <NUM> of the light source device <NUM>. The reception unit <NUM> receives the image signals, which are transmitted from the communication I/F <NUM>, and transmits the image signals to the signal processing unit <NUM>. A memory, which temporarily stores the image signals received from the reception unit <NUM>, is built in the signal processing unit <NUM>, and the signal processing unit <NUM> processes an image signal group, which is a set of the image signals stored in the memory, to generate the subject image. The reception unit <NUM> may directly transmit control signals, which are related to the light source controller <NUM>, to the system controller <NUM>. Further, a processing unit or a storage unit (for example, an irradiation region-recognizing section <NUM> or a marker table <NUM>), which are related to the length measurement mode, of the processor device <NUM> may be provided in a length measurement processor (not shown) separate from the processor device <NUM>. In this case, the length measurement processor and the processor device <NUM> are in a state where the length measurement processor and the processor device <NUM> can communicate with each other so that images or various types of information can be transmitted and received.

In a case where the endoscope <NUM> is set to the normal observation mode, signal assignment processing for assigning the blue image of the subject image to B channels of the display <NUM>, assigning the green image of the subject image to G channels of the display <NUM>, and assigning the red image of the subject image to R channels of the display <NUM> is performed in the signal processing unit <NUM>. As a result, a color subject image is displayed on the display <NUM>. The same signal assignment processing as that in the normal observation mode is performed even in the length measurement mode. On the other hand, in a case where the endoscope <NUM> is set to the special observation mode, the red image of the subject image is not used for the display of the display <NUM>, the blue image of the subject image is assigned to the B channels and the G channels of the display <NUM>, and the green image of the subject image is assigned to the R channels of the display <NUM> in the signal processing unit <NUM>. As a result, a pseudo color subject image is displayed on the display <NUM>. In a case where the endoscope <NUM> is set to the length measurement mode, the signal processing unit <NUM> may be adapted to perform structure-emphasis processing for emphasizing structures, such as blood vessels, or color difference-emphasis processing for increasing a color difference between a normal area and a lesion area of the object to be observed on the subject image.

The display controller <NUM> causes the display <NUM> to display the subject image that is generated by the signal processing unit <NUM>. The system controller <NUM> performs the control of the image pickup element <NUM> through the image pickup controller <NUM> provided in the endoscope <NUM>. The image pickup controller <NUM> also performs the control of the CDS/AGC circuit <NUM> and the A/D converter <NUM> together with the control of the image pickup element <NUM>.

As shown in <FIG>, the beam light-emitting unit <NUM> emits measurement beam light Lm obliquely with respect to the optical axis Ax (see <FIG>) of the image pickup optical system <NUM>. The beam light-emitting unit <NUM> comprises a light source 23a, a diffractive optical element (DOE) 23b, a prism 23c, and an emitting section 23d. The light source 23a is to emit light having a color that can be detected by pixels of the image pickup element <NUM> (specifically visible light), and includes a light emitting element, such as a laser light source (laser diode (LD)) or a light emitting diode (LED), and a condenser lens that condenses light emitted from the light emitting element. The light source 23a is provided on a scope electric board (not shown). The scope electric board is provided in the distal end part 12d of the endoscope, and receives power supplied from the light source device <NUM> or the processor device <NUM> and supplies power to the light source 23a.

For example, red (the color of beam light) laser light having a wavelength of <NUM> or more and <NUM> or less is used as the light emitted from the light source 23a in this embodiment, but light having a wavelength in other ranges, for example, green light having a wavelength of <NUM> or more and <NUM> or less or blue light may be used. The light source 23a is controlled by the system controller <NUM>, and emits light on the basis of an instruction given from the system controller <NUM>. The DOE 23b converts the light, which is emitted from the light source, into the measurement beam light that is used to obtain measurement information. It is preferable that the amount of measurement beam light is adjusted from a standpoint of protecting a human body, eyes, and internal organs and is adjusted to the amount of light enough to cause halation (pixel saturation) in the observation range of the endoscope <NUM>.

The prism 23c is an optical member that is used to change the travel direction of the measurement beam light converted by the DOE 23b. The prism 23c changes the travel direction of the measurement beam light so that the measurement beam light intersects with the visual field of the image pickup optical system <NUM> including the objective lens 21a. The details of the travel direction of the measurement beam light will also be described later. A subject is irradiated with the measurement beam light Lm, which is emitted from the prism 23c, through the emitting section 23d that is formed of an optical member.

In a case where the subject is irradiated with the measurement beam light, a spot SP as a beam irradiation region is formed on the subject as shown in <FIG>. The position of the spot SP is recognized by the irradiation region-recognizing section <NUM> (see <FIG>), and a measurement marker showing the size of the subject is set according to the position of the spot SP. The display controller <NUM> causes the display <NUM> to display a measurement image in which the set measurement marker is displayed in the subject image.

The position of the spot SP and an observation distance (a distance between the distal end part 12d of the endoscope and the object to be observed) are related to each other. The observation distance is shorter as the position of the spot SP is closer to the lower side, and the observation distance is longer as the position of the spot SP is closer to the upper side. The set measurement marker Ma is displayed in the subject image. The radius of the measurement marker Ma from the spot SP shows the actual size (for example, <NUM>) of the subject. Further, the size (radius) of the measurement marker Ma is changed depending on the position of the spot SP, that is, the observation distance. For example, the size of the measurement marker Ma is larger as the position of the spot SP is closer to the lower side, and the size of the measurement marker Ma is smaller as the position of the spot SP is closer to the upper side.

A plurality of types of measurement markers, such as a first measurement marker and a second measurement marker, are included in the measurement marker as described later, and a measurement marker to be displayed in the subject image among the plurality of types of measurement markers can be selected according to a user's instruction. For example, the user interface <NUM> is used for the user's instruction.

The emitting section 23d may be formed of an auxiliary measurement slit, which is formed at the distal end part 12d of the endoscope, instead of being formed of an optical member. Further, in a case where the emitting section 23d is formed of an optical member, it is preferable that an anti-reflection coating (AR coating) (anti-reflection portion) is provided on the emitting section 23d. The reason why the anti-reflection coating is provided as described above is that it is difficult for the irradiation region-recognizing section <NUM> to be described later to recognize the position of the spot SP formed on the subject by the measurement beam light in a case where the measurement beam light is reflected without being transmitted through the emitting section 23d and a ratio of the measurement beam light with which a subject is irradiated is reduced.

The beam light-emitting unit <NUM> has only to be capable of emitting the measurement beam light to the visual field of the image pickup optical system <NUM>. For example, the light source 23a may be provided in the light source device and light emitted from the light source 23a may be guided to the DOE 23b by optical fibers. Further, the prism 23c may not be used and the orientations of the light source 23a and the DOE 23b may be inclined with respect to the optical axis Ax of the image pickup optical system <NUM> so that the measurement beam light Lm is emitted in a direction crossing the visual field of the image pickup optical system <NUM>.

With regard to the travel direction of the measurement beam light, the measurement beam light is emitted in a state where the optical axis of the measurement beam light Lm intersects with the optical axis Ax of the image pickup optical system <NUM> as shown in <FIG>. In a case where the subject can be observed in a range Rx of an observation distance, it is understood that the positions (points where the respective arrows Qx, Qy, and Qz intersect with the optical axis of the measurement beam light Lm) of the spot SP, which is formed on the subject by the measurement beam light Lm, in image pickup ranges (shown by arrows Qx, Qy, and Qz) at a near end Px, an intermediate vicinity Py, and a far end Pz of the range Rx are different from each other. The image pickup angle of view of the image pickup optical system <NUM> is represented by a region between two solid lines <NUM>, and measurement is performed in a central region (a region between two dotted lines <NUM>), in which an aberration is small, of this image pickup angle of view.

Since the measurement beam light Lm is emitted in a state where the optical axis of the measurement beam light Lm intersects with the optical axis Ax as described above, the size of the subject can be measured from the movement of the position of the spot with respect to a change in observation distance. Then, the image of the subject illuminated with the measurement beam light is picked up by the image pickup element <NUM>, so that a subject image including the spot SP is obtained. In the subject image, the position of the spot SP varies depending on a relationship between the optical axis Ax of the image pickup optical system <NUM> and the optical axis of the measurement beam light Lm and an observation distance. However, the number of pixels showing the same actual size (for example, <NUM>) is increased in the case of a short observation distance, and the number of pixels showing the same actual size is reduced in the case of a long observation distance.

As shown in <FIG>, the signal processing unit <NUM> of the processor device <NUM> comprises an irradiation region-recognizing section <NUM>, a measurement marker setting section <NUM>, and a marker table <NUM>. In a case where the endoscope <NUM> is set to the normal observation mode, the subject image of the subject illuminated with the illumination light is input to the signal processing unit <NUM>. In a case where the endoscope <NUM> is set to the special observation mode, the subject image of the subject illuminated with the special light is input to the signal processing unit <NUM>. In a case where the endoscope <NUM> is set to the length measurement mode, the subject image of the subject illuminated with the illumination light and the measurement beam light is input to the signal processing unit <NUM>.

The irradiation region-recognizing section <NUM> recognizes a beam irradiation region, which has a pattern having a specific shape, from the subject image. Specifically, the pattern having a specific shape includes a white central region CR1 and a peripheral region SR1 that covers the periphery of the central region and has a feature quantity based on the measurement beam light. In a case where the beam irradiation region is the above-mentioned spot SP, the pattern having a specific shape has a circular shape as shown in <FIG>. In this case, the white central region CR1 has a circular shape, and the peripheral region SR1 has the shape of a ring.

<FIG> shows the distribution of pixel values of the respective color images of the subject image that includes a red image RP, a green image GP, and a blue image BP as a plurality of color images. Since the pixel values of the red image RP, the green image GP, and the blue image in the central region CR1 reach the maximum pixel value (for example, <NUM>), the central region CR1 is white. In a case where the measurement beam light is incident on the image pickup element <NUM> in this case, not only the red color filter RF of the image pickup element <NUM> but also all the green color filter GF and the blue color filter BF transmit the measurement beam light with a transmittance of <NUM>% in the wavelength range WMB of the measurement beam light as shown in <FIG>. On the other hand, the pixel value of the red image RP is larger than the pixel value of the green image GP or the blue image BP in the peripheral region SR1. For this reason, the peripheral region SR1 has redness. The measurement beam light is emitted with a specific amount of light in the light source unit <NUM>, so that the pixel values of the red image RP, the green image GP, and the blue image BP in the central region CR1 are set to the maximum pixel value.

The irradiation region-recognizing section <NUM> can recognize the spot SP that has the specific shape and the feature quantity described above. Specifically, it is preferable that the irradiation region-recognizing section <NUM> includes a learning model <NUM> for recognizing the spot SP by outputting the spot SP, which is a beam irradiation region, in response to the input of the subject image as shown in <FIG>. The learning model <NUM> is subjected to machine learning using many teaching data in which a subject image and beam irradiation regions having been already recognized are associated with each other. It is preferable that a convolutional neural network (CNN) is used as the machine learning.

Since the spot SP is recognized using the learning model <NUM>, not only the circular spot SP (see <FIG>) that is formed of the circular central region CR1 and the ring-shaped peripheral region SR1 but also spots SP that have patterns deformed from a circular shape, which is a specific shape, can be recognized. For example, a spot SP, which is deformed in a vertical direction as shown in (A) of <FIG>, can also be recognized. Further, a spot SP, which is deformed such that a part of a circular shape is cut out as shown in (B) of <FIG>, can also be recognized. Furthermore, the feature quantity of the peripheral region SR1, which can be recognized by the learning model <NUM>, includes a blue color or a green color other than a red color that is the color of the measurement beam light. Moreover, the feature quantity of the peripheral region SR1, which can be recognized by the learning model <NUM>, includes the luminance, brightness, chroma saturation, or hue of the measurement beam light. It is preferable that luminance conversion processing or processing for converting brightness, chroma saturation, or a hue is performed on the peripheral region of the spot SP included in the subject image to acquire the luminance, brightness, chroma saturation, or hue of the measurement beam light.

Further, as shown in <FIG>, the irradiation region-recognizing section <NUM> may include a pattern matching-processing section <NUM> that recognizes the spot SP by performing pattern matching processing on the subject image using the template image of the spot SP as a predetermined template image of a beam irradiation region. The template image of the spot SP is stored in a template image storage section 60a. Not only the template image of the circular spot SP that is formed of the circular central region CR1 and the ring-shaped peripheral region SR1 but also the template images of spots SP that have patterns deformed from a circular shape, which is a specific shape, are stored in the template image storage section 60a. Further, the pattern matching-processing section <NUM> can also perform pattern matching processing using the pattern of the feature quantity of the peripheral region SR1. A feature quantity, which can be subjected to pattern matching processing, includes the distribution of a blue color or a green color other than the distribution of a red color that is the color of the measurement beam light. Further, a feature quantity, which can be subjected to pattern matching processing, includes the distribution of the luminance, brightness, chroma saturation, or hue of the measurement beam light.

The measurement marker setting section <NUM> sets a first measurement marker, which shows the actual size (the same size as the actual size) of the subject, as a measurement marker, which is used to measure the size of the subject, on the basis of the position of the spot SP. The measurement marker setting section <NUM> sets a measurement marker image, which corresponds to the position of the spot SP, with reference to the marker table <NUM> in which a measurement marker image of which the display aspect varies depending on the irradiation position of the spot SP is stored in association with the position of the spot SP. In addition to the position of the spot SP, the measurement marker image of which the display aspect varies depending on a marker display position where the first measurement marker is displayed on the display <NUM> may be stored in the marker table <NUM> in association with the position of the spot SP and the marker display position.

For example, the size or shape of the measurement marker image varies depending on the irradiation position of the spot SP. The display of the measurement marker image will be described later. Further, the contents stored in the marker table <NUM> are maintained even in a case where the power of the processor device <NUM> is turned off. The measurement marker image and the irradiation position are stored in the marker table <NUM> in association with each other, but a distance to the subject (a distance between the distal end part 12d of the endoscope <NUM> and the subject) corresponding to the irradiation position and the measurement marker image may be stored in the marker table <NUM> in association with each other.

The display controller <NUM> causes the display <NUM> to display the measurement image in which a measurement marker is displayed in the subject image on the basis of the position of the spot SP that is a beam irradiation region. Specifically, the display controller <NUM> causes the display <NUM> to display the measurement image in which the first measurement marker is superimposed to center on the spot SP. For example, a circular measurement marker is used as the first measurement marker. In this case, as shown in <FIG>, a marker M1, which shows an actual size of <NUM> (a horizontal direction and a vertical direction of the subject image), is displayed at the center of a spot SP1 formed on a tumor tm1 of a subject in a case where an observation distance is close to the near end Px. In a case where the measurement marker is displayed on the display <NUM>, an observation distance may also be displayed on the display <NUM> together.

Further, as shown in <FIG>, a marker M2, which shows an actual size of <NUM> (the horizontal direction and the vertical direction of the subject image), is displayed at the center of a spot SP2 formed on a tumor tm2 of a subject in a case where an observation distance is close to the intermediate vicinity Py. Since the marker display position of the marker M2 is positioned at the central portion of the subject image that is less likely to be affected by distortion caused by the image pickup optical system <NUM>, the marker M2 has a circular shape without being affected by the distortion or the like.

Furthermore, as shown in <FIG>, a marker M3, which shows an actual size of <NUM> (the horizontal direction and the vertical direction of the subject image), is displayed at the center of a spot SP3 formed on a tumor tm3 of a subject. As shown in <FIG> having been described above, the size of the first measurement marker corresponding to the same actual size of <NUM> is reduced with an increase in an observation distance. Moreover, the shape of the first measurement marker also varies depending on the marker display position due to an influence of the distortion caused by the image pickup optical system <NUM>.

In <FIG>, the center of the spot SP and the center of the marker are displayed to coincide with each other. However, the first measurement marker may be displayed at a position away from the spot SP in a case where there is no problem in measurement accuracy. Even in this case, it is preferable that the first measurement marker is displayed near the spot.

The first measurement marker corresponding to the actual size of the subject, which is <NUM>, is displayed in <FIG>, but the actual size of the subject may be set to any value (for example, <NUM>, <NUM>, <NUM>, or the like) according to an object to be observed or the purpose of observation. Further, the first measurement marker has a substantially circular shape in <FIG>, but may have a cruciform shape where a vertical line and a horizontal line intersect with each other as shown in <FIG>. Furthermore, the first measurement marker may have a cruciform shape with gradations where gradations Mx are given to at least one of a vertical line or a horizontal line of a cruciform shape. Further, the first measurement marker may have a distorted cruciform shape of which at least one of a vertical line or a horizontal line is inclined. Furthermore, the first measurement marker may have a circular-and-cruciform shape where a cruciform shape and a circle are combined with each other. In addition, the first measurement marker may have the shape of a measurement point group where a plurality of measurement points EP corresponding to an actual size from a spot are combined with each other. Further, one first measurement marker may be displayed or a plurality of first measurement markers may be displayed, and the color of the first measurement marker may be changed according to an actual size.

As shown in <FIG>, three concentric circular markers M4A, M4B, and M4C having different sizes (diameters as the sizes are <NUM>, <NUM>, and <NUM>, respectively) may be displayed in the subject image as the first measurement marker to center on a spot SP4 formed on a tumor tm4. Since the three concentric circular markers are displayed as a plurality of markers, time and effort required to switch a marker can be saved and measurement can be performed even in a case where a subject has a non-linear shape. In a case where a plurality of concentric circular markers are to be displayed to center on a spot, a size and a color are not designated for each marker and combinations of a plurality of conditions may be prepared in advance and one can be selected from these combinations.

In <FIG>, all the three concentric circular markers are displayed with the same color (black). However, in a case where a plurality of concentric circular markers are to be displayed, a plurality of color concentric circular markers of which colors are different from each other may be used. As shown in <FIG>, a marker M5A is displayed by a dotted line representing a red color, a marker M5B is displayed by a solid line representing a blue color, and a marker M5C is displayed by a one-dot chain line representing a white color. Since identifiability can be improved in a case where the colors of the markers are different from each other in this way, measurement can be easily performed.

Further, as shown in <FIG>, a plurality of distorted concentric circular markers, which are distorted from the respective concentric circles, may be used as the first measurement marker other than the plurality of concentric circular markers. In this case, distorted concentric circular markers M6A, M6B, and M6C are displayed in the subject image to center on a spot SP5 formed on a tumor tm5.

Light, which forms a spot in a case where a subject is irradiated with the light, is used as the measurement beam light, but other light may be used. For example, plane measurement beam light, which forms an intersection line <NUM> on the subject as shown in <FIG> in a case where the subject is irradiated with the light, may be used. In a case where the subject is irradiated with the plane measurement beam light, the intersection line <NUM> as a line-like beam irradiation region is formed on the subject. In this case, a second measurement marker that consists of the intersection line <NUM> and gradations <NUM> formed on the intersection line <NUM> and serving as an index of the size of the subject (for example, a polyp P) is generated as a measurement marker.

In a case where the plane measurement beam light is used, the irradiation region-recognizing section <NUM> recognizes the position of the intersection line <NUM> (the irradiation position of the measurement beam light) as the beam irradiation region that has the pattern having a specific shape. As shown in <FIG>, the intersection line <NUM> includes a white central region CR2 and a peripheral region SR2 that covers the white central region R2 and has a feature quantity based on the measurement beam light. The central region CR2 has a linear shape, and the peripheral region SR2 is formed in the shape of two lines that are spaced from each other with a specific interval therebetween. The irradiation region-recognizing section <NUM> can recognize the intersection line <NUM> that has the pattern and the feature quantity described above. An intersection line <NUM> that has a shape deformed from a linear shape, such as a shape in which a part or the like of a line is cut out can also be recognized by the irradiation region-recognizing section <NUM>.

Like the spot SP, the observation distance is shorter as the intersection line <NUM> recognized by the irradiation region-recognizing section <NUM> is positioned closer to the lower side, and the observation distance is longer as the intersection line <NUM> is positioned closer to the upper side. For this reason, in a case where the second measurement marker is displayed in the measurement image, an interval between the gradations <NUM> is larger as the intersection line <NUM> is positioned closer to the lower side and an interval between the gradations <NUM> is smaller as the intersection line <NUM> is positioned closer to the upper side.

In the above description, a measurement marker corresponding to the position of a beam irradiation region, such as the spot SP or the intersection line <NUM>, has been displayed in the measurement image. However, a measurement marker corresponding to the pattern of a beam irradiation region may be displayed in the measurement image instead of this. For example, in a case where a beam irradiation region is the spot SP, a diameter DM of the white central region CR1 of the spot SP is obtained as shown in <FIG> and a measurement marker corresponding to the diameter DM is displayed in the measurement image. It is understood that the diameter DM is large in a case where an observation distance is short, and the diameter is reduced as an observation distance is increased. A measurement marker image of which the size, shape, or the like varies depending on the diameter DM is determined using a relationship between the diameter DM and an observation distance.

Next, a series of flows of the present invention will be described with reference to a flowchart shown in <FIG>. In a case where a user operates the mode changeover switch 13a to switch a mode to the length measurement mode, a subject is irradiated with the illumination light and the measurement beam light. A spot SP is formed on the subject by the measurement beam light. A subject image including the spot SP is obtained from the image pickup of the subject including the spot SP.

The reception unit <NUM> receives the subject image output from the endoscope <NUM>. The irradiation region-recognizing section <NUM> recognizes the position of the spot SP from the subject image. In this case, the irradiation region-recognizing section <NUM> recognizes the position of the spot SP that includes the white central region CR1 and the peripheral region SR1 covering the periphery of the central region CR1 and having a feature quantity based on the measurement beam light. The display controller <NUM> causes the display <NUM> to display a measurement image in which a measurement marker to be used to measure the size of the subject is displayed in the subject image on the basis of the position of the spot SP recognized by the irradiation region-recognizing section <NUM>. The measurement marker corresponds to the position of the spot SP, and the size, shape, or the like is changed depending on the position of the spot SP.

In the embodiment, the hardware structures of processing units, which perform various types of processing, such as the reception unit <NUM>, the signal processing unit <NUM>, the display controller <NUM>, the system controller <NUM>, the static image storage unit <NUM>, the irradiation region-recognizing section <NUM>, the learning model <NUM>, the pattern matching-processing section <NUM>, the template image storage section 60a, the measurement marker setting section <NUM>, and the marker table <NUM>, are various processors to be described below. Various processors include: a central processing unit (CPU) that is a general-purpose processor functioning as various processing units by executing software (program); a programmable logic device (PLD) that is a processor of which the circuit configuration can be changed after manufacture, such as a field programmable gate array (FPGA); a dedicated electrical circuit that is a processor having circuit configuration designed exclusively to perform various types of processing; and the like.

One processing unit may be formed of one of these various processors, or may be formed of a combination of two or more same kind or different kinds of processors (for example, a plurality of FPGAs, or a combination of a CPU and an FPGA). Further, a plurality of processing units may be formed of one processor. As an example where a plurality of processing units are formed of one processor, first, there is an aspect where one processor is formed of a combination of one or more CPUs and software as typified by a computer, such as a client or a server, and functions as a plurality of processing units. Second, there is an aspect where a processor fulfilling the functions of the entire system, which includes a plurality of processing units, by one integrated circuit (IC) chip as typified by System On Chip (SoC) or the like is used. In this way, various processing units are formed using one or more of the above-mentioned various processors as hardware structures.

Claim 1:
An endoscope system comprising:
an endoscope (<NUM>) that includes a beam light-emitting unit (<NUM>) irradiating a subject with measurement beam light and an image pickup optical system (<NUM>) provided in a distal end part (12d) at a position different from a position of the beam light-emitting unit and receiving light from the subject, the measurement beam light being emitted obliquely with respect to an optical axis of the image pickup optical system; and
a processor device (<NUM>),
wherein the processor device includes a processor that acquires a subject image that is obtained on the basis of the light received by the image pickup optical system and includes a beam irradiation region appearing on the subject due to irradiation with the measurement beam light, recognizes the beam irradiation region from the subject image, and causes a display to display a measurement image in which a measurement marker to be used to measure a size of the subject is displayed in the subject image on the basis of the beam irradiation region; characterized in that:
the beam irradiation region of the subject image has a pattern having a specific shape that includes a white central region and a peripheral region covering a periphery of the central region and having a feature quantity based on the measurement beam light irradiated on the subject, and
wherein the processor recognizes the beam irradiation region having the pattern having the specific shape; and wherein:
the feature quantity is at least one of a color of the measurement beam light, a color other than the color of the measurement beam light, or luminance, brightness, chroma saturation, or hue of the measurement beam light.