ENDOSCOPE SYSTEM AND PROCESSOR UNIT

An endoscope system includes an endoscope, at least one monitor, a first measuring instrument configured to measure a first direction, at least one second measuring instrument configured to measure a second direction, a reception circuit configured to receive the image from the endoscope, and a processor. The first direction corresponds to a horizontal component of an imaging direction of the endoscope. The second direction corresponds to a horizontal component of a reference direction. The reference direction is a direction in which the at least one monitor faces or is a direction toward or away from an image display surface of the at least one monitor. The processor is configured to calculate an angle between the first direction and the second direction, process the image based on the angle, and display the processed image on the at least one monitor. The processing includes a rotation processing or a horizontal flip processing.

FIELD OF THE DISCLOSURE

The present disclosure relates to an endoscope system and a processor unit.

BACKGROUND

In a surgical operation using an endoscope, the endoscope is inserted into the body of a patient and acquires an image of an organ or the like. An operator and an assistant perform a surgical operation while observing the image acquired by the endoscope. The operator and the assistant manipulate treatment tools while checking the states of the treatment tools seen in the image.

The operator and the assistant stand at different positions and face different directions. Therefore, a plurality of monitors are disposed in an operation room. Each of the monitors displays an image acquired by the endoscope.

There is a case in which the direction of the endoscope and the direction of the visual line of the operator or the assistant are greatly different from each other. In such a case, the direction of a treatment tool seen in an image is greatly different from that of the treatment tool actually viewed by the operator or the assistant. Since the moving direction of a treatment tool recognized by the operator or the assistant is greatly different from that of the treatment tool in an image displayed on each monitor, the operator or the assistant needs to be accustomed to manipulations.

Published Japanese Translation No. 2008-517703 of the PCT International Publication discloses a method of matching the direction of an organ for an operator or an assistant to the direction of the organ in an image. In the method, the operator or the assistant issues a rotation request by manipulating a manipulation unit. A monitor rotates based on the rotation request, or an image rotates based on the rotation request.Patent Literature 1: Published Japanese Translation No. 2008-517703 of the PCT International Publication

SUMMARY

According to a first aspect of the present disclosure, an endoscope system includes an endoscope configured to acquire an image of a subject, at least one monitor, a first measuring instrument configured to measure a first direction. The first direction corresponds to a horizontal component of an imaging direction of the endoscope. The endoscope system includes at least one second measuring instrument configured to measure a second direction. The second direction corresponds to a horizontal component of a reference direction. The reference direction is a direction in which the at least one monitor faces or is a direction toward or away from an image display surface of the at least one monitor. The endoscope system includes a reception circuit configured to receive the image from the endoscope and a processor. The processor calculates an angle between the first direction and the second direction. The processor processes the image based on the angle, wherein the processing includes a rotation processing or a horizontal flip processing. The processor displays the processed image on the at least one monitor.

According to a second aspect of the present disclosure, the second measuring instrument may be disposed on the monitor. The reference direction is the direction in which the at least one monitor faces.

According to a third aspect of the present disclosure, in the first aspect, the second measuring instrument is disposed on a body of the user. The reference direction is the direction toward the image display surface of the at least one monitor.

According to a fourth aspect of the present disclosure, in the first aspect, the endoscope system may further comprise a treatment tool. A third direction corresponds to a horizontal component of a reference direction of the treatment tool. The at least one second measuring instrument is further configured to measure the third direction by using an image that contains the treatment tool.

According to a fifth aspect of the present disclosure, in the first aspect, the second measuring instrument measures the second direction using an image that contains the at least one monitor.

According to a sixth aspect of the present disclosure, in the first aspect, the processor may display a value of the angle on the monitor.

According to a seventh aspect of the present disclosure, in the first aspect, the processor may display an icon on the monitor, where the icon has an orientation that corresponds to the angle.

According to an eighth aspect of the present disclosure, in the first aspect, the endoscope may acquire the image at a predetermined frame rate. The processor may calculate the angle at a lower rate than the frame rate.

According to a ninth aspect of the present disclosure, in the first aspect, the endoscope system may further comprise a sensor that measures a depression angle between the imaging direction and a horizontal plane. The processor may correct distortion of the image, which is generated in accordance with a distance between the endoscope and a subject, by using the depression angle.

According to a tenth aspect of the present disclosure, in the first aspect, the endoscope includes an image sensor in which a plurality of pixels are disposed in a matrix shape. An effective vertical pixel number of the image sensor is greater than a vertical pixel number of the image displayed on the at least one monitor.

According to an eleventh aspect of the present disclosure, in the first aspect, the endoscope system may further comprise a sensor that measures a depression angle between the imaging direction and a horizontal plane. When the depression angle is larger than a preset angle, the processor performs the rotation processing on the image based on the angle.

According to a twelfth aspect of the present disclosure, in the first aspect, a sensor may measure a depression angle between the imaging direction and a horizontal plane. When the depression angle is smaller than a preset first angle and the angle between the first direction and the second direction is larger than a preset second angle, the processor performs the horizontal flip processing on the image.

According to a thirteenth aspect of the present disclosure, in the first aspect, the image of the subject may include a left image and a right image used for displaying a stereoscopic image. The processing may include replacing the left image and the right image with each other.

According to a fourteenth aspect of the present disclosure, a processor unit includes a reception circuit connected using a wireless or a wired connection to an endoscope, and a processor. The endoscope acquires an image of a subject in a living body and the reception circuit receives the image from the endoscope. The processor calculates an angle between a first direction and a second direction, where the first direction corresponds to a horizontal component of an imaging direction of the endoscope and the second direction corresponds to a horizontal component of a reference direction, where the reference direction is a direction in which a monitor faces or is a direction in which a user faces a monitor. The processor processes the image based on the angle. The processing includes a rotation processing or a horizontal flip processing. The processer displays the processed image on the monitor.

According to a fourteenth aspect of the present disclosure, an image rotation method performed by a processor. The image rotation method includes calculating an angle between a first direction and a second direction, processing an image acquired by the endoscope based on the angle, wherein the processing includes rotation processing or horizontal flip processing; and displaying the processed image on the monitor.

The first direction corresponds to a horizontal component of an imaging direction of an endoscope and the second direction corresponds to a horizontal component of a reference direction, where the reference direction is a direction in which a monitor faces or is a direction in which a user faces a monitor.

DETAILED DESCRIPTION

First Embodiment

A first embodiment of the present disclosure will be described.FIG.1shows a schematic configuration of an endoscope system1according to the first embodiment of the present disclosure. The endoscope system1shown inFIG.1includes an endoscope insertion unit2, a universal code3, a manipulation unit4, a connector unit5, a processor unit6, and a monitor7. The endoscope insertion unit2, the universal code3, the manipulation unit4, and the connector unit5constitute a scope8(endoscope).

The endoscope insertion unit2includes an insertion unit20and an imaging unit21. The insertion unit20is inserted into a living body, which is a subject. The imaging unit21is disposed at the distal end of the insertion unit20and generates a video signal by imaging the inside of the subject. The manipulation unit4is connected to the end of the insertion unit20, which is opposite the imaging unit21. The manipulation unit4accepts various manipulations for the endoscope insertion unit2from a user.

The universal code3connects the endoscope insertion unit2and the connector unit5. The video signal generated by the imaging unit21is output to the connector unit5via a transmission cable (not shown in the drawing) inserted through the insertion unit20, the manipulation unit4, and the universal code3.

The connector unit5performs predetermined processing on the video signal output from the imaging unit21. The connector unit5is connected to the processor unit6and outputs the video signal to the processor unit6.

The processor unit6performs image processing on the video signal output from the connector unit5. Furthermore, the processor unit6centrally controls the entire endoscope system1.

The monitor7is a liquid crystal display (LCD) or the like. The monitor7displays a video based on the video signal processed by the processor unit6. In addition, the monitor7displays various kinds of information related to the endoscope system1. As described later, a plurality of monitors are used as the monitor7.

FIG.2schematically shows a layout of the endoscope system1in an operation room. Each configuration overlooked in a direction vertical to the ground is shown inFIG.2. A direction used for descriptions ofFIG.2indicates a horizontal direction that is parallel to the horizontal plane.

Three users are around a patient P1. The three users are an operator U1, an assistant U2, and a scopist U3. The operator U1, the assistant U2, and the scopist U3are at different positions. The operator U1and the assistant U2face each other. The patient P1is lying between the operator U1and the assistant U2. For example, the operator U1is on the left side of the patient P1, and the assistant U2is on the right side of the patient P1. The operator U1may be on the right side of the patient P1, and the assistant U2may be on the left side of the patient P1.

The endoscope insertion unit2and treatment tools T1to T4are inserted into an inside P2of the body cavity of the patient P1through a trocar (not shown in the drawing). The operator U1manipulates a treatment tool T1and a treatment tool T2. The assistant U2manipulates a treatment tool T3and a treatment tool T4. The operator U1and the assistant U2perform treatment on an organ OR1of the patient P1by manipulating each treatment tool. The scopist U3holds the manipulation unit4. The imaging unit21of the endoscope insertion unit2is shown inFIG.2.

A monitor7a, a monitor7b, and a monitor7ccorresponding to the monitor7are disposed. The monitor7ais a main monitor, and the monitor7band the monitor7care sub monitors. The screen of the monitor7afaces the operator U1. The screen of the monitor7bfaces the assistant U2. The screen of the monitor7cfaces the scopist U3. The monitor7afacing the scopist U3does not need to be disposed, and the scopist U3may observe an image displayed on the monitor7bfacing the operator U1or the monitor7cfacing the assistant U2.

An imaging direction De of the imaging unit21, a visual line direction Du1of the operator U1, and a visual line direction Du2of the assistant U2are shown inFIG.2. Each of the imaging direction De, the visual line direction Du1, and the visual line direction Du2indicates a horizontal component that is parallel to the horizontal plane. The visual line direction Du1is greatly different from the imaging direction De, and the visual line direction Du2is greatly different from the imaging direction De.

A direction Dh and a direction Dt are shown inFIG.2. The direction Dh and the direction Dt indicate a direction of the patient P1. The direction Dh indicates a direction from the center of the trunk of the patient P1toward the head of the patient P1. The direction Dt indicates a direction opposite to the direction Dh. Each of the directions Dh and Dt inFIG.2indicates a horizontal component that is parallel to the horizontal plane. For example, the imaging direction De is almost the same as the direction Dh. The direction Dh seen from the operator U1is the right direction, and the direction Dt seen from the operator U1is the left direction. The direction Dh seen from the assistant U2is the left direction, and the direction Dt seen from the assistant U2is the right direction.

FIG.3shows an example of an image displayed on the monitor7c. The monitor7cdisplays an image IMG10. The organ OR1, the treatment tool T1, the treatment tool T2, the treatment tool T3, and the treatment tool T4are seen in the image IMG10. Since the imaging direction De shown inFIG.2is almost the same as the direction Dh, the upward direction in the image IMG10is almost the same as the direction Dh and the downward direction in the image IMG10is almost the same as the direction Dt.

In a case in which the monitor7adisplays the same image as the image IMG10, the actual direction Dh seen from the operator U1is the right direction and the direction Dh in an image displayed on the monitor7aobserved by the operator U1is the upward direction. In a case in which the monitor7bdisplays the same image as the image IMG10, the actual direction Dh seen from the assistant U2is the left direction and the direction Dh in an image displayed on the monitor7bobserved by the assistant U2is the upward direction. Since the actual direction Dh seen from the operator U1or the assistant U2is different from the direction Dh in an image displayed on the monitor7aor the monitor7b, the operator U1or the assistant U2needs to correct the direction of each treatment tool in their heads.

In the first embodiment, the endoscope system1rotates an image in accordance with an angle between the imaging direction of the imaging unit21and the direction of each monitor. Processing of rotating an image acquired by the imaging unit21will be described by usingFIGS.4to6.

FIG.4shows an example of a positional relationship between the imaging unit21, the monitor7a, the monitor7b, the operator U1, and the assistant U2. An imaging direction De, a direction D11, and a direction D12are shown inFIG.4. The direction D11indicates a horizontal component of a reference direction indicating a direction in which the monitor7afaces. For example, the reference direction of the monitor7ais a direction perpendicular to the screen of the monitor7a. The direction D12indicates a horizontal component of a reference direction indicating a direction in which the monitor7bfaces. For example, the reference direction of the monitor7bis a direction perpendicular to the screen of the monitor7b.

The endoscope system1calculates an angle φ between the imaging direction De and the direction D11and calculates an angle θ between the imaging direction De and the direction D12. The angle φ and the angle θ are expressed as a value of −180 degrees or more and 180 degrees or less. A positive direction of the angle φ and the angle θ is a clockwise direction, and a negative direction of the angle φ and the angle θ is a counterclockwise direction. The endoscope system1rotates an image acquired by the imaging unit21by the angle +φ so as to correct the image. The endoscope system1displays the corrected image on the monitor7a. In addition, the endoscope system1rotates an image acquired by the imaging unit21by the angle −θ so as to correct the image. The endoscope system1displays the corrected image on the monitor7b.

FIG.5shows an example of an image displayed on the monitor7a. The monitor7adisplays an image IMG11. The visual line direction of the operator U1is a direction opposite to the direction D11shown inFIG.4, and is obtained by rotating the imaging direction De counterclockwise by the angle φ. The endoscope system1rotates an image acquired by the imaging unit21clockwise by the angle φ so as to obtain the image IMG11.

The actual direction Dh seen from the operator U1is the right direction, and the direction Dh in the image IMG11is a direction close to the right direction. Since the actual direction of the patient P1seen from the operator U1almost matches the direction of the patient P1in the image IMG11, the operator U1does not need to correct the direction of each treatment tool in his/her head. Therefore, the operator U1can intuitively manipulate each treatment tool.

FIG.6shows an example of an image displayed on the monitor7b. The monitor7bdisplays an image IMG12. The visual line direction of the assistant U2is a direction opposite to the direction D12shown inFIG.4, and is obtained by rotating the imaging direction De clockwise by the angle θ. The endoscope system1rotates an image acquired by the imaging unit21counterclockwise by the angle θ so as to obtain the image IMG12.

The actual direction Dh seen from the assistant U2is the left direction, and the direction Dh in the image IMG12is a direction close to the left direction. Since the actual direction of the patient P1seen from the assistant U2almost matches the direction of the patient P1in the image IMG12, the assistant U2does not need to correct the direction of each treatment tool in his/her head. Therefore, the assistant U2can intuitively manipulate each treatment tool. For example, when the operator U1instructs the assistant U2to pull the treatment tool T3, the assistant U2can intuitively understand that he/she should pull the left hand.

A configuration of the endoscope system1will be described in detail.FIG.7shows the configuration of the endoscope system1. The endoscope system1includes an azimuth sensor10and an azimuth sensor11in addition to the endoscope insertion unit2, the manipulation unit4, the processor unit6, the monitor7a, the monitor7b, and the monitor7c. The insertion unit20and the imaging unit21of the endoscope insertion unit2are shown inFIG.7. The universal code3and the connector unit5shown inFIG.1are not shown inFIG.7.

The imaging unit21includes an image sensor22and an azimuth sensor23. The processor unit6includes an image communication unit60, a sensor communication unit61, a sensor communication unit62, a sensor communication unit63, and a processor64.

A schematic function of the endoscope system1will be described. The imaging unit21acquires an image of a subject inside a living body. For example, the subject is the organ OR1shown inFIG.2. The azimuth sensor23(first measuring instrument) measures a first direction indicating a horizontal component of the imaging direction of the imaging unit21. The azimuth sensor10(second measuring instrument) measures a second direction indicating a horizontal component of the reference direction of the monitor7a, and the azimuth sensor11(second measuring instrument) measures a second direction indicating a horizontal component of the reference direction of the monitor7b.

The image communication unit60(reception circuit) receives an image from the imaging unit21. The processor64calculates an angle between the first direction and each of the two second directions. The processor64performs rotation processing on the image received from the imaging unit21based on each of the calculated angles. The processor64displays the images on which the rotation processing has been performed on the monitor7aand the monitor7b.

The function of the endoscope system1will be described in detail. The imaging unit21includes an imaging optical system not shown inFIG.7. The imaging optical system includes a lens. The imaging optical system may be separated from the imaging unit21, and the imaging optical system may be mounted on the distal endo of the imaging unit21.

The image sensor22and the azimuth sensor23are disposed inside the imaging unit21. The image sensor22is a complementary metal-oxide-semiconductor (CMOS) sensor or the like. The image sensor22includes a plurality of pixels disposed in a matrix shape and generates an image at a predetermined frame rate.

The azimuth sensor23is a geomagnetic sensor or the like. The azimuth sensor23may be disposed inside the image sensor22. The azimuth sensor23measures the imaging direction of the imaging unit21and generates imaging direction information indicating the measured imaging direction. The imaging direction corresponds to the first direction. For example, the imaging direction is parallel to the optical axis of the imaging optical system included in the imaging unit21. The imaging direction may be perpendicular to an imaging plane included in the image sensor22.

For example, the imaging direction information indicates a three-dimensional imaging direction of the imaging unit21. The processor64can calculate a horizontal component of the imaging direction by using the imaging direction information. Therefore, the imaging direction information includes information indicating the horizontal component of the imaging direction.

The processor64can calculate a depression angle of the imaging direction by using the imaging direction information. The depression angle indicates an angle between the imaging direction and the horizontal plane.

FIG.8shows the depression angle of the imaging direction.FIG.8shows the state of the imaging unit21in a plane perpendicular to the horizontal plane. A direction D13, a direction D14, and an angle β are shown inFIG.8. The direction D13indicates a horizontal direction that is parallel to the horizontal plane. The direction D14indicates the imaging direction. The angle β indicates an angle between the direction D13and the direction D14. The angle β corresponds to the depression angle of the imaging direction. The depression angle is expressed as a value of 0 degrees or more and 90 degrees or less. The imaging direction information includes depression angle information indicating the depression angle.

The azimuth sensor10and the azimuth sensor11are geomagnetic sensors or the like. For example, the azimuth sensor10is disposed on the surface of the monitor7a, and the azimuth sensor11is disposed on the surface of the monitor7b. The azimuth sensor10may be disposed inside the monitor7a. The azimuth sensor11may be disposed inside the monitor7b.

The azimuth sensor10measures the reference direction of the monitor7aand generates monitor direction information indicating the measured reference direction. The azimuth sensor11measures the reference direction of the monitor7band generates monitor direction information indicating the measured reference direction.

For example, the monitor direction information indicates a three-dimensional reference direction of the monitor7aor the monitor7b. The processor64can calculate a horizontal component of the reference direction by using the monitor direction information. Therefore, the monitor direction information includes information indicating the horizontal component of the reference direction. The reference direction of the monitor7aand the reference direction of the monitor7bcorrespond to the second direction.

The image communication unit60includes a communication circuit and is connected to the image sensor22by using wireless or wired connection. The image communication unit60performs communication with the image sensor22and receives an image from the image sensor22. The image communication unit60outputs the received image to the processor64.

The sensor communication unit61includes a communication circuit and is connected to the azimuth sensor23by using wireless or wired connection. The sensor communication unit61performs communication with the azimuth sensor23and receives the imaging direction information from the azimuth sensor23. The sensor communication unit61outputs the received imaging direction information to the processor64.

The sensor communication unit62includes a communication circuit and is connected to the azimuth sensor10by using wireless or wired connection. The sensor communication unit62performs communication with the azimuth sensor10and receives the monitor direction information related to the monitor7afrom the azimuth sensor10. The sensor communication unit62outputs the received monitor direction information to the processor64.

The sensor communication unit63includes a communication circuit and is connected to the azimuth sensor11by using wireless or wired connection. The sensor communication unit63performs communication with the azimuth sensor11and receives the monitor direction information related to the monitor7bfrom the azimuth sensor11. The sensor communication unit63outputs the received monitor direction information to the processor64.

For example, the processor64may be constituted by at least one of a central processing unit (CPU), a digital signal processor (DSP), and a graphics-processing unit (GPU). The processor64may be constituted by a logic circuit such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). The processor64may include one or a plurality of processors. The processor64may include one or a plurality of logic circuits.

The processor64may read a program and execute the read program. The program includes commands defining the operations of the processor64. In other words, the functions of the processor64may be realized by software. The program may be transmitted from a computer storing the program to the endoscope system1through a transmission medium or transmission waves in a transmission medium. The “transmission medium” transmitting the program is a medium having a function of transmitting information. The medium having the function of transmitting information includes a network (communication network) such as the Internet and a communication circuit line (communication line) such as a telephone line. The program described above may realize some of the functions described above. In addition, the program described above may be a differential file (differential program). The functions described above may be realized by a combination of a program that has already been recorded in a computer and a differential program.

The processor64calculates a horizontal component of the imaging direction based on the imaging direction information. The processor64calculates a horizontal component of the reference direction of the monitor7abased on the monitor direction information related to the monitor7a. The processor64calculates a horizontal component of the reference direction of the monitor7bbased on the monitor direction information related to the monitor7b.

The processor64calculates an angle between the horizontal component of the imaging direction and the horizontal component of the reference direction of the monitor7aso as to calculate the angle φ shown inFIG.4. The processor64calculates an angle between the horizontal component of the imaging direction and the horizontal component of the reference direction of the monitor7bso as to calculate the angle θ shown inFIG.4. The processor64performs the rotation processing on an image received from the imaging unit21based on the angle φ or the angle θ. The processor64outputs the image on which the rotation processing has been performed to the monitor7aor the monitor7b.

The endoscope system1may include a camera instead of the azimuth sensor23, the azimuth sensor10, and the azimuth sensor11. For example, the camera is disposed on the ceiling of the operation room. The camera generates an image in which the insertion unit20, the monitor7a, and the monitor7bare seen. The processor64may calculate the imaging direction of the imaging unit21, the reference direction of the monitor7a, and the reference direction of the monitor7bby analyzing the image.

In the above-described example, the processor64functions as a first measuring instrument that measures a first direction indicating a horizontal component of the imaging direction of the imaging unit21. In the above-described example, the processor64functions as a second measuring instrument that measures a second direction indicating a horizontal component of the reference direction of the monitor7aor the monitor7b.

In a case in which the insertion unit20is capable of bending in a predetermined direction, the processor64may calculate a bending direction of the insertion unit20. The processor64may calculate a direction of the insertion unit20outside the body of the patient P1by analyzing an image generated by the above-described camera. The processor64may calculate an imaging direction of the imaging unit21based on the calculated direction and the bending direction of the insertion unit20. In this example, the endoscope system1does not need to include the azimuth sensor23. In this example, the processor64functions as a first measuring instrument that measures a first direction indicating a horizontal component of the imaging direction of the imaging unit21.

The azimuth sensor23may be disposed inside the manipulation unit4. The azimuth sensor23disposed inside the manipulation unit4may measure an imaging direction of the imaging unit21and may generate imaging direction information. In a case in which the insertion unit20is capable of bending in a predetermined direction, the processor64may calculate a bending direction of the insertion unit20based on a manipulation state of a bending manipulation unit (not shown in the drawing) installed in the manipulation unit4.

An operation of the processor unit6will be described in detail by usingFIG.9.FIG.9shows a procedure of the operation of the processor unit6.

The image communication unit60receives an image output from the image sensor22(Step S100).

After Step S100, the sensor communication unit61receives the imaging direction information output from the azimuth sensor23(Step S101).

After Step S101, the sensor communication unit62receives the monitor direction information output from the azimuth sensor10, and the sensor communication unit63receives the monitor direction information output from the azimuth sensor11(Step S102). The monitor direction information output from the azimuth sensor10and the monitor direction information output from the azimuth sensor11are not necessarily received at the same time.

The order in which Steps S100to S102are executed is not limited to that shown inFIG.8. Steps S100to S102may be executed in any order.

Steps S100to S102may be executed at different frequencies. For example, Step S101or Step S102may be executed less frequently than Step S100. The frequency at which Step S102is executed may be different from that at which Step S101is executed.

After Step S102, the processor64calculates an angle −φ between the horizontal component of the imaging direction and the horizontal component of the reference direction of the monitor7a. In addition, the processor64calculates an angle +θ between the horizontal component of the imaging direction and the horizontal component of the reference direction of the monitor7b(Step S103).

After Step S103, the processor64calculates a depression angle between the imaging direction and the horizontal plane by using the imaging direction information. The processor64determines whether the depression angle of the imaging direction is within a range of 30 to 90 degrees (Step S104).

The depression angle of the imaging direction is expressed as a value of 0 degrees or more and 90 degrees or less. When the depression angle of the imaging direction is less than 30 degrees, the imaging unit21acquires an image of a subject seen in a direction close to the horizontal direction. In this case, the processor64determines that the depression angle of the imaging direction is not within the range of 30 to 90 degrees and executes Step S106described later. The rotation processing is not performed on the image acquired by the imaging unit21.

The above-described 30 degrees are an example of a threshold value used for determining the size of the depression angle. The threshold value is not limited to 30 degrees. The threshold value may be 45 degrees or the like.

When the depression angle of the imaging direction is 30 degrees or more and 90 degrees or less, the imaging unit21acquires an image of the subject seen below the horizontal direction. In this case, the processor64determines that the depression angle of the imaging direction is within the range of 30 to 90 degrees and performs the rotation processing on the image received in Step S100. At this time, the processor64rotates the received image by the angle +φ so as to correct the image and rotates the received image by the angle −θ so as to correct the image (Step S105).

After Step S105, the processor64outputs the image that has been corrected based on the angle −φ to the monitor7aand outputs the image that has been corrected based on the angle +θ to the monitor7b. Each of the monitors7aand7bdisplays the corrected image (Step S106).

Steps S100to S106are repeatedly executed. The processor64calculates an angle φ and an angle θ at the same rate as the frame rate of the image sensor22. The frequency at which the processor64calculates the angle φ and the angle θ may be different from that at which the image sensor22generates an image. For example, the processor64may calculate the angle φ and the angle θ at a lower rate than the frame rate. The rate at which the angle φ and the angle θ are calculated may be integer times as large as the frame rate. The integer is two or more.

The processor64performs the rotation processing at the same rate as the frame rate. The processor64may calculate the angle φ and the angle θ at a lower rate than that of the rotation processing. For example, Step S103and Step S105may be executed in a first frame period, and Step S105may be executed in a second frame period following the first frame period without Step S103being executed. The processor64may repeat the processing in the first frame period and the processing in the second frame period. The processor64may perform the rotation processing in Step S105in the second frame period by using the angle calculated in Step S103in the first frame period.

The processor64may perform rotation processing described below. Two or more ranges related to the angle φ and the angle θ are prepared in advance. For example, a first range, a second range, a third range, and a fourth range are prepared. The first range indicates an angle of 0 degrees or more and less than 45 degrees. The second range indicates an angle of 45 degrees or more and less than 90 degrees. The third range indicates an angle of 90 degrees or more and less than 135 degrees. The fourth range indicates an angle of 135 degrees or more and less than 180 degrees.

Hereinafter, processing related to the angle φ will be described. Processing related to the angle θ is similar to the following processing. The processor64determines which range the angle φ is included in. When the angle φ is included in the first range, the processor64does not perform the rotation processing on the image acquired by the imaging unit21. When the angle φ is included in the second range, the processor64rotates the image acquired by the imaging unit21by 45 degrees. When the angle φ is included in the third range, the processor64rotates the image acquired by the imaging unit21by 90 degrees. When the angle φ is included in the fourth range, the processor64rotates the image acquired by the imaging unit21by 135 degrees.

The processor64may display information indicating the angle φ or the angle θ on the monitor7aor the monitor7b.

FIG.10shows another example of an image displayed on the monitor7a. The monitor7adisplays an image IMG13. Differences between the image IMG13and the image IMG11shown inFIG.5will be described. The processor64superimposes information indicating the angle φ on the image IMG13. In the example shown inFIG.10, an icon IC10similar to a shape of a human is displayed on the image IMG13. The icon IC10is tilted in accordance with the angle φ. The direction of the head indicated by the icon IC10is the same as the direction Dh in the image IMG13. The operator U1can recognize a rotation direction and a rotation amount of the image IMG13displayed on the monitor7a.

FIG.11shows another example of an image displayed on the monitor7b. The monitor7bdisplays an image IMG14. Differences between the image IMG14and the image IMG12shown inFIG.6will be described. The processor64superimposes information indicating the angle θ on the image IMG14. In the example shown inFIG.11, an icon IC11similar to a shape of a human is displayed on the image IMG14. The icon IC11is tilted in accordance with the angle θ. The direction of the head indicated by the icon IC11is the same as the direction Dh in the image IMG13. The assistant U2can recognize a rotation direction and a rotation amount of the image IMG14displayed on the monitor7b.

The processor64may display a value of the angle φ or the angle θ instead of the icon IC10or the icon IC11on the monitor7aor the monitor7b. For example, when the angle φ is 45 degrees, the processor64may display characters “45” on the monitor7a. The processor64may display a value including a positive or negative sign on the monitor7aor the monitor7bdepending on whether the angle φ or the angle θ is a clockwise or counterclockwise angle. The sign indicates a rotation direction of an image.

In the first embodiment, the endoscope system1performs the rotation processing of an image in accordance with an angle between the imaging direction of the imaging unit21and the reference direction of each monitor and displays the corrected image on each monitor. Due to this, the operator U1or the assistant U2can intuitively manipulate each treatment tool.

Second Embodiment

A second embodiment of the present disclosure will be described. When the depression angle of the imaging direction is within a predetermined range, for example, the depression angle is 30 degrees or more and 60 degrees or less, an image acquired by the imaging unit21contains distortion depending on the distance between the distal end of the imaging unit21and a subject.

FIG.12shows an example of an image acquired by the imaging unit21. An image IMG20is shown inFIG.12. A subject seen in an upper region in the image IMG20is relatively far from the distal end of the imaging unit21. Therefore, an image of the region is relatively reduced. On the other hand, a subject seen in a lower region in the image IMG20is relatively close to the distal end of the imaging unit21. Therefore, an image of the region is relatively magnified.

Afigure F10shown inFIG.12schematically indicates the magnification in the image IMG20. The horizontal width of thefigure F10indicates the size of the magnification. Thefigure F10indicates that the magnification of the upper region in the image IMG20is small and the magnification of the lower region in the image IMG20is large. The difference of the magnification between regions in the image IMG20causes distortion of the image IMG20.

FIG.13shows an example of an image on which the rotation processing has been performed. An image IMG21is shown inFIG.13. The image IMG21is obtained by rotating the image IMG20shown inFIG.12by 90 degrees. Afigure F11shown inFIG.13schematically indicates the magnification in the image IMG21. Thefigure F11indicates that the magnification of a left region in the image IMG21is small and the magnification of a right region in the image IMG21is large. Since the magnification of the left region in the image IMG21and the magnification of the right region in the image IMG21are different from each other, an observer feels unnatural.

The processor64executes the following processing in order to reduce distortion of an image. The processor64calculates a depression angle of the imaging direction of the imaging unit21as in the first embodiment. The processor64performs perspective correction on an image in accordance with the depression angle. By doing this, the processor64corrects distortion of the image, which is generated in accordance with the distance between the imaging unit21and a subject.

An operation of the processor unit6will be described by usingFIG.14.FIG.14shows a procedure of the operation of the processor unit6. Descriptions of the same processing as that shown inFIG.9will be omitted.

When the processor64determines that the depression angle of the imaging direction is within the range of 30 to 90 degrees in Step S104, the processor64performs the perspective correction in accordance with the depression angle of the imaging direction by using the image received in Step S100(Step S110).

For example, the processor64enlarges the upper region in the image IMG20shown inFIG.12and reduces the lower region in the image IMG20. When the depression angle is large, in other words, the depression angle is near 90 degrees, the difference between the magnification of an upper region in an image and the magnification of a lower region in the image is small. Therefore, the difference between a correction amount of the upper region in the image IMG20and a correction amount of the lower region in the image IMG20is small. On the other hand, when the depression angle is small, in other words, the depression angle is near 30 degrees, the difference between the magnification of an upper region in an image and the magnification of a lower region in the image is large. Therefore, the difference between a correction amount of the upper region in the image IMG20and a correction amount of the lower region in the image IMG20is large.

FIG.15shows an example of an image on which the perspective correction has been performed. An image IMG22is shown inFIG.15. The image IMG22is obtained by performing the perspective correction on the image IMG20shown inFIG.12. Afigure F12shown inFIG.15schematically indicates the magnification in the image IMG22. The horizontal width of thefigure F12is uniform. In other words, the magnification is fixed in the image IMG22.

After Step S110, the processor64performs the rotation processing on the image in Step S105.FIG.16shows an example of an image on which the rotation processing has been performed after the perspective correction is performed. An image IMG23is shown inFIG.16. The image IMG23is obtained by rotating the image IMG22shown inFIG.15by 90 degrees. Afigure F13shown inFIG.16schematically indicates the magnification in the image IMG23. The vertical width of thefigure F13is uniform. In other words, the magnification is fixed in the image IMG23. Since the magnification of a left region in the image IMG23and the magnification of a right region in the image IMG23are the same, an observer does not feel unnatural.

The order in which Step S110and Step S105are executed is not limited to that shown inFIG.14. The processor64may perform the rotation processing on an image acquired by the imaging unit21and then may perform the perspective correction on the image.

In the second embodiment, the endoscope system1corrects distortion of an image, which is generated in accordance with the distance between the imaging unit21and a subject. The operator U1and the assistant U2can observe more natural image than that in the first embodiment.

Third Embodiment

A third embodiment of the present disclosure will be described. In the third embodiment, a head-mounted display (HMD) is used instead of the monitor7aand the monitor7b.

The endoscope system1shown inFIG.2is changed to an endoscope system1ashown inFIG.17.FIG.17schematically shows a layout of the endoscope system1ain an operation room. Each configuration overlooked in a direction vertical to the ground is shown inFIG.17. Descriptions of the same configuration as that shown inFIG.2will be omitted.

The endoscope system1aincludes an HMD12aand an HMD12binstead of the monitor7aand the monitor7bshown inFIG.2. The HMD12ais mounted on the head of the operator U1, and the HMD12bis mounted on the head of the assistant U2.

FIG.18shows a configuration of the endoscope system1a. Descriptions of the same configuration as that shown inFIG.7will be omitted.

For example, the azimuth sensor10is disposed on the surface of the HMD12a, and the azimuth sensor11is disposed on the surface of the HMD12b. The azimuth sensor10may be disposed inside the HMD12a. The azimuth sensor11may be disposed inside the HMD12b.

The azimuth sensor10measures a reference direction of the HMD12aand generates HMD direction information indicating the measured reference direction. For example, the reference direction of the HMD12ais a direction perpendicular to the screen of the HMD12a. The azimuth sensor11measures a reference direction of the HMD12band generates HMD direction information indicating the measured reference direction. For example, the reference direction of the HMD12bis a direction perpendicular to the screen of the HMD12b. The HMD direction information is used instead of the monitor direction information.

The azimuth sensor10may be mounted on the body of the operator U1. For example, the azimuth sensor10is mounted on the head of the operator U1. The azimuth sensor10may be mounted on the trunk of the operator U1. The azimuth sensor10measures a reference direction of the operator U1and generates user direction information indicating the measured reference direction. For example, the reference direction of the operator U1is the same as the visual line direction of the operator U1. The user direction information is used instead of the monitor direction information. For example, the reference direction of the operator U1is a direction opposite to the reference direction of the monitor7ashown inFIG.7. By using this relationship, it is possible to replace the monitor direction information with the user direction information.

The azimuth sensor11may be mounted on the body of the assistant U2. For example, the azimuth sensor11is mounted on the head of the assistant U2. The azimuth sensor11may be mounted on the trunk of the assistant U2. The azimuth sensor11measures a reference direction of the assistant U2and generates user direction information indicating the measured reference direction. For example, the reference direction of the assistant U2is the same as the visual line direction of the assistant U2. The user direction information is used instead of the monitor direction information. For example, the reference direction of the assistant U2is a direction opposite to the reference direction of the monitor7bshown inFIG.7. By using this relationship, it is possible to replace the monitor direction information with the user direction information.

An operation of the processor unit6is the same as that shown inFIG.9orFIG.14.

In the third embodiment, the endoscope system1aperforms the rotation processing of an image in accordance with an angle between the imaging direction of the imaging unit21and the reference direction of the operator U1(or the assistant U2) and displays the corrected image on each monitor. Due to this, the operator U1or the assistant U2can intuitively manipulate each treatment tool.

Fourth Embodiment

A fourth embodiment of the present disclosure will be described. In the fourth embodiment, the processor64measures a horizontal component of the reference direction of the operator U1or the assistant U2by using an image in which a treatment tool held by the operator U1or the assistant U2is seen.

The endoscope system1shown inFIG.2is changed to an endoscope system1bshown inFIG.19.FIG.19schematically shows a layout of the endoscope system1bin an operation room. Each configuration overlooked in a direction vertical to the ground is shown inFIG.19. Descriptions of the same configuration as that shown inFIG.2will be omitted.

A code unique to each treatment tool is attached to the surface of each treatment tool. A code C1is attached to the treatment tool T1, a code C2is attached to the treatment tool T2, a code C3is attached to the treatment tool T3, and a code C4is attached to the treatment tool T4. The code C1, the code C2, the code C3, and the code C4are different from each other. Each code is positioned close to the distal end of each treatment tool. Each code may be a one-dimensional code or a two-dimensional code.

The imaging unit21shown inFIG.2is changed to an imaging unit21bshown inFIG.19.FIG.20shows an example of an image acquired by the imaging unit21b. An image IMG30is shown inFIG.20. The organ OR1, the treatment tools T1to T4, and the codes C1to C4are seen in the image IMG30. The processor64analyzes the image IMG30so as to detect each code seen in the image IMG30. By doing this, the processor64detects each treatment tool. The processor64determines that the operator U1is manipulating the treatment tool T1and the treatment tool T2both extending from the right side in the image IMG30toward the center part of the image IMG30.

The processor64calculates a direction Dt1of the treatment tool T1and a direction Dt2of the treatment tool T2. The processor64calculates an average of the direction Dt1and the direction Dt2so as to calculate a direction Da1. The direction Da1indicates the reference direction of the operator U1. The direction Da1is close to the horizontal component of the reference direction of the operator U1.

An imaging direction De of the imaging unit21bis shown inFIG.20. The imaging direction De is close to the horizontal component of the imaging direction of the imaging unit21b. The imaging direction De is close to the upward direction in the image IMG30. The processor64calculates the direction Da1with respect to the imaging direction De. The direction Da1corresponds to an angle between the imaging direction De and the reference direction of the operator U1.

FIG.21shows an example of an image displayed on the monitor7a. The monitor7adisplays an image IMG31. The processor64rotates the image IMG30shown inFIG.20such that the direction Da1matches the upward direction. In the example shown inFIG.21, the processor64rotates the image IMG30clockwise by an angle φ1so as to obtain the image IMG31. The imaging direction De is changed to a direction close to the right direction in the image IMG31.

The processor64executes similar processing to the above so as to perform the rotation processing of an image displayed on the monitor7b. Hereinafter, this rotation processing will be described.

FIG.22shows an example of an image acquired by the imaging unit21b. An image IMG32is shown inFIG.22. The organ OR1, the treatment tools T1to T4, and the codes C1to C4are seen in the image IMG32. The processor64analyzes the image IMG32so as to detect each code seen in the image IMG32. By doing this, the processor64detects each treatment tool. The processor64determines that the assistant U2is manipulating the treatment tool T3and the treatment tool T4both extending from the left side in the image IMG32toward the center part of the image IMG32.

The processor64calculates a direction Dt3of the treatment tool T3and a direction Dt4of the treatment tool T4. The processor64calculates an average of the direction Dt3and the direction Dt4so as to calculate a direction Da2. The direction Da2indicates the reference direction of the assistant U2. The direction Da2is close to the horizontal component of the reference direction of the assistant U2.

An imaging direction De of the imaging unit21bis shown inFIG.22. The imaging direction De is close to the horizontal component of the imaging direction of the imaging unit21b. The imaging direction De is close to the upward direction in the image IMG32. The processor64calculates the direction Da2with respect to the imaging direction De. The direction Da2corresponds to an angle between the imaging direction De and the reference direction of the assistant U2.

FIG.23shows an example of an image displayed on the monitor7b. The monitor7bdisplays an image IMG33. The processor64rotates the image IMG32shown inFIG.22such that the direction Da2matches the upward direction. In the example shown inFIG.23, the processor64rotates the image IMG32counterclockwise by an angle θ1so as to obtain the image IMG33. The imaging direction De is changed to a direction close to the left direction in the image IMG33.

FIG.24shows a configuration of the endoscope system1b. Descriptions of the same configuration as that shown inFIG.7will be omitted.

The endoscope insertion unit2shown inFIG.7is changed to an endoscope insertion unit2bshown inFIG.24. The imaging unit21shown inFIG.7is changed to an imaging unit21bshown inFIG.24. The imaging unit21bincludes the image sensor22. The imaging unit21bdoes not include the azimuth sensor23shown inFIG.7.

The processor unit6shown inFIG.7is changed to a processor unit6bshown inFIG.24. The processor unit6bincludes the image communication unit60and the processor64. The processor unit6bdoes not include the sensor communication unit61, the sensor communication unit62, and the sensor communication unit63shown inFIG.7.

The processor64functions as a first measuring instrument that measures a first direction indicating a horizontal component of the imaging direction of the imaging unit21b. In addition, the processor64functions as a second measuring instrument that measures a second direction indicating a horizontal component of the reference direction of the monitor7aor the monitor7b.

An operation of the processor unit6bwill be described by usingFIG.25.FIG.25shows a procedure of the operation of the processor unit6b. Descriptions of the same processing as that shown inFIG.9will be omitted.

After Step S100, the following Step S120is executed. Step S101and Step S102shown inFIG.9are not executed.

The processor64calculates the direction Da1shown inFIG.20and the direction Da2shown inFIG.22. By doing this, the processor64calculates an angle between the horizontal component of the imaging direction and the horizontal component of the reference direction of a user (the operator U1or the assistant U2) (Step S120). After Step S120, Step S104is executed.

If each treatment tool frequently moves, a rotation angle of an image frequently changes. Therefore, the operator U1or the assistant U2may feel troublesome. In order to avoid this, the processor64may execute the following first processing or second processing.

The first processing will be described. When Step S120is executed for the first time, the angle calculated in Step S120is stored as an angle for processing. After Step S120is executed again, the processor64calculates an absolute value of the difference between the angle calculated this time and the angle for processing.

When the absolute value is a predetermined angle or less, the processor64performs the rotation processing by using an angle stored as the angle for processing. The predetermined angle is 5 degrees, 10 degrees, or the like. When the absolute value is greater than the predetermined angle, the processor64performs the rotation processing by using the angle calculated this time. The angle calculated this time is newly stored as an angle for processing.

The second processing will be described. When Step S120is executed for the first time, the angle calculated in Step S120is stored as an angle for processing. Step S120and Step S105are repeatedly executed, but only Step S120is executed each time a predetermined time has passed. Therefore, the rate at which Step S120is executed is lower than that at which Step S105is executed. For example, the predetermined time is 1 second or more and 5 seconds or less. The processor64performs the rotation processing by using an angle stored as the angle for processing. When Step S120is executed, the angle calculated in Step S120is newly stored as an angle for processing. The angle for processing is updated each time the predetermined time has passed.

The imaging unit21bacquires an image at a predetermined frame rate. In the above described first and second processing, the processor64calculates the angle φ or the angle θ shown inFIG.2at a lower rate than the frame rate.

The processor64may detect only one treatment tool from the right or left side in an image acquired by the imaging unit21b. In such a case, the processor64rotates the image such that the treatment tool faces in almost vertical direction in the image. The distal end of the treatment tool faces upward in the image, and the proximal end of the treatment tool faces downward in the image.

The operator U1or the assistant U2may insert or pull out a treatment tool. When the treatment tool is inserted or pulled out, the number of treatment tools detected in the above-described processing changes from 1 to 2 or changes from 2 to 1. Due to this change, a rotation angle of an image may suddenly change. Therefore, the sense of direction of the operator U1or the assistant U2may not follow an actual change of direction. In order to avoid this, the processor64may execute the following processing.

When the number of treatment tools detected from an image has changed, the processor64gradually changes the rotation angle of the image. For example, before the number of treatment tools changes, the angle calculated in Step S120is a first angle. After the number of treatment tools changes, the angle calculated in Step S120is a second angle. The processor64gradually changes the rotation angle of the image from the first angle to the second angle in a predetermined time. For example, the predetermined time is 1 second or more and 5 seconds or less.

The HMD12ashown inFIG.17may be used instead of the monitor7a, and the HMD12bshown inFIG.17may be used instead of the monitor7b.

In the fourth embodiment, the endoscope system1bcalculates a direction of each treatment tool by using an image acquired by the imaging unit21b. The endoscope system1bperforms the rotation processing of the image based on the direction of each treatment tool. The endoscope system1bcan easily perform the rotation processing of the image without using an azimuth sensor.

Fifth Embodiment

A fifth embodiment of the present disclosure will be described. In the fifth embodiment, the endoscope system1shown inFIG.7is used. After the rotation processing is performed on an image, part of the image is not displayed on the monitor7.

FIG.26andFIG.27show a range of an image displayed on the monitor7in the first to fourth embodiments.FIG.26shows a range of an image displayed on the monitor7in a case in which the rotation processing of the image is not performed. An image-forming range RL, an effective range R10, and a display range R11are shown inFIG.26.

Light passing through a lens included in the imaging optical system is incident on an imaging region of the image sensor22. The image-forming range RL indicates a range in which an optical image is formed on the image sensor22by the imaging optical system. The imaging optical system forms the optical image in a circular region, the center of which is the optical axis of the imaging optical system. The effective range R10indicates a range of an image corresponding to an effective pixel region of the image sensor22. A vertical pixel number (effective vertical pixel number) of the effective range R10is Nv1. The image sensor22outputs an image of only the effective range R10out of the image-forming range RL. The display range R11indicates a range of an image displayed on the monitor7. A vertical pixel number of the display range R11is Nv1. In a case in which the rotation processing of an image is not performed, the display range R11is the same as the effective range R10. The monitor7displays the entire image output from the image sensor22.

FIG.27shows a range of an image displayed on the monitor7in a case in which the rotation processing of the image has been performed. An image-forming range RL, an effective range R10, and a display range R11aare shown inFIG.27. The image-forming range RL shown inFIG.27is the same as the image-forming range RL shown inFIG.26, and the effective range R10shown inFIG.27is the same as the effective range R10shown inFIG.26.

In a case in which the rotation processing of the image has been performed, the display range R11shown inFIG.26is changed to the display range R11a. An image of only a range R12in which the effective range R10and the display range R11aoverlap each other is displayed on the monitor7. Therefore, part of the image on which the rotation processing has been performed is not displayed on the monitor7(blackout).

In the fifth embodiment, the effective vertical pixel number of the image sensor22is changed. The effective vertical pixel number of the image sensor22is greater than a vertical pixel number of an image displayed on the monitor7.

FIG.28andFIG.29show a range of an image displayed on the monitor7in the fifth embodiment.FIG.28shows a range of an image displayed on the monitor7in a case in which the rotation processing of the image is not performed. An image-forming range RL, an effective range R10a, and a display range R11are shown inFIG.28. The image-forming range RL shown inFIG.28is the same as the image-forming range RL shown inFIG.26, and the display range R11shown inFIG.28is the same as the display range R11shown inFIG.26.

The effective range R10shown inFIG.26is changed to the effective range R10ashown inFIG.28. A vertical pixel number (effective vertical pixel number) of the effective range R10ais Nv2. The vertical pixel number Nv2is greater than the vertical pixel number Nv1shown inFIG.26. In a case in which the rotation processing of an image is not performed, the display range R11is included in the effective range R10a. The monitor7displays an image of only the display range R11out of the image output from the image sensor22. An image of a range that is included in the effective range R10aand is not included in the display range R11is not displayed on the monitor7.

FIG.29shows a range of an image displayed on the monitor7in a case in which the rotation processing of the image has been performed. An image-forming range RL, an effective range R10a, and a display range R11aare shown inFIG.29. The image-forming range RL shown inFIG.29is the same as the image-forming range RL shown inFIG.28, and the effective range R10ashown inFIG.29is the same as the effective range R10ashown inFIG.28.

In a case in which the rotation processing of the image has been performed, the display range R11shown inFIG.28is changed to the display range R11a. An image of only a range R13in which the effective range R10aand the display range R11aoverlap each other is displayed on the monitor7. Therefore, part of the image on which the rotation processing has been performed is not displayed on the monitor7.

The image sensor22does not output an image of a range that is included in the display range R11ashown inFIG.27and is not included in the effective range R10shown inFIG.27. The monitor7does not display the image of the range. Similarly, the image sensor22does not output an image of a range that is included in the display range R11ashown inFIG.29and is not included in the effective range R10ashown inFIG.29. The monitor7does not display the image of the range. The range of the image that is not displayed on the monitor7inFIG.29is smaller than that of the image that is not displayed on the monitor7inFIG.27. The range R13shown inFIG.29is larger than the range R12shown inFIG.27.

The HMD12ashown inFIG.17may be used instead of the monitor7a, and the HMD12bshown inFIG.17may be used instead of the monitor7b.

In the fifth embodiment, the endoscope system1can reduce a range of an image that is not displayed on the monitor7when the rotation processing has been performed.

Sixth Embodiment

A sixth embodiment of the present disclosure will be described. In the sixth embodiment, the endoscope system1shown inFIG.7is used.

When the imaging direction of the imaging unit21is close to the horizontal direction, the up-and-down direction in an image acquired by the imaging unit21is close to a direction (vertical direction) that is vertical to the horizontal plane. The up-and-down direction in an image on which the rotation processing has been performed deviates from the direction vertical to the horizontal plane. When the angle φ and the angle θ shown inFIG.4are close to 180 degrees in particular, the upward direction in an image is close to the actual downward direction and the downward direction in the image is close to the actual upward direction. Therefore, the up-and-down direction recognized in the image by the operator U1and the assistant U2is opposite the actual up-and-down direction.

In the sixth embodiment, when the depression angle of the imaging direction is smaller than a preset first angle and an angle between a first direction and a second direction is larger than a preset second angle, the processor64performs horizontal flip processing (mirror inversion processing). The first direction indicates the horizontal component of the imaging direction. The second direction indicates the horizontal component of the reference direction of the monitor7. The right and left of an image is flipped through the horizontal flip processing.

An operation of the processor unit6will be described by usingFIG.30.FIG.30shows a procedure of the operation of the processor unit6. Descriptions of the same processing as that shown inFIG.9will be omitted.

The processor64determines whether the depression angle of the imaging direction is within a range of 30 degrees (first angle) to 90 degrees in Step S104. When the depression angle of the imaging direction is 30 degrees or more and 90 degrees or less, the imaging unit21acquires an image of a subject seen below the horizontal direction. In this case, the processor64determines that the depression angle of the imaging direction is within the range of 30 to 90 degrees and executes Step S105. The horizontal flip processing is not performed on the image acquired by the imaging unit21.

When the depression angle of the imaging direction is less than 30 degrees, the imaging unit21acquires an image of the subject seen in a direction close to the horizontal direction. In this case, the processor64determines that the depression angle of the imaging direction is not within the range of 30 to 90 degrees and executes the following Step S130.

The processor64determines whether the angle calculated in Step S103is larger than 120 degrees (second angle) (Step S130).

When the processor64determines that the angle calculated in Step S103is 120 degrees or less, Step S106is executed. The horizontal flip processing is not performed on the image acquired by the imaging unit21.

When the processor64determines that the angle calculated in Step S103is larger than 120 degrees, the processor64performs the horizontal flip processing on the image received in Step S100so as to correct the image (Step S131). After Step S131, Step S106is executed.

The above-described 120 degrees are an example of a threshold value used for determining the size of an angle. The threshold value is not limited to 120 degrees. The threshold value may be 135 degrees or the like.

FIG.31schematically shows a layout of the endoscope system1in an operation room. Each configuration overlooked in a direction vertical to the ground is shown inFIG.31. Descriptions of the same configuration as that shown inFIG.2will be omitted.

The monitor7a, the monitor7c, the operator U1, the scopist U3, the treatment tool T1, and the treatment tool T2are not shown inFIG.31. The angle θ shown inFIG.31is larger than 120 degrees. Therefore, the processor64performs the horizontal flip processing on an image acquired by the imaging unit21.

FIG.32shows an example of an image displayed on the monitor7c. The monitor7cdisplays an image IMG40acquired by the imaging unit21. The organ OR1, the treatment tool T3, and the treatment tool T4are seen in the image IMG40.

The direction Dh and the direction Dt are shown inFIG.32. The direction Dh indicates a direction from the center of the trunk of the patient P1toward the head of the patient P1. The direction Dh is close to a right direction Dr in the image IMG40. The direction Dt indicates a direction opposite to the direction Dh. The direction Dt is close to a left direction Dl in the image IMG40.

The treatment tool T3and the treatment tool T4are seen in the upper region of the image IMG40. The treatment tool T4is seen on the right side of the treatment tool T3.

FIG.33shows an example of an image displayed on the monitor7b. The monitor7bdisplays an image IMG41. The image IMG41is obtained by performing the horizontal flip processing on the image IMG40shown inFIG.32. The direction Dh is close to a left direction Dl in the image IMG41. The direction Dt is close to a right direction Dr in the image IMG41.

The treatment tool T3and the treatment tool T4are seen in the upper region of the image IMG41. The treatment tool T4is seen on the left side of the treatment tool T3. A positional relationship between the treatment tool T3and the treatment tool T4inFIG.33is different from that between the treatment tool T3and the treatment tool T4inFIG.32. The assistant U2manipulates the treatment tool T4with the left hand and manipulates the treatment tool T3with the right hand. A positional relationship between the treatment tool T3and the treatment tool T4in the horizontal direction in the image IMG41is the same as that between the treatment tool T3and the treatment tool T4seen from the assistant U2. Therefore, the assistant U2can intuitively manipulate each treatment tool.

FIG.34shows another example of an image displayed on the monitor7c. The monitor7cdisplays an image IMG42. Differences between the image IMG42and the image IMG40shown inFIG.32will be described. The processor64superimposes, on the image IMG42, information indicating whether the horizontal flip processing has been performed on the image IMG42. In the example shown inFIG.34, an icon IC40is displayed on the image IMG42. The icon IC40indicates that the horizontal flip processing has not been performed on the image IMG42. The scopist U3can recognize that the image IMG42displayed on the monitor7chas not been horizontally flipped.

FIG.35shows another example of an image displayed on the monitor7b. The monitor7bdisplays an image IMG43. Differences between the image IMG43and the image IMG41shown inFIG.33will be described. The processor64superimposes, on the image IMG43, information indicating whether the horizontal flip processing has been performed on the image IMG43. In the example shown inFIG.35, an icon IC41is displayed on the image IMG43. The icon IC41indicates that the horizontal flip processing has been performed on the image IMG43. The assistant U2can recognize that the image IMG43displayed on the monitor7bhas been horizontally flipped.

The HMD12ashown inFIG.17may be used instead of the monitor7a, and the HMD12bshown inFIG.17may be used instead of the monitor7b.

In the sixth embodiment, when the depression angle of the imaging direction is small and an angle between the horizontal component of the imaging direction and the horizontal component of the reference direction of each monitor is large, the endoscope system1performs the horizontal flip processing. Due to this, the vertical direction in an image displayed on the monitor7aor the monitor7balmost matches the actual vertical direction, and the left-and-right direction in the image almost matches the actual left-and-right direction. Therefore, the operator U1or the assistant U2can intuitively manipulate each treatment tool.

Seventh Embodiment

A seventh embodiment of the present disclosure will be described. In the seventh embodiment, the endoscope system1shown inFIG.7is used. The endoscope system1displays a stereoscopic image (3D image). Hereinafter, an example in which the imaging unit21acquires an image of a subject seen below the horizontal direction will be described.

The imaging unit21acquires, as an image of a subject, a left image and a right image used for displaying a stereoscopic image. The image sensor22of the imaging unit21generates the left image and the right image. The processor64performs the rotation processing on the left image and the right image related to an image displayed on the monitor7b. At this time, the processor64rotates the left image and the right image by 180 degrees in order to maintain the display of the stereoscopic image.

In this state, the perspective state of an image changes. In other words, the appearance of the image changes such that a distant subject appears close and a close subject appears distant. Therefore, the processor64performs processing of replacing the left image and the right image with each other. The processor64treats the left image on which the rotation processing has been performed as a right image. In addition, the processor64treats the right image on which the rotation processing has been performed as a left image.

An operation of the processor unit6will be described by usingFIG.36.FIG.36shows a procedure of the operation of the processor unit6. Descriptions of the same processing as that shown inFIG.9will be omitted.

The processor64receives a left image and a right image output from the image sensor22in Step S100.

When the processor64determines that the depression angle of the imaging direction is within the range of 30 to 90 degrees in Step S104, the processor64determines whether the angle calculated in Step S103is larger than 90 degrees (Step S140).

When the processor64determines that the angle calculated in Step S103is 90 degrees or less, Step S106is executed. When the processor64determines that the angle calculated in Step S103is larger than 90 degrees, the processor64rotates the left image and the right image received in Step S100by 180 degrees. By doing this, the processor64corrects the left image and the right image (Step S141).

After Step S141, the processor64replaces the left image and the right image with each other (Step S142). After Step S142, Step S106is executed.

The order in which Step S141and Step S142are executed is not limited to that shown inFIG.36. The processor64may replace the left image and the right image acquired by the imaging unit21with each other and then may perform the rotation processing on the left image and the right image.

FIG.37schematically shows a layout of the endoscope system1in an operation room. Each configuration overlooked in a direction vertical to the ground is shown inFIG.37. Descriptions of the same configuration as that shown inFIG.2will be omitted.

The monitor7c, the scopist U3, the treatment tool T1, the treatment tool T2, the treatment tool T3, and the treatment tool T4are not shown inFIG.37. A region VL and a region VR are shown inFIG.37. The region VL corresponds to a field of view of the left eye of an observer. The region VR corresponds to a field of view of the right eye of the observer. The region VL is relatively on the left side of the region VR.

The angle φ shown inFIG.37is smaller than 90 degrees. Therefore, the processor64outputs a left image and a right image acquired by the imaging unit21to the monitor7awithout performing the rotation processing on the left image and the right image. The angle θ shown inFIG.37is larger than 90 degrees. Therefore, the processor64performs the rotation processing on the left image and the right image acquired by the imaging unit21and replaces the left image and the right image with each other. The processor64outputs the processed left and right images to the monitor7b.

FIG.38shows an example of an image displayed on the monitor7a. The monitor7adisplays an image IMG50acquired by the imaging unit21. Actually, the image IMG50includes a left image and a right image. The organ OR1is seen in the image IMG50. A region VL corresponding to the region VL shown inFIG.37is shown inFIG.38, and a region VR corresponding to the region VR shown inFIG.37is shown inFIG.38. Since the rotation processing is not performed, the region VL is relatively on the left side of the region VR.

FIG.39shows an example of an image displayed on the monitor7b. The monitor7bdisplays an image IMG51. Actually, the image IMG51includes a left image and a right image. The processor64performs the rotation processing on the image IMG50shown inFIG.38and replaces the left image and the right image with each other. By doing this, the processor64generates the image IMG51. A region VL corresponding to the region VL shown inFIG.37is shown inFIG.39, and a region VR corresponding to the region VR shown inFIG.37is shown inFIG.39. Since the rotation processing is performed, the direction of the organ OR1is different from that of the organ OR1shown inFIG.38. Since the rotation processing is performed, the region VL is relatively on the left side of the region VR.

FIG.40shows another example of an image displayed on the monitor7a. The monitor7adisplays an image IMG52. Differences between the image IMG52and the image IMG50shown inFIG.38will be described. The processor64superimposes, on the image IMG52, information indicating whether the rotation processing has been performed on the image IMG52. In the example shown inFIG.40, characters CH50are displayed on the image IMG52. The characters CH50indicate that the rotation processing has not been performed on the image IMG52. The operator U1can recognize that the image IMG52displayed on the monitor7ahas not rotated.

FIG.41shows another example of an image displayed on the monitor7b. The monitor7bdisplays an image IMG53. Differences between the image IMG53and the image IMG51shown inFIG.39will be described. The processor64superimposes, on the image IMG53, information indicating whether the rotation processing has been performed on the image IMG53. In the example shown inFIG.41, characters CH51are displayed on the image IMG53. The characters CH51indicate that the rotation processing has been performed on the image IMG53. The assistant U2can recognize that the image IMG53displayed on the monitor7bhas rotated.

The HMD12ashown inFIG.17may be used instead of the monitor7a, and the HMD12bshown inFIG.17may be used instead of the monitor7b.

In the seventh embodiment, the endoscope system1performs the rotation processing of an image in accordance with an angle between the imaging direction of the imaging unit21and the reference direction of the operator U1(or the assistant U2) and replaces a left image and a right image with each other. By rotating the image by 180 degrees, the endoscope system1can maintain the display of a stereoscopic image. In addition, by replacing the left image and the right image with each other, the endoscope system1can perform the perspective correction. The operator U1or the assistant U2can intuitively manipulate each treatment tool.

Eighth Embodiment

An eighth embodiment of the present disclosure will be described. In the eighth embodiment, the endoscope system1shown inFIG.7is used. The endoscope system1displays a stereoscopic image. Hereinafter, an example in which the imaging unit21acquires an image of a subject seen in a direction close to the horizontal direction will be described.

The imaging unit21acquires a left image and a right image as an image of a subject. The image sensor22of the imaging unit21generates the left image and the right image. The processor64performs the horizontal flip processing on the left image and the right image. The processor64performs processing of replacing the left image and the right image with each other. The processor64treats the left image on which the horizontal flip processing has been performed as a right image. In addition, the processor64treats the right image on which the horizontal flip processing has been performed as a left image.

An operation of the processor unit6will be described by usingFIG.42.FIG.42shows a procedure of the operation of the processor unit6. Descriptions of the same processing as that shown inFIG.9will be omitted.

The processor64receives a left image and a right image output from the image sensor22in Step S100.

When the processor64determines that the depression angle of the imaging direction is within the range of 30 to 90 degrees in Step S104, the processor64determines whether the angle calculated in Step S103is larger than 90 degrees (Step S150).

When the processor64determines that the angle calculated in Step S103is 90 degrees or less, Step S106is executed. When the processor64determines that the angle calculated in Step S103is larger than 90 degrees, the processor64performs the horizontal flip processing on the left image and the right image received in Step S100. By doing this, the processor64corrects the left image and the right image (Step S151).

After Step S151, the processor64replaces the left image and the right image with each other (Step S152). After Step S152, Step S106is executed.

The order in which Step S151and Step S152are executed is not limited to that shown inFIG.42. The processor64may replace the left image and the right image acquired by the imaging unit21with each other and then may perform the horizontal flip processing on the left image and the right image.

When the depression angle of the imaging direction is 30 degrees or more and 90 degrees or less, Steps S140to S142and Step S106shown inFIG.36may be executed.

For example, the monitor7cdisplays the image IMG50shown inFIG.38.FIG.43shows an example of an image displayed on the monitor7b. The monitor7bdisplays an image IMG54. Actually, the image IMG54includes a left image and a right image. The processor64performs the horizontal flip processing on the image IMG50shown inFIG.38and replaces the left image and the right image with each other. By doing this, the processor64generates the image IMG54. A region VL corresponding to the region VL shown inFIG.37is shown inFIG.43, and a region VR corresponding to the region VR shown inFIG.37is shown inFIG.43. Since the horizontal flip processing is performed, the organ OR1has been horizontally flipped. Since the horizontal flip processing is performed, the region VR is relatively on the left side of the region VL.

FIG.44shows another example of an image displayed on the monitor7b. The monitor7bdisplays an image IMG55. Differences between the image IMG55and the image IMG54shown inFIG.43will be described. The processor64superimposes, on the image IMG55, information indicating whether the horizontal flip processing has been performed on the image IMG55. In the example shown inFIG.44, characters CH52are displayed on the image IMG55. The characters CH52indicate that the horizontal flip processing has been performed on the image IMG55. The assistant U2can recognize that the image IMG55displayed on the monitor7bhas been horizontally flipped.

The HMD12ashown inFIG.17may be used instead of the monitor7a, and the HMD12bshown inFIG.17may be used instead of the monitor7b.

In the eighth embodiment, the endoscope system1performs the horizontal flip processing of an image in accordance with an angle between the imaging direction of the imaging unit21and the reference direction of the operator U1(or the assistant U2) and replaces a left image and a right image with each other. By performing the horizontal flip processing, the endoscope system1can maintain the display of a stereoscopic image. In addition, by replacing the left image and the right image with each other, the endoscope system1can perform the perspective correction. Due to this, the vertical direction in an image displayed on the monitor7aor the monitor7balmost matches the actual vertical direction, the left-and-right direction in the image almost matches the actual left-and-right direction, and the far-and-near direction in the image almost matches the actual far-and-near direction. Therefore, the operator U1or the assistant U2can intuitively manipulate each treatment tool.

REFERENCE SIGNS LIST