Patent Description:
In the related art, for example, in Patent Document <NUM> below, a technology is described that assumes that an endoscope image in which the up, down, left, and right directions of an endoscope image coincide with the up, down, left, and right operation directions of an operator can be displayed on a monitor so as not to cause a sense of incongruity during operation.

<CIT> discloses a surgical system that includes, for example, a patient-side cart and a console apparatus. The patient-side cart includes an endoscope apparatus that includes three or more endoscope arms. The console apparatus manipulates the patient-side cart. In the endoscope apparatus, for example, feature points of respective images obtained by respective imaging devices of the three or more endoscopes are used to join the respective images together to generate a composite image, and the composite image is displayed on a display screen.

<CIT> discloses a robotic system for controlling an oblique-viewing endoscope, wherein the system is configured to adjust rotational speed of the oblique endoscope if the quality of reconstruction of a volume image generated based on a series of images taken during rotation is not acceptable.

In recent years, an oblique-viewing endoscope has been used as a rigid scope inserted into the human body. However, the oblique-viewing rotation by the oblique-viewing endoscope is made with respect to the scope axis, and thus there is a problem that the visual field after the rotation is less intuitive compared to the simple horizontal or vertical movement. For this reason, there have been problems such as the observation target being out of the visual field of the screen due to the oblique-viewing rotation, and losing the sight of the instrument shown in the visual field.

Therefore, it has been demanded that the sight of the observation target is not lost from the visual field of the monitor image.

According to the present disclosure, it is possible to prevent the sight of an observation target from being lost from the visual field of a monitor image.

Note that the above effects are not necessarily limited, and any of the effects shown in the present specification or other effects that can be grasped from the present specification may be exerted together with or in place of the above effects.

In addition, in this specification and drawing, constituent elements having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

The description will be made in the following order.

First, a basic configuration of an endoscope system according to an embodiment of the present disclosure will be described with reference to <FIG>.

<FIG> is a diagram illustrating an example of a schematic configuration of an endoscopic surgery system <NUM> to which the technology according to the present disclosure can be applied. <FIG> illustrates an aspect in which an operator (doctor) <NUM> is performing a surgery on a patient <NUM> on a patient bed <NUM> using the endoscopic surgery system <NUM>. As illustrated, the endoscopic surgery system <NUM> includes an endoscope <NUM>, other surgical tools <NUM>, a support arm device <NUM> for supporting the endoscope <NUM>, and a cart <NUM> equipped with various devices for endoscopic surgery.

In endoscopic surgery, a plurality of cylindrical opening instruments called trocars 5025a to 5025d are punctured into the abdominal wall instead of cutting and opening the abdominal wall. Then, a lens barrel <NUM> of the endoscope <NUM> and other surgical tools <NUM> are inserted into the body cavity of the patient <NUM> from the trocars 5025a to 5025d. In the illustrated example, as other surgical tools <NUM>, an insufflation tube <NUM>, an energy treatment tool <NUM>, and forceps <NUM> are inserted into the body cavity of the patient <NUM>. The energy treatment tool <NUM> is a treatment device that performs incision and exfoliation of tissue, sealing of blood vessels, and the like by using high-frequency current and ultrasonic vibration. However, the illustrated surgical tool <NUM> is merely an example, and various surgical tools, which are generally used in endoscopic surgery, such as a set and a retractor may be used as the surgical tool <NUM>.

An image of the operation site in the body cavity of the patient <NUM> captured by the endoscope <NUM> is displayed on a display device <NUM>. The operator <NUM> performs a process to excise an affected part using the energy treatment tool <NUM> and the forceps <NUM> while viewing the image of the operation site displayed on the display device <NUM> in real time, for example. Although not illustrated, the insufflation tube <NUM>, the energy treatment tool <NUM>, and the forceps <NUM> are supported by the operator <NUM>, an assistant, or the like during surgery.

The support arm device <NUM> includes an arm unit <NUM> extending from a base unit <NUM>. In the illustrated example, the arm unit <NUM> includes joints 5033a, 5033b, 5033c, and links 5035a and 5035b, and is driven by control from the arm control device <NUM>. The endoscope <NUM> is supported by the arm unit <NUM>, and the position and posture of the endoscope are controlled. Thereby, the position of the endoscope <NUM> can be fixed stably.

The endoscope <NUM> includes the lens barrel <NUM> of which the area having a predetermined length from the tip is inserted into the body cavity of the patient <NUM> and a camera head <NUM> connected to the proximal end of the lens barrel <NUM>. In the illustrated example, the endoscope <NUM> configured as a so-called rigid scope having the rigid lens barrel <NUM> is illustrated, but the endoscope <NUM> may be configured as a so-called flexible endoscope having a soft lens barrel <NUM>.

An opening to which an objective lens is fitted is provided at the tip of the lens barrel <NUM>. A light source device <NUM> is connected to the endoscope <NUM>, the light generated by the light source device <NUM> is guided to the tip of the lens barrel by a light guide extending inside the lens barrel <NUM>, and the light is emitted toward the observation target in the body cavity of the patient <NUM> through the objective lens. Note that the endoscope <NUM> is an oblique-viewing endoscope.

An optical system and an imaging element are provided inside the camera head <NUM>, and the reflected light (observation light) from the observation target is condensed on an imaging element by the optical system. The observation light is photoelectrically converted by the imaging element, and an electric signal corresponding to the observation light, that is, an image signal corresponding to the observation image is generated. The image signal is transmitted to a camera control unit (CCU) <NUM> as RAW data. Incidentally, the camera head <NUM> has a function of adjusting the magnification and the focal length by appropriately driving the optical system.

Note that the camera head <NUM> may be provided with a plurality of imaging elements in order to support, for example, stereoscopic viewing (3D display). In this case, a plurality of relay optical systems are provided inside the lens barrel <NUM> to guide observation light to each of the plurality of imaging elements.

The CCU <NUM> is configured by a central processing unit (CPU), a graphics processing unit (GPU), and the like, and controls the operations of the endoscope <NUM> and the display device <NUM> in an integrated manner. Specifically, the CCU <NUM> performs various types of image processing for displaying an image based on the image signal, such as a development process (demosaic process), on the image signal received from the camera head <NUM>. The CCU <NUM> provides the image signal subjected to the image process to the display device <NUM>. Also, the CCU <NUM> transmits a control signal to a camera head <NUM>, and controls the driving of the camera head. The control signal may include information on imaging conditions such as a magnification and a focal length.

The display device <NUM> displays an image based on the image signal and subjected to the image process by the CCU <NUM> under the control of the CCU <NUM>. For example, when the endoscope <NUM> supports shooting at a high resolution such as <NUM> (<NUM> horizontal pixels×<NUM> vertical pixels) or <NUM> (<NUM> horizontal pixels×<NUM> vertical pixels), and/or if the endoscope supports 3D display, a device capable of high-resolution display and/or a device capable of 3D display may be used as the display device <NUM> in response thereto. In the case of shooting at a high resolution such as <NUM> or <NUM>, the use of a display device <NUM> having a size of <NUM> inches or more can provide a more immersive feeling. Further, a plurality of display devices <NUM> having different resolutions and sizes may be provided depending on the application.

The light source device <NUM> includes a light source such as a light emitting diode (LED) and supplies the endoscope <NUM> with irradiation light when imaging the operation site.

The arm control device <NUM> is configured by a processor such as a CPU, and operates according to a predetermined program to control the driving of the arm unit <NUM> of the support arm device <NUM> according to a predetermined control method.

The input device <NUM> is an input interface for the endoscopic surgery system <NUM>. The user can input various information and input instructions to the endoscopic surgery system <NUM> via the input device <NUM>. For example, the user inputs, via the input device <NUM>, various types of information related to surgery, such as physical information of a patient and information about a surgical procedure. Further, for example, the user may input, via the input device <NUM>, an instruction to drive the arm unit <NUM>, an instruction to change imaging conditions (type of irradiation light, magnification, focal length, and the like) by the endoscope <NUM>, an instruction to drive the energy treatment tool <NUM>, or the like.

The type of the input device <NUM> is not limited, and the input device <NUM> may be various known input devices. As the input device <NUM>, for example, a mouse, a keyboard, a touch panel, a switch, a foot switch <NUM>, and/or a lever can be applied. When a touch panel is used as the input device <NUM>, the touch panel may be provided on a display surface of the display device <NUM>.

Alternatively, the input device <NUM> is a device worn by a user such as a glasses-type wearable device or a head mounted display (HMD), and various inputs are performed in accordance with a user's gesture or gaze detected by these devices. Further, the input device <NUM> includes a camera capable of detecting the movement of the user, and performs various inputs in accordance with the user's gesture or gaze detected from the video imaged by the camera. Further, the input device <NUM> includes a microphone capable of collecting a user's voice, and various inputs are performed by voice via the microphone. As described above, when the input device <NUM> is configured to be capable of inputting various kinds of information in a non-contact manner, in particular, a user (for example, the operator <NUM>) belonging to a clean area can operate a device belonging to a dirty area in a non-contact manner. In addition, since the user can operate the device without releasing the user's hand from the surgical tool that the user possesses, the convenience for the user is improved.

A treatment tool control device <NUM> controls the driving of the energy treatment tool <NUM> for cauterizing and incising a tissue, sealing a blood vessel, and the like. An insufflation device <NUM> feeds a gas into the body cavity of the patient <NUM> through the insufflation tube <NUM> to inflate the body cavity for the purpose of securing the visual field by the endoscope <NUM> and securing the working space of the operator. A recorder <NUM> is a device capable of recording various types of information related to surgery. A printer <NUM> is a device capable of printing various types of information on surgery in various formats such as text, images, and graphs.

Hereinafter, a particularly characteristic configuration of the endoscopic surgery system <NUM> will be described in more detail.

The support arm device <NUM> includes a base unit <NUM> as a base and an arm unit <NUM> extending from the base unit <NUM>. In the illustrated example, the arm unit <NUM> includes a plurality of joints 5033a, 5033b, and 5033c, and a plurality of links 5035a and 5035b connected by the joint 5033b. However, in <FIG>, the configuration of the arm unit <NUM> is illustrated in a simplified manner for simplification. Actually, the shapes, numbers and arrangements of the joints 5033a to 5033c and the links 5035a and 5035b, the directions of the rotation axes of the joints 5033a to 5033c, and the like can be appropriately set so that the arm unit <NUM> has a desired degree of freedom. For example, the arm unit <NUM> can be preferably configured to have six or more degrees of freedom. Accordingly, since the endoscope <NUM> can be freely moved within the movable range of the arm unit <NUM>, the lens barrel <NUM> of the endoscope <NUM> can be inserted into the body cavity of the patient <NUM> from a desired direction.

The joints 5033a to 5033c are provided with actuators, and the joints 5033a to 5033c are configured to be rotatable around a predetermined rotation axis by driving the actuators. The driving of the actuator is controlled by the arm control device <NUM>, whereby the rotation angles of the joints 5033a to 5033c are controlled, and the driving of the arm unit <NUM> is controlled. Thereby, the control of the position and posture of the endoscope <NUM> can be realized. At this time, the arm control device <NUM> can control the driving of the arm unit <NUM> by various known control methods such as force control or position control.

For example, when the operator <NUM> performs an appropriate operation input via the input device <NUM> (including the foot switch <NUM>), the driving of the arm unit <NUM> may be appropriately controlled by the arm control device <NUM> in accordance with the operation input, so as to control the position and posture of the endoscope <NUM>. With this control, after the endoscope <NUM> at the tip of the arm unit <NUM> is moved from an arbitrary position to an arbitrary position, the endoscope can be fixedly supported at the position after the movement. Note that the arm unit <NUM> may be operated by a so-called master slave method. In this case, the arm unit <NUM> can be remotely controlled by the user via the input device <NUM> provided at a place away from the operating room.

When the force control is applied, the arm control device <NUM> may perform so-called power assist control of driving the actuators of the joints 5033a to 5033c so that an external force is received from the user, and the arm unit <NUM> moves smoothly according to the external force. Accordingly, when the user moves the arm unit <NUM> while directly touching the arm unit <NUM>, the arm unit <NUM> can be moved with a relatively light force. Therefore, the endoscope <NUM> can be moved more intuitively and with a simpler operation, and user convenience can be improved.

Here, generally, in the endoscopic surgery, the endoscope <NUM> is supported by a doctor called a scopist. On the other hand, by using the support arm device <NUM>, the position of the endoscope <NUM> can be more reliably fixed without manual operation. Thus, the image of the operation site can be stably obtained, and the surgery can be performed smoothly.

Note that the arm control device <NUM> is not necessarily provided in the cart <NUM>. Further, the arm control device <NUM> is not necessarily one device. For example, the arm control device <NUM> may be provided in each of the joints 5033a to 5033c of the arm unit <NUM> of the support arm device <NUM>, and the drive control of the arm unit <NUM> may be realized when the plurality of arm control devices <NUM> cooperate with each other.

The light source device <NUM> supplies the endoscope <NUM> with irradiation light at the time of imaging the operation site. The light source device <NUM> includes, for example, a white light source including an LED and a laser light source, or a combination thereof. At this time, when the white light source is configured by a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision. Thus, the white balance of the captured image can be adjusted in the light source device <NUM>. In this case, the laser light from each of the RGB laser light sources is emitted to the observation target in a time-division manner, and the driving of the imaging element of the camera head <NUM> is controlled in synchronization with the irradiation timing, whereby the image corresponding to each of the RGB can be captured in a time-division manner. According to this method, a color image can be obtained without providing a color filter in the imaging element.

Further, the driving of the light source device <NUM> may be controlled so as to change the intensity of light to be output at predetermined time intervals. When the driving of the imaging element of the camera head <NUM> is controlled in synchronization with the timing of the intensity change of the light to obtain an image in a time-division manner, and the image is synthesized, it is possible to generate a so-called high dynamic range image without black spotting and white spotting.

Further, the light source device <NUM> may be configured to be able to supply light in a predetermined wavelength band corresponding to special light observation. In the special light observation, for example, by utilizing the wavelength dependence of light absorption in body tissue, light is emitted in a narrow band compared to irradiation light (that is, white light) during normal observation so as to perform a so-called narrow band imaging of imaging a predetermined tissue such as a blood vessel of a mucosal surface layer with high contrast. Alternatively, in the special light observation, fluorescence observation may be performed in which an image is obtained by fluorescence generated by emitting excitation light. In the fluorescence observation, a body tissue can be irradiated with excitation light to observe fluorescence from the body tissue (autofluorescence observation), or a reagent such as indocyanine green (ICG) is locally injected into the body tissue, and the excitation light corresponding to the fluorescence wavelength of the reagent is emitted to the body tissue, so as to obtain a fluorescent image. The light source device <NUM> can be configured to be able to supply narrowband light and/or excitation light corresponding to such special light observation.

With reference to <FIG>, the functions of the camera head <NUM> of the endoscope <NUM> and the CCU <NUM> will be described in more detail. <FIG> is a block diagram illustrating an example of a functional configuration of the camera head <NUM> and the CCU <NUM> illustrated in <FIG>.

Referring to <FIG>, the camera head <NUM> includes, as the functions thereof, a lens unit <NUM>, an imaging unit <NUM>, a drive unit <NUM>, a communication unit <NUM>, and a camera head control unit <NUM>. Also, the CCU <NUM> includes, as the functions thereof, a communication unit <NUM>, an image processing unit <NUM>, and a control unit <NUM>. The camera head <NUM> and the CCU <NUM> are communicably connected by a transmission cable <NUM>.

First, the functional configuration of the camera head <NUM> will be described. The lens unit <NUM> is an optical system provided at a connection with the lens barrel <NUM>. The observation light taken in from the tip of the lens barrel <NUM> is guided to the camera head <NUM> and enters the lens unit <NUM>. The lens unit <NUM> is configured by combining a plurality of lenses including a zoom lens and a focus lens. The optical characteristics of the lens unit <NUM> are adjusted so that the observation light is condensed on the light receiving surface of the imaging element of the imaging unit <NUM>. Further, the zoom lens and the focus lens are configured such that the positions thereof on the optical axis are movable for adjusting the magnification and the focus of the captured image.

The imaging unit <NUM> is configured of an imaging element, and is arranged at a stage subsequent to the lens unit <NUM>. The observation light that has passed through the lens unit <NUM> is condensed on the light receiving surface of the imaging element, and an image signal corresponding to the observation image is generated by photoelectric conversion. The image signal generated by the imaging unit <NUM> is provided to the communication unit <NUM>.

As the imaging element that forms the imaging unit <NUM>, for example, a color photographable image sensor having a Bayer array is used which is complementary metal oxide semiconductor (CMOS) type. Incidentally, as the imaging element, an imaging element capable of capturing an image having a high-resolution of, for example, <NUM> or more may be used. When the image of the operation site is obtained with high resolution, the operator <NUM> can grasp the state of the operation site in more detail, and can progress the surgery more smoothly.

In addition, the imaging element included in the imaging unit <NUM> is configured to include a pair of imaging elements for obtaining right-eye and left-eye image signals corresponding to 3D display. By performing the 3D display, the operator <NUM> can more accurately grasp the depth of the living tissue at the operation site. When the imaging unit <NUM> is configured as a multi-plate type, a plurality of lens units <NUM> are also provided corresponding to respective imaging elements.

Further, the imaging unit <NUM> is not necessarily provided in the camera head <NUM>. For example, the imaging unit <NUM> may be provided inside the lens barrel <NUM> immediately after the objective lens.

The drive unit <NUM> is configured by an actuator, and the zoom lens and the focus lens of the lens unit <NUM> are moved by a predetermined distance along the optical axis under the control of the camera head control unit <NUM>. Thereby, the magnification and the focus of the image captured by the imaging unit <NUM> can be appropriately adjusted.

The communication unit <NUM> is configured by a communication device for transmitting and receiving various information to and from the CCU <NUM>. The communication unit <NUM> transmits the image signal obtained from the imaging unit <NUM> as RAW data to the CCU <NUM> via the transmission cable <NUM>. At this time, it is preferable that the image signal be transmitted by optical communication in order to display the captured image of the operation site with low latency. During surgery, the operator <NUM> performs the surgery while observing the state of the affected part with the captured image. Thus, for safer and more reliable surgery, the moving images of the operation site are required to be displayed in real time as much as possible. When optical communication is performed, the communication unit <NUM> is provided with a photoelectric conversion module that converts an electric signal into an optical signal. The image signal is converted into an optical signal by the photoelectric conversion module, and then transmitted to the CCU <NUM> via the transmission cable <NUM>.

Further, communication unit <NUM> receives a control signal for controlling the driving of the camera head <NUM> from the CCU <NUM>. The control signal includes information about imaging conditions such as information that specifies the frame rate of the captured image, information that specifies the exposure value at the time of imaging, and/or information that specifies the magnification and focus of the captured image. The communication unit <NUM> provides the received control signal to the camera head control unit <NUM>. Note that the control signal from the CCU <NUM> may also be transmitted by optical communication. In this case, the communication unit <NUM> is provided with a photoelectric conversion module that converts an optical signal into an electric signal. The control signal is converted into an electric signal by the photoelectric conversion module and then is provided to the camera head control unit <NUM>.

Note that the above-described imaging conditions such as the frame rate, the exposure value, the magnification, and the focus are automatically set by the control unit <NUM> of the CCU <NUM> based on the obtained image signal. That is, a so-called auto exposure (AE) function, an auto focus (AF) function, and an auto white balance (AWB) function are mounted on the endoscope <NUM>.

The camera head control unit <NUM> controls the driving of the camera head <NUM> based on the control signal from the CCU <NUM> received via the communication unit <NUM>. For example, the camera head control unit <NUM> controls the driving of the imaging element of the imaging unit <NUM> based on the information for specifying the frame rate of the captured image and/or the information for specifying the exposure at the time of imaging. In addition, for example, the camera head control unit <NUM> appropriately moves the zoom lens and the focus lens of the lens unit <NUM> via the drive unit <NUM> based on information for specifying the magnification and the focus of the captured image. The camera head control unit <NUM> may further include a function of storing information for identifying the lens barrel <NUM> and the camera head <NUM>.

By arranging the lens unit <NUM>, the imaging unit <NUM>, and the like in a hermetically sealed structure having high airtightness and waterproofness, the camera head <NUM> can have resistance to autoclave sterilization.

Next, the functional configuration of the CCU <NUM> will be described. The communication unit <NUM> is configured by a communication device for transmitting and receiving various information to and from the camera head <NUM>. The communication unit <NUM> receives an image signal transmitted from the camera head <NUM> via the transmission cable <NUM>. At this time, as described above, the image signal can be suitably transmitted by optical communication. In this case, in response to optical communication, the communication unit <NUM> is provided with a photoelectric conversion module that converts an optical signal into an electric signal. The communication unit <NUM> provides the image signal converted to the electric signal to the image processing unit <NUM>.

The communication unit <NUM> transmits a control signal for controlling the driving of the camera head <NUM> to the camera head <NUM>. The control signal may also be transmitted by optical communication.

The image processing unit <NUM> performs various types of image processes on an image signal that is RAW data transmitted from the camera head <NUM>. For example, the image process includes various known signal processes such as a development process, a high image quality process (such as a band enhancement process, a super-resolution process, a noise reduction (NR) process, and/or a camera shake correction process), and/or an enlargement process (electronic zoom process). Further, the image processing unit <NUM> performs a detection process on the image signal for performing AE, AF, and AWB.

The image processing unit <NUM> is configured by a processor such as a CPU and a GPU, and the processor operates according to a predetermined program so as to perform the above-described image process and detection process. When the image processing unit <NUM> is configured by a plurality of GPUs, the image processing unit <NUM> divides information on an image signal as appropriate, and performs the image process in parallel by the plurality of GPUs.

The control unit <NUM> performs various controls related to the imaging of the operation site by the endoscope <NUM> and the display of the captured image. For example, the control unit <NUM> generates a control signal for controlling the driving of the camera head <NUM>. At this time, when the imaging condition is input by the user, the control unit <NUM> generates a control signal based on the input by the user. Alternatively, when the endoscope <NUM> has an AE function, an AF function, and an AWB function, the control unit <NUM> appropriately calculates the optimal exposure value, focal length, and white balance according to the result of the detection process by the image processing unit <NUM>, and generates a control signal.

Further, the control unit <NUM> causes the display device <NUM> to display an image of the operation site based on the image signal on which the image process is performed by the image processing unit <NUM>. At this time, the control unit <NUM> recognizes various objects in the operative image using various image recognition techniques. For example, the control unit <NUM> detects a shape, a color, and the like of an edge of an object included in the operative image, thereby recognizing a surgical tool such as forceps, a specific living body part, bleeding, a mist at the time of using the energy treatment tool <NUM>, and the like. When the image of the operation site is displayed on the display device <NUM>, the control unit <NUM> superimposes and displays various kinds of operation support information on the image of the operation site using the recognition result. When the operation support information is superimposed and displayed to be presented to the operator <NUM>, the surgery can be performed more safely and reliably.

The transmission cable <NUM> connecting the camera head <NUM> and the CCU <NUM> is an electric signal cable corresponding to electric signal communication, an optical fiber corresponding to optical communication, or a composite cable thereof.

Here, in the illustrated example, the communication is performed by wire using the transmission cable <NUM>, but the communication between the camera head <NUM> and the CCU <NUM> may be performed wirelessly. When the communication between the camera head and the CCU <NUM> is performed wirelessly, it is not necessary to lay the transmission cable <NUM> in the operating room. Thus, a situation can be solved in which the movement of the medical staff in the operating room is hindered by the transmission cable <NUM>.

As above, an example of the endoscopic surgery system <NUM> to which the technology according to the present disclosure can be applied has been described. Here, the endoscopic surgery system <NUM> has been described as an example, but a system to which the technology according to the present disclosure can be applied is not limited to such an example. For example, the technology according to the present disclosure may be applied to an inspection flexible endoscope system or a microscopic surgery system.

Next, a specific configuration example of the medical support arm device according to the embodiment of the present disclosure will be described in detail. The support arm device described below is an example configured as a support arm device that supports an endoscope at the tip of an arm unit, but this embodiment is not limited to this example.

First, a schematic configuration of a support arm device <NUM> according to this embodiment will be described with reference to <FIG> is a schematic view illustrating an appearance of the support arm device <NUM> according to this embodiment.

The support arm device <NUM> according to this embodiment includes a base unit <NUM> and an arm unit <NUM>. The base unit <NUM> is a base of the support arm device <NUM>, and the arm unit <NUM> extends from the base unit <NUM>. Although not illustrated in <FIG>, a control unit that integrally controls the support arm device <NUM> may be provided in the base unit <NUM>, and the driving of the arm unit <NUM> may be controlled by the control unit. The control unit includes various signal processing circuits such as a CPU and a DSP.

The arm unit <NUM> includes a plurality of active joints 1421a to 1421f, a plurality of links 1422a to 1422f, and an endoscope device <NUM> as a tip unit provided at the tip of the arm unit <NUM>.

The links 1422a to 1422f are substantially rodshaped members. One end of the link 1422a is connected to the base unit <NUM> via the active joint 1421a, the other end of the link 1422a is connected to one end of the link 1422b via the active joint 1421b, and the other end of the link 1422b is connected to one end of the link 1422c via the active joint 1421c. The other end of the link 1422c is connected to the link 1422d via a passive slide mechanism <NUM>, and the other end of the link 1422d is connected to one end of the link 1422e via a passive joint <NUM>. The other end of the link 1422e is connected to one end of the link 1422f via the active joints 1421d and 1421e. The endoscope device <NUM> is connected to the tip of the arm unit <NUM>, that is, the other end of the link 1422f via the active joint 1421f. As described above, the ends of the plurality of links 1422a to 1422f are connected to each other by the active joints 1421a to 1421f, the passive slide mechanism <NUM>, and the passive joint <NUM> with the base unit <NUM> as a fulcrum, thereby forming an arm shape extending from the base unit <NUM>.

The position and posture of the endoscope device <NUM> are controlled by driving and controlling the actuators provided in the active joints 1421a to 1421f of the arm unit <NUM>. In this embodiment, the tip of the endoscope device <NUM> enters the body cavity of the patient, which is the operation site, and the endoscope device captures a partial area of the operation site. However, the tip unit provided at the tip of the arm unit <NUM> is not limited to the endoscope device <NUM>, and various medical instruments may be connected to the tip of the arm unit <NUM> as a tip unit. As described above, the support arm device <NUM> according to this embodiment is configured as a medical support arm device including a medical instrument.

Here, the support arm device <NUM> will be described below by defining coordinate axes as illustrated in <FIG>. Also, a vertical direction, a front-back direction, and a right-left direction are defined in accordance with the coordinate axes. That is, the vertical direction with respect to the base unit <NUM> installed on the floor is defined as a z-axis direction and the vertical direction. The direction which is orthogonal to the z-axis and the direction and in which the arm unit <NUM> extends from the base unit <NUM> (that is, the direction in which the endoscope device <NUM> is located with respect to the base unit <NUM>) is defined as a y-axis direction and the front-back direction. Further, directions orthogonal to the y-axis and the z-axis are defined as an x-axis direction and the right-left direction.

The active joints 1421a to 1421f connect the links with each other so as to be rotatable. Each of the active joints 1421a to 1421f has an actuator, and has a rotation mechanism that is driven to rotate about a predetermined rotation axis by driving the actuator. By controlling the rotational driving of each of the active joints 1421a to 1421f, it is possible to control the driving of the arm unit <NUM>, for example, extending or contracting (folding) the arm unit <NUM>. Here, the driving of the active joints 1421a to 1421f can be controlled by, for example, known whole-body cooperative control and ideal joint control. As described above, since the active joints 1421a to 1421f have a rotation mechanism, in the following description, the drive control of the active joints 1421a to 1421f specifically means the control of the rotation angles and/or the generated torque (the torque generated by the active joints 1421a to 1421f) of the active joints 1421a to 1421f.

The passive slide mechanism <NUM> is one aspect of a passive form changing mechanism, and connects the link 1422c and the link 1422d to each other along a predetermined direction so as to be able to advance and retreat. For example, the passive slide mechanism <NUM> may connect the link 1422c and the link 1422d to each other so as to be able to move directly. However, the reciprocating motion between the link 1422c and the link 1422d is not limited to a linear motion, and may be a reciprocating motion in an arc-shaped direction. For example, the passive slide mechanism <NUM> is operated by a user to advance and retreat, so as to change the distance between the active joint 1421c at one end of the link 1422c and the passive joint <NUM>. Accordingly, the overall form of the arm unit <NUM> may change. Details of the configuration of the passive slide mechanism <NUM> will be described later.

The passive joint <NUM> is one aspect of the passive form changing mechanism, and connects the link 1422d and the link 1422e to each other to be rotatable. For example, the passive joint <NUM> is rotated by a user, so as to change the angle formed between the link 1422d and the link 1422e. Accordingly, the overall form of the arm unit <NUM> may change. Details of the configuration of the passive joint <NUM> will be described later.

In this specification, the "posture of the arm unit" means the state of the arm unit which can be changed by the drive control of the actuators provided in the active joints 1421a to 1421f by the control unit in a state where the distance between the active joints adjacent to each other across one or more links is constant. The "form of the arm" means the state of the arm unit that can be changed by changing the distance between the active joints adjacent to each other across the link or the angle between the links connecting the adjacent active joints according to the operation of the passive form changing mechanism.

The support arm device <NUM> according to this embodiment includes six active joints 1421a to 1421f, and has six degrees of freedom in driving the arm unit <NUM>. That is, the drive control of the support arm device <NUM> is realized by the drive control of the six active joints 1421a to 1421f by the control unit, while the passive slide mechanism <NUM> and the passive joint <NUM> is not subject to the drive control by the control unit.

Specifically, as illustrated in <FIG>, the active joints 1421a, 1421d, and 1421f are provided such that the long axis direction of each of the connected links 1422a and 1422e and the imaging direction of the connected endoscope device <NUM> are set as rotation axis directions. The active joints 1421b, 1421c, and 1421e are provided such that the x-axis direction in which the connection angle of each of the connected links 1422a to 1422c, 1422e, and 1422f and the endoscope device <NUM> is changed in a y-z plane (a plane defined by the y axis and the z axis) is set as the rotation axis direction. As described above, in this embodiment, the active joints 1421a, 1421d, and 1421f have a function of performing so-called yawing, and the active joints 1421b, 1421c, and 1421e have a function of performing so-called pitching.

With such a configuration of the arm unit <NUM>, the support arm device <NUM> according to this embodiment has six degrees of freedom in driving the arm unit <NUM>. Thus, the endoscope device <NUM> can be freely moved within the movable range of the arm unit <NUM>. <FIG> illustrates a hemisphere as an example of a movable range of the endoscope device <NUM>. If the center point RCM (remote movement center) of the hemisphere is the imaging center of the operation site to be imaged by the endoscope device <NUM>, when the endoscope device <NUM> is moved on the spherical surface of the hemisphere with the imaging center of the endoscope device <NUM> fixed at the center point of the hemisphere, the operation site can be photographed from various angles.

Further, the arm unit <NUM> may have a degree of freedom of <NUM> for rotating the endoscope device <NUM> coaxially with the link 1422f, in addition to the degree of freedom described above. Accordingly, the endoscope device <NUM> can be rotated with the longitudinal axis of the link 1422f as the rotation axis.

So far, the configuration of the support arm device <NUM> according to this embodiment has been described. Hereinafter, a configuration example of the control device for the drive control of the arm unit <NUM> in the support arm device <NUM> according to this embodiment, that is, for controlling the rotational driving of the actuators <NUM> provided in the active joints 1421a to 1421f will be described.

<FIG> is a block diagram illustrating an overall configuration example of the support arm device <NUM> including a control device <NUM>. The control device <NUM> includes a control unit <NUM>, a storage unit <NUM>, and an input unit <NUM>.

The control unit <NUM> is configured by various signal processing circuits such as a CPU and a DSP. The control unit <NUM> integrally controls the control device <NUM> and performs various calculations for controlling the driving of the arm unit <NUM> in the support arm device <NUM>. Specifically, the control unit <NUM> has a whole-body cooperative control unit <NUM> and an ideal joint control unit <NUM>. The whole-body cooperative control unit <NUM> performs various calculations in whole-body cooperative control in order to drive and control actuators <NUM> provided in the active joints 1421a to 1421f of the arm unit <NUM> of the support arm device <NUM>. The ideal joint control unit <NUM> performs various calculations in the ideal joint control that realizes an ideal response to the whole-body cooperative control by correcting the influence of disturbance. The storage unit <NUM> may be a storage element such as a Random Access Memory (RAM) or a Read Only Memory (ROM), or may be a semiconductor memory, a hard disk, or an external storage device.

The input unit <NUM> is an input interface for the user to input information, instructions, and the like regarding drive control of the support arm device <NUM> to the control unit <NUM>. The input unit <NUM> has an operating unit, which is operated by a user, such as a lever and a pedal, and according to the operation of the lever, pedal, or the like, the position, speed, or the like of each component of the arm unit <NUM> may be set for an instantaneous exercise purpose. In addition to the lever and the pedal, the input unit <NUM> may include, for example, an operation unit, which is operated by a user, such as a mouse, a keyboard, a touch panel, a button, and a switch.

The arm unit <NUM> controlled by the control device <NUM> includes an active joint <NUM>. The active joints <NUM> (1421a to 1421f) includes various components necessary for driving the arm unit <NUM>, such as a support member for connecting or supporting the links 1422a to 1422f and the endoscope device <NUM>. In the above description and the following description, the driving of the joint of the arm unit <NUM> may mean the driving of the actuator <NUM> in the active joints 1421a to 1421f.

The active joint <NUM> includes a torque sensor <NUM>, an encoder <NUM>, and an actuator <NUM>. In <FIG>, the actuator <NUM>, the encoder <NUM>, and the torque sensor <NUM> are illustrated separately, but the encoder <NUM> and the torque sensor <NUM> may be included in the actuator <NUM>.

The actuator <NUM> includes a motor, a motor driver, and a speed reducer. The actuator <NUM> is, for example, an actuator corresponding to force control. In the actuator <NUM>, the rotation of the motor is reduced at a predetermined reduction ratio by the speed reducer to be transmitted to another member at the subsequent stage via the output shaft, thereby driving the other member.

The motor is a driving mechanism that generates a rotational driving force. The motor is driven so as to generate a torque corresponding to a torque command value from the control unit under the control of the motor driver. As the motor, for example, a brushless motor is used. However, this embodiment is not limited to this example, and various known types of motors may be used as the motor.

A motor driver is a driver circuit (driver IC (Integrated Circuit)) that drives the motor to rotate by supplying current to the motor. The motor driver can control the number of rotations of the motor by adjusting the amount of current supplied to the motor. The motor driver drives the motor by supplying a current corresponding to the torque command value τ from the control unit to the motor.

In addition, the motor driver can adjust the amount of current supplied to the motor to adjust the coefficient of viscous resistance in the rotational movement of the actuator <NUM>. As a result, a predetermined resistance can be applied to the rotational movement of the actuator <NUM>, that is, the rotational movement of the active joints 1421a to 1421f. For example, the active joints 1421a to 1421f can be set in a state where the active joints are easy to be rotated by an externally applied force (that is, a state where the arm unit <NUM> is easy to be moved manually) and, conversely, also can be set in a state where the active joints are hard to be rotated by an externally applied force (that is, a state where the arm unit <NUM> is hard to be moved manually).

A speed reducer is connected to a rotation shaft (drive shaft) of the motor. The speed reducer reduces the rotation speed (that is, the rotation speed of the input shaft) of the connected rotation shaft of the motor at a predetermined reduction ratio and transmits the reduced speed to the output shaft. In this embodiment, the configuration of the speed reducer is not limited to a specific one, and various known types of speed reducers may be used as the speed reducer. However, as the speed reducer, it is preferable to use a speed reducer such as Harmonic Drive (registered trademark) in which the speed reduction ratio can be set with high accuracy. Further, the speed reduction ratio of the speed reducer can be appropriately set according to the use of the actuator <NUM>. For example, when the actuator <NUM> is applied to the active joints 1421a to 1421f of the support arm device <NUM> as in this embodiment, the speed reducer having a speed reduction ratio of about <NUM>:<NUM> can be suitably used.

The encoder <NUM> detects the rotation angle of the input shaft (that is, the rotation angle of the rotation shaft of the motor). Based on the rotation speed of the input shaft detected by the encoder <NUM> and the reduction ratio of the speed reducer, it is possible to obtain information such as the rotation angles, the rotation angular velocities, and the rotation angular accelerations of the active joints 1421a to 1421f. As the encoder <NUM>, various known rotary encoders such as a magnetic encoder and an optical encoder may be used. Note that the encoder <NUM> may be provided only on the input shaft of the actuator <NUM>, or an encoder for detecting the rotation angle or the like of the output shaft of the actuator <NUM> may be further provided at a stage subsequent to the speed reducer.

The torque sensor <NUM> is connected to the output shaft of the actuator <NUM>, and detects the torque acting on the actuator <NUM>. The torque sensor <NUM> detects the torque (generated torque) output by actuator <NUM>. The torque sensor <NUM> can also detect an external torque applied to the actuator <NUM> from the outside.

The configuration of the active joint <NUM> has been described above. Here, in this embodiment, the operation of the arm unit <NUM> is controlled by force control. In the force control, in the support arm device <NUM>, the rotation angle of each of the active joints 1421a to 1421f and the torque acting on each of the active joints 1421a to 1421f are detected by the encoder <NUM> and the torque sensor <NUM> provided for each actuator <NUM>, respectively. At this time, the torque acting on each of the active joints 1421a to 1421f detected by the torque sensor <NUM> may include the force acting on the arm unit <NUM> and/or the endoscope device <NUM>.

Further, based on the rotation angle detected by the encoder <NUM> and the torque value detected by the torque sensor <NUM>, the current state (a position, a speed, and the like) of the arm unit <NUM> can be obtained. In the support arm device <NUM>, the torque which is necessary for the arm unit <NUM> to execute a desired exercise purpose and is generated by the actuator <NUM> provided in each of the active joints 1421a to 1421f is calculated based on the acquired state (arm state) of the arm unit <NUM>, and the actuator <NUM> of each of the active joints 1421a to 1421f is driven using the torque as a control value.

As the actuator <NUM>, various known actuators can be used which are used in various devices of which the operation is generally controlled by force control. For example, as the actuator <NUM>, the actuators described in <CIT> and <CIT>, which are prior patent applications by the present applicant, can be preferably used.

In the support arm device <NUM> according to this embodiment, the configuration of the actuator <NUM> and each component configuring the actuator is not limited to the above configuration, and may be another configuration.

The basic configuration of the endoscope system has been described above. Hereinafter, a specific embodiment of the above-described endoscope system will be described.

In this embodiment, an oblique-viewing endoscope is used as the endoscope <NUM> (endoscope device <NUM>) described above. <FIG> is a schematic view illustrating the appearance of the oblique-viewing endoscope <NUM>. In the oblique-viewing endoscope <NUM>, the direction (C1) of the objective lens toward the subject has a predetermined angle ϕ with respect to the longitudinal direction (scope axis C2) of the oblique-viewing endoscope <NUM>. That is, in the oblique-viewing endoscope <NUM>, the objective optical system forms an angle with the eyepiece optical system of the scope. In the oblique-viewing endoscope <NUM>, an operation of rotating the oblique-viewing endoscope <NUM> with the scope axis C2 as a rotation axis (hereinafter, referred to as oblique-viewing rotation) is performed for observation. By performing the oblique-viewing rotation, it is possible to obtain a roundabout visual field and upper, lower, right, and left peripheral visual fields.

<FIG> is a schematic view illustrating a state in which the visual field <NUM> displayed on the display device <NUM> is changed by the oblique-viewing rotation. As illustrated in <FIG>, in a state in which the visual field <NUM> is displayed on the monitor, when the oblique-viewing endoscope is oblique-viewing-rotated by the oblique-viewing rotation angle α to control the vertical direction of the visual field <NUM>, the visual field displayed on the monitor changes from the visual field <NUM> to the visual field <NUM>'. Therefore, by controlling the two axes of the oblique-viewing rotation and the vertical direction of the camera, it is possible to obtain the upper, lower, right, and left peripheral visual fields.

<FIG> is a schematic view illustrating a state in which a visual field projected on a monitor changes due to the oblique-viewing rotation. <FIG> illustrates a state of observing the inside of the body with an oblique-viewing endoscope, and illustrates a state in which an observation target <NUM> to be viewed by a user (operator) exists in various organs <NUM>. In <FIG>, the visual field <NUM> of the monitor is illustrated in a rectangular area, and the observation target <NUM> and the scope axis C2 that the user (operator) wants to see are illustrated together with the visual field <NUM>. Further, <FIG> illustrates a state in which the operator makes a surgical tool <NUM> such as forceps contact the observation target <NUM> and grips the observation target <NUM>.

As illustrated in <FIG>, the observation target <NUM> is located below the visual field <NUM> (on the side of the operator who operates the surgical tool <NUM>) with respect to the position of the scope axis C2. <FIG> illustrates a state in which the observation target <NUM> moves with respect to the visual field <NUM> when the oblique-viewing endoscope <NUM> is rotated about the scope axis C2.

The drawing of "oblique-viewing to lower side" illustrated in <FIG> illustrates a state where the optical axis of the objective optical system of the oblique-viewing endoscope <NUM> is directed to the lower side in the vertical direction of the visual field <NUM>. In this case, the observation target <NUM> enters the visual field <NUM>, and the operator can visually recognize the observation target <NUM> on the monitor.

The drawing of "oblique-viewing to right side" illustrated in <FIG> illustrates a state where the optical axis of the objective optical system of the oblique-viewing endoscope <NUM> is directed to the right side in the right-left direction of the visual field <NUM>. In this case, the observation target <NUM> is removed from the visual field <NUM>, and the operator cannot visually recognize the observation target <NUM> on the monitor.

The drawing of "oblique-viewing to upper side" illustrated in <FIG> illustrates a state where the optical axis of the objective optical system of the oblique-viewing endoscope <NUM> is directed to the upper side in the vertical direction of the visual field <NUM>. Further, the drawing of "oblique-viewing to left side" illustrated in <FIG> illustrates a state where the optical axis of the objective optical system of the oblique-viewing endoscope <NUM> is directed to the left side in the right-left direction of the visual field <NUM>. Also in these cases, the observation target <NUM> is removed from the visual field <NUM>, and the operator cannot visually recognize the observation target <NUM> on the monitor.

As described above, in the oblique-viewing endoscope <NUM>, by performing the oblique-viewing rotation, the position of the visual field with respect to the observation target <NUM> can be changed, and a relatively wide range around the scope axis C2 can be visually recognized. On the other hand, as illustrated in <FIG>, there is a possibility that a part that the user wants to see or an instrument may be out of the visual field <NUM> of the monitor due to the oblique-viewing rotation, so that the sight of those places may be lost.

Therefore, in this embodiment, the position of the observation target <NUM> is detected, and the arm is controlled so that the observation target <NUM> moves to the screen center. In this embodiment, the rotation speed during the oblique-viewing rotation is changed according to the position of the observation target <NUM> with respect to the visual field <NUM>. Specifically, the control is performed such that the rotation speed in the oblique-viewing rotation is lower as the observation target <NUM> is further away from the center of the visual field <NUM>. As the observation target <NUM> is further away from the center of the visual field <NUM>, the observation target <NUM> deviates more easily from the visual field <NUM> when performing the oblique-viewing rotation. For this reason, as the observation target <NUM> is further away from the center of the visual field <NUM>, the rotation speed during the oblique-viewing rotation is lower. Thus, the observation target <NUM> is less likely to deviate from the visual field <NUM>, so as to suppress that the position of the observation target <NUM> is lost. However, the following function and the speed of the oblique-viewing rotation are different. Thus, when the moving speed having the degree of freedom to move the screen in the up, down, left and right directions for the following and the speed of the oblique-viewing rotation are not coordinated and controlled, the observation target may be off the screen. For this reason, while the observation target <NUM> is controlled to be at the screen center, the control is performed such that the speed of the oblique-viewing rotation is reduced according to the distance L from the screen center to the target <NUM> or the relative position.

Here, the change of the oblique-viewing rotation speed according to the distance from the center of the visual field <NUM> to the observation target <NUM> will be described. <FIG> is a schematic view illustrating a method for calculating the distance from the center of the visual field <NUM> to the observation target <NUM>. As illustrated in <FIG>, when the oblique-viewing rotation speed is ω [rad/s] and the number of pixels from the screen center O to the position p of the observation target <NUM> is (x, y) [pixels], the number of pixels l1 from the screen center to the position of the observation target <NUM> is obtained by I = √(x<NUM> + y<NUM>). From here, it is realized by adjusting with the function of the oblique-viewing rotation speed ω = f(l). For example, when the number of pixels from the screen center to the diagonal screen vertex is lmax [pixels], and the highest speed of the oblique-viewing rotation speed is ωmax, it is satisfied that ω = ωmax* (lmax - l) = ωmax* (lmax - √ (x2 + y2)).

Thus, the adjustment is performed in the form of ω = ωmax when the position of the observation target <NUM> is at the screen center, and ω = <NUM> when the position of the observation target <NUM> is at the diagonal screen vertex. In this example, the calculation is linear, but a higherorder function or the like may be used. If the distance Z [mm] from the camera to the subject and the angle of view θ are known, the distance L [mm] from the screen center to the observation target <NUM> may be calculated to set ω = f(L).

<FIG> is a schematic view illustrating an example of a map that defines a speed of the oblique-viewing rotation according to the position of the subject in the screen. A map <NUM> shows an example based on <FIG> in which the speed of the oblique-viewing rotation decreases as the distance from the screen center O increases. The density of the dots in the map <NUM> corresponds to the speed of the oblique-viewing rotation, and the map <NUM> shows the relationship between the density of the dots in the map <NUM> and the speed of the oblique-viewing rotation. As shown in the map <NUM>, the higher the dot density, the higher the speed of the oblique-viewing rotation. Therefore, in the map <NUM>, the speed of the oblique-viewing rotation decreases as the distance from the screen center O increases.

The relationship between the dot density, and the speed of the oblique-viewing rotation and the distance to the subject (the observation target <NUM>) is also shown. According to the map <NUM>, the shorter the distance to the subject, the lower the speed of the oblique-viewing rotation. As the distance to the subject is shorter, the amount of movement of the observation target <NUM> in the visual field <NUM> in response to the movement of the oblique-viewing endoscope <NUM> is greater. Thus, by decreasing the speed of the oblique-viewing rotation as the distance to the subject is shorter, it is possible to prevent the sight of the observation target <NUM> from being lost from the visual field <NUM>.

<FIG> is a characteristic diagram illustrating the relationship between the distance (horizontal axis) from the screen center O to the observation target <NUM> and the oblique-viewing rotation speed (vertical axis). As described above, the control is performed such that the speed of the oblique-viewing rotation decreases as the distance from the screen center O to the observation target <NUM> increases, but a variation illustrated in <FIG> can be assumed as a control method.

In the case of a characteristic <NUM> illustrated in <FIG>, the control is performed with priority given to the speed of the oblique-viewing rotation. If the movement of the arm in the XY direction is sufficiently fast, it is possible to follow the observation target <NUM> even with such characteristics. Since the rotation speed of the oblique-viewing endoscope is not <NUM> even at the screen end (Lmax), the observation target <NUM> may be off the screen depending on the position of the observation target <NUM> at the start of the oblique-viewing rotation.

In the case of characteristics <NUM> and <NUM> illustrated in <FIG>, a case where the speed of an XY movement of the arm is lower than the development of the visual field by the oblique-viewing rotation is assumed. When the observation target <NUM> is close to the screen center O, the oblique-viewing rotation is performed quickly, and the rotation speed of the oblique-viewing endoscope is not <NUM> even at the screen end (Lmax). Since the observation target <NUM> has a speed even when the observation target <NUM> is separated from the screen center O by a certain distance or more, the observation target <NUM> may be off the screen depending on the position of the observation target <NUM> at the start of the oblique-viewing rotation.

Further, in the case of a characteristic <NUM> illustrated in <FIG>, a case where the XY movement is considerably slow and the observation target <NUM> is not deviated from the visual field <NUM> is assumed. When the observation target <NUM> deviates to some extent from the screen center O, the oblique-viewing rotation is completely stopped, and only the XY movement is performed.

In any of the characteristics <NUM> to <NUM>, the characteristics change depending on the relationship between the maximum speed of the oblique-viewing rotation and the speed of the XY movement of the arm, whether or not the observation target <NUM> is allowed to move out of the visual field <NUM>, and the like.

<FIG> illustrates an aspect in which the direction of the optical axis of the objective optical system changes due to the rotation (oblique-viewing rotation) of the oblique-viewing endoscope <NUM> around the long axis. As illustrated in <FIG>, it is possible to change the viewing direction of the operator by the oblique-viewing rotation, but the visual field also moves as described in <FIG>. On the other hand, <FIG> illustrates a case in which the movement in the XY direction is performed by giving the rotation of the optical axis at the center of the trocar point or a movement close thereto. The trocar point means a position where the trocar is inserted into a human body. As illustrated in <FIG>, when a rotational movement (the movement of the oblique-viewing endoscope pivot) is applied to the oblique-viewing endoscope <NUM> about the trocar point, the oblique-viewing rotation angle can be changed without the deviation from the visual field. When performing such a movement, the oblique-viewing rotation angle and the XY movement are linked to move.

Hereinafter, an example of the method of moving the oblique-viewing endoscope <NUM> with respect to the visual field in combination with the oblique-viewing rotation of the oblique-viewing endoscope <NUM> will be described. <FIG> illustrates an example of parallel movement without changing the posture of the oblique-viewing endoscope, and is a schematic view illustrating a case where the oblique-viewing rotation is performed together with the parallel movement. In this case, the movement is performed without changing the posture of the oblique-viewing endoscope <NUM> with respect to the camera visual field.

When an oblique-viewing angle α (around the Y axis), an oblique-viewing rotation angle ψ (around the Z axis), a rigid scope length L, a distance R to the observation target, a point V at the center of the visual field, and the amount (ax, ay, az) of the parallel movement are set, the simultaneous conversion matrix representing the parallel movement is as follows. <MAT> <MAT> <MAT>.

Further, the simultaneous conversion matrix of the oblique-viewing angle and the oblique-viewing rotation angle can be expressed as follows. Note that the initial position of the root of the rigid scope is set as the origin of (<NUM>, <NUM>, <NUM>) for simplicity.

At this time, the point V at the center of the visual field is as follows.

As is apparent from the above equation, the visual field can be moved by the same amount of movement as that of the parallel movement of the rigid scope. Conversely, by controlling ax and ay according to the value of ψ so as to keep V constant, the target point can be continuously seen while the oblique-viewing rotation is performed as illustrated in <FIG>. Furthermore, if the amount of change of (ax, ay) that moves to keep the position of V at a fixed position is sufficiently faster than the amount of change of (Rcosψα, Rsinψsinα), the target does not deviate from the visual field due to the rotation of the oblique-viewing endoscope. Thus, when the oblique-viewing rotation speed is controlled as described above, the oblique-viewing can be rotated without losing the sight of the target from the screen. Further, since the relative difference between the change amounts of (Rcosψα, Rsinψsinα) and (ax, ay) makes an effect, the same movement can be realized by adjusting the speed of (ax, ay) in addition to the speed of the oblique-viewing rotation angle. Such a movement is limited to a case in which the trocar point is not constrained, and corresponds to, for example, a case of an oblique-viewing endoscope surgery with a small thoracotomy/small incision of the chest. Since the movement is in the Y direction, the distance between the visual field <NUM> and the camera is maintained.

<FIG> illustrates an example in which the visual field is moved up, down, left, and right by rotation about a constraint point <NUM>. The constraint point <NUM> corresponds to, for example, the trocar point. In this case, the configuration of the arm can be realized with three degrees of freedom centering on the constraint point <NUM>. Assuming that the rotation around the X axis around the constraint point <NUM> is set as ϕ, and the rotation around the Y axis is set as θ, each rotation matrix is as follows. <MAT> <MAT>.

Further, the simultaneous conversion matrix of the rotation about the constraint point <NUM> is as follows.

At this time, the point V at the center of the visual field can be expressed as follows.

If θ, ϕ, and ψ are adjusted so that the above-described V is maintained, the visual field does not deviate. When the oblique-viewing angle ψ as a target is determined, the target values of θ and ϕ for maintaining the value of V are selected. When θ, ϕ, and ψ are controlled with this target value as a command value, by adjusting the oblique-viewing rotation angle described above, it is possible to control the target value without removing the target from the visual field even in the process of following the target value.

In addition, since the amount of change of V with respect to the change of θand ϕ also changes depending on the posture of ψ, the rotation speeds of θ and ϕ may be adjusted so that the amounts of change of X and Y become the same according to the value of the oblique-viewing rotation angle ψ. Incidentally, the above calculation becomes complicated. Thus, for simple mounting, it may be set that α = <NUM>, ψ = <NUM>, and the rigid scope axis may be rotated by θ and ϕ regardless of the oblique-viewing rotation angle to simplify the movement of the visual field.

In the following, an example will be described in which the following is performed such that the observation target <NUM> is positioned at the center of the visual field <NUM> when the oblique-viewing endoscope <NUM> is moved around the constraint point <NUM>. An arm control method of recognizing the position of the target in the screen by image recognition or the like and moving the observation target in the screen based on the information may be used. In addition, it is also possible to determine an operator's observation target point by gaze detection and move the observation target point to the center of the visual field.

According to this embodiment, an information processing device is provided which includes a control unit that controls the support arm device and moves the endoscope such that the target inside the body of the patient matches the optical axis of the endoscope attached to the support arm device and inserted into the body of the patient. When the target is recognized by image recognition or the like, and the arm is controlled such that the target is displayed at the screen center, a surgical instrument or a target such as tumor can be captured at the center of the image obtained by the endoscope, so as to improve the convenience of the operator. Such an information processing device may be configured separately from the endoscopic surgery system <NUM>, or may be configured as an arbitrary device included in the endoscopic surgery system <NUM>.

Hereinafter, an outline of this embodiment will be described with reference to <FIG> and <FIG>.

<FIG> and <FIG> are diagrams for explaining the outline of this embodiment. <FIG> illustrates an aspect where the lens barrel <NUM> of the endoscope <NUM> is inserted into the body cavity of the patient <NUM> from the trocar 5025a punctured in the abdominal wall of the patient <NUM>. The endoscope <NUM> indicated by the solid line shows the current position and posture, and the endoscope <NUM> indicated by the broken line shows (that is, future) the position and posture of the destination of the movement by the endoscope control process according to this embodiment. Further, the forceps <NUM> is inserted from the trocar 5025d punctured in the abdominal wall of the patient <NUM>. In <FIG>, the image (hereinafter, also referred to as an endoscope image) obtained by the endoscope <NUM> illustrated in <FIG> is illustrated. The left drawing is the image obtained at the current position and posture, and the right drawing is the image obtained after the movement by the endoscope control process according to this embodiment.

Referring to <FIG>, at the current position and posture of the endoscope <NUM>, the tip of the forceps <NUM> is captured in the visual field <NUM> of the endoscope <NUM> but is not on the central axis (that is, the optical axis) <NUM>. Therefore, as illustrated in the left diagram of <FIG>, an endoscope image is obtained in which the tip of the forceps <NUM> is not displayed at the center. In such a situation, the endoscopic surgery system <NUM> according to this embodiment performs a process of moving the endoscope <NUM> so that the surgical tool such as the forceps <NUM> is displayed at the screen center. Specifically, the endoscopic surgery system <NUM> moves the endoscope <NUM> by the support arm device <NUM> (not illustrated) so that the tip of the forceps <NUM> is positioned on the central axis <NUM>. As a result, as illustrated in the right diagram of <FIG>, an endoscope image is obtained in which the tip of the forceps <NUM> is displayed at the center. In the following, an example is illustrated in which the tip of the forceps <NUM> is positioned on the central axis <NUM> without considering the oblique-viewing angle and the oblique-viewing rotation angle. However, the control of the oblique-viewing endoscope <NUM> can be performed in consideration with the oblique-viewing angle and the oblique-viewing rotation angle. In addition, an example in which the tip of the forceps <NUM> is positioned on the central axis <NUM> is illustrated. However, the same is performed also in a case where the observation target <NUM> as a target is positioned on the central axis <NUM> by performing image recognition on the observation target <NUM>.

As described above, the endoscopic surgery system <NUM> can provide an endoscope image in which the surgical tool is displayed at the screen center by automatically following the surgical tool. Therefore, the operator can continue the surgery comfortably without operating the endoscope <NUM>.

<FIG> is a schematic view illustrating the configuration of the control unit <NUM> of the support arm device <NUM> for performing the above-described oblique-viewing rotation operation and follow-up operation. As illustrated in <FIG>, the control unit <NUM> includes a distance acquisition unit 1351a that acquires a distance from the center of the visual field to the position of the observation target <NUM> in the visual field <NUM> obtained by imaging the observation target <NUM> and a speed calculation unit 1351b that calculates the speed of the oblique-viewing rotation of the oblique-viewing endoscope <NUM> or the moving speed of the oblique-viewing endoscope <NUM> based on the distance from the center of the visual field to the position of the observation target <NUM>. The acquisition of the distance from the center of the visual field to the position of the observation target <NUM> by the distance acquisition unit 1351a is performed based on the result of the image recognition of the observation target <NUM> by the control unit <NUM> of the CCU <NUM>.

Further, the control unit <NUM> controls the support arm device <NUM> such that the observation target <NUM> is positioned at the center of the visual field <NUM>, based on the result of the image recognition of the observation target <NUM> by the control unit <NUM> of the CCU <NUM>. At this time, the control unit <NUM> controls at least one of the speed of the oblique-viewing rotation of the oblique-viewing endoscope <NUM> and the moving speed of the oblique-viewing endoscope <NUM> according to the position of the observation target <NUM> in the visual field <NUM>. The control unit <NUM> controls the rotation angle and the rotation speed around the long axis of the oblique-viewing endoscope <NUM> according to the position of the observation target <NUM> in the visual field <NUM>. As described above, the control unit <NUM> of the CCU <NUM> can recognize the feature of the image of the observation target <NUM> and the position in the visual field <NUM> using various image recognition techniques. The control unit <NUM> obtains information on the position of the observation target <NUM> in the visual field <NUM> from the CCU <NUM>.

The operator can specify the observation target <NUM> from the operation site displayed on the display device <NUM> by operating the input device <NUM> while viewing the image of the operation site displayed on the display device <NUM> in real time. The control unit <NUM> recognizes the characteristics of the image of the observation target <NUM> and the position in the visual field <NUM> based on the specified observation target <NUM>.

Next, a process for realizing the above-described endoscope control will be described with reference to <FIG>.

<FIG> is a diagram for explaining the outline of the endoscope control process according to this embodiment. Each block illustrated in <FIG> indicates a process, and the endoscope control process includes a plurality of processes. As illustrated in <FIG>, the endoscopic surgery system <NUM> performs an image process to detect a target such as a surgical tool. Next, the endoscopic surgery system <NUM> calculates the position of the target based on the detection result. Next, the endoscopic surgery system <NUM> calculates the current posture of the endoscope based on the calculated target position and trocar position. Next, the endoscopic surgery system <NUM> calculates the target endoscope tip position based on the calculated current posture of the endoscope and the setting information of the virtual plane (plane or curved surface). Next, the endoscopic surgery system <NUM> calculates the amount of change in the posture of the endoscope based on the current posture of the endoscope and the target endoscope tip position, and generates control information (that is, a command) of the arm for realizing the posture change according to the calculated change amount. Then, the endoscopic surgery system <NUM> controls the support arm (for example, the arm unit <NUM>) to operate according to the generated command. The endoscopic surgery system <NUM> repeatedly performs a series of processes described above.

Hereinafter, the endoscope control processing according to this embodiment will be described in detail.

According to this embodiment, the endoscopic surgery system <NUM> can realize a function of recognizing a surgical tool from an endoscope image and automatically following the surgical instrument. Hereinafter, a calculation method will be described in which the arm operates the endoscope from the image processing portion (marker detection) and the detection result to move the surgical tool to the screen center while considering the trocar point.

Hereinafter, after the functional requirements are described, a method is described which detects the surgical tool (marker) by the image process, and then a calculation method is described which converts the detection result to the target movement information and posture information to operate.

Here, an example is described in which the arm is moved about the constraint point without considering the oblique-viewing angle and the oblique-viewing rotation angle.

The endoscopic surgery system <NUM> detects a surgical tool (for example, a tip position and/or a posture of the surgical tool) by an image process.

For example, when a marker serving as a mark is attached to the tip of the surgical tool, the position of the surgical tool may be detected by the image process based on the endoscope image. It is desirable that the marker is easy to detect. For example, the marker may be a color such as blue or green that stands out as compared with the color of the organ or blood vessel in the body cavity (for example, a color positioned on the opposite side of the color of the organ or blood vessel in the hue circle). Further, the marker may be a specific pattern such as a two-dimensional code or a barcode.

For example, when a marker serving as a marker is attached to a portion of the surgical tool that is outside the body, the position of the surgical tool may be detected based on the detection result of the marker by an external sensor and the information on the length and posture of the surgical tool, and the like.

The detection of the surgical tool may be performed by a method other than the image process.

For example, by creating a special trocar, the position of the surgical tool may be calculated based on the insertion amount of the surgical tool and the angle of the trocar.

For example, when the surgical tool is attached to a support arm device different from the endoscope, the position of the surgical tool may be calculated from the position and posture information of the support arm device.

The endoscopic surgery system <NUM> performs target calculation. The target calculation is a calculation for instructing the movement by calculating two of the position and the posture.

Specifically, the endoscopic surgery system <NUM> first obtains the target position from the image processing result, and then determines the amount of change in the posture based on the current posture starting from the trocar point and the posture at the time of reaching the target position. In addition, the endoscopic surgery system <NUM> performs target calculation based on the current position and posture obtained from the encoder while obtaining the movement amount from the result of the image process. However, when an actual command is issued, a finally executed command value is added with the calculated value. The reason for this is that a deviation caused by a control error occurs between the current value and the command value, and when a goal is set with the present value as a starting point when the command value is issued, the operation is not smooth and the error becomes large.

Hereinafter, an example of the flow of the target calculation process will be described with reference to <FIG>.

<FIG> is a flowchart illustrating an example of the flow of the target calculation process by the endoscopic surgery system <NUM> according to this embodiment. As illustrated in <FIG>, the endoscopic surgery system <NUM> first performs coordinate calculation.

In the coordinate calculation, the endoscopic surgery system <NUM> first calculates the coordinates based on the current value. Specifically, the endoscopic surgery system <NUM> obtains an image processing result (Step S402). Next, the endoscopic surgery system <NUM> converts the detection position into the camera coordinate system (that is, conversion from 2D to 3D) (Step S404). Next, the endoscopic surgery system <NUM> converts the camera coordinate system into the World coordinate system (Step S406). Next, the endoscopic surgery system <NUM> converts the trocar points into unit vectors (Step S408). Next, the endoscopic surgery system <NUM> obtains the length up to the intersection with a predetermined plane (that is, a virtual plane) (Step S410). Next, the endoscopic surgery system <NUM> converts the vector from the trocar point to the predetermined plane into the World coordinate system (Step S412).

After calculating the coordinates based on the current value, the endoscopic surgery system <NUM> calculates the coordinates based on the command value. Specifically, the endoscopic surgery system <NUM> performs conversion into the insertion depth based on the length of the endoscope (Step S414).

After the coordinate calculation, the endoscopic surgery system <NUM> performs posture calculation.

In the posture calculation, the endoscopic surgery system <NUM> first calculates the posture based on the current value. Specifically, the endoscopic surgery system <NUM> obtains the current posture vector (Step S416). Next, the endoscopic surgery system <NUM> obtains the posture of the calculated new target vector (Step S418). Next, the endoscopic surgery system <NUM> obtains a relative posture change amount with respect to the calculated new target vector (Step S420).

After calculating the posture based on the current value, the endoscopic surgery system <NUM> calculates the posture based on the command value. Specifically, the endoscopic surgery system <NUM> converts the posture of the final command value into a posture change amount (Step S422).

Through the process described above, the endoscopic surgery system <NUM> obtains the target position and the target posture.

<FIG> is a diagram for explaining target position calculation according to this embodiment. As illustrated in <FIG>, the image processing result is notified as values in which the position as viewed from the camera coordinate system with the screen center at the tip of the camera set as (<NUM>, <NUM>) is normalized to [<NUM> to <NUM>]. As it is a dimensionless value as it is, the endoscopic surgery system <NUM> first performs conversion to a metric system. However, since the image processing result is 2D and has no information on the depth direction, the endoscopic surgery system <NUM> assumes that the depth is, for example, <NUM> [mm] at the time of conversion, and obtains a virtual position by combination with a field angle.

The reason why the depth is assumed to be <NUM> [mm] is described. The first reason is that, when the assumed value is larger than the actual value, the moving amount of (x, y) becomes larger than the actual (x, y), and overruns (oscillates). The second reason is that the shortest distance is set by setting the photographing distance in the assumed method to <NUM> [mm] to <NUM> [mm]. The third reason is that when the actual distance is large, the movement is newly determined from the residual based on the next image processing result, so that the goal can be finally reached.

The endoscopic surgery system <NUM> obtains a target posture after a target position is determined. The control unit <NUM> controls the arm unit <NUM> by the arm unit based on the target position and the target posture.

<FIG> are schematic views for explaining a specific example in a case where an oblique-viewing is directed to the left according to the control of the oblique-viewing endoscope according to this embodiment. Similarly to <FIG>, <FIG> also illustrates a state in which an observation target <NUM> to be viewed by a user (operator) exists in various organs <NUM>. Here, <FIG> illustrates a state (a state under the oblique-viewing) in which the optical axis of the objective optical system is directed to the lower side (the side of the operator who operates the surgical tool <NUM>) of the visual field <NUM> with respect to the position of the scope axis C. <FIG> illustrates an enlarged visual field <NUM> of <FIG>. In this case, when the oblique-viewing rotation is performed to view the left side of the visual field <NUM>, in the existing technology in which the follow-up operation to the observation target <NUM> is not performed as illustrated in <FIG>, the left side of the observation target <NUM> can be observed by the oblique-viewing rotation compared to <FIG>, but the observation target <NUM> cannot be viewed.

On the other hand, in this embodiment, as illustrated in <FIG>, the arm is controlled so as to follow the observation target <NUM>, such that the observation target <NUM> is positioned at the screen center. Thus, the oblique-viewing endoscope <NUM> can be rotated to the left without losing the sight of the observation target <NUM>.

In the configuration example illustrated in <FIG> described above, an example is illustrated in which the posture and position of the oblique-viewing endoscope <NUM> are changed only by controlling the active joints 1421a to 1421f of the arm unit <NUM>. On the other hand, a holding unit with a built-in actuator for independently controlling the rotation of the oblique-viewing endoscope may be provided at the tip of the arm unit <NUM>. In this case, the rotation of the oblique-viewing endoscope <NUM> is performed by the holding unit, and the position of the entire oblique-viewing endoscope and the posture with respect to the operation site can be controlled by the active joint of the arm.

<FIG> is a schematic view illustrating a configuration in which a holding unit <NUM> that independently controls the rotation of the oblique-viewing endoscope <NUM> and the rotation of a camera head <NUM> is provided at the tip of an arm unit <NUM> in the configuration example illustrated in <FIG>. The holding unit <NUM> is mounted to an endoscope unit mounting unit <NUM> at the tip of the arm unit <NUM>, and includes a camera head mounting unit <NUM>, a camera head rotation drive unit <NUM>, an oblique-viewing endoscope mounting unit <NUM>, and an oblique-viewing endoscope rotation drive unit <NUM>.

As illustrated in <FIG>, the camera head <NUM> is mounted on the camera head rotation drive unit <NUM> via the camera head mounting unit <NUM>. The camera head rotation drive unit <NUM> includes an actuator <NUM> such as a motor, and rotates the camera head <NUM> with respect to the endoscope unit mounting unit <NUM> and the main body of the holding unit <NUM>.

Further, the oblique-viewing endoscope <NUM> is mounted to the oblique-viewing endoscope rotation drive unit <NUM> via the oblique-viewing endoscope mounting unit <NUM>. The oblique-viewing endoscope rotation drive unit <NUM> includes an actuator <NUM> such as a motor, and rotates the oblique-viewing endoscope <NUM> around the endoscope unit mounting unit <NUM> and the main body of the holding unit <NUM>.

<FIG> is a schematic view illustrating the configuration of the support arm device <NUM> including the arm unit <NUM> and the control device <NUM>, an endoscope unit <NUM>, and a CCU <NUM> in the configuration example illustrated in <FIG>. The control device <NUM> includes a CCU communication unit <NUM> that performs communication with the CCU <NUM> in addition to the configuration illustrated in <FIG>. The CCU <NUM> includes an arm communication unit <NUM> that performs communication with the control device <NUM> in addition to the configuration illustrated in <FIG>. The control device <NUM> and the CCU <NUM> can transmit and receive information to and from each other by the communication between the CCU communication unit <NUM> and the arm communication unit <NUM>. Further, the endoscope unit <NUM> is obtained by adding the function of the holding unit <NUM> to the camera head <NUM> illustrated in <FIG>, and includes an endoscope unit control unit <NUM> instead of the control unit <NUM> illustrated in <FIG>. In addition to the configuration of the camera head <NUM> illustrated in <FIG>, the endoscope unit <NUM> includes an oblique-viewing control unit (first control unit) <NUM>, a camera head control unit (second control unit) <NUM>, an oblique-viewing endoscope rotation drive unit <NUM>, and a camera head rotation drive unit <NUM>. The oblique-viewing control unit <NUM> and the camera head control unit <NUM> may be provided in the holding unit <NUM>. The functions of the endoscope unit control unit <NUM>, the oblique-viewing control unit <NUM>, and the camera head control unit <NUM> may be provided in the control unit <NUM> of the control device <NUM>.

The oblique-viewing control unit <NUM> drives the actuator <NUM> based on the command from the endoscope unit control unit <NUM>. The actuator <NUM> and the oblique-viewing rotation unit encoder <NUM> are provided in the oblique-viewing endoscope rotation drive unit <NUM>. The endoscope unit control unit <NUM> drives the actuator <NUM> and controls the rotation of the oblique-viewing endoscope <NUM> around the axis based on the rotation angle of the actuator <NUM> detected by the oblique-viewing rotation unit encoder <NUM>.

Further, the camera head control unit <NUM> drives the actuator <NUM> based on the command from the endoscope unit control unit <NUM>. The actuator <NUM> and the camera head rotation unit encoder <NUM> are provided in the camera head rotation drive unit <NUM>. The endoscope unit control unit <NUM> controls the rotation of the camera head <NUM> around the axis based on the rotation angle of the actuator <NUM> detected by the camera head rotation unit encoder <NUM>.

With the above configuration, the oblique-viewing endoscope <NUM> and the camera head <NUM> can rotate independently with respect to the endoscope unit mounting unit <NUM>. Accordingly, it becomes possible to rotate the oblique-viewing endoscope <NUM> to visually recognize a desired observation target and to rotate the camera head <NUM> to appropriately control the top and bottom of the image.

<FIG> is a sequence diagram illustrating a processing flow in the configuration illustrated in <FIG> and <FIG>. First, in Step S502, the information on the oblique-viewing rotation of the oblique-viewing endoscope <NUM> and the posture of the camera head <NUM> for top and bottom control is sent from the endoscope unit to the CCU <NUM>. In the next Step S504, the information on the oblique-viewing rotation of the oblique-viewing endoscope <NUM> and the posture of the camera head <NUM> for the top and bottom control is sent from the CCU <NUM> to the arm control device.

In the next Step S506, an arm control command and a drive command value of the endoscope unit are calculated. In the next Step S508, the drive command value of the endoscope unit is sent from the arm control device to the CCU <NUM> together with the drive command. In Step S510, the arm is controlled based on the arm control command.

In Step S512, the drive command value of the endoscope unit is sent from the CCU <NUM> to the endoscope unit together with the drive command. In the next Step S514, the endoscope unit is controlled based on the drive command value.

In the next Step S516, the information on the oblique-viewing rotation of the oblique-viewing endoscope <NUM> and the posture of the camera head <NUM> for top and bottom control is sent from the endoscope unit to the CCU <NUM>. In the next Step S504, the information on the oblique-viewing rotation of the oblique-viewing endoscope <NUM> and the posture of the camera head <NUM> for the top and bottom control is sent from the CCU <NUM> to the arm control device.

As described above, according to this embodiment, the speed of the oblique-viewing rotation decreases as the distance from the center of the visual field <NUM> of the monitor increases. Thus, it is possible to suppress the observation target <NUM> from deviating from the visual field <NUM> in the case of the visual recognition of the periphery by using oblique-viewing rotation of the oblique-viewing endoscope <NUM>.

Further, according to this embodiment, the image of the observation target <NUM> existing inside the human body of the patient is recognized, the medical arm is controlled so that the optical axis of the objective optical system of the oblique-viewing endoscope <NUM> matches the position of an observation target <NUM>, and the endoscope is moved while the tip of the endoscope is constrained on the virtual plane. Accordingly, when the oblique-viewing rotation is performed, the observation target <NUM> such as a surgical instrument or a tumor can be captured at the center of the visual field <NUM>, and the convenience for the operator is improved.

Therefore, the operator can perform the oblique-viewing rotation while always keeping a desired part, an instrument, and the like in the screen, and can suppress that the sight of the observation target <NUM> is lost due to the oblique-viewing rotation. Further, the realization can be easily made without performing the posture control based on the distance information as in the pivot operation around the subject.

The preferred embodiments of the present disclosure have been described above in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person with ordinary knowledge in the art to which the present disclosure pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that they belong to the technical scope of the present disclosure.

Note that a series of processes by each device described in this specification may be realized using any of software, hardware, and a combination of software and hardware. A program constituting the software is stored in advance in a storage medium (non-transitory medium) provided inside or outside each device, for example. Each program is read into a RAM when the computer executes the program, and is executed by a processor such as a CPU.

Further, the processes described with reference to the flowcharts and the like in this specification do not necessarily have to be executed in the illustrated order. Some processing Steps may be performed in parallel. Further, additional processing Steps may be employed, and some processing Steps may be omitted.

Further, the effects described in this specification are merely illustrative or exemplary and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of this specification in addition to or instead of the above effects.

Claim 1:
A surgical arm control system (<NUM>) for controlling an articulated arm (<NUM>) in which a plurality of joints (1422a, 1422b, 1422c) is rotatably connected by a plurality of links (1421a, 1421b, 1421c) and which is capable of supporting an oblique-viewing endoscope (<NUM>) at a tip; wherein
the control system (<NUM>) is configured to control the articulated arm to change a position and a posture of the oblique-viewing endoscope,
the control system is configured to control a rotation speed of the oblique-viewing endoscope based on information specifying a position of an observation target (<NUM>) in a visual field (<NUM>) imaged through the oblique-viewing endoscope;
wherein the control system (<NUM>) is configured to reduce the rotation speed as the distance from the center of the visual field (<NUM>) to the position of the observation target (<NUM>) in the visual field increases.