Patent Description:
In recent years, endoscopic surgery using medical robots for surgical assistance is becoming widely available. In endoscopic surgery, surgical tools such as a laparoscope, an endoscope, and forceps, attached to a medical robot are used.

In medical robots, surgical tools can be remotely controlled by a doctor. Also, medical robots that enable intuitive operation are proposed. Further, for the purpose of increasing accuracy and safety, medical robots that can present force sense to transmit an external force acting on a surgical tool to a doctor who operates the medical robot are also proposed (see, for example, Patent Documents <NUM> and <NUM>).

Patent Document <NUM> discloses receiving tactile feedback from haptic control in accordance with a measured stress received with a medical device, and displaying the measured stress and the like on a monitor so that the stress applied to a needle is felt.

Patent Document <NUM> discloses updating a three-dimensional haptic representation based on information generated by tactile feedback. In the following, three dimension will be also referred to as 3D. Specifically, Patent Document <NUM> describes updating a 3D representation, for example, a 3D graphical representation and/or a 3D haptic representation, of a patient's state when it is determined that a surgical instrument is in contact with a bone of the patient.

Patent Document <NUM> discloses a surgical assisting device having a surgical instrument and assisting a surgical operation by referring to a photographed image of an affected area, comprising: a manipulator capable of operating the surgical instrument; surgical instrument operating means for controlling the manipulator; and the surgical instrument.

Patent Document <NUM> discloses an apparatus and method for sensing at an anatomic body site and mapping or transforming the sensor signal into various forms of virtual image and feedback signals, having particular application in assisting surgeons and other operators during a medical procedure. In one embodiment, a medical system is provided that includes a medical implement, a manipulator controllable by an operator for control of the medical implement at an anatomic body site and a sensing device for sensing a non-visible field associated with a body structure at the site. A controller, intercoupling the sensing device and the manipulator, includes a mapping component for translating characteristics of the sensed field signal into a tactile feedback signal to the manipulator to warn the surgeon that he is approaching this structure with the implement. Alternatively or in addition, a virtual image of the body structure is displayed, separate or preferably together with a visual image of the site, to assist a surgeon in manipulating a medical implement.

Patent Document <NUM> discloses a grasper comprising a handle, two jaws operably connected to the handle, which jaws can be actuated by the handle, and a sensor. Further, a method for measuring an amount of force being applied by a jaw of a grasper is also provided. The method comprises the steps of: providing a grasper comprising a handle and two jaws operably connected to the handle, which jaws can be actuated by the handle; providing a sensor on the grasper; and, providing for measuring an amount of force being applied to the sensor.

Patent Document <NUM> discloses a forceps device for performing a surgical procedure. The device comprises a closeable jaw comprising jaw members having one or more force sensors deployed in a manner to measure force applied to the jaws of the device.

When a method for presenting a force sense is a tactile presentation to the hands of an operating doctor, there is an advantage that the operating doctor can intuitively grasp the force sense. On the other hand, even the same strength of the tactile presentation may be felt differently depending on a doctor receiving the strength, which may lead to lack of accuracy. In addition, since the force sense is presented only to the operating doctor and not to other doctors and the like, it is difficult to perform a third-party real-time check and ensure objectivity.

In a method for displaying the measured stress and the like, in other words, in a method for displaying the force sense, on the monitor as in the techniques disclosed in Patent Documents <NUM> and <NUM>, it is difficult, for example, to properly display a direction of an external force acting on a surgical tool such as forceps that is closed to grip an object and opened to dissect the object. In other words, it is difficult to provide a display that makes it possible to easily distinguish whether the displayed external force is a force acting when gripping the object or a force acting when dissecting the object.

In one aspect of the present disclosure, it is preferable that a system and a force sense display method of a surgical tool are provided which can easily increase accuracy and safety of operation of the surgical tool.

According to the present invention, there are provided a system having the features according to claim <NUM> and a force sense display method having the features according to claim <NUM>. The system comprises a control device corresponding to an example of a force sense display device, as further described below. A force sense display device in a first aspect of the present disclosure is a device that displays a force acting on a surgical tool configured to be closed to grip and opened to dissect an object so that the force is perceptible to an operator. The force sense display device includes an input portion, an image generator, and an output portion. The input portion receives input of measurement signals of a gripping force, which is a force to grip the object, and a dissecting force, which is a force to dissect the object, in the surgical tool. The image generator generates image information for displaying information on the gripping force as a grip display mode and information on the dissecting force as a dissection display mode different from the grip display mode based on the measurement signals. The output portion outputs the generated image information to a display portion configured to display the image information.

A force sense display method in a second aspect of the present disclosure is a display method for displaying a force acting on a surgical tool configured to be closed to grip and opened to dissect an object so that the force is perceptible to an operator. The force sense display method comprises: receiving input of measurement signals of a gripping force, which is a force to grip the object, and a dissecting force, which is a force to dissect the object, in the surgical tool; generating image information for displaying information on the gripping force as a grip display mode and information on the dissecting force as a dissection display mode different from the grip display mode based on the measurement signals; and outputting the generated image information to a display portion configured to display the image information.

With the force sense display device in the first aspect and the force sense display method in the second aspect of the present disclosure, the information on the gripping force can be displayed as the grip display mode, and the information on the dissecting force can be displayed as the dissection display mode different from the grip display mode. Thus, it becomes easy to grasp whether the displayed information is the information on the gripping force or on the dissecting force.

In the first aspect of the present disclosure, it is preferable that the grip display mode and the dissection display mode are display modes, in which different colors or hues are used based on a type of the surgical tool, and the image generator uses a first color or hue for the grip display mode and a second color or hue, which is different from the first color or hue, for the dissection display mode.

Since different colors or hues based on the type of the surgical tool are used for display as such, visual recognition of the information becomes easy. For example, the type of the surgical tool displayed in the grip display mode and the dissection display mode can be checked by the displayed color or hue. Also, since the grip display mode is displayed using the first color or hue and the dissection display mode is displayed using the second color or hue that is different from the first color or hue, it becomes easy to distinguish between the grip display mode and the dissection display mode.

In the first aspect of the present disclosure, it is preferable that the grip display mode and the dissection display mode are displays having lengths that are changed based on a magnitude of the force in the surgical tool, and the image generator generates the image information for setting the grip display mode to one of a mode having a short display length and a mode having a long display length as compared with a reference index having a specified length, and sets the dissection display mode to the other of the mode having a short display length and the mode having a long display length as compared with the reference index.

Since the information on the gripping force and the information on the dissecting force are displayed by changing the display lengths in comparison with the reference index as such, it becomes easy to visually recognize the information. For example, visual recognition of the information becomes easy by decreasing the display length as compared with the reference index to display the information on the gripping force, and increasing the display length as compared with the reference index to display the information on the dissecting force.

In addition, a magnitude of a difference between the display length and the reference index is displayed as a magnitude of the gripping force or a magnitude of the dissecting force. Therefore, visual recognition of the magnitude of the gripping force and the magnitude of the dissecting force is easy.

In the first aspect of the present disclosure, it is preferable that the grip display mode and the dissection display mode are rod-shaped or strip-shaped display modes having the lengths that are changed based on the magnitude of the force in the surgical tool.

Such rod-shaped or strip-shaped displays having the lengths changed based on the magnitude of the force in the surgical tool enable easy comparison with the reference index. Therefore, it becomes easy to grasp whether the displayed information is the information on the gripping force or the information on the dissecting force.

In the first aspect of the present disclosure, it is preferable that the grip display mode and the dissection display mode are display modes in which color or hue is changed based on the magnitude of the force in the surgical tool.

Since the color or hue of the display is changed based on the magnitude of the force in the surgical tool as such, it becomes easy to grasp the magnitude of the force in the surgical tool. Therefore, it becomes easy to grasp magnitudes of the displayed gripping force and dissecting force.

In the first aspect of the present disclosure, it is preferable that the grip display mode and the dissection display mode are displays, in which an area of a displayed figure having a specified shape is changed based on the magnitude of the force in the surgical tool, and the image generator generates the image information for setting the grip display mode to a mode of displaying the displayed figure as a single figure including the reference line, and for setting the dissection display mode to a mode of displaying the displayed figure as two figures spaced apart across the reference line.

Thus, whether the information is on the gripping force or on the dissecting force is displayed by displaying the displayed figure as the single figure or as the two figures, so it becomes easy to visually recognize the information. Further, when the displayed figure is displayed as the single figure, it is displayed to include the reference line. When the displayed figure is displayed as the two figures, they are displayed to be spaced apart across the reference line. Thus, it becomes further easy to visually recognize the information. Also, the area of the displayed figure is displayed as the magnitude of the gripping force or the magnitude of the dissecting force. Therefore, it is easy to visually recognize the magnitude of the gripping force and the magnitude of the dissecting force.

In the first aspect of the present disclosure, it is preferable that the grip display mode and the dissection display mode are fan-shaped display modes having areas that are changed based on the magnitude of the force in the surgical tool.

Display as the fan-shaped figure having the area that is changed based on the magnitude of the force in the surgical tool as such makes it easy to grasp both the magnitude of the gripping force and the magnitude of the dissecting force.

With the force sense display device in the first aspect and the force sense display method in the second aspect of the present disclosure, the information on the gripping force is displayed as the grip display mode, and the information on the dissecting force is displayed as the dissection display mode, which is different from the grip display mode. Thus, there is effect that it becomes easy to increase accuracy and safety of operation of the surgical tool.

<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. medical robot system, <NUM>. surgical tool, <NUM>. reference index, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. control device (force sense display device), <NUM>. input portion, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. image generator, <NUM>. output portion, <NUM>. display device (display portion), 260A. first display device (display portion), 260B. second display device (display portion), S12, S15, S18. input step, S14, S17, S20. image generation step, output step.

Hereinafter, a medical robot system <NUM> according to a first embodiment of the present disclosure will be described with reference to <FIG>. The medical robot system <NUM> of the present embodiment is used for surgical assistance. The medical robot system <NUM> of the present embodiment is, for example, suitable for use in assisting endoscopic surgery.

The medical robot system <NUM>, as shown in <FIG>, comprises a slave unit <NUM>, an endoscope unit <NUM>, an operation unit <NUM>, a sound generator <NUM>, a control device <NUM>, and a display device <NUM>. The control device <NUM> corresponds to an example of a force sense display device, and the display device <NUM> corresponds to an example of a display portion.

The slave unit <NUM>, as shown in <FIG> and <FIG>, supports a surgical tool <NUM> - such as forceps used in endoscopic surgery. Also, the slave unit <NUM> controls at least one of the position and posture of the surgical tool <NUM> in accordance with an operation inputted to the operation unit <NUM>. In addition, when the surgical tool <NUM> is forceps, the slave unit <NUM> opens and closes the forceps.

In the present embodiment, the slave unit <NUM> has a first slave device 10A and a second slave device 10B. The slave unit <NUM> may have more or less than two slave devices.

In the present embodiment, the first slave device 10A and the second slave device 10B are robot arms. Further, the first slave device 10A and the second slave device 10B have the same configuration, and support the same type of surgical tools <NUM>.

The first slave device 10A and the second slave device 10B only need to be able to control at least one of the position and posture of the surgical tool <NUM>, or open and close forceps as the surgical tool <NUM>, and do not have to have an arm-like configuration.

The first slave device 10A and the second slave device 10B, as shown in <FIG>, are each provided with at least a cartridge <NUM>, a shaft <NUM>, and a joint <NUM>. When it is not necessary to distinguish whether the cartridge <NUM>, the shaft <NUM>, and the joint <NUM> are provided in the first slave device 10A or in the second slave device 10B, these are referred to as cartridge <NUM>, shaft <NUM>, and joint <NUM>. When it is necessary to distinguish whether the cartridge <NUM>, the shaft <NUM>, and the joint <NUM> are provided in the first slave device 10A or in the second slave device 10B, these are referred to as cartridge 11A, shaft 12A, and joint 13A, or cartridge 11B, shaft 12B, and joint 13B, respectively.

The cartridge <NUM> supports the shaft <NUM>, the joint <NUM>, and the surgical tool <NUM>. In the present embodiment, the cartridge <NUM> has a cubic shape.

The cartridge <NUM> may store a drive source such as a pneumatic actuator that drives at least one of the shaft <NUM>, the joint <NUM>, and the surgical tool <NUM>. Also, the cartridge <NUM> may store at least one of a shaft sensor <NUM> that measures an external force applied to the shaft <NUM>, a joint sensor <NUM> that measures an external force applied to the joint <NUM>, and a surgical tool sensor <NUM> that measures opening and closing forces applied to the outside by the forceps as the surgical tool <NUM>.

When it is not necessary to distinguish whether the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> are provided in the first slave device 10A or in the second slave device 10B, these sensors are referred to as shaft sensor <NUM>, joint sensor <NUM>, and surgical tool sensor <NUM>. When it is necessary to distinguish whether the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> are provided in the first slave device 10A or in the second slave device 10B, these sensors are referred to as shaft sensor 15A, joint sensor 16A, and surgical tool sensor 17A, or shaft sensor 15B, joint sensor 16B, and surgical tool sensor 17B, respectively.

The shaft <NUM> is a member formed in a cylindrical or columnar shape that extends from the cartridge <NUM>. The shaft <NUM> is arranged to be rotatable about its central axis with respect to the cartridge <NUM>. The shaft <NUM> may be formed in a cylindrical or columnar shape, or may be formed in a prismatic shape.

The joint <NUM> supports the surgical tool <NUM>, and controls the position and posture of the surgical tool <NUM> by bending. The joint <NUM> is located between the shaft <NUM> and the surgical tool <NUM>. In other words, one end of the joint <NUM> is adjacent to an end of the shaft <NUM> that is away from the cartridge <NUM>, and the other end of the joint <NUM> is adjacent to the surgical tool <NUM>. The specific configuration of the joint <NUM> is not limited as long as the joint <NUM> is bendable.

The endoscope unit <NUM>, as shown in <FIG>, acquires video or image for use in performing endoscopic surgery. Specifically, the endoscope unit <NUM> acquires the video or image of a region within a patient's body cavity subject to surgery. Information of the acquired video or image is outputted to the control device <NUM>. The endoscope unit <NUM> only needs to be able to output the information of the video or image, and the configuration thereof is not limited.

The operation unit <NUM> is used for controlling the slave unit <NUM>. Specifically, the operation unit <NUM> is used when the first slave device 10A and the second slave device 10B are operated to perform endoscopic surgery.

The operation unit <NUM> comprises a hand operating part <NUM> and a foot operating part <NUM>. The hand operating part <NUM> is used when an operator (for example, operator of endoscopic surgery) of the medical robot system <NUM> inputs movement of the slave unit <NUM> by hand.

The hand operating part <NUM> comprises a right-hand operating part <NUM> corresponding to the right hand of the operator and a left-hand operating part <NUM> corresponding to the left hand of the operator. The right-hand operating part <NUM> and the left-hand operating part <NUM> convert information inputted by the operator to signals and output the signals to a control portion <NUM>. Also, the right-hand operating part <NUM> and the left-hand operating part <NUM> present force sense to the operator based on a control signal inputted from the control portion <NUM>.

The configuration and method of the right-hand operating part <NUM> and the left-hand operating part <NUM> are not limited. Also, the hand operating part <NUM> may comprise both the right-hand operating part <NUM> and the left-hand operating part <NUM> as described above, or may comprise only one of the right-hand operating part <NUM> and the left-hand operating part <NUM>.

The foot operating part <NUM> is used when the operator operates movement and function of the slave unit <NUM> with foot. The foot operating part <NUM> comprises a first foot switch <NUM>, a second foot switch <NUM>, and a third foot switch <NUM>. The first foot switch <NUM>, the second foot switch <NUM>, and the third foot switch <NUM> are operated by the operator stepping on the switches with foot.

Example operations can include supply and stop of high frequency current to the surgical tool <NUM>, switching of a control mode of the slave unit <NUM>, and so on. The type and detail of operation by the foot operating part <NUM> are not limited. Also, the foot operating part <NUM> may have three foot switches as described above, or may have less or more than three foot switches.

The sound generator <NUM> emits sound based on a control signal inputted from the control device <NUM> to at least some of those involved in endoscopic surgery including the operator. The configuration thereof, such as for generating a sound, is not limited.

The control device <NUM> controls movement and the like of the slave unit <NUM>, and sound generation in the sound generator <NUM>. The control device <NUM> controls contents and modes of display displayed on the display device <NUM>.

The control device <NUM>, as shown in <FIG>, is an information processor such as a computer comprising a CPU, a ROM, a RAM, an input/output interface, and so on. A program stored in a storage device such as the aforementioned ROM causes the CPU, the ROM, the RAM, and the input/output interface to cooperate with each other, and function as at least an input portion <NUM>, the control portion <NUM>, an image generator <NUM>, and an output portion <NUM>. The CPU is an abbreviation for central processing unit.

The input portion <NUM> receives a signal from the outside. Specifically, the input portion <NUM> is coupled to the operation unit <NUM> so that a signal relating to operation can be received, and is coupled to the endoscope unit <NUM> so that a signal of video and image can be received.

Further, the input portion <NUM> is coupled to the shaft sensor <NUM> that measures an external force applied to the shaft <NUM>, the joint sensor <NUM> that measures an external force applied to the joint <NUM>, and the surgical tool sensor <NUM> that measures opening and closing forces applied to the outside by the forceps as the surgical tool <NUM>, so that the measurement signals can be received.

The shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> may be arranged at any location as long as these sensors can measure the external forces as the respective measurement targets, and the locations thereof are not limited. Also, modes for the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> to measure the external forces are not specifically limited.

The control portion <NUM> generates a signal for controlling movements and performance of functions of the slave unit <NUM> and the surgical tool <NUM>. The control signal is generated based on the signal relating to operation inputted from the operation unit <NUM>.

The image generator <NUM> generates image information that is information of video and image to be displayed on the display device <NUM>. Specifically, the image generator <NUM> generates the image information based on the signal of video and image inputted from the endoscope unit <NUM>, and the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM>. The contents of the image information will be described later.

The output portion <NUM> outputs the control signal generated in the control portion <NUM> and the image information generated in the image generator <NUM>. The output portion <NUM> is coupled to the slave unit <NUM> and the surgical tool <NUM> so that the control signal can be outputted to the slave unit <NUM> and the surgical tool <NUM>, and is coupled to the display device <NUM> so that the image information can be outputted to the display device <NUM>.

The display device <NUM> displays video and image based on the image information outputted from the control device <NUM>. The display device <NUM> displays the video and image to those involved in surgery including an operator of endoscopic surgery. The method and configuration of the display device <NUM> are not limited as long as the display device <NUM> can display video and image.

Next, control in the medical robot system <NUM> having the above configuration, specifically control relating to force sense will be described. First, a flow of arithmetic processing in the control device <NUM> will be described with reference to <FIG>, and then contents to be displayed will be described.

When the control device <NUM> receives supply of electric power, the control device <NUM> performs a process to determine whether a timer counter has reached a specified time in S11, as shown in <FIG>. In the present embodiment, the specified time is <NUM>. The specified time may be <NUM>, as described above, or may be longer or shorter than <NUM>.

When it is determined that the timer counter has not reached the specified time, that is, determined NO, the control device <NUM> again performs the determination process of S11.

When it is determined that the timer counter has reached the specified time, that is, determined YES, the control device <NUM> moves the process to an input step of S12. In S12, the control device <NUM> performs a process to determine whether an external force acting on the shaft <NUM> is detected. In other words, the control device <NUM> performs a process to determine whether an arm measurement signal that is a measurement signal of the force is inputted from the shaft sensor <NUM> to the input portion <NUM>.

When it is determined that the external force acting on the shaft <NUM> is detected, that is, determined YES, the control device <NUM> moves the process to S13. In S13, the control device <NUM> performs a process to generate a control signal presenting the detected external force as a force sense. The detected external force indicates a magnitude of the force measured by the shaft sensor <NUM>. The detected external force is estimated based on the measurement signal outputted from the shaft sensor <NUM>.

Specifically, a process to make the hand operating part <NUM> corresponding to a slave device provided with the shaft sensor <NUM> that has outputted the measurement signal present the detected external force as the force sense is performed. For example, when the shaft sensor <NUM> that has outputted the measurement signal measures a force acting on the shaft <NUM> of the first slave device 10A, a process to make the hand operating part <NUM> (that is, right-hand operating part <NUM> or left-hand operating part <NUM>) used for operation of the first slave device 10A present the detected external force as the force sense is performed.

Further, a process to set a strength of the force sense to be presented in accordance with the magnitude of the force estimated based on the inputted measurement signal is performed. The corresponding hand operating part <NUM> presents the force sense corresponding to the force acting on the shaft sensor <NUM> to the operator based on the control signal outputted from the control device <NUM>.

When the process of S13 is completed, the control device <NUM> moves the process to an image generation step and an output step of S14. In S14, the control device <NUM> performs a process to present information on the external force corresponding to the detected external force as information that can be visually grasped. Specifically, the control device <NUM> performs a process to generate image information for displaying video and image corresponding to the slave device provided with the shaft sensor <NUM> that has outputted the measurement signal as well as corresponding to the magnitude of the force estimated based on the inputted measurement signal. Detail of the process to generate image information will be described later. The generated image information is outputted to the display device <NUM>. The display device <NUM> displays the video and image based on the inputted image information.

When the process of S <NUM> is completed, or when it is determined in S12 that the external force acting on the shaft <NUM> is not detected, that is, determined NO, the control device <NUM> moves the process to an input step of S15. In S15, the control device <NUM> performs a process to determine whether an external force acting on the joint <NUM> is detected. In other words, the control device <NUM> performs a process to determine whether a joint measurement signal that is a measurement signal of the force is inputted from the joint sensor <NUM> to the input portion <NUM>.

When it is determined that the external force acting on the joint <NUM> is detected, that is, determined YES, the control device <NUM> moves the process to S16. In S16, the control device <NUM> performs a process to generate a control signal presenting the detected external force as the force sense. The detected external force indicates a magnitude of the force measured by the joint sensor <NUM>. The detected external force is estimated based on the measurement signal outputted from the joint sensor <NUM>.

Specifically, a process to make the hand operating part <NUM> corresponding to the slave device provided with the joint sensor <NUM> that has outputted the measurement signal present the detected external force as force sense is performed. Further, a process to set a strength of the force sense to be presented in accordance with the magnitude of the force estimated based on the inputted measurement signal is performed. The corresponding hand operating part <NUM> presents the force sense corresponding to the force acting on the joint sensor <NUM> to the operator based on the control signal outputted from the control device <NUM>.

When the process of S16 is completed, the control device <NUM> moves the process to an image generation step and an output step of S17. In S17, the control device <NUM> performs a process to present information on the force corresponding to the detected external force as information that can be visually grasped. Specifically, the control device <NUM> performs a process to generate image information for displaying video and image corresponding to the slave device provided with the joint sensor <NUM> that has outputted the measurement signal as well as corresponding to the magnitude of the force estimated based on the inputted measurement signal. Detail of the process to generate image information will be described later. The generated image information is outputted to the display device <NUM>. The display device <NUM> displays the video and image based on the inputted image information.

When the process of S17 is completed, or when it is determined in S15 that the external force acting on the joint <NUM> is not detected, that is, determined NO, the control device <NUM> moves the process to an input step of S18. In S18, the control device <NUM> performs a process to determine whether the external force acting on the surgical tool <NUM> has been detected. In other words, the control device <NUM> performs a process to determine whether a surgical tool measurement signal that is a measurement signal of the force is inputted from the surgical tool sensor <NUM> to the input portion <NUM>.

When it is determined that the external force acting on the surgical tool <NUM> has been detected, that is, determined YES, the control device <NUM> moves the process to S19. In S19, the control device <NUM> performs a process to generate a control signal presenting the detected external force as the force sense. The detected external force indicates a magnitude of the force measured by the surgical tool sensor <NUM>. The magnitude of the force measured by the surgical tool sensor <NUM> specifically corresponds to a magnitude of a gripping force, or a dissecting force, of the forceps. The detected external force is estimated based on the measurement signal outputted from the surgical tool sensor <NUM>.

Specifically, a process to make the hand operating part <NUM> corresponding to the slave device provided with the surgical tool sensor <NUM> that has outputted the measurement signal present the detected external force as the force sense is performed. Further, a process to set a strength of the force sense to be presented in accordance with the magnitude of the force estimated based on the inputted measurement signal is performed. The corresponding hand operating part <NUM> presents the force sense corresponding to the force acting on the surgical tool sensor <NUM> to the operator based on the control signal outputted from the control device <NUM>.

When the process of S19 is completed, the control device <NUM> moves the process to an image generation step and an output step of S20. In S20, the control device <NUM> performs a process to present information on the force acting on the surgical tool corresponding to the detected external force as information that can be visually grasped. Specifically, the control device <NUM> performs a process to generate image information for displaying video and image corresponding to the slave device provided with the surgical tool sensor <NUM> that has outputted the measurement signal as well as corresponding to the magnitude of the force estimated based on the inputted measurement signal. Detail of the process to generate image information will be described later. The generated image information is outputted to the display device <NUM>. The display device <NUM> displays the video and image based on the inputted image information.

When the process of S20 is completed, the control device <NUM> moves the process to S21. In S21, the control device <NUM> performs a process to reset the timer counter to zero (<NUM>). When the reset process is completed, the control device <NUM> returns to S11, and performs the aforementioned process repeatedly. For example, at least while endoscopic surgery is assisted by the medical robot system <NUM>, the aforementioned process is repeatedly performed.

Next, the image information generated in the image generator <NUM> will be described. Specifically, a mode of display in the display device <NUM> will be described. <FIG> is a schematic diagram illustrating the mode of display by the display device <NUM> based on the image information generated by the image generator <NUM> of the present embodiment.

The image generator <NUM>, as shown in <FIG>, sets an endoscope image display section <NUM> in the center of a right-left direction on a display surface in the display device <NUM>, and sets a first force sense display section <NUM> in the lower left and a second force sense display section <NUM> in the upper right of the endoscope image display section <NUM>.

The endoscope image display section <NUM> is a region having a wider area than the first force sense display section <NUM> and the second force sense display section <NUM>. The image generator <NUM> generates image information that makes the endoscope image display section <NUM> display the information of the video or image inputted from the endoscope unit <NUM>.

The first force sense display section <NUM> and the second force sense display section <NUM> are regions having narrower areas than the endoscope image display section <NUM>. The image generator <NUM> generates the image information based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM>.

The first force sense display section <NUM> and the second force sense display section <NUM> are partitioned from other regions by frames. Also, displays are performed based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> inside the partitioned areas in the first force sense display section <NUM> and the second force sense display section <NUM>. The first force sense display section <NUM> and the second force sense display section <NUM> of the present embodiment have frames of the same color formed in the same rectangular shape. Display modes in the first force sense display section <NUM> and the second force sense display section <NUM> will be described later.

In the present embodiment, the first force sense display section <NUM> is a region where display is performed based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> corresponding to the first slave device 10A. The second force sense display section <NUM> is a region where display is performed based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> corresponding to the second slave device 10B.

The first force sense display section <NUM> may be a region where display is performed based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> corresponding to the second slave device 10B, and the second force sense display section <NUM> may be a region where display is performed based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> corresponding to the first slave device 10A.

In the present embodiment, the first force sense display section <NUM> is located in the lower left, and the second force sense display section <NUM> is located in the upper right, of the display surface in the display device <NUM>. Alternatively, as shown in <FIG>, the first force sense display section <NUM> may be located in the upper left, and the second force sense display section <NUM> may be located in the lower right, of the display surface. Or, as shown in <FIG>, the first force sense display section <NUM> may be located slightly above the left side center, and the second force sense display section <NUM> may be located slightly below the right side center, of the display surface.

In the present embodiment, the first force sense display section <NUM> and the second force sense display section <NUM> are described to have frames in the same color and formed in the same rectangular shape, as an example. Alternatively, the first force sense display section <NUM> and the second force sense display section <NUM> may have frames formed in different shapes. For example, as shown in <FIG>, the first force sense display section <NUM> may have a frame formed in a rectangular shape, and the second force sense display section <NUM> may have a frame formed in an oval shape.

Further in the present embodiment, the first force sense display section <NUM> and the second force sense display section <NUM> are described to have frames in the same color, as an example. Alternatively, the first force sense display section <NUM> and the second force sense display section <NUM> may have frames in different colors.

Next, the display modes in the first force sense display section <NUM> and the second force sense display section <NUM> will be described. Since the display modes in the first force sense display section <NUM> and the second force sense display section <NUM> are the same, the display mode in the first force sense display section <NUM> will be described as a representative example, and the description of the display mode in the second force sense display section <NUM> is omitted. Also, the first force sense display section <NUM> and the second force sense display section <NUM> may display the names of the first slave device 10A and the second slave device 10B to facilitate distinguishing between the first force sense display section <NUM> and the second force sense display section <NUM>.

The display mode of the first force sense display section <NUM> is as shown in <FIG>. In other words, in the first force sense display section <NUM>, three-dimensional models of the shaft <NUM>, the joint <NUM>, and the surgical tool <NUM> of the first slave device 10A are displayed. In other words, a three-dimensional model of shaft <NUM>, a three-dimensional model of joint <NUM>, and a three-dimensional model of surgical tool <NUM> are displayed. The displayed three-dimensional models reflect an actual posture of the shaft <NUM>, and an actual bent state of the joint <NUM>.

The shape of the surgical tool <NUM> in the three-dimensional model is determined based on type information indicating the type of the surgical tool <NUM> inputted via the input portion <NUM>. The control device <NUM> stores the type information indicating the type of the surgical tool <NUM> and the shape of the surgical tool <NUM> in the displayed three-dimensional model associated with each other. The image generator <NUM> acquires information of the shape of the surgical tool <NUM> associated with the inputted type information, and generates the image information for displaying the three-dimensional model.

The type information inputted via the input portion <NUM> may be information inputted such as by a worker that performs settings of the medical robot system <NUM> or an operator that performs endoscopic surgery, or may be a detection signal outputted from a sensor that detects the type of the attached surgical tool <NUM>.

In a part of the shaft <NUM> in the three-dimensional model, a conical shape 15f showing the magnitude of the force and a direction of the force is displayed based on the signal inputted from the shaft sensor <NUM>. Also, in a part of the joint <NUM> in the three-dimensional model, a conical shape 16f showing the magnitude of the force and a direction of the force is display based on the signal inputted from the joint sensor <NUM>.

In the present embodiment, a distance from the bottom surface to an apex of the conical shape corresponds to the magnitude of the force. Also, color in which the conical shape is displayed is changed in accordance with the magnitude of the force. For example, as the force increases, the color is changed among green, yellow, and red. Further, in the present embodiment, a direction from the bottom surface toward the apex of the conical shape indicates the direction of the force.

In a region adjacent to the surgical tool <NUM> in the three-dimensional model, for example, in a region adjacent to a leading end of the surgical tool <NUM>, a band-like shape showing a magnitude of the force to close or open the forceps as the surgical tool <NUM> is displayed based on the signal inputted from the surgical tool sensor <NUM>.

In the present embodiment, as shown in <FIG>, the force to open the forceps as the surgical tool <NUM> is represented by a display in a dissection display mode in which a length of a band-like shape 14f is longer than a preset reference index <NUM>. Also, as shown in <FIG>, the force to close the surgical tool <NUM> is represented by a display in a grip display mode in which the length of the band-like shape 14f is shorter than the reference index <NUM>.

Also, difference in length between the band-like shape 14f and the reference index <NUM> is changed depending on the magnitude of the force, and the magnitude of the force is displayed by the difference in length. For example, when the force increases, the difference in length is displayed to be larger, and, when the force decreases, the difference in length is displayed to be smaller.

Alternatively, color of the band-like shape 14f may be changed in accordance with the magnitude of the force. For example, as the force increases, the color is changed among green, yellow, and red. When the force is not measured by the surgical tool sensor <NUM>, the band-like shape 14f is displayed with the same length as the reference index <NUM>, and in white, as shown in <FIG>.

With the control device <NUM> having the above configuration, the information on the gripping force can be displayed as the grip display mode, and the information on the dissecting force is displayed as the dissection display mode that is different from the grip display mode. Thus, it becomes easy to grasp whether the displayed information is the information on the gripping force or the information on the dissecting force.

Since whether the information is on the gripping force or the dissecting force is displayed by changing the length of the band-like shape 14f in comparison with the reference index <NUM>, visual recognition of the information becomes easy. Also, since a magnitude of a difference between the length of the band-like shape 14f and the reference index <NUM> is displayed as a magnitude of the gripping force or the dissecting force, visual recognition of the magnitude of the gripping force or the dissecting force is easy.

Display as the band-like shape 14f having a length that is changed based on the magnitude of the force measured by the surgical tool sensor <NUM> makes it easy to perform comparison with the reference index <NUM>. Thus, it becomes easy to grasp whether the displayed information is the information on the gripping force or the information on the dissecting force.

Since the color or hue of the band-like shape 14f is changed based on the magnitude of the force measured by the surgical tool sensor <NUM>, it becomes easy to grasp the magnitude of the force in the surgical tool <NUM>. Thus, it becomes easy to grasp the magnitude of the displayed gripping force and dissecting force.

It is possible to generate the image information for displaying the information of the external force as an image in association with the point at which the external force is measured, and display the image information on the display device <NUM>. In other words, information of the displayed external force can be displayed in a manner that a point where the external force has been measured is identifiable.

Also, the information of the external force is displayed as an image in the first force sense display section <NUM> and the second force sense display section <NUM> which are different from the endoscope image display section <NUM>. Accordingly, the displayed information of the external force can be easily grasped, as compared with a case where the information of the external force is displayed over the endoscope image display section <NUM>. Further, visibility of the forces acting on the shaft <NUM>, the joint <NUM>, and the surgical tool <NUM> displayed in the endoscope image display section <NUM> is less likely to be interfered.

The magnitude of the value of the external force is represented by at least one of the color, shape, and area. Therefore, as compared with a case where the magnitude of the value of the external force is represented by a numerical value, it becomes easy to grasp the magnitude of the value of the external force intuitively.

The information on the force acting on the joint <NUM> and the external force acting on the shaft <NUM> which is a part other than the joint <NUM> in the first slave device 10A and the second slave device 10B is displayed. Thus, as compared with a case where the external force acting on a single part (for example, the joint <NUM> only) is displayed, it becomes easy to grasp the operation states of the first slave device 10A and the second slave device 10B.

For example, when the first slave device 10A and the second slave device 10B are brought into contact with each other, an external force by the contact acts on a part (for example, shafts <NUM>) of the first slave device 10A and the second slave device 10B other than the joints <NUM>. Since the information on this external force is displayed, the operator can grasp that the first slave device 10A and the second slave device 10B are brought into contact with each other.

The information of the forces to open the forceps and close the forceps is displayed as the information on the force acting on the surgical tool <NUM>. Thus, it becomes easy to grasp the operation state of the surgical tool <NUM>. Here, the force to open the forceps is also represented as a dissecting force, and the force to close the forceps is also represented as a gripping force.

The images showing the shapes of the surgical tools <NUM> gripped by the first slave device 10A and the second slave device 10B are displayed. Thus, it becomes easy to grasp whether the displayed information of the external force is information of the external force measured in the first slave device 10A, or information of the external force measured in the second slave device 10B.

Next, a medical robot system according to a first variation of the first embodiment of the present disclosure will be described with reference to <FIG> and <FIG>. The medical robot system of the present variation has a basic configuration similar to that of the first embodiment, but differs from the first embodiment in that there is no endoscope unit. Accordingly, for the present variation, those related to the display of force sense will be described with reference to <FIG> and <FIG>, and other descriptions are omitted.

A medical robot system <NUM> of the present variation comprises the slave unit <NUM>, the operation unit <NUM>, the sound generator <NUM>, a control device <NUM>, and the display device <NUM>, as shown in <FIG>. The control device <NUM> corresponds to an example of the force sense display device.

The control device <NUM> controls operation and the like of the slave unit <NUM> and sound generation in the sound generator <NUM>, and controls contents and modes of the display displayed on the display device <NUM>.

The control device <NUM>, as shown in <FIG>, is an information processor such as a computer comprising a CPU, a ROM, a RAM, an input/output interface, and the like. A program stored in a storage device, such as the aforementioned ROM, causes the CPU, the ROM, the RAM, and the input/output interface to cooperate with each other, and function as at least the input portion <NUM>, the control portion <NUM>, the image generator <NUM>, and the output portion <NUM>.

The image generator <NUM> generates image information that is information of video and image to be displayed on the display device <NUM>. Specifically, the image generator <NUM> generates the image information based on signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM>. The image generator <NUM> differs from the image generator <NUM> of the first embodiment in that the image generator <NUM> does not generate the image information based on the signal of video and image inputted from the endoscope unit <NUM>.

Next, control in the medical robot system <NUM> having the above configuration will be described. Control by the control device <NUM> regarding the display of force sense is similar to the control in the first embodiment, and thus the description thereof is omitted.

The image information generated in the image generator <NUM> will now be described. Specifically, description on the mode of display in the display device <NUM> will be given. <FIG> is a schematic diagram illustrating the mode of display by the display device <NUM> based on image signal generated by the image generator <NUM> of the present variation.

The image generator <NUM>, as shown in <FIG>, arranges a first force sense display section <NUM> on the left side and a second force sense display section <NUM> on the right side, on the display surface of the display device <NUM> which is divided into left and right. In other words, as compared with the first embodiment, the present variation differs in that the endoscope image display section <NUM> is not arranged on the display surface of the display device <NUM>.

In the present variation, an example of dividing the display surface into left and right is described, but the display surface may be divided into upper and lower sides. The mode of dividing the display surface is not limited. In the present variation, the first force sense display section <NUM> has a display area equal to a display area of the second force sense display section <NUM>. Alternatively, the first force sense display section <NUM> may have a smaller or larger display area than the display area of the second force sense display section <NUM>.

Further, in the present variation, the first force sense display section <NUM> and the second force sense display section <NUM> are both surrounded by the same rectangular frames. Alternatively, the first force sense display section <NUM> and the second force sense display section <NUM> may have frames different in shape from each other or may have frames having shapes other than the rectangular shape. The shape of the frame is not specifically limited. Also, the first force sense display section <NUM> and the second force sense display section <NUM> may have frames in the same color or different colors.

The display modes of the first force sense display section <NUM> and the second force sense display section <NUM> of the present variation are similar to the display modes in the first force sense display section <NUM> and the second force sense display section <NUM> of the first embodiment, and thus the description thereof is omitted.

For the above-described control device <NUM>, the endoscope unit <NUM> is not an essential element. Therefore, as compared with the medical robot system <NUM> of the first embodiment, it is easy to use the medical robot system <NUM> in surgery which does not use the endoscope unit <NUM>.

Next, a medical robot system according to a second variation of the first embodiment of the present disclosure will be described with reference to <FIG> and <FIG>. The medical robot system and the force sense display device of the present variation have basic configurations similar to those of the first embodiment, but differ from the first embodiment in the number of display devices provided. Accordingly, for the present variation, configuration related to the display of force sense will be described with reference to <FIG> and <FIG>, and other descriptions are omitted.

A medical robot system <NUM> of the present variation comprises the slave unit <NUM>, the endoscope unit <NUM>, the operation unit <NUM>, the sound generator <NUM>, a control device <NUM>, a first display device 260A, and a second display device 260B, as shown in <FIG>. The control device <NUM> corresponds to the force sense display device, and the first display device 260A and the second display device 260B correspond to an example of the display portion.

The control device <NUM> controls operation and the like of the slave unit <NUM> and sound generation in the sound generator <NUM>, and controls contents and modes of displays displayed on the first display device 260A and the second display device 260B.

The control device <NUM> controls operation and the like of the slave unit <NUM> and sound generation in the sound generator <NUM>, and controls contents and modes of display displayed on the display device <NUM>.

The image generator <NUM> generates image information that is information of videos and images to be displayed on the first display device 260A and the second display device 260B. Specifically, the image generator <NUM> generates the image information based on the signal of video and image inputted from the endoscope unit <NUM> to be outputted to the first display device 260A. The image generator <NUM> generates the image information based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> to be outputted to the second display device 260B. Detail of the image information will be described later.

The first display device 260A and the second display device 260B display the videos and images based on the image information outputted from the control device <NUM>. The first display device 260A and the second display device 260B display the videos and images to those involved in surgery including an operator of endoscopic surgery. There is no specific limitation in method and configuration of the first display device 260A and the second display device 260B as long as they display the videos and images.

The first display device 260A and the second display device 260B may have the display surfaces of the same size, or one may have a larger display surface than the other. Also, the first display device 260A and the second display device 260B may have the same method and configuration to display video and image, or may have different methods and configurations.

Next, control in the medical robot system <NUM> having the above configuration will be described. Control by the control device <NUM> regarding the display of the force sense is similar to the control in the first embodiment, and thus the description thereof is omitted.

The image information generated in the image generator <NUM> will now be described. Specifically, the modes of display in the first display device 260A and the second display device 260B will be described. <FIG> is a schematic diagram illustrating the modes of display by the first display device 260A and the second display device 260B based on the image signal generated by the image generator <NUM> of the present variation.

The image generator <NUM>, as shown in <FIG>, arranges an endoscope image display section <NUM> on a display surface in the first display device 260A. Also, the image generator <NUM> arranges a first force sense display section <NUM> on the left side and a second force sense display section <NUM> on the right side, on the display surface of the second display device 260B that is divided into left and right.

In the present variation, the endoscope image display section <NUM> is arranged in the first display device 260A, and the first force sense display section <NUM> and the second force sense display section <NUM> are arranged in the second display device 260B. Alternatively, the first force sense display section <NUM> and the second force sense display section <NUM> may be arranged in the first display device 260A, and the endoscope image display section <NUM> may be arranged in the second display device 260B.

In the above-described control device <NUM>, the display device where the endoscope image display section <NUM> is arranged differs from the display device where the first force sense display section <NUM> and the second force sense display section <NUM> are arranged. Thus, visibility of endoscope image and visibility of force sense display are easily improved.

Next, a medical robot system and a force sense display device according to a second embodiment of the present disclosure will be described with reference to <FIG>. The medical robot system and the force sense display device of the present embodiment have basic configurations similar to those of the first embodiment, but differ from the first embodiment in configuration of the slave unit and mode of force sense display. Accordingly, for the present embodiment, the configuration of the slave unit and the mode of force sense display will be described with reference to <FIG>, and other descriptions are omitted.

A medical robot system <NUM> of the present embodiment comprises a slave unit <NUM>, the endoscope unit <NUM>, the operation unit <NUM>, the sound generator <NUM>, a control device <NUM>, and the display device <NUM>, as shown in <FIG>. The control device <NUM> corresponds to an example of the force sense display device.

The slave unit <NUM>, like the slave unit <NUM> of the first embodiment, supports the surgical tool <NUM> such as forceps used in endoscopic surgery, and controls at least one of the position and posture of the surgical tool <NUM> in accordance with the operation inputted to the operation unit <NUM>. In addition, when the surgical tool <NUM> is forceps, the slave unit <NUM> opens and closes the forceps.

In the present embodiment, the slave unit <NUM> includes the first slave device 10A and the second slave device 10B, and a third slave device 10C.

In the present embodiment, the first slave device 10A, the second slave device 10B, and the third slave device 10C are robot arms. Further, the third slave device 10C has the same configuration as those of the first slave device 10A and the second slave device 10B, and supports a similar type of surgical tool <NUM>.

The third slave device 10C, like the first slave device 10A and the second slave device 10B, only needs to be able to control at least one of the position and posture of the surgical tool <NUM>, or open and close forceps as the surgical tool <NUM>. Also, the third slave device 10C need not have an arm-like configuration.

As shown in <FIG>, the third slave device 10C, like the first slave device 10A and the second slave device 10B, is provided with at least the cartridge <NUM>, the shaft <NUM>, and the joint <NUM>. When it is not necessary to distinguish whether the cartridge <NUM>, the shaft <NUM>, and the joint <NUM> are provided in the first slave device 10A, in the second slave device 10B, or in the third slave device 10C, these are referred to as cartridge <NUM>, shaft <NUM>, and joint <NUM>.

When it is necessary to distinguish whether the cartridge <NUM>, the shaft <NUM>, and the joint <NUM> are provided in the first slave device 10A, in the second slave device 10B or in the third slave device 10C, these are referred to as cartridge 11A, shaft 12A, and joint 13A, cartridge 11B, shaft 12B, and joint 13B, or cartridge 11C, shaft 12C, and joint 13C, respectively.

Further, as shown in <FIG>, the third slave device 10C, like the first slave device 10A and the second slave device 10B, is provided with the shaft sensor <NUM> that measures an external force applied to the shaft <NUM>, the joint sensor <NUM> that measures an external force applied to the joint <NUM>, and the surgical tool sensor <NUM> that measures opening and closing forces applied to the outside by the forceps as the surgical tool <NUM>.

When the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> provided in the third slave device 10C are distinguished, these sensors are referred to as shaft sensor 15C, joint sensor 16C, and surgical tool sensor 17C.

The control device <NUM>, as shown in <FIG>, is an information processor such as a computer comprising a CPU, a ROM, a RAM, an input/output interface and the like. A program stored in a storage device, such as the aforementioned ROM, causes the CPU, the ROM, the RAM, and the input/output interface to cooperate with each other, and function as at least the input portion <NUM>, the control portion <NUM>, an image generator <NUM>, and the output portion <NUM>.

The image generator <NUM> generates image information that is information of video and image to be displayed on the display device <NUM>. Specifically, the image generator <NUM> generates the image information based on the signal of video and image inputted from the endoscope unit <NUM>, and the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM>. Detail of the image information will be described later.

The image information generated in the image generator <NUM> will now be described. Specifically, the mode of display in the display device <NUM> will be described. <FIG> is a schematic diagram illustrating the mode of display displayed by the display device <NUM> based on the image signal generated by the image generator <NUM> of the present variation.

The image generator <NUM>, as shown in <FIG>, arranges an endoscope image display section <NUM> on the left and a force sense display section <NUM> on the right in a right-left direction on the display surface of the display device <NUM>. Inside the force sense display section <NUM>, a first force sense display section <NUM>, a second force sense display section <NUM>, and a third force sense display section <NUM> are arranged in an order from left to right.

The endoscope image display section <NUM> is a region having a larger area than the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM>. The image generator <NUM> generates image information for displaying the information of the video or image inputted from the endoscope unit <NUM> in the endoscope image display section <NUM>.

The first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are regions arranged inside the force sense display section <NUM>. The first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are regions having narrower areas than the endoscope image display section <NUM>. The image generator <NUM> generates the image information based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM>.

The first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are partitioned from other regions by frames. Inside the partitioned first force sense display section <NUM>, second force sense display section <NUM>, and third force sense display section <NUM>, displays are performed based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM>. The first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> of the present embodiment have frames formed in the same rectangular shapes. The first force sense display section <NUM> and the second force sense display section <NUM> have the frames of the same color, and the third force sense display section <NUM> has the frame of different color than those of the first force sense display section <NUM> and the second force sense display section <NUM>. Display modes in the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> will be described later.

In the present embodiment, the first force sense display section <NUM> is a region where the display is performed based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> corresponding to the first slave device 10A. The second force sense display section <NUM> is a region where the display is performed based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> corresponding to the second slave device 10B. The third force sense display section <NUM> is a region where the display is performed based on the signals inputted from the shaft sensor <NUM>, the joint sensor <NUM>, and the surgical tool sensor <NUM> corresponding to the third slave device 10C.

Next, the display modes in the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> will be described. Since the display modes in the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are the same, the display mode in the first force sense display section <NUM> will be described as a representative example, and description of the display modes in the second force sense display section <NUM> and the third force sense display section <NUM> is omitted.

In the present embodiment, in order to easily distinguish between the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM>, a title Left Arm is displayed in the first force sense display section <NUM>, a title Right Arm is displayed in the second force sense display section <NUM>, and a title Third Arm is displayed in the third force sense display section <NUM>.

The display modes of the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are as shown in <FIG>. In the first force sense display section <NUM>, three-dimensional models of the shaft <NUM>, the joint <NUM>, and the surgical tool <NUM> of the first slave device 10A are displayed. In other words, a three-dimensional model of shaft <NUM>, a three-dimensional model of joint <NUM>, and a three-dimensional model of surgical tool <NUM> are displayed. The displayed three-dimensional models reflect the actual posture of the shaft <NUM>, and the bent state of the joint <NUM>.

In the second force sense display section <NUM> and the third force sense display section <NUM> as well, the three-dimensional models of the shafts <NUM>, the joints <NUM>, and the surgical tools <NUM> of the second slave device 10B and the third slave device 10C are displayed, respectively.

The shape of the surgical tool <NUM> in the three-dimensional model is determined based on the type information indicating the type of the surgical tool <NUM> inputted via the input portion <NUM>. The control device <NUM> stores the type information indicating the type of the surgical tool <NUM> and the shape of the surgical tool <NUM> in the displayed three-dimensional model associated with each other. The image generator <NUM> acquires information of the shape of the surgical tool <NUM> associated with the inputted type information, and generates image information for displaying the three-dimensional model.

The type information inputted via the input portion <NUM> may be information inputted by such as a worker that performs settings of the medical robot system <NUM> or an operator that performs endoscopic surgery, or may be a detection signal outputted from a sensor that detects the type of the attached surgical tool <NUM>.

At a position adjacent to the shaft <NUM> in the three-dimensional model, a bar graph 315f showing the magnitude of the force is displayed based on the signal inputted from the shaft sensor <NUM>. In the present embodiment, "Force (Shaft)" is displayed at a position to display the bar graph 315f, and "base" is displayed at a position of a reference line of the bar graph 315f.

Also, at a position adjacent to the joint <NUM> in the three-dimensional model, a bar graph 316f showing the magnitude of the force is displayed based on the signal inputted from the joint sensor <NUM>. In the present embodiment, "Force (Wrist)" is displayed at a position to display the bar graph 316f, and "base" is displayed at a position of a reference line of the bar graph 316f.

In the present embodiment, the bar graph 315f and the bar graph 316f are graphs that extend laterally, and lengths of the bar graph 315f and the bar graph 316f correspond to the magnitudes of the forces. Also, colors of the bar graph 315f and the bar graph 316f are changed in accordance with the magnitudes of the forces. For example, as the force increases, the colors are changed among green, yellow, and red.

At a position adjacent to the surgical tool <NUM> in the three-dimensional model, a fan-shaped figure 314f showing the magnitude of the force to close or open the forceps as the surgical tool <NUM> is displayed based on the signal inputted from the surgical tool sensor <NUM>.

In the present embodiment, the fan-shaped figure 314f is displayed in a semicircular frame. The semicircular frame comprises a frame extending straight in a horizontal direction, and an upwardly convex arc-shaped frame. Also, there is a reference line extending upward from the center of the frame extending in the horizontal direction.

In the present embodiment, the force to close the surgical tool <NUM> is represented in the grip display mode in which the fan-shaped figure 314f is displayed on the left side of the reference line. Also, the force to open the forceps as the surgical tool <NUM> is represented in the dissection display mode in which the fan-shaped figure 314f is displayed on the right side of the reference line. The force to close the surgical tool <NUM> may be represented by displaying the fan-shaped figure 314f on the right side of the reference line, and the force to open the forceps as the surgical tool <NUM> may be represented by displaying the fan-shaped figure 314f on the left side of the reference line.

The magnitude of the force is represented by an area of the fan-shaped figure 314f. In other words, the magnitude of the force is represented by a magnitude of an opening angle from the reference line. For example, as the force increases, the area of the fan-shaped figure 314f increases. As the force decreases, the area of the fan-shaped figure 314f decreases.

Color of the fan-shaped figure 314f may be changed in accordance with the magnitude of the force. For example, as the force increases, the color is changed among green, yellow, and red. When the force is not measured by the surgical tool sensor <NUM>, only the semicircular frame and the reference line are displayed, and the fan-shaped figure 314f is not displayed.

With the above-described control device <NUM>, inside the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> respectively provided in accordance with the first slave device 10A, the second slave device 10B, and the third slave device 10C, the external forces measured in the respective slave devices are displayed. Thus, even if there are two or more slave devices, it is easy to grasp which slave device has measured the external force of the displayed information.

The shapes and hues of at least part of the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are differentiated in accordance with the first slave device 10A, the second slave device 10B, and the third slave device 10C. Thus, it is easy to grasp which slave device has measured the external force of the displayed information.

The images showing the shapes of the surgical tools <NUM> gripped by the respective first slave device 10A, second slave device 10B, and third slave device 10C are displayed in the corresponding first force sense display section <NUM>, second force sense display section <NUM> and third force sense display section <NUM>. Thus, it becomes easy to grasp which slave device gripping which surgical tool <NUM> has measured the external force of the displayed information.

Next, a medical robot system according to a first variation of the second embodiment of the present disclosure will be described with reference to <FIG> and <FIG>. The medical robot system and the force sense display device of the present variation have basic configurations similar to those of the second embodiment, but differ from the second embodiment in mode of display in the force sense display section. Accordingly, for the present variation, the mode of display in the force sense display section will be described with reference to <FIG> and <FIG>, and other descriptions are omitted.

The control device <NUM> controls operation and the like of the slave unit <NUM> and sound generation in the sound generator <NUM>, and controls contents and modes of display to be displayed on the display device <NUM>.

The control device <NUM>, as shown in <FIG>, is an information processor such as a computer comprising a CPU, a ROM, a RAM, an input/output interface, and the like. A program stored in a storage device, such as the aforementioned ROM, causes the CPU, the ROM, the RAM, and the input/output interface to cooperate with each other, and function as at least the input portion <NUM>, the control portion <NUM>, an image generator <NUM>, and the output portion <NUM>.

Next, control in the medical robot system <NUM> having the above configuration will be described. Control by the control device <NUM> regarding the display of force sense is similar to the control in the first embodiment, and thus the description thereof is omitted. Also, the mode of display in the display device <NUM> is similar to that in the second embodiment, and thus the description thereof is omitted.

The display modes of the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are as shown in <FIG>. In the first force sense display section <NUM>, three-dimensional models of the shaft <NUM>, the joint <NUM>, and the surgical tool <NUM> of the first slave device 10A are displayed. In other words, a three-dimensional model of shaft <NUM>, a three-dimensional model of joint <NUM>, and a three-dimensional model of surgical tool <NUM> are displayed. The displayed three-dimensional models reflect the actual posture of the shaft <NUM>, and the actual bent state of the joint <NUM>.

In the second force sense display section <NUM> and the third force sense display section <NUM> as well, three-dimensional models of the shafts <NUM>, the joints <NUM>, and the surgical tools <NUM> of the second slave device 10B and the third slave device 10C are displayed, respectively.

At the shaft <NUM> in the three-dimensional model, a cylinder graph 415f with a conical tip showing the magnitude of the force and the direction of the force is displayed based on the signal inputted from the shaft sensor <NUM>.

Also, at the joint <NUM> in the three-dimensional model, a cylinder graph 416f with a conical tip showing the magnitude of the force and the direction of the force is displayed based on the signal inputted from the joint sensor <NUM>.

In the present embodiment, lengths of the cylinder graph 415f and the cylinder graph 416f correspond to the magnitudes of the forces, and the directions in which the cylinder graph 415f and the cylinder graph 416f extend correspond to the directions of the forces. Also, colors of the cylinder graph 415f and the cylinder graph 416f may be changed in accordance of the magnitudes of the forces. For example, as the force increases, the color may be changed among green, yellow, and red.

At a position adjacent to the surgical tool <NUM> in the three-dimensional model, a fan-shaped figure 414f showing the magnitude of the force to close or open the forceps as the surgical tool <NUM> is displayed based on the signal inputted from the surgical tool sensor <NUM>.

In the present embodiment, the force to close the surgical tool <NUM> is represented in the grip display mode in which one fan-shaped figure 414f is displayed. Also, the force to open the forceps as the surgical tool <NUM> is represented in the dissection display mode in which the fan-shaped figures 414f split in two are displayed.

Further, the fan-shaped figure 414f may be displayed to include a reference line when the number of the fan-shaped figure 414f to be displayed is one. When the number of the fan-shaped figures 414f is two, the fan-shaped figures 414f may be displayed to be spaced apart across the reference line.

The magnitude of the force is represented by an area of the fan-shaped figure 414f. For example, as the force increases, the area of the fan-shaped figure 414f is increased. As the force decreases, the area of the fan-shaped figure 414f is decreased.

Alternatively, color of the fan-shaped figure 414f may be changed in accordance with the magnitude of the force. For example, as the force increases, the color is changed among green, yellow, and red. When the force is not measured by the surgical tool sensor <NUM>, the fan-shaped figure 414f is not displayed.

With the above-described control device <NUM>, visual recognition of information becomes easy since whether the displayed information relates to the gripping force or to the dissecting force is known from whether the number of fan-shaped figures 414f is one or two. Also, the area of the fan-shaped figure 414f is displayed as the magnitude of the gripping force or the dissecting force. Thus, visual recognition of the magnitude of the gripping force or the dissecting force is easy.

Further, when the number of the displayed fan-shaped figures 414f is one, the fan-shaped figure 414f is displayed to include the reference line, and, when the number of the displayed fan-shaped figures 414f is two, the fan-shaped figures 414f are displayed to be spaced apart across the reference line. Thus, visual recognition of the information is further facilitated.

Next, a medical robot system according to a second variation of the second embodiment of the present disclosure will be described with reference to <FIG> and <FIG>. The medical robot system and the force sense display device of the present variation have basic configurations similar to those of the second embodiment, but differ from the second embodiment in mode of display in the force sense display section. Accordingly, for the present variation, the mode of display in the force sense display section will be described with reference to <FIG> and <FIG>, and other descriptions are omitted.

A medical robot system <NUM> of the present embodiment comprises the slave unit <NUM>, the endoscope unit <NUM>, the operation unit <NUM>, the sound generator <NUM>, a control device <NUM>, and the display device <NUM>, as shown in <FIG>. The control device <NUM> corresponds to an example of the force sense display device.

The control device <NUM> controls operation and the like of the slave unit <NUM> and sound generation in the sound generator <NUM>. Also, the control device <NUM> controls contents and modes of display displayed on the display device <NUM>.

The display modes in the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> will now be described. Since the display modes in the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are the same, the display mode in the first force sense display section <NUM> will be described as a representative example, and description of the display modes in the second force sense display section <NUM> and the third force sense display section <NUM> is omitted.

At the shaft <NUM> in the three-dimensional model, a circular graph 515f showing the magnitude of the force and the direction of the force is displayed based on the signal inputted from the shaft sensor <NUM>.

Also, at the joint <NUM> in the three-dimensional model, a circular graph 516f showing the magnitude of the force and the direction of the force is displayed based on the signal inputted from the joint sensor <NUM>. In the present embodiment, the circular graph 515f and the circular graph 516f are partially cut out, but may have a shape without cutout.

Also, in the present embodiment, colors displayed in the circular graph 515f and the circular graph 516f correspond to the magnitudes of the forces, and also correspond to the directions in which the forces act. Specifically, in each of the circular graph 515f and the circular graph 516f, the direction in which the force acts is displayed by changing the color of a region corresponding to the direction in which the force acts. Further, as the force increases, the color of the region corresponding to the direction in which the force acts is changed among green, yellow, and red.

At a position adjacent to the surgical tool <NUM> in the three-dimensional model, a fan-shaped figure 514f showing the magnitude of the force to close or open the forceps as the surgical tool <NUM> is displayed based on the signal inputted from the surgical tool sensor <NUM>.

In the present embodiment, the force to close the surgical tool <NUM> is represented in the grip display mode in which the fan-shaped figure 514f is displayed in red. Also, the force to open the forceps as the surgical tool <NUM> is represented in the dissection display mode in which the fan-shaped figure 514f is displayed in blue.

The magnitude of the force is represented by an area of the fan-shaped figure 514f. For example, as the force increases, the area of the fan-shaped figure 514f is increased. As the force decreases, the area of the fan-shaped figure 514f is decreased.

Alternatively, shade of the color of the fan-shaped figure 514f may be changed in accordance with the magnitude of the force. For example, as the force increases, the color becomes darker. As the force decreases, the color becomes lighter. When the force is not measured by the surgical tool sensor <NUM>, the fan-shaped figure 514f is not displayed.

With the above-described control device <NUM>, visual recognition of information becomes easy since the fan-shaped figure 514f is displayed with different colors or hues. For example, it becomes possible to confirm the fan-shaped figure 514f displayed in the grip display mode and the dissection display mode by the displayed color or hue.

It becomes easy to grasp the magnitude of the force in the fan-shaped figure 514f by changing the shade of the displayed color or hue based on the magnitude of the force in the fan-shaped figure 514f. Thus, it becomes easy to grasp the magnitude of the displayed gripping force or dissecting force.

The color or hue in the fan-shaped figure 514f may be defined based on the type of the surgical tool <NUM>. This makes it easy to visually recognize the information. For the type of the surgical tool <NUM>, the type information showing the type of the surgical tool <NUM> inputted via the input portion <NUM> can be used as in the second embodiment.

Next, a medical robot system according to a third variation of the second embodiment of the present disclosure will be described with reference to <FIG> and <FIG>. The medical robot system and the force sense display device of the present variation have basic configurations similar to those of the second embodiment, but differ from the second embodiment in mode of display in the force sense display section. Accordingly, for the present variation, the mode of display in the force sense display section will be described with reference to <FIG> and <FIG>, and other descriptions are omitted.

The display modes in the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are as shown in <FIG>. In the lower parts of the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM>, circular figures 615f showing the magnitudes of the forces are displayed based on the signals inputted from the shaft sensors <NUM>. In the present embodiment, "Force (Shaft)" is displayed at a position to display the circular figures 615f.

In the middle parts of the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM>, circular figures 616f showing the magnitudes of the forces are displayed based on the signals inputted from the joint sensors <NUM>. In the present embodiment, "Force (Wrist)" is displayed at a position to display the circular figures 616f.

In the upper parts of the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM>, circular figures 614f showing the magnitudes of the forces are displayed based on the signals inputted from the surgical tool sensors <NUM>. In the present embodiment, Grip is displayed at a position to display the circular figures 614f.

In the present embodiment, colors of the circular figures 615f, the circular figures 616f, and the circular figures 614f are changed in accordance with the magnitudes of the forces. For example, as the force increases, the color is changed among green, yellow, and red (or orange).

With the above-described control device <NUM>, the magnitude of the value of the external force is represented by the color. Therefore, as compared with the case where the magnitude of the value of the external force is represented by a numerical value, it becomes easy to intuitively grasp the magnitude of the value of the external force.

Next, a medical robot system according to a fourth variation of the second embodiment of the present disclosure will be described with reference to <FIG> and <FIG>. The medical robot system and the force sense display device of the present variation have basic configurations similar to those of the second embodiment, but differ from the second embodiment in mode of display in the force sense display section. Accordingly, for the present variation, the mode of display in the force sense display section will be described with reference to <FIG> and <FIG>, and other descriptions are omitted.

The display modes in the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM> are as shown in <FIG>. In the lower parts of the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM>, pie charts 715f showing the magnitudes of the forces are displayed based on the signals inputted from the shaft sensors <NUM>. In the present embodiment, "Force (Shaft)" is displayed at a position to display the pie charts 715f.

In the middle parts of the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM>, pie charts 716f showing the magnitudes of the forces are displayed based on the signals inputted from the joint sensors <NUM>. In the present embodiment, "Force (Wrist)" is displayed at a position to display the pie charts 716f.

In the upper parts of the first force sense display section <NUM>, the second force sense display section <NUM>, and the third force sense display section <NUM>, pie charts 714f showing the magnitudes of the forces are displayed based on the signals inputted from the surgical tool sensor <NUM>. In the present embodiment, Grip is displayed at a position to display the pie charts 714f.

In the present embodiment, areas and colors of the pie charts 715f, the pie charts 716f, and the pie charts 714f are changed in accordance with the magnitudes of the forces. For example, as the force increases, the area of the pie chart is increased and the color is changed among green, yellow, and red (or orange).

Claim 1:
A systemfor displaying a force acting on a surgical tool configured to be closed to grip and opened to dissect an object so that the force is perceptible to an operator, the system comprising:
a display device (<NUM>, 260A, 260B);
a control device (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>); and
a surgical tool sensor (<NUM>, 17A, 17B) configured to measure opening and closing forces applied to an outside by the surgical tool (<NUM>);
the control device (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) comprising:
an input portion (<NUM>) configured to receive input from the surgical tool sensor (<NUM>, 17A, 17B), of a measurement signal of a gripping force, which is a force to grip the object, or a dissecting force, which is a force to dissect the object, in the surgical tool (<NUM>) depending on a direction in which the surgical tool (<NUM>) is actuated;
an image generator (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) configured to generate image information for displaying information on the gripping force as a grip display mode and information on the dissecting force as a dissection display mode different from the grip display mode based on the measurement signals; and
an output portion (<NUM>) configured to output the generated image information to the display device (<NUM>, 260A, 260B), the display device (<NUM>, 260A, 260B) being configured to display the image information.