Source: https://insight.rpxcorp.com/pat/US9089972B2
Timestamp: 2019-10-15 11:22:02
Document Index: 706935523

Matched Legal Cases: ['art12', 'art12', 'art12', 'art12', 'art12', 'arts12', 'arts12', 'arts12', 'art12', 'arts12', 'arts12', 'art12', 'art12', 'art12', 'art12', 'art12', 'art12', 'art12', 'art12', 'art12']

Patent US 9,089,972 B2
a cart, said cart includes a robot face that has a robot monitor, a robot camera, a robot speaker, a robot microphone, and an overhead camera; and
, a remote station that is coupled to said robot face and said overhead camera, said remote station includes a station monitor, a station camera, a station speaker and a station microphone.
Surgical instrument end portion
D760387S1
D767130S1
D767129S1
D768295S1
moving a cart, the cart includes a robot face that has a robot monitor, a robot camera, a robot speaker, a robot microphone, and an overhead camera; and
, transmitting to a remote station a video image captured by the robot camera or a video image captured by the overhead camera, the remote station includes a station monitor, a station camera, a station speaker and a station microphone.
FIG. 1is an illustration of a tele-presence system;
FIG. 2is a perspective view of a cart of the system;
FIG. 3is a rear view of an articulated arm and a robot face of the cart;
FIG. 4is an enlarged perspective view of an overhead camera of the cart;
FIG. 5is an illustration of a display user interface of a remote station;
FIG. 6is an illustration of the display user interface showing video images captured by a robot camera and an overhead camera being simultaneously displayed.
Referring to the drawings more particularly by reference numbers, FIGS. 1, 2and3show a tele-presence system10. The system10includes a cart12that is coupled to a remote control station14. The cart12has a robot face16and an overhead camera18. The remote control station14may be coupled to the cart12through a network20. By way of example, the network20may be either a packet switched network such as the Internet, or a circuit switched network such as a Public Switched Telephone Network (PSTN) or other broadband system. Alternatively, the cart12may be coupled to the remote station14network thru a satellite.
The remote control station14may include a computer22that has a monitor24, a camera26, a microphone28and a speaker30. The computer22may also contain an input device32such as a joystick or a mouse. The control station14is typically located in a place that is remote from the cart12. Although only one remote control station14is shown, the system10may include a plurality of remote stations14. In general any number of carts12may be coupled to any number of remote stations14or other carts12. For example, one remote station14may be coupled to a plurality of carts12, or one cart12may be coupled to a plurality of remote stations14, or a plurality of carts12. The system may include an arbitrator (not shown) that control access between the carts12and the remote stations14.
As shown inFIG. 3, the cart12may include an articulated arm40that supports and can move the robot face16. The articulated arm40may have active joints42and44that allow the robot face16to be panned and tilted, respectively. The active joints42and44may move in response to commands provided by the remote station. The joints42and44may contain position sensors46and48, respectively, that provide positional feedback of the arm40.
Referring toFIGS. 2 and 3, each robot face16includes a camera(s) 50, a monitor52, a microphone(s) 54and a speaker(s) 56. The robot camera50is coupled to the remote monitor24so that a user at the remote station14can view a video image captured by the robot camera50. Likewise, the robot monitor52is coupled to the remote camera26so personnel at the surgical site may view the user of the remote station14. The microphones28and54, and speakers30and56, allow for audible communication between the system operator and the personnel at the surgical site.
The overhead camera18may be coupled to a boom60. The boom60may include a number of joints62, either active or passive. The joints62may include positional sensors to provide feedback regarding the position of the overhead camera18.
As shown inFIG. 4, the cart12may include an overhead microphone64and a detachable handle66. The overhead microphone64may provide an alternative source of sound. The detachable handle66can be used to move the boom60and overhead camera18. If the cart12is used in a sterile field, for example in an operating room, the handle66may be replaced with a sterile handle before each medical procedure to allow a surgeon within the sterile field to position the boom during a procedure.
Referring again toFIG. 2, the cart12may include a linear actuator70that can be remotely or locally actuated to vary the height of the robot face16and overhead camera18. Varying the height allows the cart12to be rolled through doors and then actuated to move the face16and camera18to elevated positions. For example, the face16and camera18can be lowered to allow the cart12to be moved into an operating room. The camera18can then be raised to provide a desirable view over an operating table. The cart12may include a laser pointer72and/or directed lighting (not shown) located on the boom60. The cart12may also include a local control panel74to move the articulated arm40, actuator70and/or boom60. The linear actuator70is also advantageous in moving the face16to be essentially at the same level as a person whether they are standing, sitting or lying in a prone position.
The robot face16may include a processor, hard disk drive and other circuits that enable the face16to function as a computer. The face16may include an input panel76that allows a user to provide input. By way of example, the operator of the remote station may provide one or more questions through the robot face16, wherein a user of the cart provides answers through the input panel76.
The system10may have certain components and software that are the same or similar to a robotic system provided by the assignee InTouch Technologies, Inc. of Goleta, Calif. under the name RP-7 and embodies a system described in U.S. Pat. No. 6,925,357, which is hereby incorporated by reference.
FIG. 5shows a display user interface (“DUI”) 120that can be displayed at the remote station14. The DUI120may include a robot view field122that displays a video image captured by the robot camera and/or the overhead camera. The DUI120may also include a station view field124that displays a video image provided by the camera of the remote station14. The DUI120may be part of an application program stored and operated by the computer22of the remote station14.
The DUI120may include a graphical switch126that allows the user to select between the video image provided by the robot camera and the video image provided by the overhead camera. The DUI120may also have a graphical switch128that allows the user to select the simultaneous display of the video images from the robot and overhead cameras as shown inFIG. 6. The video images from both cameras can be streamed to the remote station from the cart. The images can be merged by presenting a center rectangle of each image (e.g., 320×480 center area). A zoom or highlighting feature may be utilized by manipulating a cursor on either image. The system may also automatically pan a camera when the cursor is moved out of the displayed field of view.
The DUI120may have a graphical switch130that allows the user to switch between sound captured by the robot microphone or the overhead microphone. The system may automatically switch between microphones based on a characteristic(s) of the sound captured by the microphones. For example, the system may switch to the microphone that provides the highest aural clarity, or to the microphone that is in the closest proximity to a person or object generating the sound.
The DUI120may include a location display138that provides the location of the robot face. The CHANGE button140can be selected to change the default robot face in a new session. The CHANGE button140can be used to select and control a different robot face in a system that has multiple robot faces. The user can initiate and terminate a session by selecting box142. The box142changes from CONNECT to DISCONNECT when the user selects the box to initiate a session. System settings and support can be selected through buttons144and146.
Both the robot view field122and the station view field124may have associated graphics to vary the video and audio displays. Each field may have an associated graphical audio slide bar148to vary the audio level of a selected microphone and another slide bar152to vary the volume of the speakers.
The DUI120may have slide bars150, 154and156to vary the zoom, focus and brightness of a selected camera, respectively. A still picture may be taken at either the robot face or remote station by selecting one of the graphical camera icons158. The still picture may be the image presented at the corresponding field122or124at the time the camera icon158is selected. Capturing and playing back video can be taken through graphical icons160. A return to real time video can be resumed, after the taking of a still picture, captured video, or reviewing a slide show, by selecting a graphical LIVE button162.
A still picture can be loaded from disk for viewing through selection of icon164. Stored still images can be reviewed by selecting buttons166. The number of the image displayed relative to the total number of images is shown by graphical boxes168. The user can rapidly move through the still images in a slide show fashion or move through a captured video clip by moving the slide bar170. A captured video image can be paused through the selection of circle174. Play can be resumed through the same button174. Video or still images may be dismissed from the active list through button172. Video or still images may be transferred to the robot by selecting icon176. For example, a doctor at the remote station may transfer an x-ray to the screen of the robot.
The system may provide the ability to annotate184the image displayed in field122and/or124. For example, a doctor at the remote station may annotate some portion of the image captured by the robot face camera. The annotated image may be stored by the system. The system may also allow for annotation of images sent to the robot face through icon176. For example, a doctor may send an x-ray to the robot face which is displayed by the robot screen. The doctor can annotate the x-ray to point out a portion of the x-ray to personnel located at the robot site. This can assist in allowing the doctor to instruct personnel at the robot site.
The display user interface may include graphical inputs186that allow the operator to turn the views of the remote station and remote cameras on and off.
Referring toFIG. 1, the cart12can be used in an operating room. By way of example, the boom60can be moved to place the overhead camera18above an operating table200. The overhead camera18may be located above a sterile field. The robot face16may be placed adjacent to the sterile field. With such a configuration, personnel may conduct two-way video conferencing through the robot face16. The overhead camera18may provide a more desirable view of the patient and operating procedure. This would allow a physician at the remote station to view the procedure and have a video conference to provide instructions, mentoring, etc. to personnel at the surgical site.
Stuart, David, Sanchez, Daniel Steven, Lai, Fuji, Hanrahan, Kevin, Jordan, Charles S., Roe, David, Rosenthal, James, Mangaser, Amante, Whitney, Blair, Walters, Derek
294/070.4, 294/50, 244/3.15, 348/14.05, 348/87, 348/135, 348/143, 348/14.16, 348/86, 381/122, 382/153, 700/247, 700/250, 700/259, 700/260, 705/2, 705/400, 705/1.1, 705/14.16, 706/11