Abstract:
An unmanned systems operator control system includes a hand held controller with a set of switches and control enumeration software specially configured to report a superset of virtual switches based on the physical switches. A core unit includes a first unmanned system control application subscribing to a first switch subset of the superset and outputting commands controlling a first unmanned system based on activation of the set of switches. A second unmanned system control application subscribes to a second switch subset of the superset and outputs commands controlling a second unmanned system based on activation of the set of switches. A mode switching subsystem is configured, in a first state, to map the set of switches to the first switch subset and, in a second state, to map the set of switches to the second switch subset.

Description:
GOVERNMENT RIGHTS 
       [0001]    This invention was made with U.S. Government support under Contract No. #N00178-11-C-1025 awarded by Naval Surface Warfare Center. The Government may have certain rights in aspects of the invention. 
     
    
     FIELD OF THE INVENTION 
       [0002]    The subject invention relates to robot controllers. 
       BACKGROUND OF THE INVENTION 
       [0003]    There are a variety of remotely controlled mobile robots available and each typically include their own specific “Operator Control Unit” (OCU) used by an operator to wirelessly drive and steer the robot, operate its arm, maneuver its cameras, and the like. That is, the OCU for the well-known QinetiQ-NA, Inc. “Talon” robot cannot be used as an OCU to control the iRobot Corp. “PackBot” robot and vise-versa. The same is sometimes true with respect to different model robots of even the same company or vendor and also true with respect to unmanned aerial and watercraft vehicles, different remote sensors, weapons, and the like. For example, the OCU of the “Raven” drone cannot be used to control the “Talon” robot. 
         [0004]    The result is an incentive for a so-called “common controller”. The U.S. Army program is called the “AN/PSW-2 Common Controller”. 
         [0005]    U.S. Pat. No. 8,119,109 of iRobot Corp., incorporated herein by this reference, discloses a “twin grip design” input device connected to a rugged laptop computer for controlling teleoperative remote controllable vehicles. The software of the computer is stated to be proprietary but also includes “common OCU software”. How the signals output by the switches of the device are processed by the computer is not disclosed but different button function modes are possible. 
         [0006]    Others are developing tactical control units with scalable war fighter-machine interfaces. Still others are attempting to adapt game type controllers for controlling unmanned aerial and ground vehicles. 
         [0007]    Those skilled in the art have also studied controlling multiple robots or payloads. One example is a soldier controlling a flying drone and, at the same time, controlling a ground robot using one controller. The handheld controller could have one switch set for the ground robot and another switch set for the drone but then the handheld controller would be large and complex. 
         [0008]    U.S. Pat. No. 5,394,168, incorporated herein by this reference, discloses a game controller with buttons for controlling movement of one object and an optical subsystem for controlling movement of a different object. U.S. Pat. No. 8,199,109 states that the controller has “mode changing software” which is not disclosed. The “109 patent also states the controller can be used to control two or more payloads. 
       SUMMARY OF THE INVENTION 
       [0009]    A problem occurs when a “common” OCU includes proprietary hardware and/or software and the different teleoperatable remotely controlled vehicles each include proprietary hardware and software. Arriving in a true common OCU which is able to remotely control vehicles of different types and from different vendors is challenging. 
         [0010]    In one example of this invention, vendors of different remote control vehicles can keep their controlling hardware (e.g., radios and the like) and their controlling software proprietary. The different radios can even transmit at different frequencies, use different protocols, and the like. Source code of proprietary software is not required. The common controller system of one example of the invention can be used to control any remotely controlled vehicle because the common controller system accepts any remote control vehicle&#39;s radio pack and proprietary software for controlling the radio pack. 
         [0011]    The invention features, in one example, the ability to use a commercially available handheld controller without the need for mode changing software. The radios and radio software applications for different unmanned system are accommodated. When the controller system will be used to control a drone, robot, or the like, its radio and software applications are incorporated. 
         [0012]    Featured is an unmanned systems operator control system comprising a set of switches and control enumeration software configured to report a superset of virtual switches. A first unmanned system control application subscribes to a first switch subset of the superset and outputs commands controlling a first unmanned system based on activation of the set of switches. A second unmanned system control application subscribes to a second switch subset of the superset and outputs commands controlling a second unmanned system based on activation of the set of switches. A mode switching subsystem is configured, in a first state, to map the set of switches to the first switch subset and, in a second state, to map the set of switches to the second switch subset. 
         [0013]    In one example, the set of switches are associated with a handheld controller and include at least one joystick and a plurality of X buttons and the superset includes at least two joysticks and 2X buttons. The mode switching subsystem is also typically associated with the handheld controller. One handheld controller further includes a first screen for the first unmanned system and a second screen for the second unmanned system. In one version, the first unmanned system control application and the second unmanned system control application are associated with a core module. A first radio is for transmitting control commands to the first unmanned system. A second radio is for transmitting control commands to the second unmanned system. 
         [0014]    The system may also include a video server and transmitter for wirelessly transmitting video displayed on the video screen(s). 
         [0015]    Also featured is a method of controlling a plurality of unmanned systems with one handheld controller having a set of switches. The set of switches are mapped to a first switch subset of a superset of virtual switches. The method includes subscribing to the first switch subset when controlling a first unmanned system using the set of switches. The set of switches are mapped to a second switch subset of a superset of virtual switches and the second switch subset is subscribed to when controlling the second unmanned system using the set of switches. Preferably, video associated with the first unmanned system is displayed on a first screen and video associated with the second unmanned system is displayed on a second video screen. 
         [0016]    Also featured is a tactical robot controller for first and second robots comprising a first unmanned system control application is configured to output commands controlling a first unmanned system via a first radio. At least a second unmanned system control application system is configured to output commands controlling a second unmanned system via a second radio. A handheld controller includes a set of switches for controlling the first and second unmanned systems and also software delivering switch data to the first unmanned system control application and to the second unmanned system control application. A mode selection is switchable between the first and second unmanned systems. In one preferred embodiment, the handheld controller includes software configured to report a superset of virtual switches, the first unmanned system control application subscribes to a first switch subset of the superset, the second unmanned system control application subscribes to a second switch subset of the superset, and the handheld controller software is configured to map the set of handheld controller switches to the first switch subset when the mode selection is switched to the first unmanned system and to map the set of handheld controller switches to the second switch subset when the mode selection is switched to the second unmanned system. 
         [0017]    Also featured is a tactical robot control method for first and second robots comprising controlling a first unmanned system by delivering switch data to a first unmanned system control application issuing commands to the first unmanned system via a first radio, controlling a second unmanned system by delivering switch data to a second unmanned system control application issuing commands to the second unmanned system via a second radio, and switching between control of the first unmanned system and the second unmanned system. 
         [0018]    The method may further include the step of reporting a superset of virtual switches based on a set of physical switches generating the switch data, subscribing the first unmanned system control application to a first switch subset of the virtual superset, subscribing the second unmanned system control application to a second switch subset of the virtual superset, and switching between control of the first unmanned system and the second unmanned system by mapping the set of physical switches to the first switch subset when controlling the first unmanned system and mapping the set of physical switches to the second switch subset when controlling the second unmanned system. 
         [0019]    The subject invention, however, in other embodiments, need not achieve all these objectives and the claims hereof should not be limited to structures or methods capable of achieving these objectives. 
     
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
         [0020]    Other objects, features and advantages will occur to those skilled in the art from the following description of a preferred embodiment and the accompanying drawings, in which: 
           [0021]      FIG. 1  is a highly schematic view showing an example of a handheld controller in accordance with the invention; 
           [0022]      FIG. 2  is a block diagram showing the primary components associated with a tactical robot controller in accordance with one example of the invention; 
           [0023]      FIG. 3  is a schematic view of the core module shown in block diagram form in  FIG. 2 ; and 
           [0024]      FIG. 4  is a flow chart depicting the primary steps associated with a method in accordance with an example of the invention and the programming of the software disclosed herein. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0025]    Aside from the preferred embodiment or embodiments disclosed below, this invention is capable of other embodiments and of being practiced or being carried out in various ways. Thus, it is to be understood that the invention is not limited in its application to the details of construction and the arrangements of components set forth in the following description or illustrated in the drawings. If only one embodiment is described herein, the claims hereof are not to be limited to that embodiment. Moreover, the claims hereof are not to be read restrictively unless there is clear and convincing evidence manifesting a certain exclusion, restriction, or disclaimer. 
         [0026]      FIG. 1  depicts an example of a handheld controller  20  with first screen  22   a , second screen  22   b , and housing  26  for a set of switches including, in this particular example, joysticks  28   a  and  28   b  and  20  buttons such as button  28   c ,  28   d ,  28   e  and the like. Also featured is mode button  32 . 
         [0027]    In general, the user can control two or more robots such as ground robot  34  depicted on screen  22   a  and flying drone  36  as depicted on screen  22   b . Video from cameras on the robots is typically delivered to screen  22   a  and  22   b  via the robot&#39;s radio set. In one example, in a first mode, joystick  28   a  may be used to control (e.g., turn) ground robot  34  while, in a second mode, joystick  28   a  is used to control (e.g., turn) drone  36 . The different modes are activated by mode button  32 . In other examples, there is only one screen and video from the robot cameras is shown in an alternate fashion on the screen in conjunction with the activation of mode button  32  and/or by splitting the screen. 
         [0028]    One preferred architecture for a tactical robot controller is shown in  FIG. 2  where controller  20  includes one or more touch screens  22 , a set of switches  28 , and processor  40  running control enumeration software  42  as well as other software. Processor  40  may be or include one or more microprocessors, application specific integrated circuits, programmable logic devices, controllers and the like. The control enumeration software is configured to deliver switch data to and output a message (via processor  40  and USB cable  44 ) to core  50  concerning the switch set of handheld controller  20 . In this particular example, control enumeration software  42  would normally generate a USB control enumeration message indicating handheld controller  20  has two joysticks and 20 buttons. 
         [0029]    But, in order to control multiple robots, control enumeration software  42  is specifically configured to report a superset of virtual switches. e.g., four joysticks and 40 buttons. 
         [0030]    Core  50  includes central processing unit  52  (configured for a windows environment, for example) running application software modules such as first unmanned system control application  54   a  and second unmanned system control application  54   b . These applications, in general, generate control signals for their respective robots based on the activation of switches associated with handheld controller  20 . 
         [0031]    In this invention, application  54   a  specially subscribes to a first switch subset of the aforementioned superset of virtual switches. Application  54   b  specially subscribes to a second switch subset of the aforementioned superset of virtual switches. In the example above, application  54   a  may subscribe to the first and second joysticks and buttons  1 - 20 . Application  54   h  then subscribes to the third and fourth joysticks and buttons  21 - 40 . 
         [0032]    When mode selection switch  32  is activated to control the first robot, software  42  maps the actual physical set of controller switches to the first virtual switch subset, here the first and second joysticks and buttons  1 - 20 . When the mode selection  32  is activated to control the second robot, software  42  maps the actual physical set of switches to the second virtual switch subset, here the third and forth joysticks and buttons  21 - 40 . 
         [0033]    In this way, application software  54   a  which subscribes only to the first virtual switch subset controls the first robot  34  (via signals transmitted by radio  56   a ) when the mode for the first robot is selected and application software  54   a  ignores signals from handheld controller  20  when the mode for the second robot is selected since now the handheld controller has mapped its physical switches to the second virtual switch subset which application  54   a  does not subscribe to. Further, application software  54   b  which subscribes only to the second virtual switch subset, controls the second robot  36  (via signals transmitted by radio  56   b ) when the mode for the second robot is selected and application software  54   b  ignores signals from the handheld controller  20  when the mode for the first robot is selected since now the handheld controller has mapped the physical switches to the first virtual switch subset which application  54   b  does not subscribe to. 
         [0034]    If three robots are to be controlled, the superset of virtual switches could be larger and there may be three switch subsets and three control applications each subscribed to a different subset and transmitting signals to three different radios. Note that the subsets need not, when combined, coincide with the superset, there may be overlap between the subsets, and the superset need not coincide with all the physical buttons. 
         [0035]    In one particular example, when the mode for control of robot  34  is selected and joystick  28   a ,  FIG. 1  is toggled to the right, software  42  has mapped joystick  28   a  to joystick No. 1. Software  42  reports to CPU  52  a switch data message that joystick No. 1 has toggled to the right and application  54   a , which subscribes to joystick No. 1, sends a command via radio  56   a  to turn robot  34  to the right. When the mode for the control of robot  36  is selected and joystick  28   a ,  FIG. 1  is toggled to the right, software  42  has mapped joystick  28   a  to joystick No. 3. Software  42  reports to CPU  52  a message that joystick No. 3 has toggled to the right and application  54   b , which subscribes to joystick No. 3, sends a command via radio  56   b  to turn drone  36  so it banks to the right. 
         [0036]    Note how radios  56   a  and control software  54   a  may be provided from the vendor of robot  34  and radio  56   b  and application software  54   b  may be provided from the vendor of robot  36 . By loading a vendor software and carrying out the subscription and mapping steps described above for the physical switches, a true common controller is realized. 
         [0037]      FIG. 3  shows an example of electronic subsystem core module  50  physically and electrically coupled, in one particular example, to power supply module  60  which includes swappable batteries  62   a  and  62   b . Weighing less than 5.5 lbs, this combination can be placed in a small molle pack or pouch coupled to a soldier&#39;s existing molle pack. Core  50  includes heat sink  64 , a Wi-Fi/Bluetooth antenna, and various ports or connectors. Handheld controller  20 ,  FIG. 2  is electrically coupled to core module  50 ,  FIG. 3  via USB cable plugged into connector  66 . Various radio packs such as radio pack  56   a  and  56   b ,  FIG. 2  can be electrically coupled to core  50 ,  FIG. 3  via a cable plugged into connectors  66   a  and  66   b . Other ports are for a headset, other radio packs, and/or a personal computer interface (USB, Ethernet, or the like). Power supply module  60  is shown physically as well as electrically coupled to core module  50  but this is not a necessary limitation of the invention. Power supply module  60  can be coupled to core module  50  via a cable, for example. 
         [0038]    A solider using handheld controller  20 ,  FIG. 1  may have core module  50  and power supply module  60  located in the bottom half of a molle pouch while the radio packs can be located in the upper half of the molle pouch. The molle pouch can be coupled to an assault pack. 
         [0039]    As discussed above, by changing the radio pack and executing different software programs operable on the core module, the solider can use the handheld controller to control different ground robots, different UAVs, remotely controlled watercraft, various pods of an aircraft, sensors, and the like. 
         [0040]    The vendors of such devices typically provide their own radio packs plugged into and software programs loaded onto core module. In order to make a software program operate correctly, interface control software modules can be included using a USB button map provided to the various vendors. A vendor can keep its software programs, protocols, and the configuration of a radio pack proprietary. 
         [0041]    Core module  50  may also include video server  51  electrically connected to a video out lead of CPU  52  (e.g., a 1-2 GHz dual core processor operating Windows XP OS and employing 4 GB of RAM) for digitizing to an MPEG video stream the analog video signal processed by CPU  52 . This MPEG video signal is transmitted by Wi-Fi transmitter  53  so other personnel can view, on a smart phone, for example, the video feed displayed on screens  22   a  and/or  22   b.    
         [0042]    Note that in some embodiments the software  42 ,  54   a ,  54   b  and the like can be combined, distributed in various modules, and/or reside in firmware. Other software is typically associated with the system. In general, there is software configured, as discussed above, to report to the CPU or other processor(s) a superset of virtual switches, step  80   FIG. 4 . One robots control application subscribes to a first switch subset, step  82  and another robots control application subscribes to a second switch subset, step  84 . For a first mode, the set of physical switches are mapped to the first switch subset, step  84  and, for the other mode controlling another robot, the set of physical handheld controller switches are mapped to the second switch subset, step  88 . 
         [0043]    Also note that the functionality of processor  40  and CPU  52  may be combined or reside in circuitry and distributed other than as shown. Microprocessors, controllers, application specific integrated circuits, programmable logic devices and the like may be used. Various signal processing and other electronic and electrical circuitry and printed circuit boards are not shown. 
         [0044]    Therefore, although specific features of the invention are shown in some drawings and not in others, this is for convenience only as each feature may be combined with any or all of the other features in accordance with the invention. The words “including”, “comprising”, “having”, and “with” as used herein are to be interpreted broadly and comprehensively and are not limited to any physical interconnection. Moreover, any embodiments disclosed in the subject application are not to be taken as the only possible embodiments. 
         [0045]    In addition, any amendment presented during the prosecution of the patent application for this patent is not a disclaimer of any claim element presented in the application as filed: those skilled in the art cannot reasonably be expected to draft a claim that would literally encompass all possible equivalents, many equivalents will be unforeseeable at the time of the amendment and are beyond a fair interpretation of what is to be surrendered (if anything), the rationale underlying the amendment may bear no more than a tangential relation to many equivalents, and/or there are many other reasons the applicant can not be expected to describe certain insubstantial substitutes for any claim element amended. 
         [0046]    Other embodiments will occur to those skilled in the art and are within the following claims.