Abstract:
Described herein is a robot for testing touch-sensitive displays. The test robot may have a test surface holding a touch-sensitive display. The test robot may also have a first robotic unit that can translate in only two dimensions relative to the touch-sensitive display, where the first robotic unit secures a first plurality of finger units. The test robot may also have a second robotic unit that can translate in only the two dimensions relative to the touch-sensitive display, where the second robotic unit secures a second plurality of finger units. The test robot may also have a control unit controlling the first robotic unit, the second robotic unit, the first plurality of finger units, and the second plurality of finger units.

Description:
BACKGROUND 
       [0001]    With increasing use of touch-sensitive devices such as smart phones, tablets, laptops, and others, there has been increasing need to test such touch-sensitive devices. For example, it may be desirable to verify physical attributes of a touch-sensitive device, such as sensitivity and accuracy. In addition, it may be desirable to test the correctness of software running on a touch-sensitive device using physical test inputs to interact with software being tested. With regard to testing physical traits of a touch-sensitive device, to test for compliance with a certification standard, for example, human testers generally cannot duplicate their test behaviors on diverse target devices to equally measure the same physical qualities of different devices. The test results of touch devices have been judged with an individual&#39;s subjectivities and without specific criteria. Furthermore, human finger methodology is prone to inconsistency due to variables in finger pressure, line-straightness, tracing speed, etc. 
         [0002]    Techniques and devices related to robotic testing of touch-sensitive devices are described below. 
       SUMMARY 
       [0003]    The following summary is included only to introduce some concepts discussed in the Detailed Description below. This summary is not comprehensive and is not intended to delineate the scope of the claimed subject matter, which is set forth by the claims presented at the end. 
         [0004]    Described herein is a robot for testing touch-sensitive displays. The test robot may have a test surface holding a touch-sensitive display. The test robot may also have a first robotic unit that can translate in only two dimensions relative to the touch-sensitive display, where the first robotic unit secures a first plurality of finger units. The test robot may also have a second robotic unit that can translate in only the two dimensions relative to the touch-sensitive display, where the second robotic unit secures a second plurality of finger units. The test robot may also have a control unit controlling the first robotic unit, the second robotic unit, the first plurality of finger units, and the second plurality of finger units. 
         [0005]    Many of the attendant features will be explained below with reference to the following detailed description considered in connection with the accompanying drawings. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0006]    The present description will be better understood from the following detailed description read in light of the accompanying drawings, wherein like reference numerals are used to designate like parts in the accompanying description. 
           [0007]      FIG. 1  shows an overhead of robotic hands and a touch-sensitive device. 
           [0008]      FIG. 2  shows a side view of a test robot. 
           [0009]      FIG. 3  shows a detailed overhead view of a test robot, in particular an X-Y Cartesian robot. 
           [0010]      FIG. 4  shows a test surface. 
           [0011]      FIG. 5  shows a side view of a test robot. 
           [0012]      FIG. 6  shows a detailed view of robotic hands. 
           [0013]      FIG. 7  shows detailed views of a moveable finger unit. 
           [0014]      FIG. 7A  shows a detailed view of a finger unit. 
           [0015]      FIG. 8A  shows a perspective view, off-center and from above, of a robotic hand. 
           [0016]      FIG. 8B  shows a side view of a robotic hand. 
           [0017]      FIG. 8C  shows a perspective view, off-center and from below, of a robotic hand. 
           [0018]      FIG. 8D  shows a top view of a robotic hand. 
           [0019]      FIG. 8E  shows a front view of a robotic hand. 
           [0020]      FIG. 8F  shows a bottom view of a robotic hand. 
           [0021]      FIG. 9  shows a block diagram of control hardware. 
           [0022]      FIG. 10  shows a block diagram of a master controller. 
           [0023]      FIG. 11  shows a block diagram of a finger module. 
           [0024]      FIG. 12  shows a firmware architecture for the master controller. 
           [0025]      FIG. 13  shows a firmware architecture for a finger module. 
           [0026]      FIG. 14  shows a test framework architecture. 
       
    
    
     DETAILED DESCRIPTION 
       [0027]    Embodiments discussed below relate to techniques and devices for robotic testing of touch-sensitive devices. Various mechanical details of a test robot will be discussed, followed by discussion of a software framework for using test robots to test touch-sensitive devices. 
         [0028]      FIG. 1  shows an overhead of robotic hands  102 A,  102 B and a touch-sensitive device  104 . Robotic hands  102 A and  1028  have similar construction and control; the following description of the robotic hand  102 A will describe parallel features of the robotic hand  1028 .  FIG. 1 , an overhead view, may be thought of as a reference plane (i.e., an x-y plane), with various robot elements moving in x-y directions parallel to the reference plane. The robotic hand  102 A moves in x-y directions above a touch device  104 . 
         [0029]    The robotic hand  102 A has two or more finger units  106 A,  1068 . The finger units  106 A  106 B naturally move as the robotic hand  102 A to which they are attached moves. If a finger unit  106 A,  1068  is contacting the touch device  104 , the finger unit  106 A  1068  will trace a path on the surface of the touch device  104  that corresponds to a translation path of the robotic hand  102 A in a plane parallel to the reference plane. The finger units  106 A,  106 B are designed to move in a direction substantially perpendicular to the reference plane (e.g., up and down if the touch device  104  is lying flat), thus allowing them to be individually arranged such that one finger unit  106 A may be contacting the touch device  104  (e.g., down) while the other finger unit  1068  is not contacting the touch device  104  (e.g., up). 
         [0030]    One or more of the finger units, for example finger unit  1068 , may translate relative to the robotic hand  102 A in a direction parallel to the reference plane. For example, the robotic hand  102 A may have a rail that is parallel to the reference plane, and the finger unit  1068  may be movably mounted on the rail, thus allowing the finger unit  1068  to be translated by a dedicated servo along the rail in a direction parallel to the reference plane (as indicated by the dashed arrow on the finger unit  106 B). If the finger unit  1068  is contacting the touch device  104 , the finger unit  106 B&#39;s translation relative to the robotic hand  102 A causes the finger unit to trace a corresponding contact path on the surface of the touch device  104 . 
         [0031]    As indicated by the dashed arcs in  FIG. 1 , the robotic hand  102 A may also be configured to rotate in a plane parallel to the reference plane (parallel to the surface of the touch device  104 ), thereby causing any finger units that are contacting the surface of the touch device  104  to trace a circular or arced path, according to possible additional translation of the robotic hand  102 A and/or translation of a finger unit  106 A,  1068  relative to the robotic hand  102 A. In sum, it is possible that a finger unit  106 A,  1068 , at any given time, may be (i) translating perpendicular to the reference plane (e.g., the touch device  104  or a surface holding same), thus allowing the finger unit  106 A,  1068  to move into or out of contact with the touch device  104 , (ii) translating parallel to the reference plane according to translation of the robotic hand  102 A, and (iii) translating parallel to the reference plane according to rotation of the robotic hand  102 A. 
         [0032]      FIG. 2  shows a side view of a test robot  130 . A test surface  132  may include one or more lifts and jigs to hold the touch device  104 . The test surface  132  may be thought of as a reference plane that is perpendicular to  FIG. 2 . An X-Y Cartesian robot  134  provides movement of one or two of the robotic hands  102 A,  1028  in directions parallel to the test surface  132 . For an example, see arm unit  136 A in  FIG. 3 . An electrical controller box  138  may house various control components discussed later, such as control circuits, power supplies, communications buses, etc. 
         [0033]      FIG. 3  shows a detailed overhead view of test robot  130 , in particular X-Y Cartesian robot  134 . In the embodiment shown in 
         [0034]      FIG. 3 , robotic hands  102 A,  1028  each have five finger units  106  ( 106 A/ 106 B), one or more of which may move relative to robotic hands  102 A,  102 B as described above in reference to  FIG. 1 . The robotic hands  102 A,  102 B may be moved laterally in a plane parallel to test surface  132  (e.g., a reference plane). The X-Y movement of robotic hands may be provided by arm units  136 A,  1368 . An arm unit  136 A,  136 B may have a motorized movement unit  138 A,  1388 , which moves an arm  140 A,  1408  in the X direction by moving the arm  140 A,  1408  in the direction of its length, and in the Y direction by the movement unit  138 A,  138 B moving itself along a beam  142 A,  1428 . In addition, the robotic hands  102 A,  102 B may be rotated by servos  144 A,  144 B. 
         [0035]    The rotational and translational movement of the robotic hands  102 A,  1028  may be implemented by a variety of known mechanical techniques, including, for example, servos moving gears engaging toothed rails, servos driving belts, rotating threaded rods (i.e., ball screws), chains, etc. Moreover, other arrangements may be used. For example, rather than the movement unit  140 A,  1408  actually moving, the movement unit  140 A,  140 B, may have a servo that rotates to move the arm  140 A,  140 B in sweeping motions, in which case the movement unit  140 A,  1408  may also have another servo that moves the arm  140 A,  1408  toward and away from the movement unit  140 A,  1408  (i.e., along the length of the arm  140 A,  140 B). Other designs may be used, including reticulated arms, a single movement unit moving two attached arms, etc. 
         [0036]      FIG. 4  shows test surface  132 . In an embodiment shown in the upper part of  FIG. 4 , a first jig  150 , with first moveable holders  152 , holds a second jig  154 . The second jig  154  has second moveable holders  156 . The first moveable holders  152  are adjusted to hold the second jig  154 , and the second moveable holders  156  are adjusted to hold the test device  104 . In embodiment shown in the lower part of  FIG. 4 , the second jig  154  has been removed and a larger touch device  104  is held directly by first moveable holders  152 . 
         [0037]      FIG. 5  shows a side view of test robot  130 . In  FIG. 5 , test surface  132  is a planar surface perpendicular to the figure. Relative to the figure, the arms  140 A,  140 B move to the left and the right, and the movement units  138 A,  138 B move perpendicular to the figure. Each type of movement causes corresponding movement of the robotic hands  102 A,  102 B. 
         [0038]      FIG. 6  shows a detailed view of robotic hands  102 A,  1028 . The robotic hand  102 A,  1028  has from two to five finger units  106 . Each finger unit  106  may have a removable stylus or finger  182 . One or more of the finger units  106  may be fixed, for example, central finger unit  106 . The other finger units  106  may move laterally (left and right with respect to the figure) along rails  184 . The lateral translation of the finger units may be accomplished by belts  186 , which are driven by respective servos  188  and anchored between rotating units  190 . In one embodiment, the belts  186  are timing belts, and the rotating units  190  are toothed cylinders to engage teeth of the timing belts. In other embodiments, the translational movement of the finger units  106  relative to the robotic hand  102 A,  102 B is driven by other mechanical means, such as pistons, rotating threaded shafts (ball gears), elastic loops and pulleys, etc. Control software, described further below, can monitor locations of finger units  106  and pending movement commands to prevent collisions between finger units  106 . 
         [0039]    In the embodiment shown in  FIG. 6 , it may be convenient for some of the rotating units  190  to be attached to the non-moving finger unit  190 . Each paired belt  186  and finger unit  106  are attached by coupled by an attachment  192 , for example, a clamp, a piece of metal affixed to both the belt  186  and the finger unit  106 , a nut and bolt through the belt  186  and finger unit  106 , etc. It can be seen from the design of  FIG. 6  that finger units  106  (and consequently fingers  182 ) can be individually moved in a direction relative to the robotic hand  102 A,  1028 . 
         [0040]    As will be described later, each finger unit  106  may have a pressure sensor  194  to measure pressure of the corresponding finger  182  on the touch device  104 . Moreover, as describe next, each finger unit  106  may have a mechanism to move its corresponding finger  182  along the length of the finger unit  106  (i.e., perpendicular to the test surface  132 ). That is, if the test surface  132  is horizontally level, the fingers  182  may be individually moved up and down. 
         [0041]      FIG. 7  shows detailed views of a moveable finger unit  106 . The left view in  FIG. 7  (front view), is perpendicular to the right view in  FIG. 7  (side view). The finger unit  106  may have a main body  200 , which may hold a finger servo  202  and finger wheels  204  around which turns a finger belt  206  driven by the finger servo  202 . A carriage  208  may be moveably mounted on rails (not shown). The carriage  208  is attached to the finger belt  206  by an attachment  210  of any variety discussed earlier. In one embodiment, the attachment  210  may protrude and travel between two stoppers (not shown) on the carriage  208  to limit movement of the finger  182 . Again, movement of finger  182  may be implemented by known means of robotic movement other than belts. 
         [0042]    Each finger  182  may have a detachable tip  209 , to allow use of different materials and shapes to contact the touch device  104 . For example, a brass detachable tip  209  may be suitable for a capacitive type touch device  104 . A silicon cover or detachable tip  209  may be called for when other types of touch devices  104  are to be tested. 
         [0043]    In addition, each finger unit  106  may have a pressure sensor  212 . It is assumed that the construction of the finger unit  106  allows the finger  182  to move with some degree of freedom, and the pressure sensor  212  is interposed between finger  182  and the carriage  208 , thereby allowing the pressure sensor  212  to measure the pressure of the finger  182  contacting the touch device  104 , due to force from the servo  202  and belt  206 . In other words, the pressure sensor may measure pressure between the finger  182  and the carriage  208 . 
         [0044]      FIG. 7A  shows a detailed view of a finger unit  106 . In addition to features mentioned with reference to  FIG. 7 , also shown are a belt holder  213 , for lateral movement via belt  186 , and a rail  214 , along which the finger unit  106  moves up and down. A sensor target object  215  and detector  216  function such that if the motor  202  drives the finger belt  206  upward, the detector  216  senses the sensor target object  215  and further movement is prevented. To measure pressure, when a detachable tip  209  is blocked by a surface such as a touch screen while the finger belt  206  is driving downward (depicted by downward arrow) along the rail  214 , the tip  209  will move slightly upward along a second rail  217  (indicated by upward arrow) and then a load point  218  pushes a load cell  219  which can sense how much pressure is being applied by the load point  218 . A spring  221  can be included to help the load point  218  push the load cell  219  with regular force. 
         [0045]      FIG. 8A  shows a perspective view, off-center and from above, of either robotic hand  102 A,  1028 .  FIG. 8B  shows a side view of either robotic hand  102 A,  102 B. An optional camera  209  may be included.  FIG. 8C  shows a perspective view, off-center and from below, of either robotic hand  102 A,  1028 .  FIG. 8D  shows a top view of either robotic hand  102 A,  1028 .  FIG. 8E  shows a front view of either robotic hand  102 A,  1028 .  FIG. 8F  shows a bottom view of either robotic hand  102 A,  1028 . It may be appreciated that the angled attachments  207  attaching fingers  182  to finger units  106  allow the fingers  182  to be placed close if not in contact with each other when the finger units  106  are positioned toward the center of the hand. This can allow a wide range of multi-touch gestures to be simulated. 
         [0046]    The camera  209  can be helpful in performing an initial orientation process when the touch device  104  is to be tested. When the touch device  104  is in place for testing, a signal from the camera  209 , which can be located based on the location of the corresponding robot hand, allows the robot to locate a test area boundary, such as the edges of a touch surface. For example, the camera signal allows the robot to place the hand at a corner of a test area boundary. The hand is then traced, guided by the camera signal, along the test area boundary to allow the location/coordinates of the boundary sides to be ascertained. In one embodiment, one hand starts at one corner, another hand starts the opposite corner, and the hands trace out respective halves of the test area or test surface boundary, and the trace path (boundary) is recorded. It may be sufficient to locate only corners and calculate the connecting boundary lines. In sum, a program such as MFStudio.exe  304  (discussed below) can implement a calibration process that finds the initial corner positions of a target touch screen for two XY Cartesian Robots and measures screen size and how well the target touch screen is aligned. This information can allow an operator to adjust the target touch screen calibration to the robot  130 . 
         [0047]      FIG. 9  shows a block diagram of control hardware  220  for the various embodiments of the robot  130 . The control hardware of the test robot  130  has two primary parts; a motion controller  222  and finger controller  224 . Note that the term “MF” will be used as a synonym for “test robot”  130 . The term “PC” will refer to a computer. Any components shown in  FIGS. 9-13  not specifically discussed are self-explanatory according to their labels. 
         [0048]    Regarding the motion controller  222 , the motion controller  222  preferably uses PCI-motion controllers to control six AC servo motor AMPs, thereby providing Application Programming Interfaces (APIs) to drive six AC servo motors by programming languages such as C++, Visual Basic™, etc. The motion controller  222  may support up to six AC servo motor AMPs, but uses six channels to drive the X-Y Cartesian robot  134 , and the two robotic hands  102 A,  102 B. 
         [0049]      FIG. 10  shows a block diagram of a master controller  226 . Primarily, the master controller  226  acts as a UART-CAN bridge (Universal Asynchronous Receiver/Transmitter, Controller Area Network) to bypass data packets from the motion controller  222  (PC) on a serial UART to the finger modules  240  (see  FIG. 11 ) on a CAN network. Also, the master controller  226  exposes external inputs and outputs to connect other devices (e.g., the motion controller  222 ). It may be helpful for the external inputs and outputs to trigger complex, subsequent operations with minimal delay time. 
         [0050]      FIG. 11  shows a block diagram of a finger module  240 . Finger module  240  controls two DC motors and measures the currents and voltages of each motor, as well as the pressure of one loadcell (e.g., pressure sensor  212 ). Each of ten modules on the same CAN network have unique respective address IDs. Thus, an operator can control the movement of each finger unit  106 . Particularly, the circuit and firmware have several protection features that measure a drive motor&#39;s current temperature and voltage in real time. 
         [0051]      FIG. 12  shows a firmware architecture  260  for the master controller  226 . The master controller  226  has a device initialization (interrupt vector tables, watch dogs, and so forth) and also includes codes which allow updating of flash memory via the network. A Message Process and Command Process may define main protocols to control the finger units  106 . A Controller API &amp; Control state machine provide an abstraction layer to invoke driver functions by the Command Process routines. Moreover, this state machine manages and processes the requests from other devices, such as the master controller  226  PC and the finger modules  240 . A CAN interface and UART Interface are responsible for UART-CAN bridging and therefore, if packets from the motion controller  222  (control PC) are not destined for the master controller  226 , then they are bypassed to the finger modules  240 . Regarding External I/O, the master controller  226  provides external inputs and external outputs to interact with other devices. 
         [0052]      FIG. 13  shows a firmware architecture  280  for a finger module  240 . A Controller API &amp; Control State Machine provides an abstraction layer for invocation of driver functions by Command Process routines. This State Machine manages and processes all of the requests from other devices, such as the control PC and the finger modules  240 . A PID (Proportional-Integral-Derivative) Controller drives DC motors with feedbacks, as the test robot  130  includes cascade PID routines for position, speed, and pressure control. An H-Bridge driver has one module that includes two HBridge ICs (integrated circuits) to control two DC motors; i.e., this component is a driver to support the DC motors. The microcontroller includes quadrature encoder counters and PWM (Pulse Width Modulation) generator to control DC motors. A Limit Switch may be provided because when a motor or servo runs, the controller should know the start and end limits to prevent collisions. An ADC Driver reads the acquisition data including temperature, current, voltage, and pressure (loadcell). The ADC Driver also includes a software filter, like Low Pass Filter and Moving Average Filter, to compensate ADC output. An electrical switch can change the electrical state of a finger tip for a capacitive-type touch device. A finger tip is connected to GND, and the operator can change the finger&#39;s electrical state from GND to Floating or vice versa. 
         [0053]      FIG. 14  shows a test framework architecture. A DCOM (Distributed COM) server  300  (MFService.exe) is an operating system based background process that monitors the movement of the test robot  130  and protects against unexpected collision of robot arms. The DCOM server  300  monitors the positions of the test robot  130  from the AC motor AMPs and from the finger controllers. The monitoring DCOM server  300  also initializes the configuration of the motion controller with data that would likely result in unexpected accidents. In addition, the DCOM server  300  is responsible for controlling the X-Y Cartesian robot  134  and finger units  106 , and is also responsible for logging/verifying actual results and expected results. Thus, the DCOM server  300  may be considered the core of the test software framework. In addition to this function, the DCOM server  300  provides a COM API through a TCP/IP socket with the target touch device  104  (specifically, an MFClient.exe  301  application) to transfer touch raw data such as pressure data from the touch device  104 . 
         [0054]    A management tool MFStudio.exe  304  can provide test case execution/reviewer UI (user interface), dash board for the status of the robot, and target touch device calibration process. The management tool can create/run/verify the test case jobs and generate the reports of each test case. The management tool is also responsible for providing configuration data, status information, and manual control of the test robot  130 ; the X-Y Cartesian robot  134 , the robotic hands  102 A,  1028 , and the finger units  106 . 
         [0055]    The client application MFClient.exe  301  manages each test case to be invoked by MFStudio.exe  304  when the connection is available. The client application MFClient.exe  301  is also responsible for capturing Raw touch HID (Human Interface Device) information, which can be facilitated with HID class drivers, which provide an interface for extracting data from raw HID reports. The class drivers supply a report descriptor that details the format of the different reports that it creates. Therefore, the client application MFClient.exe  301  captures these raw reports and then copies them into the robot controller (server).