Source: https://patents.google.com/patent/US9785281B2/en
Timestamp: 2019-06-27 08:43:54
Document Index: 778574075

Matched Legal Cases: ['Application No. 201210018527', 'Application No. 201210052934', 'Application No. 201210018527', 'Application No. 201210029859', 'Application No. 201210018527', 'Application No. 201210029859', 'Application No. 201210052934', 'Application No. 201210446236', 'Application No. 100135900', 'Application No. 101107036', 'Application No. 103124288', 'Application No. 101107036', 'Application No. 102100728', 'Application No. 102100728', 'Application No. 12744496', 'Application No. 12744979', 'Application No. 12754354', 'Application No. 201210446236', 'Application No. 201210052934']

US9785281B2 - Acoustic touch sensitive testing - Google Patents
Acoustic touch sensitive testing Download PDF
US9785281B2
US9785281B2 US13/293,060 US201113293060A US9785281B2 US 9785281 B2 US9785281 B2 US 9785281B2 US 201113293060 A US201113293060 A US 201113293060A US 9785281 B2 US9785281 B2 US 9785281B2
US13/293,060
US20130113751A1 (en
Andrey B. Batchvarov
2011-11-09 Priority to US13/293,060 priority Critical patent/US9785281B2/en
2011-11-10 Assigned to MICROSOFT CORPORATION reassignment MICROSOFT CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BATCHVAROV, ANDREY B., UZELAC, ALEKSANDAR
2013-05-09 Publication of US20130113751A1 publication Critical patent/US20130113751A1/en
2017-10-10 Publication of US9785281B2 publication Critical patent/US9785281B2/en
Acoustic touch sensitive testing techniques are described. In one or more implementations, a touch-sensitive surface of a touch-sensitive device is tested by detecting contact made with the touch sensitive surface using an acoustic sensor and comparing data describing the contact that is received from the acoustic sensor with data describing the contact that is received from the touch-sensitive device.
Display and input techniques have continued to evolve, such as to sense touch using a touchscreen display of a computing device to recognize gestures. A user, for instance, may interact with a graphical user interface by inputting a gesture using the user's hand that is detected by the touchscreen display or other touch-sensitive device. However, traditional techniques that were utilized to test touchscreen displays and other touch-sensitive devices were often inaccurate and therefore were typically inadequate to test the touchscreen displays as suitable for intended use of the device.
In one or more implementations, an apparatus includes an acoustic sensor configured for placement proximal a touch-sensitive surface and one or more modules implemented at least partially in hardware to use a signal received from the acoustic sensor and a signal received from the touch-sensitive surface to test the touch-sensitive surface.
In one or more implementations, a system includes one or more modules that are implemented at least partially in hardware and configured to test latency of a touchscreen of a touchscreen device using a signal received from an acoustic sensor that is disposed proximal to a touchscreen of the touchscreen device to detect contact made with the touchscreen and data received from the touchscreen device that describes the contact.
FIG. 1 is an illustration of an environment in an example implementation that is operable to utilize acoustic testing techniques described herein.
FIG. 2 is an illustration of a system in an example implementation showing a test apparatus of FIG. 1 in greater detail.
FIG. 3 depicts a graph showing an electrical voltage drop on a resistor as being used as a reference as to when contact occurred.
FIG. 4 depicts a graph showing latency induced by physical processes producing a sound wave as defined as a difference between a moment the signal from acoustic wave crosses a predefined threshold and a moment a voltage drop occurred.
FIG. 5 depicts a graph as showing an example measurement taken using an oscilloscope.
FIG. 6 depicts a system in an example implementation showing a measuring setup for detection of touch event time via software post processing.
FIG. 7 is a flow diagram depicting a procedure in an example implementation in which an acoustic testing technique is described.
FIG. 8 illustrates various components of an example device that can be implemented as any type of computing device as described with reference to FIGS. 1, 2 and 6 to implement embodiments of the techniques described herein.
Conventional techniques that were utilized to test touchscreens and other touch-sensitive devices were often difficult to reproduce. Consequently, test results from these conventional techniques could be inaccurate and difficult to interpret and thus often failed for their intended purpose.
Acoustic touch sensitive testing techniques are described herein. In one or more implementations, techniques are described in which acoustic techniques are utilized to test a touch-sensitive device, such as a touchscreen. For example, an acoustic sensor may be used to detect “when” contact is made with a touchscreen of a touchscreen device. The input detected using the acoustic sensor may then be compared with an input generated by the touchscreen device to test functionality of the touchscreen device, such as to determine latency. Further discussion of these and other testing techniques may be found in relation to the following sections.
In the following discussion, an example environment is first described that may employ the testing techniques described herein in relation to a touchscreen device. However, it should be readily apparent that a variety of touch sensitive devices are also contemplated, such as track pads and other sensing devices. Example procedures are then described which may be performed in the example environment as well as other environments. Consequently, performance of the example procedures is not limited to the example environment and the example environment is not limited to performance of the example procedures.
In one or more implementations described herein, an ability of a touchscreen 106 to detect contact (e.g., by a finger of a user's hand 108) is tested by the test apparatus 102. For example, the test apparatus 102 may include a test module 114 and acoustic sensor 116. The acoustic sensor 116 may be utilized to test the touchscreen device 104 in a variety of ways. For instance, the test module 114 may leverage the acoustic sensor 116 to measure response latency the touchscreen device 104, such as an ability of the touch module 112 to detect contact by one or more fingers of the user's hand 108 (or other objects) with the touchscreen 106. This may be performed in a variety of ways, an example of which may be found in relation to FIG. 2.
FIG. 2 is an illustration of a system 200 in an example implementation showing the test apparatus 102 of FIG. 1 in greater detail. The test apparatus 102 in this example is configured to leverage acoustic wave detection to evaluate latency of a touch-sensitive device, such as the touchscreen 106 of the touchscreen device 104. The test module 114, for instance, may be configured to detect a sound wave produced when contact 202 is made with a surface, e.g., the touchscreen 106 in this example, which may be used to define a moment at which physical touch has occurred.
Processing of one or more contacts by a touchscreen device 104 may involve the following. First, physical contact 202 is detected by one or more touch sensors 110 of the touchscreen device 104, resulting in an electrical signal. The electrical signal is then processed by the touch module 112 to determine “what” the signal describes, e.g., characteristics of the contact 202 such as where the contact occurred. This processing may then be reported to the test module 114 as a system touch event time 204 (Ts) 204 for testing purposes. Thus, detection and processing of the contact 202 may take time to complete, which may therefore be thought of as latency of the touchscreen device 104 in detecting the contact 202.
In order to test latency, the test module 114 may employ one or more acoustic sensors 116 (e.g., a piezo-transducer) that are positioned at or near the touch-sensitive surface, e.g., the touchscreen 106. A signal from the acoustic sensors 116 may then be amplified by an amplifier 206 and processed by a touch detection module 208 to determine an acoustic touch event time (Ta) 210. This processing, for instance, may include identification of an actual signal from background noise detected by the acoustic sensors 116. A latency evaluation module 212 may then determine a time associated with a touch event, e.g., the moment of contact. This may be determined by detecting a difference in the amount of time between the system touch event time 204 and the acoustic touch event time 210.
As should be readily apparent, operation of the test apparatus 102 may also involve latency. This may include duration and latency of the physical process occurring during interaction of the touching object with the touch sensitive surface which produces acoustic waves, speed of the propagation of the acoustic waves on that surface, distance between touch point and acoustic sensor, latency in producing electrical signal in the electronic acoustic detector, and latency of an electronic registration system used to store results of the latency evaluation module 212. However, this latency is generally below one millisecond.
For example, production of an acoustic vibration on the touch surface from the contact 202 is generally between 30 and 100 uS in case of hard surface as a glass or wood and depends on the speed of the contact. As the speed of sound on a solid surface is in the order of over a thousand of meters per second, the time taken for the acoustic vibrations to reach the sensor may be within few hundreds microseconds when the acoustic sensor 116 is positioned within few centimeters from the point of contact 202. Hence, if a piezo-transducer is used as acoustic detector, then the time used to produce an electrical signal from acoustic vibrations may be below one microsecond. The amount of time consumed by the touch detection module 208 to identify the contact 202 that produced acoustic signal from the background noise may be dependent on a level of that noise, frequency and magnitude of the touch event signal, and so on.
In an implementation example, the system 200 may be configured as follows. In order to produce the contact 202, a thin conductive plate may be positioned on the surface of the touch-sensitive surface to be tested to act as the contact 202. The conductive plate and the human body are connected to an electrical circuit so that when the finger touches the plate and electrical current flows through the finger and plate. The electrical current is amplified and a voltage drop on a resistor is measured in order to detect the moment and dynamics of the contact 202. Although use of a user is described, this process may also be automated using non-human motion.
An acoustic sensor 116 is positioned close to the conductive plate. A signal from the acoustic sensor 116 is amplified and recorded by the test module 114 as described earlier. An electrical voltage drop on the resistor is then used as reference to when the actual touch contact occurred, an example of which is shown in the graph 300 of FIG. 3.
Latency induced by physical processes producing sound wave is defined as difference between the moment the signal from acoustic wave crosses a predefined threshold and the moment the voltage drop occurred as shown by the graph 400 of FIG. 4. The threshold, for instance, may be defined to be greater than a maximum magnitude of the existing background noise.
An example measurement as may be taken using an oscilloscope is shown in the example graph 500 of FIG. 5. Since triggering may occur either during first or second phase of the acoustic signal, the signal may be passed through a rectifier to convert the original signal into single polarity to minimize error in latency evaluation. Comparison with the predefined threshold may be made in real time using hardware and/or via software post processing. Electronic comparison with the threshold can be accomplish by using analog comparator or using digital comparator after digitizing the signal and comparing it digitally with the threshold value.
FIG. 6 depicts a system 600 in an example implementation showing a measuring setup for detection of touch event time via software post processing. In this example, a system under test 602 (e.g., the touchscreen device 104) is tested by the test apparatus 102 based on a recorded signal.
The test module 114, for instance, may include a recorder 604 to continuously record a signal from the acoustic sensor 116. The recording may be set to last for at least a duration that is greater than expected or known latency of the system under test 602.
Upon receiving a system touch event 606 from the system under test 602, the latency evaluation module 212 may send a stop recording 608 signal to the recorder 604. In response to that signal, the recorder 604 may stop recording and send back the recorded signal 610 to the latency evaluation module 212. The latency evaluation module 212 may then process the data and identify a first moment of time, at which, the recorded acoustic signal has crossed the threshold value as shown in the graphs. Further discussion of acoustic touch sensitive testing may be found in relation to the following procedures.
Although a single acoustic sensor 116 was described above, a plurality of acoustic sensor may be utilized. For example, a plurality of acoustic sensors 116 may be used to perform trilateration to estimate a position of a contact and thus enable an estimation of an amount of time consumed for a wave to reach the acoustic sensors 116. This may be used to provide a better estimate for latency, and hence improve the process. A variety of other examples are also contemplated, such as to employ feedback from both acoustic sensors 116 (e.g., averaging) for processing.
The following discussion describes touchscreen testing techniques that may be implemented utilizing the previously described systems and devices. Aspects of each of the procedures may be implemented in hardware, firmware, or software, or a combination thereof. The procedures are shown as a set of blocks that specify operations performed by one or more devices and are not necessarily limited to the orders shown for performing the operations by the respective blocks. In portions of the following discussion, reference will be made to the environment 100 of FIG. 1, the systems 200, 600 of FIGS. 2 and 6, and the graphs 300-500 of FIGS. 3-5.
FIG. 7 is a flow diagram depicting a procedure 900 in an example implementation in which a touch-sensitive surface is tested using acoustic techniques. A touch-sensitive surface of a touch-sensitive device is tested (block 702) by detecting contact made with the touch sensitive surface using an acoustic sensor (block 704). As previously described, the touch-sensitive surface may be configured in a variety of ways, such as a touchscreen device, track pad, and so on. Thus, the touch-sensitive surface may employ a variety of techniques to detect contact, such as capacitive, imaging (e.g., IR, cameras, and so on), and so forth.
Data describing the contact that is received from the acoustic sensor with data describing the contact that is received from the touch-sensitive device (block 706). A test module 114 may employ a variety of different techniques to test a touch-sensitive surface. As shown in FIGS. 2 and 6, for instance, a latency evaluation module 212 may be employed to detect latency in an ability of a touchscreen device 104 to recognize contact. A variety of other examples are also contemplated.
FIG. 8 illustrates various components of an example device 800 that can be implemented as any type of computing device as described with reference to FIGS. 1, 2, and 6 to implement embodiments of the techniques described herein. Device 800 includes communication devices 802 that enable wired and/or wireless communication of device data 804 (e.g., received data, data that is being received, data scheduled for broadcast, data packets of the data, etc.). The device data 804 or other device content can include configuration settings of the device, media content stored on the device, and/or information associated with a user of the device. Media content stored on device 800 can include any type of audio, video, and/or image data. Device 800 includes one or more data inputs 806 via which any type of data, media content, and/or inputs can be received, such as user-selectable inputs, messages, music, television media content, recorded video content, and any other type of audio, video, and/or image data received from any content and/or data source.
Computer-readable media 814 provides data storage mechanisms to store the device data 804, as well as various device applications 818 and any other types of information and/or data related to operational aspects of device 800. For example, an operating system 820 can be maintained as a computer application with the computer-readable media 814 and executed on processors 810. The device applications 818 can include a device manager (e.g., a control application, software application, signal processing and control module, code that is native to a particular device, a hardware abstraction layer for a particular device, etc.). The device applications 818 also include any system components or modules to implement embodiments of the techniques described herein. In this example, the device applications 818 include an interface application 822 and an input/output module 824 (which may be the same or different as input/output module 84) that are shown as software modules and/or computer applications. The input/output module 824 is representative of software that is used to provide an interface with a device configured to capture inputs, such as a touchscreen, track pad, camera, microphone, and so on. Alternatively or in addition, the interface application 822 and the input/output module 824 can be implemented as hardware, software, firmware, or any combination thereof. Additionally, the input/output module 824 may be configured to support multiple input devices, such as separate devices to capture visual and audio inputs, respectively.
testing a touch-sensitive surface of a touchscreen device by:
receiving at the touch-sensitive surface a contact;
detecting the contact by both a first sensor part of the touch-sensitive surface and a second sensor of a test apparatus, the second sensor being an acoustic sensor, the first sensor being a type other than acoustic, the touch-sensitive surface not containing an acoustic sensor usable to detect the contact, and the second sensor being separate and distinct from the touchscreen device and not used by the touchscreen device to detect the contact;
sending first data from the touchscreen device to the test apparatus, the first data describing the contact as detected by the first sensor; and
determining an ability of the touchscreen surface to accurately detect the contact by comparing the first data to second data, the second data describing the contact as detected by the second sensor.
2. The method of claim 1, wherein the testing is configured to determine a latency associated with detection of the contact.
3. The method of claim 1, wherein detecting the contact by both further comprises:
receiving signals from the second sensor continuously for a prescribed duration;
analyzing the received signals for a first time point at which at least one signal exceeds a threshold; and
utilizing the first time point as the first data.
4. The method of claim 3, wherein the threshold is based on a maximum magnitude associated with background noise.
5. The method of claim 1, wherein the touchscreen device is a mobile computing device.
6. The method of claim 1, wherein the touch-sensitive surface is operable as a display for a mobile communications device.
7. The method of claim 1, wherein the first data is based on a time associated with the contact as detected by the first sensor, and wherein the second data is based on a time associated with the contact as detected by the second sensor.
8. The method of claim 1, wherein determining the contact by both further comprises:
receiving signals from a plurality of additional acoustic sensors of the test apparatus;
using the signals from the plurality of additional acoustic sensors to determine a position of the contact; and
determining a latency of the touch-sensitive surface based on the determined position of the contact.
an acoustic sensor configured for placement proximal to a touch-sensitive surface of a touchscreen device, the acoustic sensor separate and distinct from the touchscreen device and not used by the touchscreen device to detect a contact made to the touch-sensitive surface; and
at least one module that detects the contact by both the acoustic sensor and a first sensor part of the touch-sensitive surface, the at least one module configured to:
receive first data from the touchscreen device describing the contact as detected by the first sensor, the first sensor being a type other than acoustic; and
determine an ability of the touch-sensitive surface to accurately detect the contact by comparing the first data to second data, the second data describing the contact as detected by the acoustic sensor.
10. The test apparatus of claim 9, wherein the comparison of the first data to the second data is used to determine latency associated with detection of the contact.
11. The test apparatus of claim 9, wherein the first data is based on a time associated with the contact as detected by the first sensor, and wherein the second data is based on a time associated with the contact as detected by the second sensor.
12. The test apparatus of claim 9, wherein the touchscreen device is a mobile computing device.
13. The test apparatus of claim 9, wherein the contact is associated with a non-user contact.
14. The test apparatus of claim 9, wherein the first sensor is a capacitive sensor.
15. The test apparatus of claim 9, wherein the at least one module is implemented as one or more of software, firmware, or hardware.
a first sensor part of a touch-sensitive surface of a touchscreen device, the first sensor being other than an acoustic sensor, the touch-sensitive surface not containing an acoustic sensor usable to detect a contact with the touch-sensitive surface;
a second sensor, the second sensor being an acoustic sensor, the second sensor being separate and distinct from the touch-sensitive surface and not used to detect the contact;
the memory including at least one module including executable instructions that when executed on the at least one processor is configured to detect the contact by both the first sensor and the second sensor by:
receive first data associated with the touch-sensitive surface describing the contact as detected by the first sensor; and
determine an ability of the touch-sensitive surface to accurately detect the contact by comparing the first data to second data, the second data associated with the second sensor describing the contact detected by the second sensor.
17. The system of claim 16, wherein the comparison determines a latency associated with detection of the contact.
18. The system of claim 16, wherein the touchscreen device is a mobile electronic device.
19. The system of claim 16, wherein the first data is based on a time point associated with the contact as detected by the first sensor, and wherein the second data is based on a time point associated with the contact as detected by the second sensor.
20. The system of claim 16, wherein the second sensor is associated with a test apparatus.
US13/293,060 2011-11-09 2011-11-09 Acoustic touch sensitive testing Active US9785281B2 (en)
US13/293,060 US9785281B2 (en) 2011-11-09 2011-11-09 Acoustic touch sensitive testing
CN201210446236.9A CN102929453B (en) 2011-11-09 2012-11-09 Acoustic touch sensitive test
US20130113751A1 US20130113751A1 (en) 2013-05-09
US9785281B2 true US9785281B2 (en) 2017-10-10
ID=47644276
US13/293,060 Active US9785281B2 (en) 2011-11-09 2011-11-09 Acoustic touch sensitive testing
US (1) US9785281B2 (en)
CN (1) CN102929453B (en)
TW561685B (en) 2001-05-28 2003-11-11 Sanyo Electric Co Method of controlling a stepping motor
US20040120459A1 (en) 2002-12-18 2004-06-24 Crowley John P. Industrial machine vision system having a direct conversion X-ray detector
US20050277323A1 (en) 2004-06-15 2005-12-15 Formfactor, Inc. Mechanically reconfigurable vertical tester interface for IC probing
US20060007177A1 (en) 2004-07-07 2006-01-12 Mclintock Kevin S Method and apparatus for calibrating an interactive touch system
US20080024790A1 (en) 2006-07-31 2008-01-31 Raintree Scientific Instruments (Shanghai) Corporation Image Rotation Devices and Their Applications
JP2010055576A (en) 2008-08-29 2010-03-11 Sharp Corp Image processor, image processing program, computer-readable recording medium, electronic apparatus and image processing method
CN101681219A (en) 2007-06-21 2010-03-24 泰科电子公司 Method and system for calibrating an acoustic touchscreen
CN101825970A (en) 2010-05-13 2010-09-08 友达光电股份有限公司 Tracking method and tracking system of touch-control points
US20100266993A1 (en) 2009-04-16 2010-10-21 Redbird Flight Simulations, Inc. Interchangeable instrument panel, throttle quadrant, and control device system
TWM401767U (en) 2010-11-05 2011-04-11 Chroma Ate Inc Optical measurement system structure
TWM403688U (en) 2010-07-26 2011-05-11 Chroma Ate Inc A multi-stages tester for touch panel
TW201128198A (en) 2010-02-10 2011-08-16 Nas Technologies Corps Touch panel testing equipment
TW201133412A (en) 2010-03-19 2011-10-01 Cct Co Ltd Method of using radar vehicle detector to determine vehicle type, speed, and radar detection zone width
US20120218205A1 (en) 2011-02-28 2012-08-30 Samsung Electronics Co. Ltd. Touchscreen-enabled mobile terminal and text data output method thereof
US20120218307A1 (en) 2011-02-25 2012-08-30 Asustek Computer Inc. Electronic device with touch control screen and display control method thereof
US20130253871A1 (en) 2012-03-20 2013-09-26 Hurco Companies, Inc. Method for measuring a rotary axis of a machine tool system
2011-11-09 US US13/293,060 patent/US9785281B2/en active Active
2012-11-09 CN CN201210446236.9A patent/CN102929453B/en active IP Right Grant
US20110122072A1 (en) 2009-11-26 2011-05-26 Jao-Ching Lin Touch device
US20150160781A1 (en) 2011-01-24 2015-06-11 Microsoft Microsoft Technology Licensing, LLC Touchscreen Testing
"Actuation Force of Touch Screen", Solutions @ Mecmesin, retrieved from <http://www.ArticleOnePartners.com/index/servefile?fileId=188971>,(Dec. 31, 2010), 1 page.
"Capacitive Touch Sensors-Application Fields, Technology Overview and Implementation Example", Fujitsu Microelectronics Europe GmbH; retrieved from http://www.fujitsu.com/downloads/MICRO/fme/articles/fujitsu-whitepaper-capacitive-touch-sensors.pdf on Jul. 20, 2011, (Jan. 12, 2012),12 pages.
"Corrected Notice of Allowance Issued in U.S. Appl. No. 13/152,991", dated Dec. 12, 2016, 2 Pages.
"Corrected Notice of Allowance Issued in U.S. Appl. No. 13/152,991", dated Nov. 21, 2016, 2 Pages.
"Fifth Office Action Issued in Chinese Patent Application No. 201210018527.8", dated Jun. 28, 2016, 15 Pages.
"Final Action Issued in U.S. Appl. No. 13/659,777", dated Apr. 29, 2015, 10 Pages.
"Final Office Action Issued in U.S. Appl. No. 13/152,991", dated Jul. 20, 2016, 11 Pages.
"Final Office Action Issued in U.S. Appl. No. 13/198,036", dated Jun. 16, 2016, 41 Pages.
"Final Office Action Issued in U.S. Appl. No. 13/198,036", dated May 4, 2017, 62 Pages.
"Final Office Action Issued in U.S. Appl. No. 13/659,777", dated Aug. 13, 2015, 11 Pages.
"Final Office Action Issued in U.S. Appl. No. 13/664,840", dated Mar. 11, 2016, 16 Pages.
"Final Office Action Issued in U.S. Appl. No. 13/664,840", dated Mar. 27, 2015, 15 Pages.
"Final Office Action Issued in U.S. Appl. No. 13/664,840", dated May 4, 2017, 18 Pages.
"Final Office Action Issued in U.S. Appl. No. 13/905,093", dated Jun. 15, 2016, 18 Pages.
"Final Office Action Issued in U.S. Appl. No. 14/620,819", dated Jan. 27, 2017, 9 Pages.
"First Office Action and Search Report Issued in Chinese Patent Application No. 201210052934.0", dated Dec. 24, 2015, 13 pages.
"Fourth Office Action and Search Report Issued in Chinese Patent Application No. 201210018527.8", dated Jan. 18, 2016, 15 Pages.
"Fourth Office Action Issued in Chinese Patent Application No. 201210029859.6", dated Oct. 21, 2015, 7 Pages.
"Haptic-Actuator Controllers", retrieved from <http://www.maxim-ic.com/products/data-converters/touch-interface/haptic-actuator.cfm> on May 4, 2011, 1 page.
"Input Testing Tool", U.S. Appl. No. 13/659,777, filed Oct. 24, 2012, 45 pages.
"International Preliminary Report on Patentability Issued in PCT Application No. PCT/US2014/064483", dated Jan. 27, 2016, 8 Pages.
"International Search Report & Written Opinion for PCT Patent Application No. PCT/US2014/064483", dated Mar. 16, 2015, 10 Pages.
"International Search Report and Written Opinion Issued in PCT Application No. PCT/US2012/053621", dated Feb. 20, 2013, 10 Pages.
"Moss User Permissions and 'Modify Shared Webpart' Link", Retrieved from <<https://social.technet.microsoft.com/forums.sharepoint/en-US/bf893562-7f49-48bd-8d12-5befca34dba5/moss-user-permissions-and-modify-shared-webpart-link?forum=sharepointgenerallegacy>>, Retrieved Date: Aug. 8, 2011, 3 Pages.
"Non Final Office Action Issued in U.S. Appl No. 13/229,121", dated Jun. 7, 2013, 14 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 13/152,991", dated Dec. 14, 2015, 15 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 13/198,036", dated Sep. 9, 2016, 52 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 13/228,283", dated Aug. 27, 2013, 13 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 13/659,777", dated Nov. 4, 2014, 11 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 13/664,840", dated Jul. 15, 2014, 15 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 13/664,840", dated Jul. 5, 2016, 16 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 13/664,840", dated Sep. 18, 2015, 17 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 13/905,093", dated Jan. 29, 2016 18 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 14/620,819", dated Aug. 12, 2016, 23 Pages.
"Non Final Office Action Issued in U.S. Appl. No. 14/662,512", dated Oct. 5, 2015, 9 Pages.
"Notice of Allowance Issued in Chinese Patent Application No. 201210018527.8", dated Jan. 9, 2017, 4 Pages.
"Notice of Allowance Issued in Chinese Patent Application No. 201210029859.6", dated Feb. 6, 2016, 4 Pages.
"Notice of Allowance Issued in Chinese Patent Application No. 201210052934.0", dated Dec. 28, 2016, 4 Pages.
"Notice of Allowance Issued in Chinese Patent Application No. 201210446236,9", dated Apr. 26, 2016, 4 Pages.
"Notice of Allowance Issued in Taiwan Patent Application No. 100135900", dated Mar. 28, 2016, 4 Pages.
"Notice of Allowance Issued in Taiwan Patent Application No. 101107036", dated Feb. 16, 2016, 4 Pages.
"Notice of Allowance Issued in Taiwan Patent Application No. 103124288", dated Apr. 14, 2016, 4 Pages.
"Notice of Allowance Issued in U.S. Appl. No. 13/152,991", dated Aug. 25, 2016, 8 Pages.
"Notice of Allowance Issued in U.S. Appl. No. 13/659,777", dated Dec. 18, 2015, 5 Pages.
"Notice of Allowance Issued in U.S. Appl. No. 14/662,512", dated Apr. 19, 2016, 8 Pages.
"Office Action Issued in Taiwan Patent Application No. 101107036", dated Oct. 27, 2015, 9 Pages.
"Office Action Issued in Taiwan Patent Application No. 102100728", dated Jul. 7, 2016, 10 Pages.
"Office Action Issued in Taiwan Patent Application No. 102100728", dated Nov. 9, 2016, 4 Pages.
"OptoFidelity Touch & Test", retrieved from <http://www.ArticleOnePartners.com/index/servefile?fileId=188969, (Feb. 20, 2012), 2 pages.
"OptoFidelity Touch and Test", retrieved from <http://www.ArticleOnePartners.com/index/servefile?fileId=188420>, (May 4, 2012), 2 pages.
"OptoFidelity Two Fingers-robot", video available at <http://www.youtube.com/watch?v=YppRASbXHfk&feature=player-embedded#!section>, (Sep. 15, 2010), 2 pages.
"Public or Private Articles", Retrieved from<<http://web.archive.org/web/20111112135351/http://www.presspublisher.com/features/public-or-private-articles/>>, Retrieved Date: Aug. 11, 2011, 3 Pages.
"Resistive Touch Screen-Resistance Linearity Test", video available at <http://www.youtube.com/watch?v=hb23GpQdXXU>, (Jun. 17, 2008), 2 pages.
"Search Report Issued in European Patent Application No. 12744496.6", dated Oct. 7, 2016, 8 Pages.
"Search Report Issued in European Patent Application No. 12744979.1", dated Mar. 28, 2017, 10 Pages.
"Searh Report Issued in European Patent Application No. 12754354.4", dated Aug. 6, 2014, 8 Pages.
"Second Office Action and Search Report Issued in Chinese Patent Application No. 201210446236.9", Mailed Date: Nov. 13, 2015, 13 Pages.
"Second Office Action Issued in Chinese Patent Application No. 201210052934.0", dated Aug. 4, 2016, 13 Pages.
"Second Written Opinion Issued in PCT Application No. PCT/US2014/064483", dated Oct. 2, 2015, 7 Pages.
"Setting Sharing Permissions for Google Docs and Google Sites", Retrieved from <<http://www.library.kent.edu/files/SMS-Sharing-Permissions.pdf>>, Retrieved Date: Aug. 8, 2011, 8 Pages.
"Share Office Documents in SkyDrive", Retrieved from http://office.microsoft.com/enus/web-apps-help/share-office-documents-in-skydrive-HA101820121.aspx>>, Retrieved Date: Aug. 8, 2011, 3 Pages.
"Shared Folder Permissions", Retrieved from <<http://web.archive.org/web/20111020095512/http://www.tech-faq.com/shared-folder-permissions.html>>, Aug. 8, 2011, 7 Pages.
"Smartphone Automatic Testing Robot at UEI Booth", video available at <www.youtube.com/watch?v=f-Qfns-b9sA>, (May 9, 2012), 2 pages.
"Technology Comparison: Surface Acoustic Wave, Optical and Bending Wave Technology", Retrieved at <<http://multimedia.3m.com/mws/mediawebserver?mwsId=66666UuZjcFSLXTtnXT2NXTaEVuQEcuZgVs6EVs6E666666--&fn=DST-Optical-SAW%20Tech%20Brief.pdf>>, 2009, pp. 4.
"Touch Panel Inspection & Testing Solution", retrieved from <http://www.ArticleOnePartners.com/index/servefile?fileId=188967>, (Dec. 31, 2010),1 page.
"Touch Panel Semi-Auto Handler Model 3810", retrieved from <http://www.chromaus.com/datasheet/3810-en.pdf>, (Dec. 31, 2010), 2 pages.
"Touch Quality Test Robot", U.S. Appl. No. 13/530,692, filed Jun. 22, 2012, 40 pages.
"TouchSense Systems Immersion", retrieved from <http://www.ArticleOnePartners.com/index/servefile?fileId=188486>, (Jun. 19, 2010), 20 pages.
"Capacitive Touch Sensors—Application Fields, Technology Overview and Implementation Example", Fujitsu Microelectronics Europe GmbH; retrieved from http://www.fujitsu.com/downloads/MICRO/fme/articles/fujitsu-whitepaper-capacitive-touch-sensors.pdf on Jul. 20, 2011, (Jan. 12, 2012),12 pages.
"OptoFidelity Two Fingers-robot", video available at <http://www.youtube.com/watch?v=YppRASbXHfk&feature=player—embedded#!section>, (Sep. 15, 2010), 2 pages.
"Setting Sharing Permissions for Google Docs and Google Sites", Retrieved from <<http://www.library.kent.edu/files/SMS—Sharing—Permissions.pdf>>, Retrieved Date: Aug. 8, 2011, 8 Pages.
Asif , Muhammad et al., "MPEG-7 Motion Descriptor Extraction for Panning Camera Using Sprite Generated", In Proceedings of AVSS 2008, Available at <http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4730384>,(Sep. 2008), pp. 60-66.
Binnis, Francis S., "Multi-"Touch" Interaction via Visual Tracking", Bachelor of Science in Computer Science with Honours, The University of Bath, available at <<http://www.cs.bath.ac.uk/˜mdv/courses/CM30082/project.bho/2008-9/Binns-FS-dissertation-2008-9.pdf>>,(May 2009), 81 pages.
Brodkin, Jon "Windows 8 hardware: Touchscreen, sensor support and robotic fingers", <<http://arstechnica.com/business/news/2011/09/windows-8-hardware-touch-screens-sensor-support-and-robotic-fingers.ars>>, (Sep. 13, 2011), 1 Page.
Buffer, Y "Robot Touchscreen Analysis", <<http://ybuffet.posterous.com/labsmotocom-blog-archive-robot-touchscreen-an>>, (Apr. 19, 2010), 2 Pages.
Cao, Xiang et al., "Evaluation of an On-line Adaptive Gesture Interface with Command Prediction", In Proceedings of GI 2005, Available at <http://citeseerx.ist.psu.edu/viewdoc/download;jessionid=DAB1B08F620C23464427932BAF1ECF49?doi=10.1.1.61.6749&rep=rep1&type=pdf>,(May 2005), 8 pages.
Dang, Chi T., et al., "Hand Distinction for Multi-Touch Tabletop Interaction", University of the Augsburg: Institute of Computer Science; Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, (Nov. 23-25, 2009), 8 pages.
Dillow, Clay "Liquid-Filled Robot Finger More Sensitive to Touch Than a Human's", retrieved from <www.popsci.com/technoloqy/article/2012-06/new-robot-finger-more-sensitive-touch-human> on Jul. 27, 2012, (Jun. 19, 2012), 3 pages.
Fritz, et al., "Using Low Power Mode on the MPR083 and MPR084", Retrieved at <<http://cache.freescale.com/files/sensors/doc/app-note/AN3583.pdf>>, PAIR '10, Nov. 2011, pp. 5.
Fritz, et al., "Using Low Power Mode on the MPR083 and MPR084", Retrieved at <<http://cache.freescale.com/files/sensors/doc/app—note/AN3583.pdf>>, PAIR '10, Nov. 2011, pp. 5.
Hoggan, Eve et al., "Mobile Multi-Actuator Tactile Displays", In 2nd international conference on Haptic and audio interaction design, retrieved from <http://www.dcs.gla.ac.uk/˜stephen/papers/HAID2.pdf >,(Oct. 29, 2007),12 pages.
Hoshino, et al., "Pinching at finger tips for humanoid robot hand", Retrieved at <<http://web.mit.edu/zoz/Public/HoshinoKawabuchiRobotHand.pdf>>, (Jun. 30, 2005), 9 Pages.
Kastelan, et al., "Stimulation Board for Automated Verification of Touchscreen-Based Devices", 22nd International Conference on Field Programmable Logic and Applications, Avaiable at <https://www2.lirmm.fr/lirmm/interne/BIBLI/CDROM/MIC/2012/FPL-202/Papers/PHD7.pdf>,(Aug. 29, 2012), 2 pages.
Kastelan, et al., "Stimulation Board for Automated Verification of Touchscreen-Based Devices", 22nd International Conference on Field Programmable Logic and Applications, Avaiable at <https://www2.lirmm.fr/lirmm/interne/BIBLI/CDROM/MIC/2012/FPL—202/Papers/PHD7.pdf>,(Aug. 29, 2012), 2 pages.
Li, et al., "Role Based Access Control for Social Network Sites", In Proceedings of Joint Conferences on Pervasive Computing (JCPC), Dec. 3, 2009. pp. 389-393.
McGlaun, Shane "Microsoft's Surface 2.0 Stress Testing Robot Called Patty Shown off for First Time", Retrieved at <<http://www-.slashgear.-com/microsofts-surface-2--0-stress-testing-robot--called-patty-shown-off--for--first-time-19172971/>>, (Aug. 19, 2011), 1 Page.
McGlaun, Shane "Microsoft's Surface 2.0 Stress Testing Robot Called Patty Shown off for First Time", Retrieved at <<http://www—.slashgear.—com/microsofts-surface-2—-0-stress-testing-robot—-called-patty-shown-off—-for—-first-time-19172971/>>, (Aug. 19, 2011), 1 Page.
McMahan, William et al., "Haptic Display of Realistic Tool Contact via Dynamically Compensated Conntrol of a Dedicated Actuator", International Conference on Intelligent Robots and Systems, St. Louis, MO, Oct. 11-15, 2009, retrieved from <http://respiratory.upenn.edu/meam-papers/222>,(Dec. 15, 2009), 9 pages.
McMahan, William et al., "Haptic Display of Realistic Tool Contact via Dynamically Compensated Conntrol of a Dedicated Actuator", International Conference on Intelligent Robots and Systems, St. Louis, MO, Oct. 11-15, 2009, retrieved from <http://respiratory.upenn.edu/meam—papers/222>,(Dec. 15, 2009), 9 pages.
Takeuchi, et al., "Development of a Muti-fingered Robot Hand with Softness changeable Skin Mechanism", International Symosium on and 2016 6th German Conference on Robotics (ROBOTIK), Retrieved at <<http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=05756853>>,(Jun. 7, 2010), 7 Pages.
Tao, Yufei et al., "An Efficient Cost Model for Optimization of Nearest Neighbor Search in Low and Medium Dimensional Space", Knowledge and Data Engineering, vol. 16 Issue: 10, retrieved from <<http://www.cais.ntu.edu.sg/˜jzhang/papers/ecmonns.pdf>> on Mar. 16, 2011,(Oct, 2004),16 pages.
Tsuchiya, Sho et al., "Vib-Touch: Virtual Active Touch Interface for Handheld Devices", In Proceedings of the 18th IEEE International Symposium on Robot and Human Interactive Communication, Available at <http://www.mech.nagoya-u.ac.jp/asi/en/member/shogo—okamoto/papers/tsuchiyaROMAN2009.pdf>,(Oct. 2009), pp. 12-17.
Westman, Tapani et al., "Color Segmentation by Hierarchical Connected Components Analysis with Image Enhancement by Symmetric Neighborhood Filter", Pattern Recognition, 1990. Proceedings., 10th International Conference on Jun. 16-21, 1990, retrieved from <<http://ieexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=118219>> on Mar. 16, 2011,(Jun. 16, 1990), pp. 796-802.
Wilson, Andrew D., "TouchLight: An Imaging Touch Screen and Display for in Gesture-Based Interaction", In Proceedings of ICIM 2004, Available at <http://citeseerx.ist.psu.edu/viewdoc/download?doi=doi=10.1.1.95.3647&rep=rep1&type=pdf>,(Oct. 2004), 8 pages.
Wimmer, et al., "Modular and Deformable Touch-Sensitive Surfaces Based on Time Domain Reflectometry", Retrieved at <<http://www.medien.ifi.lmu.de/pubdb/publications/pub/wimmer2011tdrTouch/wimmer2011tdrTouch.pdf>>, UIST '11, Oct. 16-19, 2011, pp. 10.
US20130113751A1 (en) 2013-05-09
CN102929453A (en) 2013-02-13
CN102929453B (en) 2016-08-17
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:UZELAC, ALEKSANDAR;BATCHVAROV, ANDREY B.;REEL/FRAME:027203/0477