Abstract:
Using ground truth events and sensors available on a first mobile device, an algorithm executed at the first mobile device determines the mobile device&#39;s position; the first mobile device also obtains wireless signal information; the wireless signal information is associated with the position determined via the sensor data and ground truth events; the associated data is stored and may be sent to or used by a second device; the second device may then, for example, detect then-current wireless signal information and may locate the second device by looking up the then-current wireless signal information in the associated data.

Description:
RELATIONSHIP TO OTHER APPLICATIONS AND PRIORITY CLAIM 
       [0001]    This application claims the benefit of and incorporates by this reference, provisional patent application No. 61552092, filed Oct. 27, 2011. 
     
    
     BACKGROUND 
       [0002]    Stationary and mobile computing devices—including cellular telephones and a range of other portable computers—include many electro-magnetic radiation-based one- and two-way wireless communication technologies, such as cellular telephone technologies, GPS, WIFI, Bluetooth, and Near-Field Communication (“NFC”). These wireless systems emit and receive electro-magnetic radiation, typically in the radio frequency bands, with various characteristics, such as signal strength and wireless channel (or frequency or frequency-block) utilization. Fixed and wireless data networks also commonly include information in the networks&#39; bit stream such as “Media Access Control address” (“MAC address”), “Ethernet hardware address” (“EHA”), or “Physical Address,” which, generally, are unique identifiers assigned to a network interface or network node. As used herein, “Wireless Signal Information” is any radio frequency (e.g., WiFi, Bluetooth, NFC, etc.) or timing signal (e.g., Ultra-wideband or “UWB”) that is detectable by a mobile device and can be associated with a unique terminal, such as via MAC address. 
         [0003]    Many of the wireless systems mentioned above enable communication between a first computing device and other computing devices and/or they may be used to locate the first computing device in a network or in physical space. 
         [0004]    Systems exist which create digital images utilizing pixels and which display digital images on computing devices with pixel-based display technologies. 
         [0005]    Many mobile computing devices now also contain one or more sensors , such as multi-axis gyros, compasses, barometers, accelerometers, microphones or other hardware or software base technologies, which technologies are used within mobile devices to detect changes in position and orientation. 
         [0006]    However, not known are systems which determine a user&#39;s position in a pixel-based image from user interaction and from sensors and which correlate this position information with wireless signal information, the resulting information being stored and made available to other computing devices so that the other computing devices can, for example, locate themselves based on then-available wireless signal information. 
         [0007]    Needed is a method and system to determine the position of a mobile device in a pixel-based image based on user input and sensor data, to contemporaneously receive and log wireless signal information, to correlate and store the resulting information, and to make the correlated information available to other computing devices. 
       SUMMARY 
       [0008]    Using ground truth events and sensors available on a first mobile device, an algorithm executed at the first mobile device determines the mobile device&#39;s position; the first mobile device also obtains wireless signal information; the wireless signal information is associated with the position determined via the sensor data and ground truth events; the associated data is stored and may be sent to or used by a second device; the second device may then, for example, detect then-current wireless signal information and may locate the second device by looking up the then-current wireless signal information in the associated data. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0009]      FIG. 1  is a diagram of a user traveling in a building, illustrating a “Ground Truth Event,” a sensor-determined path of the user, and wireless base stations “A,” “B,” and “C.” 
           [0010]      FIG. 2  is a network and device diagram illustrating two mobile devices, two wireless base stations, a Network, and a Server. 
           [0011]      FIG. 3  is an outline of a process in which a first mobile device&#39;s coordinate position is determined based on user input and sensor data, in which wireless signal information is received, in which the determined coordinate position and the wireless signal information are sent to a server and stored, and in which the correlated information is used to determine coordinate position by a second wireless device. 
           [0012]      FIG. 4  is a functional block diagram of exemplary computing devices and some data structures and/or components thereof. 
       
    
    
     DETAILED DESCRIPTION 
       [0013]    As used herein, “sensors” comprise multi-axis gyros, compass, barometer, accelerometer, GPS, microphone or other hardware or software base technologies used within the mobile device, which technologies are used within the mobile device to detect changes in position and orientation. “Dead reckoning” is a method of determining motion and distance traveled including vector information. “Wireless signal information” is any radio frequency (e.g., WiFi, Bluetooth, NFC, etc.) or timing signal (e.g., UWB) that is detected by the Mobile Device and can be associated with a unique terminal (e.g., MAC address or similar). A “pixel coordinate” identifies a location in space corresponding to pixel coordinates in an image. A “ground truth event” is an explicit act on behalf of or by the user of the mobile device to specify the location of the user or the mobile device. A ground truth event may comprise a user touching an image of a map (or otherwise identifying coordinates on a map) or may comprise taking a photo of a barcode, QR Code, at location with a known coordinate position, or pixel coordinate, relative to an image or may comprise becoming proximate to an NFC or other wireless device at location with a known coordinate position or pixel coordinate relative to an image. 
         [0014]    Referring to  FIGS. 1 ,  2 , and  3 , an image, such as Image  226 , has been sent to a mobile device, such as Mobile Device 1,  205 , or is otherwise present on Mobile Device One  205 , as Image  207 ; step  305  in  FIG. 3 . Image  207  may be rendered at Mobile Device One  205  as image  100  in a display with which the user can interact (such as via touch screen, a moveable cursor) with to indicate the user&#39;s position in the image. 
         [0015]    At location  105 , a ground truth event occurs; step  310  in  FIG. 3 . The ground truth event may be the user of Mobile Device One  205 , touching the Image  100 , such as a touch-screen in Mobile Device One  205 , to indicate the user&#39;s position (and that of Mobile Device One  205 ); the ground truth event may also occur when the user becomes proximate to a Ground Truth Event Sensor, such as “GTE  230 ” in  FIG. 2 . As noted above, this may involve the user of the Mobile Device One  205  taking a photograph of the GTE  230  or it may involve the detection of proximity by sensors in Mobile Device One  205  and/or in the GTE Sensor  230 . The GTE  230  may comprise data for its Known Position  231  (which data may be communicated to the Mobile Device One  205 ) or the known position of GTE  230  may be known by Server  225  (or by the Mobile Device One  205 ), such as GTE Sensor Known Position  227 , in which case the GTE  230  and/or the Mobile Device One  205 , may report (such as to the Mobile Device One  230 , and/or to the Server  225 ) that the Mobile Device One  205  and GTE  230  became proximate, which report may be reported back to the Mobile Device One  205  as a location. The ground truth event occurs and is stored at step  310  as a location, such as a set of coordinates in the Image  207 , a latitude and longitude, an address, or similar. 
         [0016]    At step  320 , the Mobile Device One  205  receives sensor data, such as from Sensors  206 . At step  315 , the Mobile Device One  205  (or the Server  225 , using sensor data received from the Mobile Device One  205 ) may execute Dead Reckoning Algorithm  208  to determine the position of the user (or, equivalently, the Mobile Device One  205 , carried by the user) relative to the ground truth event  105  and the coordinate positions in Image  207  ( 100 , in  FIG. 1 ), which coordinate positions may be stored in the Mobile Device One  205  as Sensor Determined Locations  209 ; illustrated in  FIG. 3  as step  325 . The Dead Reckoning Algorithm  208  may determine the position of the user, for example, based on vector analysis of changes in the sensor data. The sequence of circles in  FIG. 1  represents coordinate positions determined by the Dead Reckoning Algorithm  208 . More than one set of sensor data may be received, which sensors operate at different sample rates, all of which may be input into the Dead Reckoning Algorithm  208 . 
         [0017]    At step  330 , the Mobile Device One  205  receives and stores Wireless Signal Information  210 , received from, for example, Base Station A  212  ( 120  in  FIG. 1 ), Base Station B  215  ( 125  in  FIG. 1 ), and Base Station C ( 130  in  FIG. 1 ; not shown in  FIG. 2 ). At step  335 , the Wireless Signal Information  210  received at the Sensor Determined Locations  209  is stored in association with the Sensor Determined Locations  209 . 
         [0018]      FIG. 1  illustrates the user passing through a building, through a security perimeter at the dotted line at  140 , and then along a path illustrated by a sequence of circles, passing by Base Stations B ( 125 ), C ( 130 ), and A ( 120 ), while performing the process discussed above. 
         [0019]    At step  340 , the stored and correlated Wireless Signal Information  210  and Sensor Determined Locations  209  are transmitted, for example, to the Server  225 , where the correlated information may be stored as Correlated WSI and Position  228 . Transmission to the Server  225  may be in batches or in real time. 
         [0020]    At step  345 , the Server  225  may transmit the Correlated WSI and Position  228  to a recipient, such as Mobile Device Two  240  (represented in  FIG. 2  as Correlated WSI and Position  242 ). 
         [0021]    At step  350 , the recipient, such as Mobile Device Two  240  receives Wireless Signal Information  241  from the environment, such as from, for example, Base Station A  212  ( 120  in  FIG. 1 ), Base Station B  215  ( 125  in  FIG. 1 ), and Base Station C ( 130  in  FIG. 1 ). 
         [0022]    At step  355 , the recipient, such as Mobile Device Two  240 , uses the Wireless Signal Information  241  and the Correlated WSI and Position  242  to determine the location of Mobile Device Two  240 . Examples of how to determine location include a best-fit of the Wireless Signal Information  241  to the wireless signal information in the Correlated WSI and Position  242  and extracting the corresponding position from the Correlated WSI and Position  242 . 
         [0023]    Capturing the Wireless Signal Information  210  and  241  may be by a routine or application on the mobile devices, which captures the Wireless Signal Information as frequently as the rate of change in, for example, the Sensor Determined Locations  209 . If the user moves at a pace of 100 pixels per second then the Wireless Signal Information could be captured at this same rate or a lesser rate. 
         [0024]    Following is Table 1, containing position events, locations in pixel coordinates in an image (the pixel coordinates do not necessarily map onto the image in  FIG. 1 ), and Wireless Signal Information (the locations of Base Stations A, B, and C in  FIG. 1  are provided as approximate examples). Table 1 may be a sample of Correlated Wireless Signal Information and Position. 
         [0000]    
       
         
               
               
               
             
               
               
               
               
             
           
               
                 TABLE 1 
               
               
                   
               
               
                   
                   
                 Wireless Signal Information, where 
               
               
                   
                   
                 A = FIG. 1, 120 
               
               
                   
                   
                 B = FIG. 1, 125 
               
               
                 Position Event 
                 Location 
                 C = FIG. 1, 130 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                 Initial Ground  
                 Pixel Coordinate 
                 A.  
                 Physical Address: 70-1A-04-A6-A4-C0;  
               
               
                 Truth Event 
                 (345, 819) 
                   
                 Channel 6; Signal Strength: −89dBm 
               
               
                 FIG. 1, 105 
                   
                 B.  
                 Physical Address: 00-E0-8C-6C-8F-61;  
               
               
                   
                   
                   
                 Channel 1; Signal Strength: −45dBm 
               
               
                   
                   
                 C.  
                 Physical Address: 00-19-C0-61-9D-62;  
               
               
                   
                   
                   
                 Channel 11; Signal Strength: −73dBm 
               
               
                 Dead Reckoning  
                 Pixel Coordinate 
                 A.  
                 Physical Address: 70-1A-04-A6-A4-C0;  
               
               
                 Event 1 
                 (346, 810) 
                   
                 Channel 6; Signal Strength: −83dBm 
               
               
                 FIG. 1, 110 
                   
                 B.  
                 Physical Address: 00-E0-8C-6C-8F-61;  
               
               
                   
                   
                   
                 Channel 1; Signal Strength: −54dBm 
               
               
                   
                   
                 C.  
                 Physical Address: 00-19-C0-61-9D-62;  
               
               
                   
                   
                   
                 Channel 11; Signal Strength: −76dBm 
               
               
                 Dead Reckoning  
                 Pixel Coordinate 
                 A.  
                 Physical Address: 70-1A-04-A6-A4-C0;  
               
               
                 Event 2 
                 (345, 800) 
                   
                 Channel 6; Signal Strength: −62dBm 
               
               
                 FIG. 1, 115 
                   
                 B.  
                 Physical Address: 00-E0-8C-6C-8F-61;  
               
               
                   
                   
                   
                 Channel 1; Signal Strength: −66dBm 
               
               
                   
                   
                 C.  
                 Physical Address: 00-19-C0-61-9D-62;  
               
               
                   
                   
                   
                 Channel 11; Signal Strength: −92dBm 
               
               
                   
               
             
          
         
       
     
         [0025]    In  FIG. 2 , Mobile Device One  205  and Mobile Device Two  240  may be portable computers, cell phones, tablet computers, laptops, or other similar computers. The Mobile Devices can at least receive transmissions from at least one of the Base Stations (Base Station A,  212 , and Base Station B,  215 ). The Mobile Devices may or may not be able to form a network connection with one or both of the Base Stations. The Base Stations may or may not have a connection to the Network  220 . The Mobile Devices need to be able to obtain an identifier for the Base Stations, such as the “Physical Address” listed above in Table 1, in addition to the signal strength. Server  225  may be a computing device. Server  225  may represent more than one computing device. 
         [0026]      FIG. 4  is a functional block diagram of exemplary computing devices and some data structures and/or components thereof, such as the computing devices shown in the other figures. In some embodiments, the computing device  400  may include many more components than those shown in  FIG. 4 . However, it is not necessary that all of these generally conventional components be shown in order to disclose an illustrative embodiment. As shown in  FIG. 4 , the computing device  400  includes a network interface  405  for connecting to the network  220 . 
         [0027]    The computing device  400  also includes at least one processing unit  415 , memory  435 , and an optional display  410 , all interconnected along with the network interface  405  via a bus  425 . The memory  435  generally comprises a random access memory (“RAM”), a read only memory (“ROM”), and a permanent mass storage device, such as a disk drive or SDRAM (synchronous dynamic random-access memory). The memory  435  stores program code for routines  445 , such as, for example, the dead reckoning algorithms, as well as web browsing applications, web serving applications, email servers and client applications, and database applications. In addition, the memory  435  also stores an operating system  440 . These software components may be loaded from a non-transient computer readable storage medium  430  into memory  435  of the computing device  400  using a drive mechanism (not shown) associated with a non-transient computer readable storage medium  430 , such as a floppy disc, tape, DVD/CD-ROM drive, memory card, or other like storage medium. In some embodiments, software components may also or instead be loaded via a mechanism other than a drive mechanism and computer readable storage medium  430  (e.g., via network interface  405 ). 
         [0028]    The computing device  400  may also comprise hardware supporting optional input modalities, Optional Input  420 , such as, for example, a touchscreen, a keyboard, a mouse, a trackball, a stylus, a microphone, and a camera. 
         [0029]    Computing device  400  also comprises or communicates via bus  425  with workflow data store  465 . In various embodiments, bus  425  may comprise a storage area network (“SAN”), a high speed serial bus, and/or via other suitable communication technology. In some embodiments, computing device  400  may communicate with workflow data store  465  via network interface  405 . 
         [0030]    The above Detailed Description of embodiments is not intended to be exhaustive or to limit the disclosure to the precise form disclosed above. While specific embodiments of, and examples are described above for illustrative purposes, various equivalent modifications are possible within the scope of the system, as those skilled in the art will recognize. For example, while processes or blocks are presented in a given order, alternative embodiments may perform routines having operations, or employ systems having blocks, in a different order, and some processes or blocks may be deleted, moved, added, subdivided, combined, and/or modified. While processes or blocks are at times shown as being performed in series, these processes or blocks may instead be performed in parallel, or may be performed at different times. Further, any specific numbers noted herein are only examples; alternative implementations may employ differing values or ranges.