Source: https://patents.justia.com/patent/9386417
Timestamp: 2020-07-04 06:32:44
Document Index: 613076636

Matched Legal Cases: ['§119', 'Application No. 61', 'in fine', 'in fine', 'in fine', 'in fine', 'in fine']

US Patent for Two-pass copresence Patent (Patent # 9,386,417 issued July 5, 2016) - Justia Patents Search
Justia Patents Computer ConferencingUS Patent for Two-pass copresence Patent (Patent # 9,386,417)
Two-pass copresence
May 24, 2015 - Google
The disclosure includes a system and method for detecting fine grain copresence between users. The system includes a processor and a memory storing instructions that when executed cause the system to: process one or more signals to determine coarse grain location information of a first device and a second device; determine whether the first device and the second device are copresent based on the coarse grain location information; in response to determining that the first device and the second device are copresent based on the coarse grain location information, transmit a signal to the second device to alert the second device to listen for a fine grain copresence token from the first device; and refine copresence based on receiving an indication that the second device has received the fine grain copresence token.
This application is a continuation of and claims priority to U.S. application Ser. No. 14/313,949, filed Jun. 24, 2014, titled “Two-Pass Copresence,” which claims priority under 35 U.S.C. §119, of U.S. Provisional Patent Application No. 61/941,467, filed Feb. 18, 2014 and entitled “Two-Pass Copresence,” each of which is incorporated by reference in its entirety.
According to one innovative aspect of the subject matter described in this disclosure, a system for determining fine grain copresence of user devices includes a processor and a memory storing instructions that, when executed, cause the system to: receive one or more signals associated with a first user device and a second user device; process the one or more signals to determine coarse grain location information; determine whether the first and the second user device are copresent based on the coarse grain location information; in response to determining that the first and second device are copresent based on the coarse grain location information: transmit a signal to the second device to alert the second device to listen for a fine grain copresence token, and initiate transmission of the fine grain copresence token from the first user device; refine copresence based on receiving an indication that the second device has received the fine grain copresence token; and performing an action based on copresence of the first and second device.
These and other embodiments may each optionally include one or more of the following features. For instance, the coarse grain location information may be supplemented by location information provided by one or more of GPS, Wi-Fi, a cellular network, IP address information, or other sensor information (e.g., accelerometer patterns, barometer, temperature, or the like). For example, by looking at accelerometer patterns, barometer, or temperature information, it may be determined that two devices are located in the same vehicle. The fine grain copresence token may be transmitted using one or more of inaudible audio, audible audio Bluetooth, BLE, Wi-Fi, or near field communications. Coarse grain copresence may be determined using signals including one or more of a text message, an email message, an instant message, a calendar event, a social media post, or other similar signals that may indicate a wider range of proximity.
FIG. 4 is a flowchart of an example method for determining fine grain copresence between user devices.
FIG. 5 is an example block diagram depicting signals transmitted and received grain one embodiment for determining fine grain copresence of user devices.
FIG. 1 illustrates a block diagram of one embodiment of a system 100 for determining copresence of two or more users or devices. It should be understood that the system 100 illustrated in FIG. 1 is representative of an example system for determining copresence of two or more devices, and that a variety of different system environments and configurations are contemplated and are within the scope of the present disclosure. For instance, various functionality may be moved from a server to a client, or vice versa and some implementations may include additional or fewer computing devices, services, and/or networks, and may implement various functionality client or server-side. Further, various entities of the system may be integrated into to a single computing device or system or additional computing devices or systems, etc.
The illustrated system 100 includes user devices 115a . . . 115n that can be accessed by users 125a . . . 125n, one or more social network servers 101, a Short Messaging Service (SMS)/Multi-media Messaging Service (MMS) sever 111, a micro-blogging server 113, an Instant Messaging (IM) server 117 and a copresence server 107. In FIG. 1 and the remaining figures, a letter after a reference number, e.g., “115a,” represents a reference to the element having that particular reference number. A reference number in the text without a following letter, e.g., “115,” represents a general reference to embodiments of the element bearing that reference number. In the illustrated embodiment, the entities of the system 100 are communicatively coupled via a network 105. In some embodiments, the system 100 may include other servers or devices not shown in FIG. 1. For example, the system 100 may include a global positioning system (GPS) to aid in determining the location of a user device 115.
In some embodiments, a proximity application 103a is operable on the copresence server 107, which is coupled to the network 105 via signal line 106. The copresence server 107 can be a hardware server that includes a processor, a memory and network communication capabilities. The copresence server 107, for example, sends and receives data to and from other entities of the system 100 via the network 105. While FIG. 1 illustrates one copresence server 107, the system 100 may include one or more copresence servers 107.
Copresent user devices may be devices that are currently within a proximate distance to each other or within a certain area coverage proximate to one another. Copresence may be determined, for example, by determining that user devices are proximate to each other within a particular distance or area. Further, copresence may be determined on various scales defined by coarse grain copresence determination and fine grain copresence determination. For example, two copresent devices may be within a first proximate distance to each other based on coarse grain copresence determination. Furthering on this example, the two devices may be within may be within a second proximate distance to each other based on fine grain copresence determination, the second proximate distance being a distance within the first proximate distance. As another example, coarse grain copresence may be determined when user devices are proximate within a particular distance and fine grain copresence may be determined when user devices are proximate within a particular distance that is relatively smaller than the distance for determining coarse grain copresence. Coarse grain copresence may be determined using technologies having a relatively wide copresence threshold, for example, GPS techniques, Wi-Fi positioning systems, cellular network location services, or the like. On the other hand, fine grain copresence may be determined using technologies that have a relatively smaller communication area, and therefore a relatively smaller copresence threshold. For example, fine grain copresence may be determined using Bluetooth, Bluetooth Low Energy (BLE), Wi-Fi, infrared, inaudible audio signals, audible audio signals, near field communication, and the like. Additionally, the technologies used to determine fine grain copresence can be effective indoors and can even be used to determine copresence in a particular enclosed area, e.g., a room within a building. Further, determining coarse grain copresence of devices may provide and initial location identification of devices within a wider range. This determination of coarse grain location information may provide location information to determine whether the promixity of the devices may be further specified or refined through fine grain location determination, thereby determining whether the devices may communicate with each other or transmit data to one another via near-field communication technologies such as Bluetooth, Bluetooth Low Energy (BLE), Wi-Fi, infrared, inaudible audio signals, audible audio signals described above.
In some embodiments, the copresence server 107 may determine a coarse grain location (or coarse grain copresence) of user devices 115 based on signals received from other entities of the system 100. For example, the copresence server 107 may receive signals from a user device 115, the SMS/MMS server 111, the micro-blogging server 113, the IM server 117, and/or the social network server 101. Using these signals, the copresence server may determine a coarse grain location of each of the user devices 115 in communication with the copresence server 107. For example, in one embodiment, the copresence server 107 may receive a signal from a user device 115 that includes the location of the device. In another embodiment, the copresence server 107 may receive a signal, such as a text message, an email message, an instant message, a calendar event, a social media post, or other similar signals that may indicate a wider range of proximity from another entity of the system 100. The copresence server 107 may then process the signals to determine a coarse grain location of a user device.
In one embodiment, the copresence server 107 may determine coarse grain copresence of two or more user devices 115 based on the signals even though a physical location of the user devices 115 may be unknown. For example, the copresence server 107 may receive an IM, SMS, or MMS sent from a first user device (e.g., user device 115a) to a second user device (e.g., user device 115n) that reads “I am here.” The copresence server 107 may then receive a second signal sent from the second user device to the first user device that reads “I am here too. Where are you?” Using these signals, the copresence server 107 can determine that the first user and the second user are located in the same general area (i.e., the first and second user device have coarse grain copresence). Based on this determination, the copresence server can initiate a process to refine copresence as described in further detail below.
User devices 115a . . . 115n are computing devices having data processing and communication capabilities. For example, the user devices 115a . . . 115n may be laptop computers, desktop computers, tablet computers, smartphones, portable game players, portable music players, e-readers, televisions, or the like. In some implementations, a user device 115 may include a processor (e.g., virtual, physical, etc.), a memory, a power source, a communication unit, and/or other software and/or hardware components, including, for example, a display, graphics processor, wireless transceivers, keyboard, camera, sensors, firmware, operating systems, drivers, various physical connection interfaces (e.g., USB. HDMI, etc.). The user devices 115a . . . 115n may couple to and communicate with one another and the other entities of the system 100 via the network 105 using a wireless and/or wired connection. While FIG. 1 illustrates two user devices 115a and 115n, the disclosure applies to a system architecture having any number of user devices 115.
The memory 237 includes one or more non-transitory computer-usable (e.g., readable, writeable, etc.) mediums, which can be any tangible apparatus or device that can contain, store, communicate, propagate or transport instructions, data, computer programs, software, code, routines, etc., for processing by or in connection with the processor 235. In some implementations, the memory 237 may include one or more of volatile memory and non-volatile memory. For example, the memory 237 may include, but is not limited, to one or more of a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, an embedded memory device, a discrete memory device (e.g., a PROM, FPROM, ROM), a hard disk drive, an optical disk drive (CD, DVD, Blue-Ray™, etc.). It should be understood that the memory 237 may be a single device or may include multiple types of devices and configurations.
The communication unit 241 may include one or more interface devices for wired and/or wireless connectivity with the network 105 and the other entities and/or components of the system 100 including, for example, the social network server 101, the copresence server 107, the user devices 115, the SMS/MMS server Ill, the micro-blogging server 113, the IM server 117, etc. For instance, the communication unit 241 may include, but is not limited to, CAT-type interfaces; wireless transceivers for sending and receiving signals using Wi-Fi™, Bluetooth®, cellular communications, etc.; USB interfaces; various combinations thereof; etc. The communication unit 241 may be coupled to the network 105 and may be coupled to the other components of the computing device via the bus 220. In some implementations, the communication unit 241 can link the processor 235 to the network 105, which may in turn be coupled to other processing systems. The communication unit 241 can provide other connections to the network 105 and to other entities of the system 100 using various standard communication protocols.
The device detector 202 may then refine copresence by determining fine grain copresence for user devices 115 using Bluetooth, Bluetooth Low Energy (BLE), Wi-Fi, infrared, inaudible audio signals, audible audio signals, near field communication, etc., to transmit or receive a copresence token and/or transmit or receive a response to a copresence token as described in more detail below. For example, after determining coarse grain copresence of one or more user devices 115 based on, for example, the coarse grain location information, the copresence server 107 may select a user device 115 to transmit a fine grain copresence token for determining fine grain copresence. The copresence server 107 may initiate the fine grain copresence detection by sending a signal to the transmitting user device 115. In one embodiment, the signal may be sent to the transmitting user device 115 via network 105. The copresence server 107 may also transmit a signal to a receiving user device to wake up and instruct the user device to listen for a fine grain copresence token.
In some embodiments, the device detector 202 on the copresence server 107 generates a token (for example after registration with the proximity application 103) and permanently assigns the token to the user device 115. In other embodiments, the device detector 202 on the copresence server 107 periodically issues new tokens so that the tokens are also not misused. In yet other embodiments, a token may be temporarily assigned to a user device 115 for the duration of a fine grain copresence determination and then recycled by a different user device for use in a separate fine grain copresence determination. In other embodiments, the device detector 202 on the user device 115 generates its own token (e.g. one time or periodically) and transmits the token to the copresence server 107 for identification purposes. The device detector 202 transmits the token to the communication unit 241 on the user device 115 to be used in the discovery process with other user devices 115. In some embodiments, a token is a random set of bits sufficient to ensure uniqueness and security (e.g., 40 bits), not-guessable, and have a server-enforced lifespan (e.g., 10 minutes). In some embodiments, a token may be a proxy identifier for the location and time a user device requested the token, with some smearing on both since the user device can move during the enforced lifespan. In one embodiment, a single device may be allowed to broadcast a particular token and only for the server-enforced lifespan of that token. Within the server-enforced lifespan of a token, the copresence server 107 may guarantee the token to be unique. However, as described above, a token may be recycled—assigned to another user device 115—after the token's server-enforce lifespan has passed.
In some embodiments, the device detector 202 wakes up all other user devices 115 within a possible communication range of the first device, based on coarse grain location and/or copresence information, to listen for the fine grain copresence token. In some embodiments, the device detector 202 may wake up only a subset of user devices 115 within range of the first user device 115 due to the memory limitations, communication technology limitations, and/or battery life limitations. Criteria may be specified by a user or determined by the copresence server 107 for selecting the subset of user devices to wake up. Example criteria can include social connections between users, common interests based on user profiles or activities, a historical status of devices, signals from other applications, etc. For example, although the copresence server 107 maintains or has access to coarse grain location and/or copresence information for a large number of user devices, it is not necessary to involve all user devices that are roughly copresent based on this information in fine grain copresence detection. In the scenario described above where the copresence server 107 determines coarse grain copresence based on non-location signals (e.g., text messages, emails, calendar notices, etc.) the copresence server may send a transmit signal to one user device and a wake up signal to the other user device even though multiple other devices may be copresent based on coarse grain location information. In some embodiments, the filter engines 204 or the list generator 206 may determine which user devices to wake up for fine grain copresence detection based on one or more of the certain criteria (e.g., the user of the first user device 115 may know the users associated with the list of other user devices 115). In other embodiments, the device detector 202 receives a list of other user devices 115 filtered by the permission engine 208 that are within proximity to the first user device 115 to wake up for fine grain copresence detection. In some other embodiments, the device detector 202 receives a list of other devices filtered by the ranking engine 212 and wakes up those user devices in the list. For example, the ranking engine 212 may pick the top 50 users in a list and the device detector 202 wakes up the top 50 user devices 115 for fine grain copresence detection.
In some embodiments, the permission engine 206 allows users to select which other users (or user devices) or which type of other users can learn of the user's copresence. For example, the permission engine 208 cooperates with the user interface engine 210 to provide a user interface for a first user to give permission to one or more other users to detect the first user's copresence. In some embodiments, the permission engine 208 can generate suggestions for a first user about other users to be given permission based on the first user's social connections and provide the suggestions to the first user through the user interface. For example, the permission engine 208 provides the first user with the option of permitting types of connections (e.g., friends, family, etc.) in a certain social network to detect the first user's proximity. In some embodiments, the permission engine 208 receives an input from a first user specifying other users allowed to detect the first user's proximity and configures privacy settings for the first user device 115 specifying the other users allowed to detect the first user device 115. In some embodiments, the permission engine 208 notifies the first user and the permitted other users or obtains their confirmations before configuring privacy settings for the first user device 115. For example, the permission engine 208 cooperates with the server (e.g., social network server 101, copresence server 107, etc.) to generate emails to send to the first user and the permitted other users for notification or confirmation.
FIG. 3 is a flowchart of an example method 300 for determining fine grain copresence between user devices 115. At 302, the copresence server 107 processes one or more signals to determine coarse grain location information of a first user device and a second user device. For example, as described above, the copresence server 107 may determine coarse grain copresence based on GPS, Wi-Fi, or cellular network signals. In other embodiments, the copresence server 107 may determine coarse grain copresence based on the contents of a text message, an email message, an instant message, a calendar event, a social media post or other similar signals that may indicate a wider range of proximity associated with the first device and the second device. In some embodiments, coarse grain copresence may be determined by user devices 115 performing periodic Wi-Fi scans. When the Wi-Fi environment that two or more user devices 115 detect is similar enough, the copresence server 107 may determine that the two or more user devices 115 are copresent, in a coarse grain sense.
At 304, the copresence server 107 determines whether the first device and the second device are copresent based on the coarse grain location information. For example, the copresence server 107 may determine that two devices are copresent based on coarse grain location information when the first and the second device are within a threshold distance of each other. If location information is not available, as described in the example above relating to text messages sent between a first user and a second user, the copresence server 107 may determine coarse grain copresence from the context of the text messages or other signals.
FIG. 4 is a flowchart of an example method 400 for determining fine grain copresence between user devices. At 402, the copresence server 107 may receive one or more signals associated with a first device and a second device. For example, as described above the signals may be coarse grain location signals or other signals from which the coarse grain copresence server 107 can determine copresence. At 404, the copresence server 107 may process the one or more signals to determine coarse grain location or copresence information of a first device and a second device as described above with reference to 302. At 406, the copresence server 107 may determine whether the first device and the second device are copresent based on the coarse grain location information as described above with reference to 304. At 408, as described above with reference to 306, the copresence server 107 transmits a signal to the second device to alert the second device to listen for a fine grain copresence token.
At 410, the copresence server 107 initiates transmission of the fine grain copresence token from the first device. For example, the copresence server 107 may transmit a fine grain copresence token to the first device for use in determining fine grain copresence of user devices. In another embodiment, the copresence server 107 may transmit a signal that instructs to the first device to generate a fine grain copresence token for use in determining fine grain copresence of user devices. In response to receiving the signal, the first device may transmit the fine grain copresence token using one or more transmission methods including, for example, inaudible audio, audible audio, Bluetooth, BLE, Wi-Fi, or near field communication.
At 412, the copresence server 107 refines copresence based on receiving an indication that the second device has received the fine grain copresence token as described above with reference to 306. Additionally, at 414, the copresence server 107 may perform an action based on fine grain copresence of the first and second device. In one embodiment, as described above, the action may be initiating the display of copresent user devices to the first and second device. Detection and display of copresent user devices may be determined based on privacy settings as described elsewhere herein.
FIG. 5 is an example block diagram depicting signals transmitted and received grain one embodiment for determining fine grain copresence of user devices. As described herein, the copresence server may transmit a wake up signal 502 to user devices to be included in the fine grain copresence determination, such as user device 115n. The copresence server 107 also transmits a signal 504 to the transmitting user device, for example user device 115a, to initiate transmission of a fine grain copresence token. The signal 504 may include the fine grain copresence token as described above or the signal may include instructions to generate a fine grain copresence token.
Furthermore, the modules, routines, features, attributes, methodologies and other aspects of the disclosure can be implemented as software, hardware, firmware, or any combination of the foregoing. Also, wherever a component, an example of which is a module, of the present disclosure is implemented as software, the component can be implemented as a standalone program, as part of a larger program, as a plurality of separate programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and/or in every and any other way known now or in the future. Additionally, the disclosure is in no way limited to implementation in any specific programming language, or for any specific operating system or environment. Accordingly, the disclosure is intended to be illustrative, but not limiting, of the scope of the subject matter set forth in the following claims.
receiving, at one or more processors, a signal associated with a first device, the first device being associated with a first user;
processing, using the one or more processors, the signal to determine location information associated with the first device;
determining, using the one or more processors and the location information associated with the first device, a second device in coarse grain copresence with the first device and a second user associated with the second device;
determining whether the second user satisfies a threshold of interaction with the first user;
responsive to the second user satisfying the threshold of interaction with the first user, transmitting a wake-up signal to the second device to instruct the second device to begin listening for a fine grain copresence token;
transmitting the fine grain copresence token from to the second device; and
responsive to receiving an indication that the second device has received the fine grain copresence token, confirming that the second device is in fine grain copresence with the first device.
2. The computer-implemented method of claim 1, wherein determining the second device in coarse grain copresence with the first device includes determining that the second device is within a first distance from the first device; and wherein transmitting the fine grain copresence token includes transmitting the fine grain copresence token using a communication range smaller than the first distance.
3. The computer-implemented method of claim 1, wherein transmitting the fine grain copresence token includes transmitting the fine grain copresence token using one or more of inaudible audio, audible audio, Bluetooth, Bluetooth Low Energy (BLE), Wi-Fi, and Near Field Communication.
4. The computer-implemented method of claim 1, wherein the signal includes historical location data associated with the first device.
5. The computer-implemented method of claim 1, wherein the signal includes one or more of a message, a calendar event, and a social media item associated with the first user and wherein determining the second device in coarse grain copresence with the first device is based on content of the one or more of the message, the calendar event, and the social media item.
6. The computer-implemented method of claim 1, wherein determining the second device in coarse grain copresence with the first device is based on a third device, the third device being in coarse grain copresence with the first device.
7. The computer-implemented method of claim 1, further comprising performing an action responsive to determining that the second device is in fine grain copresence with the first device.
8. A computer program product comprising a non-transitory computer usable medium including a computer readable program, wherein the computer readable program, when executed on a computer, causes the computer to perform:
receiving a signal associated with a first device, the first device being associated with a first user,
processing the signal to determine location information associated with the first device;
determining, based on the location information associated with the first device, a second device in coarse grain copresence with the first device and a second user associated with the second device;
transmitting the fine grain copresence token to the second device; and
9. The computer program product of claim 8, wherein determining the second device in coarse grain copresence with the first device includes determining that the second device is within a first distance from the first device; and wherein transmitting the fine grain copresence token includes transmitting the fine grain copresence token using a communication range smaller than the first distance.
10. The computer program product of claim 8, wherein transmitting the fine grain copresence token includes transmitting the fine grain copresence token using one or more of inaudible audio, audible audio, Bluetooth, Bluetooth Low Energy (BLE), Wi-Fi, and Near Field Communication.
11. The computer program product of claim 8, wherein the signal includes historical location data associated with the first device.
12. The computer program product of claim 8, wherein the signal includes one or more of a message, a calendar event, and a social media item associated with the first user and wherein determining the second device in coarse grain copresence with the first device is based on content of the one or more of the message, the calendar event, and the social media item.
13. The computer program product of claim 8, wherein the computer readable program when executed on the computer also causes the computer to perform performing an action responsive to determining that the second device is in fine grain copresence with the first device.
a memory storing instructions that, when executed, cause the system to perform operations comprising:
15. The system of claim 14, wherein determining the second device in coarse grain copresence with the first device includes determining that the second device is within a first distance from the first device; and wherein transmitting the fine grain copresence token includes transmitting the fine grain copresence token using a communication range smaller than the first distance.
16. The system of claim 14, wherein transmitting the fine grain copresence token includes transmitting the fine grain copresence token using one or more of inaudible audio, audible audio, Bluetooth, Bluetooth Low Energy (BLE), Wi-Fi, and Near Field Communication.
17. The system of claim 14, wherein the signal includes historical location data associated with the first device.
18. The system of claim 14, wherein the signal includes one or more of a message, a calendar event, and a social media item associated with the first user and wherein determining the second device in coarse grain copresence with the first device is based on content of the one or more of the message, the calendar event, and the social media item.
19. The system of claim 14, wherein determining the second device in coarse grain copresence with the first device is based on a third device, the third device being in coarse grain copresence with the first device.
20. The system of claim 14, wherein the instructions, when executed, further cause the system to perform operations including performing an action responsive to determining that the second device is in fine grain copresence with the first device.
Patent number: 9386417
Inventors: Andrew Ames Bunner (Belmont, CA), Alan Lee Gardner, III (San Mateo, CA), Mohammed Waleed Kadous (Santa Clara, CA), Brian Patrick Williams (Mountain View, CA), Marc Stogaitis (San Mateo, CA), Nadav Aharony (Sunnyvale, CA), Brian Duff (Santa Clara, CA), Pascal Tom Getreuer (San Francisco, CA), Zhentao Sun (Sunnyvale, CA), Daniel Estrada Alva (Mountain View, CA), Ami Patel (Mountain View, CA), Benjamin Razon (Mountain View, CA), Richard Daniel Webb (Redwood City, CA), Tony Weber (San Jose, CA), Thomas Yuchin Chao (Burlingame, CA), Ryan Michael Rifkin (San Francisco, CA), Richard Francis Lyon (Los Altos, CA), Liem Tran (Milpitas, CA), Joseph A. Farfel (San Francisco, CA)
Application Number: 14/720,825
International Classification: H04W 24/00 (20090101); H04W 4/02 (20090101); H04L 29/08 (20060101);