Client device orientation

Systems and methods are provided for determining an orientation of one or more client devices relative to a computing device. In various embodiments a method and system is provided for receiving, via a client device, data being displayed on a computing device. The data is used to determine an orientation of the client device relative to a computing device. The determined orientation of the client device is used to orient a user interface of a computing device.

BACKGROUND OF THE INVENTION

Computing devices may communicate with other devices for multiple reasons. The communication may occur via Local Area Networks (LANs) using various protocols, for example Wi-Fi. Data may be exchanged on these networks for various reasons and in some examples may enable multiple devices to participate in an event, for example a game.

DETAILED DESCRIPTION

Local Area Networks (LANs) using various protocols, near field communication (NFC) standards, and other forms for communication generally fail to capture spatial relationships between computing devices. Solutions involving ultrasound techniques for understanding spatial relationships between objects are uncommon. Further, the use of ultrasound in mobile consumer devices and computing devices is costly and unlikely. Affording the enhanced feature of capturing the spatial relationship of client devices to improve a user experience would be advantageous.

It would be useful to obtain information regarding the orientation or spatial relationship of participants joining an activity such as a virtual board game or a virtual card game or even a side-by-side interacting experience. For example, card and board games include activities that may utilize both mobile devices and a stationary display. These activities may benefit from both private screens and a shared screen. Knowing the orientation of the mobile devices with respect to the stationary screen allows the user interface of the stationary screen to indicate the proper virtual placement of the participants in such an activity as well as to alter the shared screen based upon the active participant in turn based operations. This information can be gained substantially simultaneously with the joining of an activity through the communication of data like a quick response (QR) code, barcode, aura, watermark, or other image that has an orientation.

When using mobile devices around a stationary screen for a collaborative activity, such as a virtual card game or virtual board game where the mobile devices are utilized as private screens and the computing device is utilized as a shared screen, it is desirable to know how the mobile devices are positioned with respect to the stationary screen. Information may be used to represent the location of participants in a virtual board game and orient the main virtual board toward the participant whose turn is currently underway. Visual observation from the mobile devices can provide that information by knowing what is displayed on the computing device screen and using the mobile device orientation sensors along with its camera to determine the mobile placement with respect to the shared screen.

For example, a QR code could be used to establish a linkage between a mobile device and a computing device. The QR code could be read upside down by a mobile device, for example, and information that the QR code was read upside down could be passed back from the mobile device to the computing device to allow the computing device to understand that the particular mobile device was located to the top of the computing device (assuming that the computing device's display was flat on a table). In this way, linkage between the mobile device and computing device could be simultaneously affected as well as the information regarding orientation communicated to the computing device. Accomplishing both the linking and orienting with the same process simplifies the user experience. In addition to using computing device sensors to know its orientation, using a mobile device's sensors, for example a camera, while aimed at the computing device's display allows information regarding the client device's orientation to be gained.

In various embodiments, methods, systems, and configurations for capturing spatial relationships of client devices, such as mobile devices, are provided.FIGS. 1A-Dschematically illustrate a computing system100according to an embodiment of the present disclosure. Client devices101and103may be any of several mobile devices such as tablets, smartphones, netbooks, and/or notebooks. Computing device102may be a personal computer (computing device), television, tablet, or laptop, and includes a user interface120. In an embodiment of the present disclosure, client and computing devices are in close proximity to each other and may be connected over a wireless network or LAN104for sharing resources such games, files, and other applications or services. The network104may be configured using various protocols such as Wi-Fi or pairing techniques.

InFIG. 1A, user interface120of computing device102displays a visual code130that may be sensed (110,111) by one or more mobile devices (101,103) and used to determine an orientation of the one or more mobile devices relative to a shared computing device or computing device102. In one example of the present disclosure, visual code130is a quick response (QR) code, barcode, aura, watermark, character, symbol, or other image having an orientation. In addition to being utilized for determining an orientation of mobile devices, visual code130may be utilized for pairing client and computing devices. Information for pairing the devices may be encoded in the visual code130that is displayed by a computing device102and capable of being sensed (110,111) by any of several mobile devices (101,103). Alternatively, a client device having a user interface is capable of generating and displaying a visual code, such as needed when a mobile device does not have a means of sensing the visual code from a computing device. In this way, a visual code may be sensed by either a client device or computing device.

The visual code130may be dynamically generated, including time-sensitive information such as Dynamic Host Configuration Protocol (DHCP) assigned network address and/or a connection password having an expiration date. The visual code130may be encoded with a key to identify a specific computing device or other computing device. In addition to including information that is used to identify a device, the visual code may include information that enables communication over a network, such as a network address (Internet Protocol (IP) address) network port, security credentials (password, cryptographic, keys, security certificates), and so on. The visual code130may also include a pointer and/or shortcut to additional information such as a website address that may be used to access additional information associated with the visual code. The visual code130may encode additional information, such as executable instructions and/or software modules to instruct a device to configure itself for pairing. The visual code130may be generated, encoded, and decoded based on functionality provided by JavaScript and/or ZXing (zebra crossing) libraries for visual code generating, encoding, and decoding, for example.

FIGS. 1B-1Dillustrate various arrangements of user interface120, having displayed user interface elements121,122, and123, according to various determined orientations of client devices101and103with respect to computing device102. User interface elements121and123may include icons, avatars, objects, or list items that represent users of client devices101and103, respectively. User interface element122may include an icon, object, avatar, or list item that represents shared computing device102. As an example, inFIG. 1B, user interface120displays user interface elements121and123according to an initial position of client devices101and103, respectively. For example, user interface element122in a virtual game may represent computing device102, where user interface element122is displayed as a virtual table in a virtual card game with avatars121and123representing card players utilizing client devices101and103, respectively. In this example, avatars121and123are positioned in the virtual game around virtual card game table122similar to the placement of players or users of client devices101and103as positioned around computing device102. Users of client device101and103may lay computing device102flat on the ground or on another surface to mimic a card game table, or computing device102may be placed in an upright position or at an incline.

InFIG. 1C, an arrangement of user interface120is displayed according to a second position of client devices101and103. In this example, client device101has transmitted an indicia of orientation to computing device102, e.g. indicating that client device101has rotated. An indicia of orientation as used herein is data that indicates an orientation of the transmitting device. In an alternate embodiment, client device101has transmitted, to computing device102, a signal to rotate user interface120from an initial position. Similarly, inFIG. 1D, an arrangement of user interface120is displayed according to a second position of client devices101and103. In this example, client device101has transmitted an indicia of orientation to computing device102, e.g. indicating that client device101has moved to a different position with respect to computing device102. In this example client device101has transmitted an indicia of orientation to computing device102causing computing device102to relocate avatar121from position1as inFIG. 1Bto position2as inFIG. 1D. In various examples, client deices101,103may continually monitor their position and continually update the computing device102.

Referring toFIGS. 2 and 3, flow diagrams are illustrated in accordance with various embodiments. The flow diagrams may illustrate embodiments of a method associated with the various systems and apparatuses discussed with reference toFIGS. 1A-Dand3-6. While illustrated a sequence of operations, the flow diagrams should not be construed to require that all operations are required for all embodiments, or that the operations must be performed in the same order as described. Additionally, one or more of the operations may be embodied in the form of computer readable instructions stored on a computer readable medium.

The method200may begin where, in at least one embodiment, data is generated and displayed on a computing device. The data may be dynamically generated at the request of either a client or computing device. The data may be encoded with a unique key that locally identifies the computing device. The data may comprise any of a quick response (QR) code, barcode, aura, watermark, or other image capable of having an orientation when displayed on a computing device.

At step201, a client device receives data locally identifying, a computing device. The data may be displayed on the computing device screen and received via a sensor of the client device. Receiving the data may comprise using a camera of the client device to capture the data. Receiving the data may also comprise receiving data that enables communication over a network. The data received at the client device may be decoded to identify the computing device and establish a communication link between the computing device and the client device.

At step202, client device determines an orientation of the client device relative to the computing device using the received data. In various embodiments, determining an orientation of the client device relative to the computing device comprises determining an orientation of the data received. The data may include a quick response (QR) code, barcode, aura, watermark, or other image having an orientation and displayed on a computing device user interface.

At step203, based on a determined orientation of the client device, client device transmits an indicia of orientation to the computing device to enable the computing device to orient a user interface for the client device user. For example, the indicia of orientation may include data detailing an avatar mapping of a client device user in relation to a computing device. In various embodiments, the client device transmits the indicia of orientation to allow the computing device to rotate its user interface to represent the virtual placement of the client device with respect to the computing device. The transmitted indicia of orientation enables the computing device to position a user interface element, such as an avatar representing a player of a game session, on the computing device screen. The client device may transmit the indicia in response to any of several events such as a timer, a detected motion of the client device, or a gaming session event such as in a card game when one player's turn ends and another player's turn begins. The transmitted indicia of orientation may also enable the computing device to rotate the user interface from 0 to 360 degrees.

Referring toFIG. 3, the method300may begin where, in at least one embodiment of the present disclosure, at step301, a client device may receive image data displayed on a computing device via a camera. In other embodiments, client device may not comprise a camera; therefore, the image data may be exchanged between devices over an established network connection. At302, the image data received is then decoded in order to identify the computing device and establish a unique pairing of the devices or communication link.

At303, orientation of the client device is determined based on a determined orientation of the image data. For example, if a camera of smartphone takes an upside down snapshot of a QR code displayed on a computing device that is positioned flat on a table, then the client device may determine that if the QR code was read upside down from the smartphone, then the smartphone must be located above the computing device. Optionally, the client device may pass the image data back to the computing device in order to allow the computing device to determine the orientation of the client device relative to the computing device based on the determined orientation of the image data.

Finally, at304, an indicia of orientation is transmitted to the computing device. The transmitted indicia of orientation enables the computing device to arrange the user interface of the computing, device. In various embodiments of the present disclosure, arranging the user interface of the computing device may involve rotating the user interface or arranging an avatar, icon, or list item on the computing device screen to represent the orientation of the one or more client devices relative to the computing device. In various examples, transmitting the indicia of orientation to the computing device may include continually updating the computing device with an orientation of the client device as the client device moves positions or alters its orientation. The continual updating may occur at predefined times, in response to various triggers, based upon turn based operations, or other events.

Referring toFIG. 4, client device401is illustrated in accordance with an example of the present disclosure. Client device401comprises a sensor410for sensing a visual code having an orientation from a computing device an orientation module420for determining an orientation of the client device based on the visual code; and a transmitter430for transmitting an indicia of client device orientation.

In various examples sensor410may receive data locally identifying a computing device, wherein the data is displayed via the computing device. The sensor may include a camera, video sensor, light sensor, proximity sensor, accelerometer, gyroscope, compass, microphone, and other sensors. The receipt of the data may be in response to a trigger from a user of the client device.

The orientation module420may be embodied within instructions stored on a non-transitory computer-readable medium, which may be executed by a processor, or alternatively, may be embodied in an integrated circuit, for example an application specific integrated circuit (ASIC). The orientation module420is to determine an orientation of the client device relative to a computing device displaying the data based on an orientation of the data. The orientation module420may also output an orientation signal to be transmitted to the computing device.

The transmitter430is to transmit the orientation signal, or alternatively, an indicia of orientation to the computing device. The transmitter may be independent or combined with a receiver. The transmitter430may be one of multiple transmitters within a client device430, such as a local area network transmitter, a broadband transmitter, or a NFC device. Other transmitters are contemplated.

Referring toFIG. 5, another example of a client device501comprises a sensor510, an orientation module520, a transmitter530, a communication interface540, and a detection module550. The orientation module520of client device501is configured to determine an orientation of the client device based on a visual code received and is further configured to determine an orientation of the client device based on a change in the orientation of the client device. Detection module550is configured to detect movement of the client device and output a signal, denoting a change in orientation or placement of the client device, to the orientation module in order to determine a change in orientation of the client device. The detection module550may be configured to continually detect movement.

The detection module550may include sensors similar to and included in the orientation module. The detection module550may be configured to movement of the client device. The movement may be determined via a camera and image differentiation hardware and software, gyroscopes noting movement in various directions or for prolonged periods of time, or a light sensor determining a different angle of incident light, in various examples. The detection module550may be configured to continually update the computing device on the location of the client device. In various examples, the detection module may be configured to send updates at predetermined instances, in response to various events, or based upon a determination of movement. Other examples are contemplated.

In one example, a user of smart phone (e.g. a client device) may wish to join a virtual card game being played on a shared computer (e.g computing device). The smart phone may serve as a client device requesting that a visual code be generated by a computing device602in order to determine an orientation of a client device501for use in a virtual game or other application. For example, a smart phone may serve as a requesting client device seeking to join a virtual card game being played on a shared computing device. Smart phone501may be one of several client devices seeking to join the virtual game.

Communication interface540of client device is enabled to handle communication requests or pairing requests from a client device which may be initiated or established in either direction to/from either of the client device501or computing device602. Establishing the communication pairing enables the client device and the computing device to communicate over a communication channel, such as a network connection including network connections to the Internet. In an embodiment of the present disclosure, communication interface540may receive a request regarding communication pairing. The request may be initiated on the computing device602. For example, based on a request from a client device, the computing device exchanges a visual code to establish communication pairing with client device. Devices may attempt various different pairing combinations, trading off displaying or receiving the visual code between the clients and computing device.

Referring toFIG. 6, an embodiment of a computing device602is illustrated in accordance with the present disclosure. Computing device602may include a plurality of components for executing the client device orientation system. Computing device602includes memory620having program instructions630, user interface610, processor640, communication interface650, receiver660, and controller670that will be described in more detail. The memory620includes a non-transitory computer readable medium comprising program instructions630that when executed cause a processor640to perform various functions. The functions may include generating a visual code locally identifying the computing device; displaying, on the user interface610of the computing device, the visual code having an orientation, where the visual code can be sensed by a client device and used by the client device to determine an orientation of the client device relative to the computing device; to arrange a user interface610of the computing device602based on the receipt, via a receiver660of a computing device, of an indicia of orientation of the client device relative to the computing device.

In various examples, generating data includes generating a visual code such as a quick response (QR) code, barcode, aura, watermark, or other image to be displayed on the user interface610of the computing device602. The program instructions630may enable dynamic generation of data encoded with a key for identifying the computing device and utilizing a communications interface650. Arranging the user interface610of the computing device, in various examples, may comprise rotating the user interface of the computing device or arranging an avatar, icon, or list item on the computing device screen to represent the orientation of the client device relative to the computing device. In addition, receiving an indicia of orientation via a receiver660may comprise receiving, an indicia of orientation in response to a detected movement of a client device, and the instructions for arranging the user interface further comprises instructions for rotating the user interface in response to the detected movement. In this manner a user interface may be continually updated with a location or orientation of the client device.

Communication interface650of computing device, similar to communication interface540of client device, is enabled to handle communication requests or pairing requests from a client device which may be initiated or established in either direction to/from either of the client device501or computing device602. Establishing the communication pairing enables the client device and the computing device to communicate over a communication channel, such as a network connection including network connections to the Internet. In an embodiment of the present disclosure, communication interface540or650may receive a request regarding communication pairing. The request may be initiated on the client device501and/or the computing device602. For example, based on a request from a client device, the computing device exchanges a visual code to establish communication pairing with client device. Devices may attempt various different pairing combinations, trading off displaying or receiving the visual code between the clients and computing device.

Receiver660of computing device660receives an input signal indicating, a determined orientation of a client device relative to a computing device. The receiver660outputs an orientation signal to a controller670. Controller670is coupled to receiver660and is configured to arrange the user interface of the computing device based on the orientation signal. Controller670may be configured to control the appearance, layout, arrangement of a user interface and user interface elements displayed. Further, the user interface610is configured to display the user interface elements corresponding the relative placement or orientation of a client devices with respect to a computing device.

The foregoing disclosure has been presented for purposes of illustration and description. Although certain embodiments have been illustrated and described herein, the disclosure is not intended to be exhaustive or to limit the invention to the precise forms disclosed. It will be appreciated by those of ordinary skill in the art that a wide variety of alternate and/or equivalent embodiments or implementations to achieve the same purposes may be substituted for the embodiments shown and described without departing from the scope of this disclosure. The scope of the invention is to be defined by the claims appended hereto, and by their equivalents.