Patent Description:
The present disclosure relates to augmented reality (AR), and in particular to a tiered network architecture for providing low-latency shared AR experiences.

In parallel-reality gaming systems, players interact via computing devices in a shared virtual world that parallels at least a portion of the real world. The location of a player in the virtual world is based on the player's location the real world. However, when actions are primarily undertaken with reference to the virtual world, the virtual world can act as a barrier to players engaging with each other. Players interact with the virtual world, making other players seem remote or unreal. As a result, even when many individuals are playing in a geographic region of the real world, the gaming experience may feel individual rather than social.

A parallel-reality gaming system may use one or more protocols to update the game state of the virtual world with which a player interacts to reflect changes (e.g., the results of other players' actions in the game). Communication protocols, such as the User Datagram Protocol (UDP), define systems of rules for exchanging data using computer networks. UDP adheres to a connectionless communication model without guaranteed delivery, ordering, or non-duplicity of datagrams. Computing devices communicating using UDP transmit datagrams, which are basic units for communication each including a header and a payload, to one another via the computer network.

Connectionless communication protocols such as UDP generally have lower overhead and latency than connection-oriented communication protocols like the Transmission Control Protocol (TCP), which establish connections between computing devices before transmitting data. However, existing connectionless communication protocols are inadequate for data transfers that require less latency than is accommodated by the existing art. For example, a parallel-reality game session streaming at <NUM> frames per second (FPS) may require latency an order of magnitude lower than provided by current techniques. In such a game session, the frames are spaced at approximately sixteen millisecond intervals, while current communication protocols typically provide latency of approximately one hundred milliseconds (or more).

Thus, the latency of these existing connectionless communication protocols provides a barrier between the player and the virtual world in parallel-reality gaming systems. With these existing communication protocols, a player does not interact with the current game state, only a recent game state. For example, in a parallel-reality game, a player may see a virtual object at an old location (e.g., where the object was <NUM> milliseconds previously), while the virtual positional data in fact has a new location for the virtual object (e.g. the virtual object has been moved by another player). This latency in communication between the client and a server hosting or coordinating the parallel-reality game may lead to a frustrating user experience. This problem may be particularly acute where more than one user is participating in the game because the latency may cause a noticeable delay between the actions of one player showing up in other players' views of the virtual world.

<CIT> concerns computer network protocols, and in particular computer network protocols for providing low-latency wireless communication between devices within physical proximity of each other. <NPL> discloses a few examples of consistency maintenance and state update dissemination for creating a shared reality virtual world, where a large number of concurrent users are connected via server-based network architectures.

Aspects of the invention are defined in the accompanying claims. According to a first aspect there is provided a method in accordance with claim <NUM>. Preferred optional features are defined in the dependent claims.

Augmented reality (AR) systems supplement views of the real world with computer-generated content. Incorporating AR into a parallel-reality game may improve the integration between the real and virtual worlds. AR may also increase interactivity between players by providing opportunities for them to participate in shared gaming experiences in which they interact. For example, in a tank battle game, players might navigate virtual tanks around a real-world location, attempting to destroy each other's tanks. In another example, in a tag battle game, players may attempt to tag each other with energy balls to score points.

Conventional AR session techniques involve a game server maintaining a master game state and periodically synchronizing the local game state of players devices to the master game state via a network (e.g., the internet). However, synchronizing a player's local game state may take a significant amount of time (e.g., ~<NUM> of milliseconds), which is detrimental to the gaming experience. The player is, in effect, interacting with a past game state rather than the current game state. This problem may be particularly acute where more than one user is participating in the AR game because the latency causes a noticeable delay between the actions of one player showing up in other players' views. For example, if one player moves an AR object in the virtual world, other players may not see it has moved until one hundred milliseconds (or more) later, which is a human-perceptible delay. As such, another player may try to interact with the object in its previous location and be frustrated when the game corrects for the latency (e.g., by declining to implement the action requested by the player).

This and other problems may be addressed by performing some game state processing at an edge node of the network (e.g., a cell tower). As a result, computation of the game state may be sharded naturally based on real-world location, with the master game state maintained by the server providing conflict resolution (e.g., where actions of players connected to nearby cell towers potentially interfere with each other). Latency may also be reduced using a peer-to-peer (P2P) protocol that exchanges game updates between clients connected to the same edge node without routing the updates via the game server. For example, using these approaches, latency may be reduced to ~<NUM> milliseconds or less. Furthermore, this may increase bandwidth availability enabling a greater number of players to share a common AR experience.

In one embodiment, a method for providing a shared AR experience by an edge node includes receiving a connection request from a client at the edge node and identifying a shared AR experience for the client based on the connection request. The method also includes providing map data and a local game state for the shared AR experience to the client. The edge node receives an action request from the client that indicates a desired interaction with a virtual item in the shared AR experience and determines an outcome of the action request based on a local game state maintained by the edge node. The method further includes providing the outcome to a plurality of clients connected to the edge node and validating the outcome with a master game state maintained by a server. The outcome may be validated after the outcome is initially provided to the plurality of clients.

The figures and the following description describe certain embodiments by way of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods may be employed without departing from the principles described. Reference will now be made to several embodiments, examples of which are illustrated in the accompanying figures. Wherever practicable similar or like reference numbers are used in the figures to indicate similar or like functionality. Where elements share a common numeral followed by a different letter, the elements are similar or identical. The numeral alone refers to any one or any combination of such elements. Although the embodiments described below relate to a parallel-reality game, one of skill in the art will recognize that the disclosed techniques may be used for other types of shared AR experience.

<FIG> illustrates one embodiment of a networked computing environment in which AR content is provided to clients <NUM>. The AR content may be part of a parallel-reality game in which players interact in a shared virtual world that parallels at least a portion of the real world. In the embodiment shown, the networked computing environment includes a server <NUM>, a cell tower <NUM>, and two clients <NUM>. Although only two clients <NUM> are shown for convenience, in practice, more (e.g., tens or hundreds of) clients <NUM> may be connected to the cell tower <NUM>. In other embodiments, the networked computing environment may include different or additional components. For example, the networked computing environment may be a local area network (LAN) using a WiFi router as an edge node rather than a cell tower <NUM>. In addition, the functions may be distributed among the elements in a different manner than described.

The server <NUM> is one or more computing devices that provide services to clients <NUM> in a communications network. The server <NUM> communicates with the clients <NUM> via edge nodes of the communications network (e.g., cell towers <NUM>). In one embodiment, the server <NUM> maintains a master game state for a parallel-reality game that is, ultimately, the ground truth state of the parallel reality game. In this context, the master game state corresponds to the ground truth sate in that, where conflicts arise between the master game state and a local game state, the master game state governs. This may be particularly useful where two local game states (e.g., two local game states maintained by adjacent cell towers <NUM> in a network) correspond to overlapping geographic areas. Thus, devices connected to both cell towers <NUM> may interact with the same game content (in the master game state) while being connected to different AR sessions.

The cell tower <NUM> is an edge node via which clients <NUM> connect to the communications network. As described above, the computer network may include other edge nodes in addition to or replacing a cell tower <NUM> but enabling similar communication. In one embodiment, a cell tower includes one or more computing devices configured to store AR data and provide AR services to connected clients <NUM>. Because the clients <NUM> are located relatively close to the cell tower and connect directly to it, the lag time in providing data and services to the clients may be significantly shorter than for the server <NUM>. Various embodiments of the cell tower <NUM> are described in greater detail below, with reference to <FIG>.

Clients <NUM> are computing devices such as personal computers, laptops, tablets, smartphones, or the like. In embodiments involving a parallel-reality game, the clients <NUM> are typically smartphones or tablets that have connected or built-in cameras and which players can easily carry. A client (e.g., client 130A) may communicate by sending P2P messages to the cell tower <NUM> which forwards them to other clients (e.g., client 130B) connected to the same cell tower. Clients <NUM> may also communicate with one another by sending messages to the server <NUM> (via the cell tower <NUM>), which in turn forwards them to the recipient. For example, a client 130A may send a message to another client 130B that is connected to a different cell tower <NUM> or the server <NUM> in this manner.

In one embodiment, client communications may be routed through the server <NUM> or peer-to-peer (P2P). Communications routed through the server <NUM> may go from a first client 130A to the server <NUM> via the cell tower <NUM> and then back through a cell tower <NUM> to a second client 130B. In contrast, P2P communication may go from the first client 130A to the cell tower <NUM> and then directly to the second client 130B. Note that in some cases, the communications may pass through other intermediary devices (or nodes), such as signal boosters. As used herein, a communication is considered P2P if it is routed from a sending client 130A to a target client 130B without passing through the server <NUM>. This may reduce latency by bypassing the need to send the communication to the server <NUM> before the communication is sent to the target client 130B. For example, a message (e.g., a datagram) may be sent P2P if the target client 130B is connected to the same cell tower <NUM> as the sending client 130A and may be routed via the server <NUM> otherwise. In another embodiment, clients <NUM> communicate entirely using P2P communications.

In one embodiment, the clients <NUM> use a coordination service (e.g., hosted at the server and communicated with via TCP) to synchronize IP addresses. The clients <NUM> can thus communicate (e.g., via UDP) using public facing IP addresses or a local area network (LAN). For example, a first client 130A can send a request via TCP to the coordination service to join a local AR shared experience. The coordination service may provide the first client 130A with the IP address of a second client 130B connected to the AR session providing the AR experience (e.g., via the same cell tower <NUM>). The coordination service may also provide the first client's IP address to the second client 130B or the first client 130A may provide it directly using the second client's IP address (as provided by the coordination service). In some embodiments, the coordination service may prompt the second client 130B to approve the first client 130A (e.g., by requesting user confirmation or checking a list of approved clients <NUM> to connect with the second client 130B) before the second client's IP address is provided.

Among other advantages, structuring the networked computing environment in the manner shown in <FIG> enables efficient distribution of computation. Information may be exchanged P2P between clients <NUM> with a short lag time, enabling players to interact within a shared AR experience in a similar manner than they would interact in the real world. Similarly, the cell tower <NUM> can determine what map data to provide to clients <NUM> and synchronize game state between connected clients <NUM>. The cell tower <NUM> may synchronize the game state with less lag than would result from synchronizing same state globally with the server <NUM>. The server <NUM> may then be used to handle longer term processing and resolve conflicts. For example, the server <NUM> may manage connections between geographic regions handled by different cell towers <NUM>, double check determinations made by cell towers (and make corrections if needed), perform additional security checks, analyze data received from clients <NUM> to detect cheating, maintain global data (e.g., total team score for regions that are larger than the coverage of a single cell tower), and the like.

<FIG> illustrates one embodiment of a datagram <NUM> suitable for use in the networked computing environment of <FIG>. As previously described, a datagram <NUM> is a basic unit for communication. In the embodiment shown, the datagram <NUM> includes a payload <NUM> and a header <NUM>, the latter including a P2P flag <NUM>, also known as an indicator. The header <NUM> is metadata specifying aspects of the datagram <NUM>, such as a source port, a destination port, a length of the datagram <NUM>, and a checksum of the datagram <NUM>. The payload <NUM> is the data communicated by the datagram <NUM>. In other embodiments, the datagram <NUM> may include different or additional elements.

The payload <NUM> includes the content of the datagram <NUM> that is intended for delivery to the recipient client or clients <NUM>. In one embodiment, the header <NUM> may be similar to a UDP header with the addition of the P2P flag <NUM>. The header <NUM> may also contain additional metadata. The P2P flag <NUM> is used to determine whether the datagram <NUM> is sent to the server <NUM> or is sent P2P to another client <NUM>. In other embodiments the P2P flag <NUM> is replaced with one or more other indicators within the header providing similar functionality.

<FIG> is a flowchart illustrating a process for using a P2P communication protocol at an edge node (e.g., a cell tower <NUM>), according to one embodiment. In <FIG>, the cell tower <NUM> receives <NUM> a datagram <NUM> from a client 130A. The cell tower <NUM> analyzes <NUM> the datagram <NUM> to determine whether it should be sent P2P or via the server <NUM>. In one embodiment, the P2P flag <NUM> indicates the datagram <NUM> is a P2P message, and the cell tower <NUM> sends <NUM> the datagram <NUM> to one or more clients <NUM> connected to the client 130A. For example, if the header <NUM> of the datagram <NUM> indicates the destination port is that of client 130B, the cell tower <NUM> sends <NUM> the datagram <NUM> to client 130B. Alternatively, the cell tower may maintain a list of connected clients <NUM> that are engaged in a local AR session and send <NUM> the datagram <NUM> to all clients <NUM> (or a subset of clients, such as those corresponding to a player's teammates) engaged in the local AR session. In contrast, if the P2P flag <NUM> indicates the datagram <NUM> is directed to the server <NUM>, the cell tower <NUM> sends <NUM> the datagram <NUM> to the server <NUM>.

In another embodiment, the sending client 130A may set the P2P flag <NUM> to indicate that the datagram <NUM> should be sent P2P if possible. The cell tower <NUM> receives <NUM> and analyzes <NUM> the datagram <NUM> and, assuming the P2P flag <NUM> indicates the datagram <NUM> should be sent P2P, determines whether a target client 130B is currently connected to the cell tower <NUM> (e.g., by comparing an identifier of the target client 130B to the list of currently connected clients <NUM>). If the target client 130B is connected to the cell tower <NUM>, the cell tower <NUM> sends <NUM> the datagram <NUM> straight to the target client 130B instead of going via the server <NUM>. In contrast, if the target client 130B is not connected to the cell tower <NUM>, the cell tower <NUM> sends <NUM> the datagram <NUM> to the server <NUM> to be sent on to the target client 130B (e.g., via a second cell tower <NUM> to which it is currently connected). For example, the server <NUM> might maintain a database or other list of which cell towers <NUM> are currently or have been recently connected to which client devices <NUM>. In some embodiments, the cell tower <NUM> may send the datagram <NUM> to both the target client 130B and the server <NUM>.

In other embodiments, the P2P flag <NUM> may be an identifier of an entity such as an AR session, a user, a device, a game account, or the like. The cell tower <NUM> maintains a list of P2P flags <NUM> for which the datagram <NUM> should be sent P2P (or P2P if possible). The cell tower <NUM> receives <NUM> and analyzes <NUM> the datagram <NUM> to determine whether it should be sent via the server <NUM> or P2P. If the P2P flag <NUM> includes an identifier of an entity on the list, the datagram <NUM> is a P2P message and the cell tower <NUM> sends <NUM> the datagram <NUM> to one or more clients 130B associated with the identified entity. For example, if the header <NUM> of the datagram <NUM> includes an identifier of the target client 130B, the cell tower <NUM> sends <NUM> the datagram <NUM> to the target client 130B. To give other examples, if the P2P flag <NUM> identifies an AR session, the datagram <NUM> is sent to all clients <NUM> connected to that session, and if it is a game account, the datagram is sent to one or more clients associated with that game account, etc. In contrast, if the P2P flag <NUM> identifies an entity that is not on the list, the cell tower <NUM> sends <NUM> the P2P flag <NUM> to the server <NUM> to be forwarded to clients <NUM> associated with the entity. Alternatively, the list may indicate P2P flags <NUM> for messages that are not to be sent P2P, in which case the default behavior if the P2P flag <NUM> is not on the list is to send the corresponding datagram P2P to the target client 130B.

<FIG> illustrates one embodiment of an edge node in a communications network. In the embodiment shown, the edge node is a cell tower <NUM> that includes a routing module <NUM>, a data ingest module <NUM>, an AR session module <NUM>, a map processing module <NUM>, an authority check module <NUM>, and a local data store <NUM>. The cell tower <NUM> also includes hardware and firmware or software (not shown) for establishing connections to the server <NUM> and clients <NUM> for exchanging data. For example, the cell tower <NUM> may connect to the server <NUM> via a fiberoptic or other wired internet connection and clients <NUM> using a wireless connection (e.g., <NUM> or <NUM>). In other embodiments, the cell tower <NUM> may include different or additional components. In addition, the functions may be distributed among the elements in a different manner than described.

The routing module <NUM> receives data packets and sends those packets to one or more recipient devices. In one embodiment, the routing module <NUM> receives datagrams <NUM> from clients <NUM> and uses the method described previously with reference to <FIG> to determine where to send the received datagrams. The routing module <NUM> may also receive data packets from the server addressed to either particular clients <NUM> or all clients that are connected to the cell tower <NUM>. The routing module <NUM> forwards the data packets to the clients <NUM> to which the data packets are addressed.

The data ingest module <NUM> receives data from one or more sources that the cell tower <NUM> uses to provide a shared AR experience to players via the connected clients <NUM>. In one embodiment, the data ingest module <NUM> receives real-time or substantially real-time information about real-world conditions (e.g., from third party services). For example, the data ingest module <NUM> might periodically (e.g., hourly) receive weather data from a weather service indicating weather conditions in the geographic area surrounding the cell tower <NUM>. As another example, the data ingest module <NUM> might retrieve opening hours for a park, museum, or other public space. As yet another example, the data ingest module <NUM> may receive traffic data indicating how many vehicles are travelling on roads in the geographic area surrounding the cell tower <NUM>. Such information about real-world conditions may be used to improve the synergy between the virtual and real worlds.

The AR session module <NUM> manages AR sessions in which players in the geographic area surrounding the cell tower <NUM> may engage in shared AR experiences. In one embodiment, a client <NUM> connects to the cell tower <NUM> while executing an AR game and the AR session module <NUM> connects the client <NUM> to an AR session for the game. All players of the game who connect to the cell tower <NUM> may share a single AR session or players may be divided among multiple AR sessions. For example, there may be a maximum number of players in a particular AR session (e.g., ten, twenty, one hundred, etc.). Where there are multiple AR sessions, newly connecting clients <NUM> may be placed in a session randomly or a client <NUM> may provide a user interface (UI) to enable the player to select which session to join. Thus, a player may elect to engage in an AR session with friends. In some embodiments, players may establish private AR sessions that are access protected (e.g., requiring a password or code to join).

In various embodiments, to enable AR objects (e.g., creatures, vehicles, etc.) to appear to interact with real-world features (e.g., to jump over obstacles rather than going through them), the AR session module <NUM> provides connected clients <NUM> with map data representing the real world in the proximity of the client (e.g., stored in the local data store <NUM>). The AR session module <NUM> may receive location data for a client <NUM> (e.g., a GPS location) and provide map data for the geographic area surrounding the client (e.g., within a threshold distance of the client's current position).

The map data can include one or more different types of representations of the real world. For example, the map data can include a point cloud model, a plane matching model, a line matching model, a geographic information system (GIS) model, a building recognition model, a landscape recognition model, etc. The map data may also include more than one representation of a given type at different levels of detail. For example, the map data may include two or more point cloud models, each including different number of points.

The client <NUM> may compare the map data to data collected by one or more sensors on the client <NUM> to refine the client's location. For example, by mapping the images being captured by a camera on the client <NUM> to a point cloud model, the client's location and orientation may be accurately determined (e.g., to within one centimeter and <NUM> degrees). The client <NUM> provides the determined location and orientation back to the AR session module <NUM> along with any actions taken by the player (e.g., shooting, selecting a virtual item to interact with, dropping a virtual item, etc.). Thus, the AR session module <NUM> can update the status of the game for all players engaged in the AR session to accurately reflect players' locations in the AR session.

The map processing module <NUM> updates map data based on current conditions (e.g., data from the data ingest module <NUM>). Because the real world is not static, the map data in the local data store <NUM> may not represent current real-world conditions. For example, the same park trail in Vermont may look very different in different seasons. In summer, the trail may be clear and the surrounding trees may be covered in foliage. In contrast, in winter, the trail may be blocked by snow drifts and the trees may be bare. The map processing module <NUM> may transform the map data to approximate such changes.

In one embodiment, the map processing module <NUM> retrieves current condition data to identify a transformation and applies that transformation to the map data. The transformations for different conditions may be defined by heuristic rules, take the form of trained machine-learning models, or use a combination of both approaches. For example, the map processing module <NUM> might receive current weather condition data, select a transformation for the current weather conditions, and apply that transformation to the map data. Alternatively, the map processing module <NUM> may pre-calculate the transformed maps and store them (e.g., in the local data store <NUM>). In this case, when a client <NUM> connects to the cell tower <NUM>, the map processing module <NUM> determines the current conditions, selects the appropriate pre-calculated version of the map data, and provides that version to the client <NUM>.

The authority check module <NUM> maintains synchronization between game states of different clients <NUM>. In one embodiment, the authority check module <NUM> confirms that game actions received from clients <NUM> are consistent with the game state maintained by the AR session module <NUM>. For example, if two players both try to pick up the same in-game item, the authority check module <NUM> determines which player receives the item (e.g., based on timestamps associated with the requests). As described, the use of a P2P protocol and local processing at the cell tower <NUM> may significantly reduce the latency of a player's actions being seen at other players' clients <NUM> during an AR session by providing the outcome of an action requested by a player (e.g., capturing a virtual object) directly to other players' clients <NUM>. Therefore, the likelihood (and number) of instances of conflicts between actions arising and being resolved by the authority check module <NUM> is reduced and the AR experience may be improved.

The authority check module <NUM> may also maintain synchronization between its local copy of the game state and a master game state maintained by the server <NUM>. In one embodiment, the authority check module <NUM> periodically (e.g., every one to ten seconds) receives global updates regarding the game state from the server <NUM>. The authority check module <NUM> compares these updates to the local copy of the game state and resolves any discrepancies. For example, if a player's request to pick up an item was initially approved by the authority check module <NUM> but a game update from the server <NUM> indicates the item was picked up by another player (or otherwise made unavailable) before the player attempted to pick it up, the authority check module <NUM> might send an update to the player's client <NUM> indicating the item should be removed from the player's inventory.

This process may provide value for clients <NUM> located close to a boundary between coverage provided by different cell towers <NUM>. In this case, players connected to different cell towers <NUM> may both be able to interact with the same virtual element. Thus, each individual cell tower <NUM> might initially approve conflicting interactions with the element, but the sever <NUM> would detect the conflict and send updates to resolve the conflict (e.g., instructing one of the cell towers <NUM> to revoke its initial approval of the action and update its local game state accordingly).

The local data store <NUM> is one or more non-transitory computer-readable media configured to store data used by the cell tower <NUM>. In one embodiment, the stored data may include map data, current conditions data, a list of currently (or recently) connected clients <NUM>, a list of P2P flags, a local copy of the game state for the geographic region, etc. Although the local data store <NUM> is shown as a single entity, the data may be split across multiple storage media. Furthermore, some of the data may be stored elsewhere in the communication network and accessed remotely. For example, the cell tower <NUM> may access current condition data remotely (e.g., from a third-party server) as needed.

<FIG> illustrates a method <NUM> of providing a shared AR experience, according to one embodiment. The steps of <FIG> are illustrated from the perspective of an edge node (e.g., a cell tower <NUM>) performing the method <NUM>. However, some or all of the steps may be performed by other entities or components. In addition, some embodiments may perform the steps in parallel, perform the steps in different orders, or perform different steps.

In the embodiment shown in <FIG>, the method <NUM> begins with the cell tower <NUM> receiving <NUM> a connection request from a client <NUM>. The connection request may include one or more of an identifier of the client <NUM>, an identifier of a game, a player identifier (e.g., a username), current location data for the client <NUM> (e.g., a GPS position), etc. The connection request may be split into more than one portion. For example, the client <NUM> may first establish a communication channel with the cell tower <NUM> and then an AR game executing on the client <NUM> may send a request to join a shared AR session to provide the shared AR experience.

The cell tower <NUM> identifies <NUM> a shared AR session for the client <NUM>. The particular shared AR session may be selected based on the particular AR game executing on the client <NUM>, information about the player (e.g., the cell tower may favor connecting players with their contacts or other players with similar in-game profiles, etc.), numbers of players already connected to ongoing AR sessions, etc. In one embodiment, the client <NUM> identifies which shared AR session to join (e.g., based on player input), which is communicated to the cell tower <NUM>. Note that, in some instances, there may only be a single shared AR session available. In which case, the cell tower <NUM> may automatically connect the client <NUM> to that shared AR session.

The cell tower <NUM> provides <NUM> map data and a local game state for the shared AR session to the client <NUM>. In one embodiment, the cell tower uses position data received from the client <NUM> (e.g., GPS data) to identify a subset of the available map data to provide <NUM> to the client. For example, the cell tower <NUM> may provide map data describing real-world locations within a threshold distance of the client <NUM> (e.g., within one hundred meters). As described previously, the map data may include one or more models representing the real-world geographic area around the client <NUM>. The client <NUM> uses the map data to locate the client <NUM> within the shared AR session and presents an AR experience including AR content based on the local game state (e.g., virtual items, creatures, signs, etc.).

The cell tower <NUM> receives <NUM> an action request from the client <NUM>. The action request may represent any desired interaction between the player and content in the AR experience, such as a virtual item, virtual character, virtual creature, or other player. For example, the player may desire to: pick up a virtual item, drop a virtual item, use a virtual item, interact with a virtual item, talk to a virtual character, attack a virtual creature, attack another player, etc. One of skill in the art will recognize a wide range of possible actions that may be performed, depending on the specific nature of the AR game.

The cell tower <NUM> determines <NUM> an outcome for the action request based on the game state. The outcome may be based on a series of rules that may have various levels of complexity, depending on the particular action. For example, if the action is picking up a virtual item, the action may simply succeed unless the cell tower determines another player has already picked up, moved, or destroyed the item. As noted previously, due to the reduced latency achieved using the disclosed techniques, this may be a rare occurrence. In contrast, if the action is an attack on a virtual creature, the outcome may be based on a set of calculations and virtual die roles. One of skill in the art will recognize a wide range of possible rule sets and approaches that may be used to determine the outcome of an action.

The cell tower <NUM> provides <NUM> the outcome of the action to connected clients <NUM>. In one embodiment, the cell tower <NUM> provides the result to all clients <NUM> connected to the AR session. In another embodiment, the cell tower <NUM> provides the result to only those clients <NUM> within a threshold real-world distance of the player performing the action (e.g., one hundred meters). As the latency resulting from the disclosed techniques is relatively low (e.g., ten milliseconds or less), players may experience the results of the action substantially in real time. Thus, the players may get the impression that they are directly interacting with each other in the shared AR experience.

The cell tower <NUM> validates <NUM> the outcome of the action with a master game state maintained by the server <NUM>. In some embodiments, the cell tower <NUM> sends an indication to the server <NUM> to update the master game state reflect any actions taken by players in its local game state, provided the actions are proved valid in view of the master game state. In one embodiment, the cell tower <NUM> periodically (e.g., every one to ten seconds) verifies that its local game state is consistent with the master game state. As described previously, this may be particularly useful at the edge of the region covered by the cell tower <NUM> to handle cases where the actions of players connected to different cell towers <NUM> may interfere with each other. If a discrepancy between the local game state and the master game state is detected, the cell tower <NUM> may push an update to connected clients <NUM> to synchronize their game states to the master game state. Further, in some embodiments, the cell tower <NUM> may send an action and its outcome to the server for validation, and if the server indicates that the outcome conflicts with the master game state, the cell tower <NUM> updates the local game state to revoke the outcome.

<FIG> is a high-level block diagram illustrating an example computer <NUM> suitable for use within the networked computing environment <NUM> shown in <FIG>, according to an embodiment. The example computer <NUM> includes at least one processor <NUM> coupled to a chipset <NUM>. For convenience, the processor <NUM> is referred to as a single entity but it should be understood that the corresponding functionality may be distributed among multiple processors using various ways, including using multi-core processors, assigning certain operations to specialized processors (e.g., graphics processing units), and dividing operations across a distributed computing environment. Any reference to a processor <NUM> should be construed to include such architectures.

The chipset <NUM> includes a memory controller hub <NUM> and an input/output (I/O) controller hub <NUM>. A memory <NUM> and a graphics adapter <NUM> are coupled to the memory controller hub <NUM>, and a display <NUM> is coupled to the graphics adapter <NUM>. A storage device <NUM>, keyboard <NUM>, pointing device <NUM>, and network adapter <NUM> are coupled to the I/O controller hub <NUM>. Other embodiments of the computer <NUM> have different architectures.

In the embodiment shown in <FIG>, the storage device <NUM> is a non-transitory computer-readable storage medium such as a hard drive, compact disk read-only memory (CD-ROM), DVD, or a solid-state memory device. The memory <NUM> holds instructions and data used by the processor <NUM>. The pointing device <NUM> is a mouse, track ball, touch-screen, or other type of pointing device, and is used in combination with the keyboard <NUM> (which may be an on-screen keyboard) to input data into the computer system <NUM>. The graphics adapter <NUM> displays images and other information on the display <NUM>. The network adapter <NUM> couples the computer system <NUM> to one or more computer networks.

The types of computers used by the entities of <FIG> can vary depending upon the embodiment and the processing power required by the entity. For example, a server <NUM> might include a distributed database system comprising multiple servers working together to provide the functionality described. Furthermore, the computers can lack some of the components described above, such as keyboards <NUM>, graphics adapters <NUM>, and displays <NUM>.

Some portions of above description describe the embodiments in terms of algorithmic processes or operations. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs comprising instructions for execution by a processor or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of functional operations as modules, without loss of generality.

As used herein, any reference to "one embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment.

For example, some embodiments may be described using the term "connected" to indicate that two or more elements are in direct physical or electrical contact with each other. In another example, some embodiments may be described using the term "coupled" to indicate that two or more elements are in direct physical or electrical contact.

In addition, use of the "a" or "an" are employed to describe elements and components of the embodiments. This is done merely for convenience and to give a general sense of the disclosure. This description should be read to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise.

Claim 1:
A method for providing a shared augmented reality, AR, experience by an edge node of a communications network, the method characterized by comprising:
receiving (<NUM>), at the edge node, a connection request from a client, the connection request including a player identifier;
identifying (<NUM>) a shared AR session for the client based on the connection request;
providing (<NUM>), to the client, map data and a local state, maintained by the edge node, of an AR experience provided by the shared AR session;
receiving (<NUM>) an action request from the client, the action request indicating a desired interaction with a virtual item in the AR experience;
determining (<NUM>) an outcome of the action request based on the local state of the AR experience;
providing (<NUM>) the outcome to a plurality of clients connected to the edge node; and
validating (<NUM>), after providing the outcome to the plurality of clients, the outcome with a master state of the AR experience maintained by a server.