Adding latency to improve perceived performance

Techniques described herein provide a system and methods for adding latency to improve the perceived performance of computing devices. For instance, the techniques may calculate transaction latencies for a given session based on the time between sending requests for content receiving the requested content. The calculated latencies may be aggregated or averaged in such a way that a specific latency may be selected for improving the perceived performance. A client device may then render subsequent content based on the selected latency or a server may serve subsequent content based on the selected latency. This artificial addition of latency may enhance the user experience by creating a more consistent environment.

This application is related to U.S. patent application Ser. No. 12/885,296, filed on Sep. 17, 2010, entitled ACCOMODATING LATENCY IN A SERVER-BASED APPLICATION, which is incorporated herein by reference in its entirety.

BACKGROUND

In the world of computers and the Internet, many different types of devices transmit and receive digital content both internally as well as over wired and wireless communication networks. For instance, a computer processor on a local device may read data from and/or write data to a memory storage location that resides locally on the device. The processor may also read from and/or write to networked storage locations that are located remotely on other devices. That is, many computing devices often interconnect via wired and/or wireless networks for transmitting digital content from one device to another, sometimes over vast distances.

On a local device, a processor may retrieve an instruction from a local memory location, and may subsequently write some data to a different location in the local memory. These processes may occur in response to a user request or other action. For instance, when a user selects an icon on a screen of a computing device, the processor may execute many read and/or write instructions involving interaction with the memory. Each of these actions, however, may involve data traveling along a physical wire to and from the processor. Although it may often seem immediate to the user (as the user's selection is highlighted in seemingly no time), these data transmissions are not instantaneous. For example, some time may pass during the execution of instructions. In general, this passing of time between a request and receipt of an indication that the request has been executed is called latency.

Further, it is often the case that latency may be variable for any given session. For instance, a user may experience variations in latency, even for identical scenarios, during a single time-frame or on-line session. That is, a user may experience changes in latency from one instance to the next. However, this can be rather frustrating and can cause a poor user experience. Unfortunately, latencies of the current systems may be too unpredictable and variable.

DETAILED DESCRIPTION

Overview

This disclosure is directed, in part, to techniques for calculating and selecting latencies from a multitude of different scenarios to smooth out variable latencies found in electronic devices. This disclosure is also related to injecting selected latencies into scenarios involving multiple different types of digital devices. By doing so, the techniques allow for the perceived improvement of a device's performance with or without actually reducing latency. For instance, a device may calculate and aggregate transaction latencies that occur with another device, or within the device itself, such that it may inject a certain amount of latency into subsequent related transactions. For instance, the device may inject latency equal to some average latency of the previous transactions, a latency that is near the highest calculated latency, or the like. In this way, device transactions may achieve a consistent latency for improving the perceived performance and, thus, a user's experience.

As such, when a user operates a device implementing the techniques described herein, regardless of the specific environment, the user may be unaware of any latency variance found between one digital transaction and the next. Further, by injecting higher than normal latencies to create a more consistent user experience, the devices described may gradually increase noticed latency over time and/or may level the playing field for on-line gaming. Service providers may then create optimal latencies for different days, or times of the day, in order to improve customer relations or provide more evenly distributed services.

In some instances, a client device or server may implement the latency injection techniques. In this way, the client or server device may calculate each transaction latency and record the calculated latencies in a pool of aggregated latencies for each given session or game. The client or server device may use the pool of latencies to keep track of the variations in latency and, in some instances, to track aggregate statistics such as a running average latency for the session, or for a predetermined amount of time. The client or server device may then select a latency for injection. The selection may be made from either, or any combination of, the pool of latencies, a running average or some other aggregate statistic, a threshold latency based on the pool, or a latency based on a user's, or operator's, instruction. Further, the client or server device may inject the selected latency for each subsequent transaction by delaying the rendering or serving of subsequent content, respectively, or buffering the content until the selected latency is achieved. Alternatively, the server device may inject the selected latency

In yet other instances, a client device and a server device may work together to implement the latency injection techniques together. In this way, either the client device, the server device, or both devices may calculate transaction latencies. For example, the client device may calculate the transaction latencies by tracking the entire transaction from content request to content receipt. On the other hand, the server device may calculate the transaction latencies by tracking the entire transaction based on reported content request times and reported content receipt times. Additionally, the two devices may work together by communicating latency portions to construct a total transaction latency. Regardless of the implementation of latency calculation, the client device may inject a selected latency by delaying the rendering of received content, the server device may inject the selected latency by delaying the serving of the content, or they may each delay the content. For instance, the client and the server may each delay the content according to the latencies experienced during respective transmissions to each device. That is, a request latency may be injected by the client device, while a service latency may be injected by the server device.

Still, in other instances, the techniques described herein may be implemented outside of the client-server environment altogether. For example, one or more processors on a single computing device may inject latency by delaying rendering of content to a user of the computing device. Similar to the client-server model described above, variations in latency may occur during reads and writes to local memory. As such, the processor may calculate, pool, and/or average the transaction latencies before selecting a latency with which to inject into the user experience. In one example, a processor that usually takes 100 milliseconds to display a drop-down menu may instill a 200 millisecond latency if the processor occasionally takes 200 milliseconds. This way, the user becomes accustomed to a consistent 200 millisecond delay and does not notice when the processor inadvertently hangs for 220 milliseconds. Peer-to-peer networks, IP and cellular telephony networks, interactive television devices, on-line auction Web sites, on-line poker Web sites, and multiple other environments may also practice the techniques described above.

Illustrative Architectures

FIG. 1illustrates an example architecture100in which a device102may receive and transmit digital content to a server104over a network106. For instance, the device102may send a request108for content over the network106to the server104. In one example, this device102may request108for the server104to send on-line game content to be rendered to a user. In another example, the device102may request108for the server to effectuate a game action or game component selection, such as firing a virtual weapon at an opponent or moving a virtual character. Additionally, in response to the request108, the server104may serve110the requested content to the device102. For example, if the device102requests108that a user's character jump onto a virtual block in the game environment, the server104may serve110the content that allows the device102to render the jumping action to the user.

Additionally, in some instances, the device102may send a location112of a game feature to the server104. In this way, the server104may track the location112of the game feature when the user last acted. For example, to avoid issues with latency, it may be advantageous for the server104to know the location112of a game feature when the user perceived that they acted upon it. In other words, if a user fires a virtual weapon at an opponent in a first location112, that location112may be reported to the server104in case, due to latency, the server104is unaware whether the user “hit” the intended target or not. This way, the server104may record that the user actually “hit” the target even though the server104perceived it as a “miss.”

As discussed above, these network transactions, namely the request108and the service110of content, may involve some inherent, and even variable, latencies. For example, a slow network connection for either the device102or the server104may create latencies. Additionally, spikes in network congestion or even processor hangs may cause latencies. While this is generally unavoidable and expected among Internet users, variance within the latencies can lessen the user experience. In other words, users may find higher latencies that are constant more tolerable than variable latencies (even when the latency is low).

Thus, in some instances, the device102and/or the server104may calculate transaction latencies114(1),114(2), through114(N) (represented inFIG. 1by clocks), where N is an integer greater than zero. In this instance, each transaction latency114(1) through114(N) (collectively “calculated latencies114”) may represent an individually calculated latency114from individual transactions. By way of example, each individual transaction may represent a transaction between device102and server104for a single on-line session or game. The device102and/or the server104may perform the latency calculations114for each transaction that takes place during the session or game. As noted above, the device102may perform the latency calculations114, the server104may perform the latency calculations114, or the device102and the server104may perform the latency calculations114together. Also, as noted above, latency calculations114may be performed with respect to the request portion108, the content service portion110, or the entire transaction including both the request108and the serving110as signified by the two clocks in each latency calculation114.

Additionally, the device102and/or the server104may be configured to create a latency pool116for aggregating each calculated latency114. The latency pool116may contain each calculated latency114for the session or the game, or the latency pool116may only contain calculated latencies114for a specified time period that is shorter or longer than the on-line session or game. In some aspects, the latency pool116may contain the calculated latencies114as a ranked list (i.e., ranked from the highest latency to the lowest latency or vice versa), or as a form of aggregate statistic such as a running or exponential average for the entire session or game, or as a running or exponential average for some predetermined time period. For example, the device102and/or server104may continuously calculate a running average as each latency114is calculated and then store the running average in the latency pool.

The device102and/or server104may also be configured to select118a calculated latency114from the latency pool116to be injected in subsequent transactions. For example, the device102may select118a selected latency120from the latency pool116based on several factors. In one aspect, the device102may select the highest latency, the second or third highest latency, some other nthlatency, an aggregate statistic such as a type of average, or some other latency with which to inject into subsequent transactions. As shown inFIG. 1, and by way of example only, device102may inject selected latency120into subsequent transactions or server104may inject selected latency120into subsequent transactions. Additionally, both device102and server104may inject selected latency120into subsequent transactions together.

In this way, a calculated latency114that has been observed over network106may be selected118and injected to smooth out latency variance and improve the perceived performance of the system. As such, when the injected latency is at or near the highest calculated latency114for the system, occasional high latency spikes may go unnoticed. Further, by making the latency consistent across multiple or even all users, service providers and on-line game providers may be able to provide new types of games and services that have a leveled playing field. In other words, users with slower connections and/or slower computers may be more willing to participate on multi-user on-line games or activities.

WhileFIG. 1illustrates each transaction with a single device102and a single server104, the architecture100may implemented with multiple devices102and/or multiple servers104. Additionally, while transactions are shown as a combination of request108and service of content110, many other types of events or instructions may take place in a transaction over network106. Additionally, while only one network106is shown inFIG. 1, any number of networks, whether public or private, can be used to implement example architecture100.

FIG. 2illustrates an example architecture200detailing two alternative on-line game renderings202(A) and202(B) for any number of a multitude of devices204(1) through204(M), where M is an integer greater than zero, interacting over a network206with a server208. Devices204(1) through204(M) (collectively “devices204”), may include any type of digital device that can display graphical renderings to a user, such as, a personal computer, a handheld device, a game console, a smart phone, a laptop, etc. Additionally, similar to that discussed with reference toFIG. 1, devices204may send requests208, or other instructions, to a server208and may receive content served212from the server208in response to the request210. In particular, in one aspect, the content served212may be the game renderings202(A) and/or202(B).

Game renderings202(A) and202(B) are each examples of a game scene that may be played by a user of one or more of the devices204. In both game rendering202(A) and202(B), the device204may render four scenes in sequential order. The game scene renders the user's actions of moving a crosshair from the left side of the screen214(1) and216(1) over a game character214(2) and216(2), shooting at the game character214(3) and216(3), and then moving the crosshair to the right side of the screen214(4) and216(4). Additionally, the latencies observed, or calculated, between game scenes are represented as L1through L3and L4through L6, respectively. For example, the user may experience a latency of L1time between activating an input device and seeing the crosshairs move from game instance214(1) to214(2), a latency of L2time between activating the input device and seeing the crosshairs indicate a weapons fire from game instance214(2) to214(3), and a latency of L3time between activating the input device and seeing the crosshairs move away from the game character from game instance214(3) to214(4).

In one instance, a device204and/or a server208may calculate latencies as described above. For example, with respect toFIG. 2, a device204may calculate the following latencies for the transactions related to game rendering202(A): L1=200 milliseconds, L2=245 milliseconds, and L3=150 milliseconds. By aggregating these calculated latencies in a pool, the device may select a latency for injection. In this example, the device204may select the highest latency of 245 milliseconds so that it may be less likely for later latencies to exceed this mark. In this case, the device204may delay the display of subsequent game renderings, such as game rendering202(B), such that L4, L5, and L6are each equal to 245 milliseconds. As discussed above, this can be done in several different ways, including but not limited to, delaying rendering for the difference of the current calculated latency and the selected latency or buffering the content to be rendered until 245 millisecond has passed since the user's last interaction.

In another example, the device204may select the second highest latency for injection, in which case, game rendering202(B) may set L4through L6to be equal to 200 milliseconds. Clearly, this will make game play faster and more responsive for the user; however, latency spikes at or above 245 milliseconds may be more noticeable and bothersome. In yet another example, the device204may calculate an aggregate statistic such as a running average, an exponential average, or some other type of average. In this case, the device204may select the running average as the injection latency or it may select an inflated or deflated running average. That is, the device204may determine that the running average is changing too frequently and not producing a constant latency for the user, in which case, the device204may inflate or deflate the running average by a certain percentage or by a predetermined constant amount and select this inflated or deflated running average as the injection latency. By way of example and not limitation, the inflated running average may be 125%, 150%, 175%, or even 200% of the running average, or the inflated running average may be the running average plus 100 milliseconds, 200 milliseconds, or even 300 milliseconds, based on current network conditions or the specific application for the which the latency injection is being used.

WhileFIG. 2illustrates the sample transaction with a single server208, the architecture200may implemented with multiple servers208. While the transactions is shown as a combination of request210and service of content212, many other types of events or instructions may take place in a transaction over network206. Additionally, while only one network206is shown inFIG. 2, any number of networks, whether public or private, can be used to implement example architecture200. Further, any type of aggregate statistic may be used prior to selecting an injection latency and this disclosure shall not be limited to running and/or exponential averages.

FIG. 3illustrates one possible square graph300representing calculated latencies302as a function of time304for a set of transactions as described above with reference toFIGS. 1 and 2. In this example, latencies are calculated in milliseconds and are recorded (i.e., stored in the pool) every second, represented by the solid line. As shown here, several sample selected latencies that a device and/or server may use for latency injection are shown. For example, a device and/or server may select the highest latency306or the second highest latency308. Once selected, the device and/or server may be configured to inject the selected latency in one or more of the manners described above. For example, a device may delay the rendering of content or buffer the content to keep the perceived latency constant. It is important to note, however, that the highest latency, or any nthhighest latencies selected, may change as more latencies are calculated. For example, while the highest latency306inFIG. 3is shown as 250 milliseconds, that was not the highest latency until the time interval between seconds 2 and 3, and it may change if a latency above 250 milliseconds is calculated during a later transaction. Additionally, the same may be true for running average310which will change over time assuming that the calculated latencies vary. Additionally, as shown here, and as described briefly above, the device and/or server may select the inflated running average312for injection. As described above, the inflated running average312may be the running average310plus some constant.

WhileFIG. 3illustrates one possible square graph300with calculated latencies302measured in milliseconds, any other type of graph may be used to represent the calculated latencies. Additionally, while time is measured in seconds inFIG. 3, any sampling time may be used for calculating latencies. Further, while a running average310and an inflated running average312are shown, any other type of aggregate statistic may be used to track the calculated latencies.

Illustrative Alternative Embodiments

FIGS. 4A through 4Dillustrate four different embodiments for implementing the latency injection architecture100described inFIGS. 1 and 2. However, while only four embodiments are shown here, additional embodiments are possible, some of which have been described above. In one aspect, and as described in detail regardingFIG. 1, latency injection may be implemented over a client-server network. Here,FIG. 4Aillustrates one or more devices402(1) through402(K), where K is an integer greater than zero, in communication via a network with one or more servers404(1) through404(L), where L is an integer greater than zero. In this example, latencies406may be calculated, stored, and selected for injection as described above.

In another aspect, a single device408may implement latency injection to improve the perceived performance of local processing410. Here,FIG. 4Billustrates one or more processors412(1) through412(P), where P is an integer greater than zero, in communication with one or more local memory devices414(1) through414(Q), where Q is an integer greater than zero. In this example, latencies416may be calculated, stored, and selected for injection as described above as well.

FIG. 4Cillustrates a peer-to-peer network similar to the implementations described above, however, without any servers. In this example, a multitude of devices418(1) through418(S) may communicate with one another over a network. Latencies420may be calculated by one or all of the devices, collectively devices418, for storing and selecting. In this example, each device418may control its own latency with respect to the other devices418and may calculate its own latencies420or may share information with each other to facilitate in the calculations of each other's latencies420.

In another aspect,FIG. 4Dillustrates a telephone system which may be implemented in a wired or wireless network. In this example, one or more telephone devices422(1) through422(X), where X is an integer greater than zero, may communicate over a cellular or other network with one or more cellular towers424(1) through424(Y), where Y is an integer greater than zero. As such, latencies426may be calculated and injected as described above to effectuate the perceived performance enhancements described herein. WhileFIG. 4Dillustrates cellular telephone devices, collectively telephone devices422, in communication with cellular towers, collectively cellular towers424, other types of telephone systems may be used. For example, a wired or wireless IP telephony network or any other type of wireless network, such as 802.11 networks, may implement the latency injection methods described.

In yet another aspect, combinations of any of the afore-mentioned embodiments may be envisioned. For example, a telephone device422may be configured to implement the local processing type of latency injection426internally to create a constant delay before dialing a number. Additionally, the telephone device422may also be configured to inject latency into other internal processing transactions, and at the same time be configured to inject latency into networked transactions ofFIG. 4D. In other words, the four embodiments illustrated inFIGS. 4A through 4D, and the other implementations described herein, are not intended to be mutually exclusive. Further, other combinations of either wired and/or wireless networks are possible as well. For examples scenarios may be envisioned where a computing device is communicating with other devices via a network made up of both wired and wireless connections.

Illustrative Computing Device

FIG. 5illustrates an example computing device500that might be configured to implement the functions and features of latency injection, as described above. In a very basic configuration, the computing device500may include one or more processors502, a communications interface504, and memory506. Depending on the configuration of the computing device500, the memory506is an example of computer storage media and may include volatile and nonvolatile memory. Thus, the memory506may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, or any other medium which can be used to store media items or applications and data which can be accessed by the computing device500.

The memory506may be used to store any number of functional components that are executable on the processor(s)502, as well as data and content items that are rendered by the computing device500. Thus, the memory506may store an operating system and several modules containing logic.

A latency calculation module508located in memory506and executable on the processor(s)502may facilitate calculation of latency between devices over a network, or between local components of a device. The latency calculation module508may also be configured to calculate latencies based at least in part on content requests sent by one device and content received in response to the request. In one aspect, the latency calculation module508may be configured to operate a single device, while in other aspects it may span multiple devices in communication with one another. In other words, at least with respect to a client-server network, the latency calculations may be based on an amount of time between a content request and receipt determined by the client, an amount of time between content request and receipt determined by the server, or an amount of time between content request and receipt determined by the client and the server together.

The memory506may further store a latency selection module510to select an appropriate latency from an aggregated pool of calculated latencies. In one aspect, the latency calculation module508may be configured to store the calculated latencies in a pool of latencies for later selection by the latency selection module510. In another aspect, however, the latency selection module510may be configured to perform the aggregation of calculated latencies. Additionally, the latency selection module510may be configured to select a running average of the aggregated latencies, a predetermined threshold based on the aggregated latencies, a predetermined nthhighest latency of the aggregated latencies, or any other latency that is based at least in part on the selected latencies.

The memory506may also store a latency injection module512to facilitate injecting latency into a system to improve the perceived performance. As discussed above, the latency injection module512may be responsible for delaying and/or buffering content at a client prior to rendering the content based on the latency selected by the latency selection module. Alternatively, the latency injection module512may be configured to delay or buffer serving content from a server based on the latency selection. In yet another aspect, the latency injection module512may facilitate cooperation between a client and a server to cooperatively delay or buffer the content such that the perceived performance is improved by making the latency consistent.

The computing device500, in one configuration, may also include a current state module514stored in the memory506and executed on the processor(s)502responsible for recording and transmitting a current state of a system. In one aspect, the current state may include a location of a game feature that a user is intending to activate or move. In another aspect, however, the current state may represent a time within a game that the user perceives. In this way, the current state module514may report game feature locations and/or game times to other devices so that the devices may be synchronized. Additionally, by reporting a game feature location to an on-line game server, the computing device500can ensure that, regardless of latency, the game server will be notified if a user actually “hit” a target, even if the game server perceives a “miss.”

Various instructions, methods and techniques described herein may be considered in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. for performing particular tasks or implementing particular abstract data types. These program modules and the like may be executed as native code or may be downloaded and executed, such as in a virtual machine or other just-in-time compilation execution environment. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments. An implementation of these modules and techniques may be stored on some form of computer-readable storage media.

Illustrative Processes

FIGS. 6-7are flow diagrams showing respective processes600and700for injecting latency to improve perceived computing device performance. These processes are illustrated as logical flow graphs, each operation of which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the process.

FIG. 6illustrates an example flow diagram of process600for implementing a latency injection system that adds latency to improve the perceived performance of computing devices and networks, as discussed above.

The process600includes a remote device sending an instruction to a server hosting an on-line game at602. At604, the remote device may (optionally) send a location of a game feature to the server as well. In some instances, as noted above, the instruction sent to the server may be a request for content. At606, the remote device receives content representing the on-line game to be rendered to a user in response to the instruction or the request. At608, the remote device may calculate the latency of the transaction based on the time between sending the instruction or request and receiving the content from the server. In one instance, the remote device may calculate the latency based on information about the timing of sent content from the server.

At610, the remote device may store the calculated latency in a pool of latencies. The pool of latencies may contain previously calculated latencies from the same game or on-line session. In one aspect, the remote device may determine a subset of the highest calculated latencies at612. In other words, a few of the highest calculated latencies may be included in the subset. In another aspect, the remote device may calculate a running average or an inflated running average at612. At614, the remote device may select a game latency from the pool of calculated latencies or from the subset of highest latencies. In yet another aspect, the remote device may select the running average or the inflated running average. At616, the remote device may render subsequent received content to a user based on the selected latency.

FIG. 7illustrates an example flow diagram of process700for an alternative implementation of a latency injection system that adds latency to improve the perceived performance of computing devices and networks, as discussed above.

The process700includes receiving an instruction or a request for content from a remote device at702. At704, the process700may also (optionally) receive a location of a game element. At706, the process700may receive a time-sent indicator associated with the instruction or request from the remote device. In one aspect, this time-sent indicator may be used to aid in calculating the transaction latency. At708, the process700may (optionally) receive a client-side latency (i.e., a latency calculation made by the remote device).

At710, the process700may calculate the latency of the given transaction based on the time between the time-sent indication and receipt of the instruction (i.e., the server-side latency). In another aspect, the process700may calculate the latency of the transaction based in part on the client-side latency at710. In yet another aspect, at710, the process700may calculate the latency of the given transaction based on a combination of the server-side latency and the client-side latency. At712, the process700may select a game latency based on the calculated latency from710. At714, the process700may serve the requested content to the remote device based on the selected latency.

Additionally, whileFIG. 6generally describes a process from the perspective of a remote device andFIG. 7generally describes a process from the perspective a server device, any combination of process600and process700may be possible. For example, a process may perform any combination of the operations listed above, including, but not limited to, all of the operations of the two processes600and700combined or merely a subset of the operations listed above.

CONCLUSION