Computer architecture for animation of a model in a simulation

Methods for improving movement animation of a model in a simulation are provided. In one aspect, a method includes receiving a movement input for the model, the model includes interconnected joints having respective current locations. Updated locations for each joint are determined based on the movement input and the current locations. Modified locations for each joint are determined based on parameters of a control loop that introduces an error between the current locations and the updated locations. The model is rendered based on the modified locations. Systems and machine-readable media are also provided.

TECHNICAL FIELD

The present disclosure generally relates to computer architecture for animation of a model in a simulation of real-world events, and more specifically relates to computer architecture for movement animation of the model. The present disclosure describes improved resource utilization to produce movement animations without requiring significant extra processing or hardware.

BACKGROUND

Video games, such as sports video games, provide ever increasing realistic game-playing experiences. Although a video game engine may recreate objects and characters in a simulation, the objects and characters may be animated to move unrealistically. The objects and characters may move as if real world physics were not applicable.

SUMMARY

The disclosed system provides for animating a model in a simulation using parameters in a control loop to introduce errors in the movement. The parameters may be applied to calculate updated locations for each joint of the model.

According to certain aspects of the present disclosure, a computer-implemented method for improving movement animation of a model in a simulation is provided. The method includes receiving a movement input for the model. The model includes a plurality of interconnected joints having respective current locations. The method includes determining, based on the movement input and the current locations, updated locations for each of the plurality of joints. The method includes determining, based on at least one parameter of a control loop configured to introduce an error between the current locations and the updated locations, modified locations for each of the plurality of joints. The method includes rendering the model based on the modified locations.

According to certain aspects of the present disclosure, a system for improving movement animation of a model in a simulation is provided. The system includes a memory comprising instructions, and a processor configured to execute the instructions. The processor is configured to execute the instructions to receive a movement input for the model. The model includes a plurality of interconnected joints having respective current locations. The processor is configured to execute the instructions to determine, based on the movement input and the current locations, updated locations for each of the plurality of joints. The processor is configured to execute the instructions to determine modified locations for each of the plurality of joints, based on at least one parameter of a proportional-integral-derivative (PID) controller. The PID controller is configured to introduce an error between the current locations and the updated locations. The processor is configured to execute the instructions to render the model based on the modified locations.

According to certain aspects of the present disclosure, a non-transitory machine-readable storage medium comprising machine-readable instructions for causing a processor to execute a method for improving movement animation of a model in a simulation is provided. The method includes receiving a movement input for the model. The model includes a plurality of interconnected joints having respective current locations. The method includes determining, based on the movement input and the current locations, updated locations for each of the plurality of joints. The method includes determining modified locations for each of the plurality of joints, based on at least a proportional modifier, an integral modifier, and an input modifier of a proportional-integral-derivative (PID) controller. The PID controller is configured to introduce an error between the current locations and the updated locations. The proportional modifier corresponds to a present deviation, the integral modifier corresponds to an accumulation of past deviations, and the input modifier corresponds to a prediction of future deviations. The method includes rendering the model based on the modified locations.

According to certain aspects of the present disclosure, a system for improving movement animation of a model in a simulation includes means for receiving a movement input for the model. The model includes a plurality of interconnected joints having respective current locations. The system also includes means for determining, based on the movement input and the current locations, updated locations for each of the plurality of joints. The means for determining are further configured for determining modified locations for each of the plurality of joints, based on at least a proportional modifier, an integral modifier, and an input modifier of a proportional-integral-derivative (PID) controller. The PID controller is configured to introduce an error between the current locations and the updated locations. The proportional modifier corresponds to a present deviation, the integral modifier corresponds to an accumulation of past deviations, and the input modifier corresponds to a prediction of future deviations. The means for determining are further configured for rendering the model based on the modified locations.

DETAILED DESCRIPTION

General Overview

The disclosed system provides for improving movement animation of a model in a simulation. For example, by using a PID controller to introduce an error, corresponding to an overlap exhibited by real-world physics, to the movement animation, the movement animation may simulate real-world movement.

The disclosed system addresses a technical problem tied to computer technology and arising in the realm of computer networks, namely the technical problem of efficient and accurate movement animation of a model in a simulation. The disclosed system solves this technical problem by a technical solution of introducing error in the movement. The error may correspond to overlap, for example overshooting and/or undershooting the desired movement. The movement animation may be made more accurate through additional processing, such as implementing a robust physics engine. However, the additional processing may require additional resources, such as additional time to complete the processing, additional memory for running the robust physics engine, or hardware capable of such processing without requiring the additional time, which may affect a user's access to the simulation and usability of the simulation. The disclosed system uses a control loop, such as a PID controller, to improve the simulation results, to produce movement animation that simulates overlap exhibited in real-world movement, and requiring less processing than implementing the robust physics engine.

While many examples are provided herein in the context of a video game, the principles of the present disclosure contemplate other types of simulations as well. For example, military simulations, medical simulations, emergency response simulations, choreography simulations, motion picture simulations, and other simulations involving moving objects or people simulating realistic movement are all considered within the scope of the present disclosure.

Example System Architecture

FIG. 1illustrates an example architecture100for improving simulation of a sporting event by using real-world data to seed the simulation. The architecture100includes one or more servers130and devices110connected over a network150.

One of the many servers130is configured to host application data, such as application data for a simulation or video game. For purposes of load balancing, multiple servers130can host the application data. The server130may further be configured to host simulations for multiple devices110. For example, the server130may host a multiplayer simulation for multiple devices110to connect to, such that the multiple devices110experience the same simulation at approximately the same time.

The devices110include one or more computing devices. The devices110may include devices capable of running a simulation engine, such as a sports game, for simulating sporting events, a first person shooter (FPS) game, a first person video game, a driving simulator game, an open world (or free roam or sandbox) game, or any other type of simulation or game having objects move according to simulated real-world physics. For example, the devices110may include stationary video game consoles, tablets, mobile devices, laptop computers, desktop computers, and/or other devices capable of running a sports game.

The disclosed system uses application data to coordinate the simulations across the devices110. The server130sends the portions of the application data to one or more devices110, which uses the application data to run similar instances of the simulation. As such, the disclosed system can, for example, create a consistent simulation across the devices110.

In addition, according to certain aspects of the disclosure, the simulation may be hosted locally, for example on one of the devices110, or between two or more devices110such that application data is shared between the devices110rather than being generated by the server130.

The servers130can be any device having an appropriate processor, memory, and communications capability for hosting real-world data. The devices110to which the servers130are connected over the network150can be, for example, desktop computers, mobile computers, tablet computers (e.g., including e-book readers), mobile devices (e.g., a smartphone or PDA), set top boxes (e.g., for a television), video game consoles, or any other devices having appropriate processor, memory, and communications capabilities. The network150can include, for example, any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, and the like. Further, the network150can include, but is not limited to, any one or more of the following network topologies, including a bus network, a star network, a ring network, a mesh network, a star-bus network, tree or hierarchical network, and the like.

Example System for Simulation of a Sporting Event

FIG. 2is a block diagram200illustrating an example server130and device110in the architecture100ofFIG. 1according to certain aspects of the disclosure.

The device110and the server130are connected over the network150via respective communications modules218and238. The communications modules218and238are configured to interface with the network150to send and receive information, such as data, requests, responses, and commands to other devices on the network. The communications modules218and238can be, for example, modems or Ethernet cards.

The server130includes a processor236, a communications module238, and a memory232that includes application data252. The application data252corresponds to data for hosting a simulation, such as a multiplayer video game. The application data252may include data for generating the simulation, such as models of the environment (e.g., maps), textures, environment variables and parameters, and other data for generating the simulation. Although the present disclosure describes the simulation as a multiplayer video game, in other implementations the simulation may allow spectators to view the simulation without controlling objects or characters within the simulation.

The processor236of the server130is configured to execute instructions, such as instructions physically coded into the processor236, instructions received from software in memory240, or a combination of both. For example, the processor236of the server130executes instructions to send at least a portion of the application data252to one or more devices110. The application data252includes model data254, which includes joint data256.

The model data254includes data for defining models of every object in the simulation, which may include characters, vehicles, and other player controllable objects. The joint data256includes data on joints—corresponding to rotation or translation points—such as between “bones,” used herein to describe solid portions of the models connected at the joints. The joint data256includes location data, defining each joint's location in reference to the simulation environment, which bones are connected to the joint, and may further include additional parameters, such as types of rotations or translations associated with the joint and any limitations such as maximum angle or minimum angle or other threshold angle.

The model data254may be based on an animation technique used in the simulation. For example, a skeletal animation technique uses a hierarchical set of interconnected bones (e.g. a skeleton or rig) with a surface representation (e.g. skin or mesh). Each bone has a three-dimensional transformation, including position, scale, and orientation. The transformation of each bone includes transformations of any parent bones in the hierarchy. For example, moving an arm bone would also move a hand bone which is a child of the arm bone. Each bone is skinned to complete the visual representation of the model, for example by applying a polygonal mesh based on vertices associated with the bone. Although the model data254is described with respect to skeletal animation, other animation techniques and corresponding model data may be used, and the control loop transformations described herein may be accordingly applied to the model data.

AlthoughFIG. 2depicts the application data252, the model data254, and the joint data256in the memory232of the server130, in other implementations the memory220of the device110may include the application data252, the model data254, and the joint data256, or local copies thereof.

For each frame of animation, the models corresponding to the model data254have gamestates, which may include for each joint and/or bone, pitch, roll, yaw, directional movement, left and right, etc. The gamestates may incorporate user movement inputs or other movement inputs, such as object movement in response to the environment or other moving objects. Based on the gamestates, joint transformations are calculated for each joint and/or bone. The joint transformations may be further modified, for example based on limits on the joint and/or bone, environmental limits or restrictions, environmental effects, etc. Based on the joint transformations, the joints and/or bones are repositioned to render the next frame.

According to certain aspects of the disclosure, one or more PID controllers are implemented. For each frame, each incoming gamestate may have a corresponding PID controller. Each PID controller takes each incoming gamestate and introduces PID error to produce a corresponding outgoing gamestate. The joint transformations are modified using the PID error, and the joint transforms are then modified per state. Based on the modified joint transformations, the joints and/or bones are repositioned to render the next frame.

The device110includes a processor212, the communications module218, and the memory220that includes the application222. The application222may be a simulation engine, or software capable of simulating moving objects, such as a video game. The device110also includes an input device216, such as a keyboard, mouse, touchscreen and/or game controller, and an output device214, such as a display. The processor212of the device110is configured to execute instructions, such as instructions physically coded into the processor212, instructions received from software in memory220, or a combination of both. The processor212of the device110executes instructions from the application222causing the processor212to receive a movement input for a model in a simulation corresponding to the application222. The model includes interconnected joints having current locations. The received movement input may correspond to a user input from the input device216, or may correspond to a movement based on a contact with another model in the simulation. Executing the instructions also causes the processor212to determine, based on the movement input and the current locations, updated locations for each of the joints. Executing the instructions also causes the processor212to determine, based on at least one parameter of a control loop configured to introduce an error between the current locations and the updated locations, modified locations for each of the joints. The control loop may correspond to a PID controller. The at least one parameter may include a proportional modifier corresponding to a present deviation, an integral modifier corresponding to an accumulation of past deviations, an input modifier corresponding to a prediction of future deviations, and a threshold angle corresponding to a threshold joint movement. Executing the instructions also causes the processor212to render the model based on the modified locations.

Executing the instructions also causes the processor212to modify joint transformations using the proportional modifier, integral modifier, and input modifier. The joint transformations may be set according to the threshold angle when the modification exceeds the threshold angle. Executing the instructions also causes the processor212to perform the determining the updated locations, the determining the modified locations, and the rendering the model for each frame of the movement animation. Executing the instructions also causes the processor212to interpolate between the updated location and a location corresponding to the threshold angle when the joint transformation approaches the threshold angle.

Executing the instructions for receiving the player positional data may also cause the processor212to determine the modified locations of the joints based on a distance of the model from a virtual camera. For example, the model may be beyond a threshold distance from the virtual camera. The error introduced by the PID controller may not significantly impact the rendering of the model beyond the threshold distance, and therefore may be reduced or disabled. In other words, the model may be far away enough from the virtual camera that the movement animation may not require the PID controller modification. Processing may be reduced by disabling the PID controller modification beyond the threshold distance.

FIG. 3illustrates an example process300for improving movement animation of a model in a simulation, using the example device110ofFIG. 2. WhileFIG. 3is described with reference toFIG. 2, it should be noted that the process steps ofFIG. 3may be performed by other systems. In addition, the process300is not limited to the specific order of steps presented inFIG. 3.

The process300begins by proceeding to step301when, for example, the device110receives a movement input for a model. The model corresponds to the model data254. The model includes interconnected joints having current locations, which correspond to the joint data256. The current locations may correspond to a current pose or animation frame of the model. The movement input may be received from the input device216. Alternatively, the movement input may correspond to contact with another model in the simulation. For example, another model may have moved—from a user input or another moving object in the environment—hereby causing contact with the model, which also causes the model to move.

FIG. 4shows an example graph400of PID error according to certain aspects of the disclosure. The graph400includes a start value410, a stop value420, an input430, a model curve440, a drag error450, an integral error460, and a proportional error470. The start value410corresponds to the current location of a joint in step301. The input430corresponds to the movement input in step301. The model curve440shows the angular value of the joint over time, for example each frame of animation. The input430may correspond to a user input for moving a virtual camera of the simulation, which may be a first-person video game, and the model curve440may correspond to a model of a weapon of the first-person video game, which can be seen by the virtual camera. More specifically, the virtual camera may correspond to the user's first-person view, and the weapon may be seen in the view. However, in other implementations the input430may correspond to another movement input, and the model curve440may correspond to another model in the simulation. The movement input may correspond to any kind of input that can derive or cause the model to move. For example, the input430may correspond to a force applied to the model, and the model curve440may correspond to a component of the model, such as a body part, limb, or other object component. In addition, althoughFIG. 4depicts angular values, corresponding to gamestates such as pitch, roll, or yaw which utilize angular values, in other implementations other dimensions, such as distance, may be used.

Returning toFIG. 3, at step302, the device110determines, based on the movement input and the current locations, updated locations for the joints. InFIG. 4, based on the start value410and the input430, the stop value420is determined. The stop value420corresponds to the updated locations, which also corresponds to the destination indicated by the movement input.

The destination may correspond to a new location from the current location based on a magnitude and direction of the input430. Alternatively, the destination may correspond to a specific pose for the model, for example as part of an animation sequence corresponding to the movement input. The pose may include joint locations. The animation sequence may include poses for each time slice of the animation sequence.

At step303, the device110determines, based on at least one parameter of a control loop configured to introduce an error between the current locations and the updated locations, modified locations for the joints. For example, a PID controller may be configured to introduce error between the current location and the updated location. Parameters for the PID controller may include one or more of a proportional modifier, an integral modifier, an input modifier, and a threshold angle. The proportional modifier corresponds to a present deviation from the updated location. The integral corresponds to an accumulation of past deviation. The input modifier corresponds to a prediction of future deviations. The threshold angle corresponds to a threshold joint movement, such as a minimum angle or maximum angle which the joint may move. Any joint transformations which exceeds the threshold angle may be instead be set to the threshold angle rather than incorporating the PID error. In addition, if the joint transformation approaches the threshold angle, for example if the joint transformation is within a threshold offset from the threshold angle, the joint transformation may instead interpolate between the current location and the updated location rather than incorporating the PID error.

InFIG. 4, the start value410may correspond to the user's first person view. The weapon normally remains centered in the view, for example when the user's player is standing still or otherwise not moving, to allow the user to aim. When the user inputs the input430to move the weapon to re-aim, the weapon may follow exactly the input430to the stop value420. Although such movement keeps the weapon centered throughout the animation for the re-aiming, such movement does not look realistic. For instance, because objects in the real world have mass and exhibit inertia when moving and stopping, the weapon in the real world would not follow exactly the input430to the stop value420.

The PID controller introduces PID error to the movement to simulate the effects of inertia without requiring additional computations to estimate actual inertia for the model having a simulated mass. As seen inFIG. 4, the PID controller adds the drag error450, the integral error460, and the proportional error470such that the model curve440does not follow exactly the input430. The PID controller adds lagging behavior, from the drag error450, such that the model lags behind the input430. The PID controller also adds overlap behavior, from the integral error460and the proportional error470, such that the model overshoots and undershoots the stop value420several times before stopping at the stop value420.

The proportional modifier may adjust the proportional error470, which corresponds to a stiffness of the movement. The stiffness corresponds to how many oscillations before the model settles on the stop value420. The integral modifier may adjust the integral error460, which corresponds to magnitude of overshoot. The input modifier may adjust the drag error450, which corresponds to magnitude of lag behind the input430.

For a given model, there may be a PID controller for each joint of the model, such that there is at least one PID calculation per joint. Thus, there may be multiple PID controllers running concurrently for each of the models in the simulation.

At step304, the device110renders the model based on the modified locations. The device110displays the rendered model on the output device214.FIG. 5Ashows a frame500A of an animation without PID error, andFIG. 5Bshows a frame500B of the animation with PID error. The frames500A and500B may correspond to the same frame of animation, for example a frame in which a character model520, holding a weapon model510, has just touched down on the ground after a jumping motion.

InFIG. 5A, the weapon model510is level. However, a person holding a weapon having mass would normally not be able to keep the weapon level when the person lands from a jump. InFIG. 5B, the weapon model510is not level, but rather tilted downwards, due to the introduction of the PID error. For example, the integral modifier may have introduced an overshoot such that the weapon model510tilts downward. The integral modifier and the input modifier may also affect the final tilt angle of the weapon model510inFIG. 5B. In addition, the character model520is also affected by the tilting of the weapon model510. For example, the hands and arms of the character model520are accordingly modified based on the tilting of the weapon model510. The PID error simulates inertia, such that a more realistic pose is rendered for the weapon model510and the character model520.

In another example, the model data254may correspond to a vehicle model and a character model driving the vehicle model. The vehicle model may be driving over terrain. The PID error may be introduced to change how the vehicle model responds to the terrain. For example, if the terrain is bumpy, the PID error may be applied to simulate a suspension of the vehicle model such that a body of the vehicle model may oscillate independently of tires of the vehicle model. Portions of the vehicle model may tilt at different speeds. For example, when the vehicle model stops, the body of the vehicle model may tilt forward and oscillate before settling to a stop. In addition, based on the motion of the vehicle model, the character model may also move. For example, if the body of the vehicle model is oscillating, the character model will also oscillate with the body, and further portions of the character model, such as limbs, will further move based on additional PID error introduced to the limbs.

The steps301-304may repeat for each frame of animation. In other words, the movement input may correspond to an animation sequence. For each frame of the animation sequence, the current pose is evaluated (step301), the pose is updated (step302), the PID controller input is added and transformation applied to the pose (step303) and the model is rendered for the frame based on the transformation (step304).

Because the PID error may be introduced as a post process for the animation rendering, the PID error may be further adjusted based on performance or user considerations. For example, in certain situations PID error may not be applied in order to reduce processing. PID error may be adjusted based on line of sight. The PID error may not be applied to models that are outside of the user's view. In a multiplayer simulation, in which the server130manages the simulation and the devices110recreates local instances of the simulation based on the server130, the devices110may locally apply PID error to models in the corresponding views. In a peer-to-peer simulation, each peer device110may apply PID error to the same models.

If a model is beyond a threshold distance from the camera, the PID error may be reduced as the distance is increased and may be disabled beyond a second threshold distance from the camera. In other situations, such as when accurate and consistent collision detection is required across the devices110, the PID error may also be disabled. In addition, if a threshold number of models in view is reached, a threshold number of joints is reached—which may correspond to the threshold number of models—or a threshold number of PID controllers (or other control loops) is reached, additional PID controllers may be disabled.

In certain other situations, such as when the user requires responsiveness or precise input, the PID error may be disabled. For example, the PID error-based animation may be immediately disabled based on specific user inputs, including if the user is firing the weapon in the first-person video game.

Hardware Overview

FIG. 6is a block diagram illustrating an example computer system600with which the device110and server130ofFIGS. 1 and 2can be implemented. In certain aspects, the computer system600may be implemented using hardware or a combination of software and hardware, either in a dedicated server, or integrated into another entity, or distributed across multiple entities.

Computer system600(e.g., device110and server130) includes a bus608or other communication mechanism for communicating information, and a processor602(e.g., processor212and236) coupled with bus608for processing information. According to one aspect, the computer system600is implemented as one or more special-purpose computing devices. The special-purpose computing device may be hard-wired to perform the disclosed techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are persistently programmed to perform the techniques, or may include one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networking devices or any other device that incorporates hard-wired and/or program logic to implement the techniques. By way of example, the computer system600may be implemented with one or more processors602. Processor602may be a general-purpose microprocessor, a microcontroller, a Digital Signal Processor (DSP), an ASIC, a FPGA, a Programmable Logic Device (PLD), a controller, a state machine, gated logic, discrete hardware components, or any other suitable entity that can perform calculations or other manipulations of information.

Computer system600can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them stored in an included memory604(e.g., memory232or220), such as a Random Access Memory (RAM), a flash memory, a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable PROM (EPROM), registers, a hard disk, a removable disk, a CD-ROM, a DVD, or any other suitable storage device, coupled to bus608for storing information and instructions to be executed by processor602. The processor602and the memory604can be supplemented by, or incorporated in, special purpose logic circuitry. Expansion memory may also be provided and connected to computer system600through input/output module610, which may include, for example, a SIMM (Single In Line Memory Module) card interface. Such expansion memory may provide extra storage space for computer system600, or may also store applications or other information for computer system600. Specifically, expansion memory may include instructions to carry out or supplement the processes described above, and may include secure information also. Thus, for example, expansion memory may be provided as a security module for computer system600, and may be programmed with instructions that permit secure use of computer system600. In addition, secure applications may be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner.

Computer system600further includes a data storage device606such as a magnetic disk or optical disk, coupled to bus608for storing information and instructions. Computer system600may be coupled via input/output module610to various devices (e.g., input device216and/or output device214). The input/output module610can be any input/output module. Example input/output modules610include data ports such as USB ports. In addition, input/output module610may be provided in communication with processor602, so as to enable near area communication of computer system600with other devices. The input/output module610may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used. The input/output module610is configured to connect to a communications module612. Example communications modules612(e.g., communications modules238and/or218) include networking interface cards, such as Ethernet cards and modems.

The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. The communication network (e.g., communication network150) can include, for example, any one or more of a PAN, a LAN, a CAN, a MAN, a WAN, a BBN, the Internet, and the like. Further, the communication network can include, but is not limited to, for example, any one or more of the following network topologies, including a bus network, a star network, a ring network, a mesh network, a star-bus network, tree or hierarchical network, or the like. The communications modules can be, for example, modems or Ethernet cards.

For example, in certain aspects, communications module612can provide a two-way data communication coupling to a network link that is connected to a local network. Wireless links and wireless communication may also be implemented. Wireless communication may be provided under various modes or protocols, such as GSM (Global System for Mobile Communications), Short Message Service (SMS), Enhanced Messaging Service (EMS), or Multimedia Messaging Service (MMS) messaging, CDMA (Code Division Multiple Access), Time division multiple access (TDMA), Personal Digital Cellular (PDC), Wideband CDMA, General Packet Radio Service (GPRS), or LTE (Long-Term Evolution), among others. Such communication may occur, for example, through a radio-frequency transceiver. In addition, short-range communication may occur, such as using a BLUETOOTH, WI-FI, or other such transceiver.

In any such implementation, communications module612sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information. The network link typically provides data communication through one or more networks to other data devices. For example, the network link of the communications module612may provide a connection through local network to a host computer or to data equipment operated by an Internet Service Provider (ISP). The ISP in turn provides data communication services through the world-wide packet data communication network now commonly referred to as the “Internet.” The local network and Internet both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on the network link and through communications module612, which carry the digital data to and from computer system600, are example forms of transmission media.

Computer system600can send messages and receive data, including program code, through the network(s), the network link and communications module612. In the Internet example, a server might transmit a requested code for an application program through Internet, the ISP, the local network and communications module612. The received code may be executed by processor602as it is received, and/or stored in data storage606for later execution.

In certain aspects, the input/output module610is configured to connect to a plurality of devices, such as an input device614(e.g., input device216) and/or an output device616(e.g., output device214). Example input devices614include a keyboard and a pointing device, e.g., a mouse or a trackball, by which a user can provide input to the computer system600. Other kinds of input devices614can be used to provide for interaction with a user as well, such as a tactile input device, visual input device, audio input device, or brain-computer interface device. For example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, tactile, or brain wave input. Example output devices616include display devices, such as a LED (light emitting diode), CRT (cathode ray tube), LCD (liquid crystal display) screen, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display) or an OLED (Organic Light Emitting Diode) display, for displaying information to the user. The output device616may include appropriate circuitry for driving the output device616to present graphical and other information to a user.

According to one aspect of the present disclosure, the device110and server130can be implemented using a computer system600in response to processor602executing one or more sequences of one or more instructions contained in memory604. Such instructions may be read into memory604from another machine-readable medium, such as data storage device606. Execution of the sequences of instructions contained in main memory604causes processor602to perform the process steps described herein. One or more processors in a multi-processing arrangement may also be employed to execute the sequences of instructions contained in memory604. In alternative aspects, hard-wired circuitry may be used in place of or in combination with software instructions to implement various aspects of the present disclosure. Thus, aspects of the present disclosure are not limited to any specific combination of hardware circuitry and software.

As used in this specification of this application, the terms “computer-readable storage medium” and “computer-readable media” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals. Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media. For example, transmission media includes coaxial cables, copper wire and fiber optics, including the wires that include bus608. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications. Furthermore, as used in this specification of this application, the terms “computer,” “server,” “processor,” and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms display or displaying means displaying on an electronic device.

Terms such as “top,” “bottom,” “front,” “rear” and the like as used in this disclosure should be understood as referring to an arbitrary frame of reference, rather than to the ordinary gravitational frame of reference. Thus, a top surface, a bottom surface, a front surface, and a rear surface may extend upwardly, downwardly, diagonally, or horizontally in a gravitational frame of reference. Furthermore, to the extent that the term “include,” “have,” or the like is used in the description or the claims, such term is intended to be inclusive in a manner similar to the term “comprise” as “comprise” is interpreted when employed as a transitional word in a claim.

The title, background, brief description of the drawings, abstract, and drawings are hereby incorporated into the disclosure and are provided as illustrative examples of the disclosure, not as restrictive descriptions. It is submitted with the understanding that they will not be used to limit the scope or meaning of the claims. In addition, in the detailed description, it can be seen that the description provides illustrative examples and the various features are grouped together in various implementations for the purpose of streamlining the disclosure. The method of disclosure is not to be interpreted as reflecting an intention that the claimed subject matter requires more features than are expressly recited in each claim. Rather, as the claims reflect, inventive subject matter lies in less than all features of a single disclosed configuration or operation. The claims are hereby incorporated into the detailed description, with each claim standing on its own as a separately claimed subject matter. The claims are not intended to be limited to the aspects described herein, but are to be accorded the full scope consistent with the language claims and to encompass all legal equivalents. Notwithstanding, none of the claims are intended to embrace subject matter that fails to satisfy the requirements of the applicable patent law, nor should they be interpreted in such a way.