Touchscreen game user interface

A swipe input mechanism for a computer-implemented game presented on a touchscreen allows for user-selection of different menu options by swipe gestures in different directions via a swipe menu dedicated to receiving such swipe input. In a third person shooter game or a first-person shooter game user-selection of different character abilities are enabled via the swipe menu. The swipe menu is a floating user interface element that auto-adjusts its on-screen position to register with the position of user touch.

BACKGROUND

Computer games are increasingly being made available for play across a variety of different types of devices. By virtue of the different natures of different types of devices (e.g., gaming consoles, PCs, and mobile touchscreen devices), the user interfaces for such different platforms are also different and provide different respective advantages and drawbacks.

Traditionally, games such as first-person shooters (FPS), in which speed and precision of user control is a great importance and in which a number of different character abilities are available for selective deployment, have been primarily available on devices with separate manual input devices such as mouse and keyboard (for PCs) or a dedicated controller (for gaming consoles). Recently, such games have increasingly been made available for play on mobile devices with touchscreen input, such as mobile phones, tablets, or touchscreen laptops.

Existing game user interfaces on touchscreen devices, however, have a number of disadvantages over console or PC input mechanisms. One such example is where different character abilities are selectable by haptic selection of different respective soft buttons. In particular, the speed and accuracy of button selection is problematic on small screens where the buttons to be selected are often obscured by the user's hand, where haptic feedback from elements such as control sticks and physical buttons are unavailable, and where limited screen space is available for display of all selectable ability buttons.

INTRODUCTION

This disclosure provides for a swipe input mechanism that allows for user selection of different menu options on a touchscreen by swipe gestures in different directions via a user interface element or widget dedicated to receiving such swipe input. This enables the depth and complexity of inputs offered by PC or Console input devices to be achieved without the loss in speed and tap accuracy typical to touch screen devices.

One aspect of the disclosure thus provides for a method comprising: generating on a touchscreen of a mobile user device a game display that shows in-game action of a computer-implemented game, the in-game action comprising user-controlled movement of a player character within a virtual world;providing a movement control mechanism that enables user control of movement of the player character via touchscreen input;providing a swipe input mechanism that enables selective deployment of a plurality of abilities of the player character by different respective swipe inputs via the touchscreen in a corresponding plurality of trigger directions;detecting a swipe input via the swipe input mechanism;identifying the trigger direction selected by the swipe input; andcausing in-game deployment for the player character of that one of the plurality of abilities which corresponds to the selected trigger direction.

Providing the swipe input mechanism may comprise displaying as part of a heads-up display (HUD) overlaid on the game display an abilities widget, the plurality of trigger directions extending radially relative to a center defined by the abilities widget, with at least two of the plurality of trigger directions being transverse to one another and intersecting at the center. In some embodiments, the trigger directions may be the four cardinal directions oriented horizontally and vertically (i.e., up, down, left, and right). In other embodiments, more or fewer swiping directions may be supported.

In some embodiments, the abilities widget comprises a plurality of ability elements arranged circumferentially about the center, each ability element being a graphic user interface element that is user-selectable by a radially outward swipe input extending from the center in a corresponding one of the plurality of trigger directions. The ability elements may in some embodiments be movable responsive to swiping (e.g., buttons or tactile sliding badges), but it may in other embodiments be selectable simply by swiping over them.

In some embodiments, the method further comprises:

switching one or more of the plurality of ability elements between a ready state in which the corresponding ability is available for selection, and an unready state in which the corresponding ability is unavailable for selection; and

responsive to switching a particular ability element between the ready state and the unready state, providing a visual indication of the switching by varying one or more visual attributes of the particular ability element.

In one example embodiment, the visual indication comprises applying different color schemes based on readiness.

The unready state for at least one of the plurality of ability elements may comprise a charging mode in which a charge level for the corresponding ability progressively increases towards a predefined charge threshold at which the ability element automatically switches to the ready state. The method may in such embodiments further comprise displaying in association with the respective ability element a dynamic charge indicator that graphically indicates the charge level for the corresponding ability. In some embodiments, each ability element comprises a slidable trigger button. In such embodiments, each dynamic charge indicator may comprise a charge ring about the corresponding trigger button, the charge ring being progressively filled annularly to indicate the charge level for the associated ability.

As mentioned, the plurality of ability elements may comprise, for each of the plurality of trigger directions, a respective trigger button that is radially slidable relative to the center in the corresponding trigger direction. In such cases, the method may further comprise displaying animated sliding movement of the trigger button corresponding to the selected trigger direction responsive to the detected swipe input. Switching a particular ability element to the ready state made some embodiments comprise extending a firing lane radially outwards from the particular ability element, the corresponding trigger button being slidable along the firing lane. The firing lane may be automatically retracted when the ability is in the unready state.

The abilities widget in some embodiments has a dynamically variable on-screen position that automatically adjusts to an origin point for swipe input. The method may in such embodiments comprise:

prior to receiving the swipe input, detecting initiation of touch contact for swipe control of the abilities widget;

identifying on-screen coordinates of the touch contact initiation as the origin point for swipe input; and

automatically adjusting the on-screen position of the abilities widget such the center of the abilities widget is located at the identified origin point.

The detection of touch contact for swipe control of the abilities widget may be limited to a predefined abilities control area that forms a subset of a total input-receptive area of the touchscreen, the method comprising detecting a touch input as initiation of touch contact for swipe control of the abilities widget responsive to and conditional upon satisfaction of on a set of criteria. In one embodiment, the set of criteria for identifying initiation of swipe contact comprises: that the touch input is received in the abilities control area; and that the touch input is sustained for longer than a predefined threshold interval.

Some of the selectable abilities may support an aiming function, such as to allow user selection of the particular direction and/or distance an object is to travel. In such instances, in-game deployment of the selected ability may comprise:

entering an aiming mode in which a trajectory or a target area for the selected ability is displayed while touch contact that provided the swipe input is sustained;

changing the displayed trajectory or target area responsive to user-provided change in position of the sustained touch contact; and

upon release of the touch contact, deploying the selected ability according to the displayed trajectory or target area.

DETAILED DESCRIPTION

FIG. 1shows a schematic depiction of a mobile phone102having a touchscreen104on which a game display106is provided. In this example embodiment, the game display106is for a computer-implemented third-person shooter (FPS) game. In conventional fashion, the game display106thus shows in-game action comprising user-controlled movement of a player character108within a virtual world in which gameplay is set. Gameplay comprises moving through the virtual three-dimensional world to achieve predefined objectives against other player characters (PCs) or nonplayer characters (NPCs). In this example embodiment, user-controlled functions of the character108include movement, controlling direction of view/aim, jumping, melee, shooting, and the deployment of a number of predefined special skills or abilities. As will be described below, these abilities include, inter alia, switching to a map view, deploying a grappling hook, throwing a grenade, and deploying a special weapon such as a rocket or the like. It will be appreciated that the above-listed actions and abilities provide a nonexhaustive list of functions that are controllable in this or in other embodiments by means of a user interface according to the disclosure.

The game display106is presented on the touchscreen104, which also serves as an exclusive input device to control the in-game behavior of the player character108via manual haptic or touch input by the user. The game display106on the touchscreen104thus provides a user interface for both displaying in game action and for receiving user input. The game display106includes a heads up display (HUD) that comprises a number of graphical user interface (GUI) elements with which the user can interact to control operation of the character108.

In the example embodiment ofFIG. 1, the HUD comprises a movement control mechanism in the form of a movement element110that enables user control of movement of the character108within the virtual world. The HUD further comprises a swipe input mechanism that enables selective deployment of a plurality of different abilities of the character108by swipe gestures in different respective directions. In this example embodiment, the swipe input mechanism comprises an abilities widget502that provides for triggering four different abilities of the character108by swiping in one of the four cardinal directions (up, down, left or right). The particular behaviors and functions the abilities widget502of the example embodiment ofFIG. 1will be described in greater detail below with reference toFIG. 2throughFIG. 4.

Before discussing in-depth the functionalities provided by the abilities widget502, we briefly turn toFIG. 5, which illustrates a mechanism for automatically discriminating between touch input provided for different respective purposes. In the example embodiment ofFIG. 5, the user interface provided by the game display106automatically discriminates, based on onscreen location of touch initiation, between touch input for controlling (a) character movement, (b) look direction, and (c) deployment of special abilities by means of swipe gestures.

FIG. 5shows a stylized schematic view of a game display500according to another embodiment, in which the abilities widget502has a different shape and configuration from that discussed with reference toFIG. 1throughFIG. 4. The mechanism for discriminating between the inputs for the different respective purposes, however, functions similarly for the embodiments ofFIG. 1andFIG. 5. In these example embodiments, distinguishing between different types of touch input is based on predefined segregated regions or areas of the touchscreen104, indicated in different colors inFIG. 5exclusively for ease of description. The game display500includes a movement control area504(hatched diagonally descending rightwards), a look control area506(hatched diagonally ascending rightwards), and an abilities control area508(unhatched). Touch input received in the movement control area504is automatically identified as input directed to controlling movement of the character108, while touch input received in the look control area506is automatically identified as input directed to controlling the direction in which the character108is looking or aiming, as the case may be. Only touch input initiated in the abilities control area508, however, is identified as input received for swipe control of the abilities widget502. As will be described later herein with reference to other example embodiments, the abilities widget502has a dynamically variable on-screen position that dynamically adjusts to an origin point for swipe input. In this example embodiment, responsive to detecting initiation of touch contact for swipe control of the abilities widget502(e.g., responsive to detecting the initiation of touch contact in the abilities control area508), the mobile phone102automatically identifies as an origin point the on-screen coordinates of that touch input, and automatically adjusts the on-screen position of the abilities widget502such that it is centered on the origin point. As will become evident from what follows, such dynamically responsive auto positioning of the abilities widget502promotes ease of use and accuracy of interpreting user intentions communicated through swipe gestures directed at the abilities widget502.

FIG. 2throughFIG. 4schematically illustrate operation of the abilities widget112according to the example embodiment ofFIG. 1, the abilities widget112being shown in isolation. As can be seen inFIG. 2, the abilities widget is an interactive user interface element that comprises a plurality of swipable ability elements in the example form of four trigger buttons that can be swiped in different respective trigger directions to deploy different respective in-game abilities or special skills for the character102. (See, for example,FIG. 4in which one of the abilities supported by the abilities widget112is deployed by a swipe gesture in the corresponding trigger direction). In the example embodiment ofFIG. 2, the trigger buttons provided by the abilities widget112are:a ping button202that can be swiped upwards to look at a map of the game world or to view an enemy position;grappling hook button204that can be swiped horizontally rightwards to deploying a grappling hook;a grenade button206that can be swiped vertically downwards to throw a grenade or, in this example embodiment, enter a grenade aim and release mode as will be described later; anda rocket button208that can be swiped horizontally leftwards (seeFIG. 4) to launch a rocket.

A center and swiping origin point of the abilities widget112is provided by a central pad210located at the intersection of the horizontal and vertical trigger directions, so that the four trigger directions extend radially relative to the central pad210. As mentioned previously, the abilities widget112is in this example embodiment a floating HUD element, being dynamically variable in on-screen position to automatically register with the user's finger placement for providing swipe input to the abilities widget112. Thus, when the user initiates touch contact in the abilities control area508(FIG. 5) and holds contact for more than a predefined lower threshold period (so as to distinguish between tap input and the initiation of a swipe gesture), the mobile phone102automatically identifies the on-screen coordinates of the touch contact, and immediately moves the abilities widget112such that the central pad210is located at the identified coordinates. In this manner, the mobile phone102automatically and responsively ensures that the central pad210is located at the origin point of any swipe gesture inputted to the abilities widget112. As will be described below, the central pad210in this example embodiment also functions as a help button for launching a helper overlay panel by holding the central pad210for longer than a predefined upper threshold without swiping one of the trigger buttons.

Each of the trigger buttons has a respective icon212that graphically indicates the particular ability or skill that can be triggered by that button. In this example embodiment, each icon212additionally indicates whether or not the corresponding ability is available for selection. Each trigger button is operable between an unready state (in which the ability is not available) and a ready state in which the ability is available for selection. Switching of a trigger button from unready to ready is indicated by varying visual attributes of the icon212of that trigger button. In this example embodiment, the color schemes of ready buttons and unready buttons are inverted relative to one another. Thus, in the view ofFIG. 2, for example, the ping button202is in the ready state, having a darker background and a lighter foreground image. In contrast, the rocket button208, grappling hook button204, and grenade button206are in the unready state, having darker backgrounds and lighter foreground images. In this example embodiment the trigger buttons in the unready state are in a cooldown period charging mode where a charge level for the ability progressively increases towards a charge level at which the ability becomes available. In some instances, each ability thus has a recharge period subsequent to deployment before the ability can be used again. Each trigger button in the charging mode includes a dynamic charge indicator in the form of a charge ring214that extends circumferentially around the periphery of the trigger button, the charge ring214progressively filling annually to indicate the charge level for the corresponding ability. Thus, when the charge ring214for a particular ability is filled, the color scheme for the corresponding icon212is inverted to indicate that the relevant ability is available. Compare, for example, the appearance of the rocket button208while charging (FIG. 2) and when available for deployment (FIG. 3).

In this example embodiment, the abilities widget112automatically switches between a dormant mode (FIG. 2) when the user does not interact with it, and an active mode (FIG. 3andFIG. 4) during haptic engagement of the abilities widget112by the user. The major difference between the abilities widget112in the dormant mode and the active mode in this example embodiment is the display of a gesture lane or firing lane for each fully charged trigger button. In the active mode, responsive to touch input in the abilities control area508, each trigger button in the ready mode is provided with a respective firing lane302projecting radially outwardly from the trigger button in the corresponding trigger direction. The firing lane302indicates the direction and extent of a swipe input required for triggering deployment of the relevant ability. Each firing lane302is in this example embodiment capped by a semicircular bumper304at the distal end of the firing lane302. Display of the firing lane302promotes ready identification by the user of those abilities that are available for deployment. Note that when an ability becomes available (i.e., reaches a full charge level) while the abilities widget112is in the active mode, it automatically results in immediate extension of the corresponding firing lane302. If the abilities widget112is released without firing any of the available abilities, the firing lane302of each availability is automatically retracted, returning it to the abilities widget112to the configuration shown inFIG. 2. Selective deployment of an in-game ability by use of the abilities widget112will now be described by way of example with reference to launching a rocket using the rocket button208. To deploy any of the available abilities or skills, the user swipes from the central pad210in the trigger direction of the chosen ability (indicated by the radial direction of the associated firing lane302). In this instance, the user thus slides a finger in contact with the touchscreen104horizontally leftwards from the central pad210. Responsive to this swipe gesture, the selected trigger button slides radially outwards towards the lane bumper304. In this instance, the button thus slides leftwards with the swipe gesture, as shown schematically inFIG. 4.

Every firing lane302other than that of the selected ability is automatically retracted into the abilities widget112. In the present example, the firing lane302of the ping button202is thus retracted when animated sliding of the rocket button208begins, as again shown inFIG. 4. In some embodiments, deployment of the selected ability is performed upon release of the selected trigger button. Thus, in the example embodiment ofFIG. 4, the user would slide the rocket button208to the extreme left position shown, would optionally delay deployment of the rocket by maintaining contact with the touchscreen104, and can precisely time the instant of rocket deployment by releasing the rocket button208at the desired moment. In this embodiment, however, abilities other than that of the grenade button206are deployed immediately when the selected trigger button has traveled a predefined threshold distance along the corresponding firing lane302. Thus, the selected rocket launch is automatically performed when the rocket button208is brought up against the bumper304of its firing lane302. When the selected ability is deployed, a feedback flash is produced by the touchscreen104to indicate or confirm that the option has been activated. In some embodiments, triggering of the ability can be confirmed by a tactile indication such as a phone vibration. After firing a particular ability, the selected trigger button and firing lane302are retracted inwards, and the fired trigger button is returned to charging mode for the predefined cooldown period. The operations described above in engaging the abilities widget112and firing a selected ability by swipe gesture typically consumes less than one second. Note that the firing lanes302for abilities in the cooldown period (and in some instances for locked abilities that are not merely recharging but are persistently unavailable to the user) do not expand. Swiping in the direction of a locked or charging trigger button has no effect. The user can in such case return touch contact to the central pad210, or can release contact to no effect. For some skills or abilities, the abilities widget112can in some embodiments provide aiming functionality. In this example embodiment, aiming functionality via the abilities widget112is provided for the grenade button206. To cause the player character108to throw a grenade, the user thus swipes the grenade button206(when fully charged) downwards and continues to hold contact with the touchscreen104. An aiming mode is automatically entered, in which a visual trajectory and/or landing zone for the grenade within the virtual world is shown in the game display106. The user can now aim the grenade by vertical finger movement to vary the distance of the throw, and by horizontal movement to change the direction of the throw. When the user releases the grenade button206, the grenade is launched according to the selected trajectory. As mentioned, the central pad210in this example embodiment provides a help functionality, as indicated by the help icon displayed thereon. In this instance, tapping and holding contact with the abilities widget112for longer than a predefined threshold duration (in this example, one second) automatically invokes a helper overlay panel explaining character abilities and location of the abilities on the abilities widget112. Lifting contact immediately dismisses the overlay. Repeated tapping in the abilities control area508without holding (e.g., an above-threshold number of taps within a predefined period, for example two or more taps within one second) causes display of a message saying “tap and hold for help.”

FIG. 6is a schematic block diagram of (in the example embodiment ofFIG. 1throughFIG. 5being provided by a portable electronic device in the form of a mobile phone102), in accordance with an example embodiment. The system600comprises a number of different hardware-implemented modules, units, or other means configured for automated performance of associated operations, as described in greater detail elsewhere herein. The various components may, in some embodiments, be provided by permanently configured circuit arrangements, and may, in other embodiments, be provided by software executing on one or more dynamically reconfigurable processor devices, thereby to configure the processor devices sequentially, or in parallel, to provide the respective hardware-implemented modules.FIG. 10andFIG. 11provide software and hardware architectures one such embodiment. In some embodiments, the system600may be provided by server-side components, for example being provided by a game server communicating with a mobile electronic device such as the mobile phone102to execute the game on the mobile phone102as a client device. In other embodiments, the system600is provided by the mobile phone102executing custom software.

The system600includes a game management module602configured for hosting the computer-implemented game, controlling in game behavior of the player character108and providing the game display106on the touchscreen104.

An input interpreter604is provided to receive and interpret user input provide with respect to the game via the touchscreen104. In particular, the input interpreter604is configured automatically to receive touch input via the touchscreen104, to identify whether the touch input is directed to controlling movement, aim direction, or ability deployment (consistent in one example embodiment with the methodologies described with reference toFIG. 5), and to interpret swipe gestures received in the abilities control area508to identify the swipe direction of such swipe gestures.

The system600further includes an abilities widget module606configured to automatically control behavior of the abilities control widget as described with reference to the example embodiment ofFIG. 1throughFIG. 5, and to implement an example method as described below.

FIG. 7shows a high-level flowchart of an example method according to an example embodiment. The method700is in this example embodiment performed by the mobile phone102(which provides a system600consistent withFIG. 6and the architecture ofFIG. 10andFIG. 11), consistent with the example embodiment illustrated inFIG. 1throughFIG. 5.

At operation702, method700generates on the touchscreen104of a mobile user device (e.g., the mobile phone102,FIG. 1) a game display106that shows in-game action of a computer-implemented game, the in-game action comprising user-controlled movement of the player character108within a virtual world. At operation704, a movement control mechanism is provided that enables user control of movement of the player character108via touchscreen input. In this example embodiment, the movement control mechanism enables control of player movement using touch input in the movement control area504(FIG. 1).

At operation706, method700provides a swipe input mechanism that enables selective deployment of a plurality of abilities of the player character108by different respective swipe inputs via the touchscreen104in a corresponding plurality of trigger directions. In the example embodiment under discussion, the swipe input mechanism comprises the abilities control abilities widget112(FIG. 1throughFIG. 4), with the plurality of trigger directions being the four cardinal directions associated with the abilities indicated by the four trigger button icons212discussed with reference toFIG. 2. At operation708, method700detects a swipe input via the swipe input mechanism, e.g. comprising a leftward swipe on the abilities widget112as illustrated inFIG. 4. At operation710, method700identifies the trigger direction selected by the swipe input, in the example embodiment ofFIG. 4identifying the horizontal leftward swipe direction. At operation712, method700causes in-game deployment for the player character108of that one of the plurality of abilities which corresponds to the selected trigger direction, in the example embodiment ofFIG. 4being deployment of a rocket, as described previously.

FIG. 8shows a more detailed flowchart of a method800of providing a game user interface according to an example embodiment. The method800is in this example embodiment performed by the mobile phone102(which provides a system600consistent withFIG. 6and the architecture ofFIG. 10andFIG. 11), consistent with the example embodiment illustrated inFIG. 1throughFIG. 5.

At operation802, there is no touch contact in the abilities control area508(FIG. 5) and the abilities widget112is thus displayed in the dormant mode (FIG. 2) in which it is radially contracted. At operation804, touch input is detected in the abilities control area508, and responsive to the touch contact being sustained for an above-threshold interval, the touch contact is identified as being intended for initiation of a swipe gesture.

Responsive to detecting the touch contact for swiping, the abilities widget112is immediately moved, at operation804, such that the center of the abilities widget112(in this example embodiment, the central pad) is located at the location of the touch input. At operation808, the abilities widget112is switched to the active mode, in this example embodiment comprising expanding the firing lane302for each ability that is available for selection. At operation810, a swipe gesture is received extending radially outwards from the central pad210, and the system automatically identifies the trigger direction thus selected. It will be appreciated that such trigger direction identification is based in part on predefined tolerances for the precise direction swiped, and can employ a triage algorithm for automated disambiguation in borderline cases. If the selected ability is a non-aiming ability (e.g., a rocket, map look, or grappling hook in the example embodiment ofFIG. 2) then the selected ability is deployed, at operation818when the selected trigger button (e.g., the rocket button208inFIG. 4) is swiped to the firing position (or, in some embodiments, when the swiped button is released). If, however, the selected ability is an ability supported by aiming (e.g., the grappling hook button204), the aiming mode is entered, at operation812.

As described previously, the aiming mode comprises displaying a trajectory or the target area. At operation814, the aim is adjusted manually by the user by sustaining touch contact and moving the point of contact to correspond with the desired trajectory and/or target area). At operation816, the touch contact is released, responsive to which the selected ability is deployed (e.g., the grenade is thrown), at operation818. After deployment of the selected ability at operation818(or upon release of the sustained touch contact before operation810), the abilities widget112is returned to the dormant mode, at operation802. Benefits of the disclosed techniques include improved accuracy and speed of receiving and interpreting user input when compared to existing techniques. Allowing selection of different abilities by swiping in different directions is more tolerant and less prone to error than requiring the selection of the precise on-screen position of a soft button, for example. There is also a greater subjectively experienced difference between swiping in two directions that are at least 90° apart than by selecting soft buttons that may be located right next to each other on-screen.

Input accuracy is further promoted by the auto-adjusting floating location of abilities widget112, so that the user need not select the precise location of the widget center, as is the case with conventional button presses. In this respect, it is to be appreciated that the user interface elements at the lateral end of the touchscreen is typically obscured by the user's thumb during gameplay, which makes accurate option selection in conventional interfaces challenging, but is a virtual irrelevancy with the floating swipe input mechanisms disclosed herein.

It will be appreciated that the particular shape and configuration of the abilities widget112discussed previously is but one example configuration for a swipe input mechanism consistent with this disclosure, and that different shapes, configurations, and selectable skills/abilities/functions are possible in other embodiments.FIG. 9shows one such alternative embodiment of a diamond-shaped swipe menu900.

The swipe menu900functions similarly to the example abilities widget112discussed previously, a main distinction being that the respective swipable abilities elements are provided by four squares902oriented at a45angle. The swipe menu900further has no selectable UI element like the central pad210at its center. Instead, the swipe menu900is in the dormant mode (FIG. 9A) fully contracted so that the four selectable squares are in abutment to form a compact unified square. Upon receiving touch input in the abilities control area508, as described before, the four selectable squares separate and move apart radially, being centered on the touch point. As before, firing lanes302capped off by chevrOn-shaped bumpers extend radially from each square that is available for selection.

It will be appreciated that many such variations are possible without diverging from the scope of the techniques described herein.

Modules, Components, and Logic

Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Examples of such components include the game management module602, the input interpreter604, and the abilities widget module606, described with reference toFIG. 6, it will be appreciated that different modules may in other embodiments be used to perform the techniques described with reference toFIG. 1throughFIG. 9. Such components or modules may constitute either software modules (e.g., code embodied on a machine-readable medium) or hardware modules/components. A “hardware module/component is a tangible unit capable of performing certain operations and may be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein. In such cases, the various described hardware modules of a system or machine to perform the disclosed operations may not at any time have all of the modules described as forming part of the system or machine. Instead, a reconfigurable computer processor (e.g., a CPU) may, at various times, be configured by execution of specific software to form different corresponding modules.

Machine and Software Architecture

The modules, methods, applications and so forth described in conjunction withFIG. 1throughFIG. 9are implemented in the context of a machine and an associated software architecture. The sections below describe representative software architecture(s) and machine (e.g., hardware) architecture that are suitable for use with the disclosed embodiments.

Software Architecture

FIG. 10is a block diagram1000illustrating a representative software architecture1002, which may be used in conjunction with various hardware architectures herein described.FIG. 10is merely a non-limiting example of a software architecture and it will be appreciated that many other architectures may be implemented to facilitate the functionality described herein. The software architecture1002may be executing on hardware such as machine1100ofFIG. 11that includes, among other things, processors1110, memory1130, and I/O components1150. A representative hardware layer1004is illustrated and can represent, for example, the machine1100ofFIG. 11. The representative hardware layer1004comprises one or more processing units1006having associated executable instructions1008. Executable instructions1008represent the executable instructions of the software architecture1002, including implementation of the methods, modules and so forth ofFIG. 1throughFIG. 9. Hardware layer1004also includes memory and/or storage modules1010, which also have executable instructions1008. Hardware layer1004may also comprise other hardware as indicated by1012which represents any other hardware of the hardware layer1004, such as the other hardware illustrated as part of machine1100.

In the example architecture ofFIG. 10, the software1002may be conceptualized as a stack of layers where each layer provides particular functionality. For example, the software1002may include layers such as an operating system1014, libraries1016, frameworks/middleware1018, applications1060and presentation layer1044. Operationally, the applications1060and/or other components within the layers may invoke application programming interface (API) calls1024through the software stack and receive a response, returned values, and so forth illustrated as messages1026in response to the API calls1024. The layers illustrated are representative in nature and not all software architectures have all layers. For example, some mobile or special purpose operating systems1014may not provide a frameworks/middleware layer1018, while others may provide such a layer. Other software architectures may include additional or different layers.

The operating system1014may manage hardware resources and provide common services. The operating system1014may include, for example, a kernel1028, services1030, and drivers1032. The kernel1028may act as an abstraction layer between the hardware and the other software layers. For example, the kernel1028may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, and so on. The services1030may provide other common services for the other software layers. The drivers1032may be responsible for controlling or interfacing with the underlying hardware. For instance, the drivers1032may include display drivers, camera drivers, Bluetooth® drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi® drivers, audio drivers, power management drivers, and so forth depending on the hardware configuration.

The libraries1016may provide a common infrastructure that may be utilized by the applications1060and/or other components and/or layers. The libraries1016typically provide functionality that allows other software modules to perform tasks in an easier fashion than to interface directly with the underlying operating system1014functionality (e.g., kernel1028, services1030and/or drivers1032). The libraries1016may include system1034libraries (e.g., C standard library) that may provide functions such as memory allocation functions, string manipulation functions, mathematic functions, and the like. In addition, the libraries1016may include API libraries1036such as media libraries (e.g., libraries to support presentation and manipulation of various media format such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG), graphics libraries (e.g., an OpenGL framework that may be used to render 2D and 3D in a graphic content on a display), database libraries (e.g., SQLite that may provide various relational database functions), web libraries (e.g., WebKit that may provide web browsing functionality), and the like. The libraries1016may also include a wide variety of other libraries1038to provide many other APIs to the applications1060and other software components/modules.

The frameworks1018(also sometimes referred to as middleware) may provide a higher-level common infrastructure that may be utilized by the applications1060and/or other software components/modules. For example, the frameworks1018may provide various graphic user interface (GUI) functions, high-level resource management, high-level location services, and so forth. The frameworks1018may provide a broad spectrum of other APIs that may be utilized by the applications1060and/or other software components/modules, some of which may be specific to a particular operating system1014or platform.

The applications1060includes built-in applications1040and/or third party applications1042. Examples of representative built-in applications1040may include, but are not limited to, a contacts application, a browser application, a book reader application, a location application, a media application, a messaging application, and/or a game application. Third party applications1042may include any of the built in applications1040as well as a broad assortment of other applications. In a specific example, the third party application1042(e.g., an application developed using the Android™ or iOS™ software development kit (SDK) by an entity other than the vendor of the particular platform) may be mobile software running on a mobile operating system such as iOS™, Android™, Windows® Phone, or other mobile operating systems1014. In this example, the third party application1042may invoke the API calls1024provided by the mobile operating system such as operating system1014to facilitate functionality described herein.

The applications1060may utilize built in operating system functions (e.g., kernel1028, services1030and/or drivers1032), libraries1016(e.g., system1034, APIs1036, and other libraries1038), and frameworks/middleware1018to create user interfaces to interact with users206of the system202. Alternatively, or additionally, in some systems, interactions with a user206may occur through a presentation layer, such as presentation layer1044. In these systems, the application/module “logic” can be separated from the aspects of the application/module that interact with a user.

Some software architectures utilize virtual machines. In the example ofFIG. 10, this is illustrated by virtual machine1048. A virtual machine1048creates a software environment where applications/modules can execute as if they were executing on a hardware machine (such as the machine1100ofFIG. 11, for example). A virtual machine1048is hosted by a host operating system (operating system1014inFIG. 10) and typically, although not always, has a virtual machine monitor1046, which manages the operation of the virtual machine1048as well as the interface with the host operating system (i.e., operating system1014). A software architecture executes within the virtual machine1048such as an operating system1050, libraries1052, frameworks/middleware1054, applications1056and/or presentation layer1058. These layers of software architecture executing within the virtual machine1048can be the same as corresponding layers previously described or may be different.

Example Machine Architecture and Machine-Readable Medium

FIG. 11is a block diagram illustrating components of a machine1100, according to some example embodiments, able to read instructions1008from a machine-readable medium (e.g., a machine-readable storage medium) and perform any one or more of the methodologies discussed herein. Specifically,FIG. 11shows a diagrammatic representation of the machine1100in the example form of a computer system, within which instructions1116(e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine1100to perform any one or more of the methodologies discussed herein may be executed. For example the instructions1116may cause the machine1100to execute the flow diagrams ofFIG. 7andFIG. 8, and may cause the machine1100to provide the abilities widget functionality ofFIG. 1throughFIG. 5, in which embodiment the machine1100is provided in the example form of the mobile phone102. Additionally, or alternatively, the instructions1116may implement the respective modules ofFIG. 6and so forth. The instructions1116transform the general, non-programmed machine1100into a particular machine1100programmed to carry out the described and illustrated functions in the manner described. In alternative embodiments, the machine1100operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine1100may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine1100may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions1116, sequentially or otherwise, that specify actions to be taken by machine1100. Further, while only a single machine1100is illustrated, the term “machine” shall also be taken to include a collection of machines1100that individually or jointly execute the instructions1116to perform any one or more of the methodologies discussed herein.

The machine1100may include processors1110, memory1130, and I/O components1150, which may be configured to communicate with each other such as via a bus1102. In an example embodiment, the processors1110(e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processor1112and processor1114that may execute instructions1116. The term “processor” is intended to include a multi-core processor1110that may comprise two or more independent processors1112,1114(sometimes referred to as “cores”) that may execute instructions1116contemporaneously. AlthoughFIG. 11shows multiple processors1112,1114, the machine1100may include a single processor1110with a single core, a single processor1110with multiple cores (e.g., a multi-core process), multiple processors1110with a single core, multiple processors1110with multiples cores, or any combination thereof.

The memory/storage1130may include a memory1132, such as amain memory, or other memory storage, and a storage unit1136, both accessible to the processors1110such as via the bus1102. The storage unit1136and memory1132store the instructions1116, embodying anyone or more of the methodologies or functions described herein. The instructions1116may also reside, completely or partially, within the memory1132, within the storage unit1136, within at least one of the processors1110(e.g., within the processor1110's cache memory), or any suitable combination thereof, during execution thereof by the machine1100. Accordingly, the memory1132, the storage unit1136, and the memory of processors1110are examples of machine-readable media.

Communication may be implemented using a wide variety of technologies. The I/O components1150may include communication components1164operable to couple the machine1100to a network1180or devices1170via coupling1182and coupling1172respectively. For example, the communication components1164may include a network interface component or other suitable device to interface with the network1180. In further examples, communication components1164may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices1170may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a Universal Serial Bus (USB)).

Transmission Medium

The instructions1116may be transmitted or received over the network1180using a transmission medium via a network interface device (e.g., a network interface component included in the communication components1164) and utilizing any one of a number of well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions1116may be transmitted or received using a transmission medium via the coupling1172(e.g., a peer-to-peer coupling) to devices1170. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying instructions1116for execution by the machine1100, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.

Language