Overloading app icon touchscreen interaction to provide action accessibility

A method of operating a user device includes, in response to a set of app icons being displayed, monitoring for user interaction with one of the app icons. In response to detecting user interaction with a first app icon corresponding to a first app, the method determines whether the detected user interaction satisfies first criteria or second criteria. In response to determining that the detected user interaction satisfies the first criteria, the method displays an action list corresponding to the first app. The action list includes a set of action items for user selection. A first item corresponds to a specific state of the first app. In response to the user selecting the first item, the method opens the first app to the specific state. In response to determining that the detected user interaction satisfies the second criteria, the method performs a default action for the first app.

FIELD

This disclosure relates to mobile device software and more particularly to user interface interaction with application icons on touchscreen-enabled mobile devices.

BACKGROUND

In recent years, use of computers, smartphones, and other Internet-connected devices has grown exponentially. Correspondingly, the number of available software applications for such devices has also grown. Today, many diverse native and web software applications can be accessed on any number of different devices, including, but not limited to, smartphones, personal computers, automobiles, and televisions. These diverse applications can range from business driven applications, games, educational applications, news applications, shopping applications, messaging applications, media streaming applications, social networking applications, and so much more. Users may install apps, use them occasionally, and eventually delete them, without being aware of useful features they make available. Further, users may rely on specific features of apps that are not quickly accessible within the apps themselves.

SUMMARY

A method includes receiving, at data processing hardware, a first indication of a first user selection of a first application. The method includes associating, by the data processing hardware, the first indication with a quick-access menu includes user selectable links, each user selectable link associated with an action access mechanism having a reference to the first application and indicating a performable operation for the first application. The method includes displaying, by the data processing hardware, on a screen in communication with the data processing hardware the quick-access menu. The method includes receiving, at the data processing hardware, a second indication of a second user selection of a user selectable link. The method includes executing, by the data processing hardware, the action access mechanism associated with the selected user selectable link.

In other features, the method includes displaying, by the data processing hardware, on the screen a graphical user interface includes one or more application access mechanisms. Each application action mechanism has a reference to a corresponding application executable on the data processing hardware and indicating a performable operation of the corresponding application. The method includes receiving, at the data processing hardware, a third indication of a third user selection of an application access mechanism having a reference to the first application. The third indication is different than the first indication. The method includes executing, at the data processing hardware, the selected application access mechanism having the reference to the first application.

In other features, the first indication is a long selection including a user interaction with the screen for a first period of time greater than or equal to a selection time or a short selection including a user interaction with the screen for a second period of time less than the selection time. The third indication is the other one of the long selection or the short selection. Displaying the quick-access menu includes displaying a first threshold number of user selectable links and displaying a second threshold number of user selectable links when receiving a scroll indication.

In other features, displaying the quick-access menu includes displaying the user selectable links as icons overlaid on the display. Receiving the second indication includes receiving a swipe indication indicating a user interaction over the displayed overlaid icons and receiving a stop indication indicating cessation of the user interaction over or on a selected overlaid icon associated with the selected user selectable link. In other features, the method includes displaying the quick-access menu on a bottom portion or a top portion of the screen. In other features, the quick-access menu includes a scroll bar configured to move at least one of the user-selectable links into and out of a viewing area of the screen. In other features, displaying the quick-access menu includes displaying a pop-up menu adjacent to the selected application and displaying the user selectable links inside the pop-up menu.

A system includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations including receiving a first indication of a first user selection of a first application. The operations include associating the first indication with a quick-access menu includes user selectable links. Each user selectable link is associated with an action access mechanism having a reference to the first application and indicating a performable operation for the first application. The operations include displaying on a screen in communication with the data processing hardware the quick-access menu. The operations include receiving a second indication of a second user selection of a user selectable link. The operations include executing the action access mechanism associated with the selected user selectable link.

In other features, the operations include displaying on the screen a graphical user interface includes one or more application access mechanisms. Each application action mechanism has a reference to a corresponding application executable on the data processing hardware and indicating a performable operation of the corresponding application. The operations include receiving a third indication of a third user selection of an application access mechanism having a reference to the first application. The third indication is different than the first indication. The operations include executing the selected application access mechanism having the reference to the first application.

In other features, the first indication is one of a long selection including a user interaction with the screen for a first period of time greater than or equal to a selection time or a short selection including a user interaction with the screen for a second period of time less than the selection time. The third indication is the other one of the long selection or the short selection. In other features, displaying the quick-access menu includes displaying a first threshold number of user selectable links or displaying a second threshold number of user selectable links when receiving a scroll indication.

In other features, displaying the quick-access menu includes displaying the user selectable links as icons overlaid on the display. Receiving the second indication includes receiving a swipe indication indicating a user interaction over the displayed overlaid icons and receiving a stop indication indicating cessation of the user interaction over or on a selected overlaid icon associated with the selected user selectable link. In other features, the operations include displaying the quick-access menu on a bottom portion or a top portion of the screen. In other features, the quick-access menu includes a scroll bar configured to move at least one of the user-selectable links into and out of a viewing area of the screen. In other features, displaying the quick-access menu includes displaying a pop-up menu adjacent to the selected application and displaying the user selectable links inside the pop-up menu.

A method of operating a user device includes, in response to a set of app icons being displayed, monitoring for user interaction with one of the app icons. Each of the set of app icons corresponds to a different app installed on the user device. The method includes, in response to detecting user interaction with a first app icon corresponding to a first app, determining whether the detected user interaction satisfies first criteria or second criteria. The method includes, in response to determining that the detected user interaction satisfies the first criteria, preparing and displaying an action list corresponding to the first app. The action list includes a set of action items for user selection. A first item of the set of action items corresponds to a specific state of the first app. The method includes monitoring for user interaction with the action list and, in response to determining user selection of the first item, opening the first app to the specific state. The method includes, in response to determining that the detected user interaction satisfies the second criteria, performing a default action for the first app.

In other features, performing the default action for the first app includes launching the first app. In other features, the first criteria includes a length of a press on a touchscreen of the user device at a location corresponding to the first app icon exceeding a predetermined threshold. In other features, the first criteria is a force of a press on a touchscreen of the user device at a location corresponding to the first app icon exceeding a predetermined threshold. In other features, the method includes, (i) in response to detecting user interaction with the first app icon, determining whether the detected user interaction satisfies third criteria and (ii) in response to determining that the detected user interaction satisfies the second criteria, performing a deletion process for the first app.

In other features, the method includes downloading a default set of action items for the first app from a remote overlay server via a network connection. In other features, the method includes downloading default sets of action items for a plurality of apps including the first app. In other features, the plurality of apps includes apps corresponding to each of the set of displayed app icons. In other features, the method includes downloading an access mechanism for the specific state of the first app from a remote overlay server via a network connection.

In other features, the method includes tracking usage of the first app and generating the set of action items based on most frequently used states of the first app. In other features, opening the first app to the specific state includes at least one of invoking a view controller corresponding to the specific state, providing a uniform resource identifier to the first app that indicates the specific state, and replaying user interface events corresponding to the specific state.

A user device includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations including those described for the above methods.

DETAILED DESCRIPTION

A system of the present disclosure implements an overlay system on a user device or remotely, allowing the user to access one or more application functionalities from a home screen that displays the application icons, without having to launching the application. In other words, the overlay system provides the user with a shortcut menu to application functionalities associated with the corresponding application, where each functionality is associated with a user-selectable link. For example, a user may select an application that includes navigation functionalities. In this case, the overlay system displays on a graphical user interface (GUI) of a display of the user device, a quick-access menu with at least one option allowing the user to access navigation functionalities without having actually launched the app. Some of the functionalities may include, but are not limited to, “Home” that provides a map and directions to the user's house without the user entering his home address, or “Work” that also provides a map and directions to the user's work, or “Estimated Time to Home” that provides an estimated time of arrival from the user's current location to his home. Other examples are possible as well.

Overview

Referring toFIGS. 1 and 2, in some implementations, an example system100includes a user device200associated with a user10in communication with a remote system110via a network120. The remote system110may be a distributed system (e.g., cloud environment) having scalable/elastic computing resources112and/or storage resources114. The user device200and/or the remote system110may implement an overlay system300that optionally receives data from one or more data sources130or one or more other user devices200. In some implementations, the overlay system300communicates with the one or more user devices200and the data sources(s)130via the network120. The network120may include various types of networks120, such as a local area network (LAN), wide area network (WAN), and/or the Internet. As shown and as will be discussed, the overlay system300is separate from the user device200. However, in some implementations, the overlay system300(e.g., an overlay data processing hardware of the overlay system300) executes on the user device200(e.g., a device data processing hardware of the user device200), as will be shown inFIG. 8.

In some implementations, when a user10wants to execute a function of an application210, the user10selects an icon220displayed on the display202of the user device200by way of the GUI204. Usually, the user10desires a specific function to be executed by the application210associated with the selected application icon220. For example, if the user10wants to navigate to a specific location, the user10selects an application icon220associated with a navigation application210. The navigation application210would then execute allowing the user to enter the address he/she wants to navigate to. It is desirable to implement a system100that saves the user10one or more steps before reaching the desired functionality (e.g., navigating to a specific destination).

The system100, more specifically, the overlay system300associates a quick-access menu330with one or more application icons220displayed on the display202of the user device200, by overlaying an interface over the displayed application icons220which allows the user10to access the quick-access menu330. As shown, a user10selects an application icon220(e.g., APP3) associated with an application210executable on the user device200. Upon selection, the GUI204displays the quick-access menu330including menu items332, where each menu item332is associated with a user-selectable link that executes a functionality or action of the selected application210(e.g., APP3).

Menu items may also contain dynamic information that can be retrieved by the quick-access menu. For example, if each menu item represents a different type of UBER ride sharing service (X, Black, XL, etc.), each menu item may display the dynamically-retrieved wait time for that type of vehicle to arrive at the user location. For example, one or more menu items may be a deep view card (DVC).

A DVC for an app state shows additional information beyond just the identification of the app or app state. For example, the information may include a title of the app state or a description of the app state, which may be a snippet of text from the app state. Other metadata may be provided from the app state, including images, location, number of reviews, average review, and status indicators. For example, a status indicator of “open now” or “closed” may be applied to a business depending on whether the current time is within the operating hours of the business.

Some DVCs may emphasize information that led to the DVC being selected as a search result. For example, text within the DVC that matches a user's query may be shown in bold or italics. The DVC may also incorporate elements that allow direct actions, such as the ability to immediately call an establishment or to transition directly to a mapping app to get navigation directions to the establishment.

Other interactions with the DVC (such as tapping or clicking any other area of the DVC) may take the user to the indicated state or app. As described in more detail below, this may be accomplished by opening the relevant app or, if the app is not installed, opening a website related to the desired app state. In other implementations, an app that is not installed may be downloaded, installed, and then executed in order to reach the desired app state.

In other words, a DVC includes identifying information for the app or state as well as additional content from the app or state itself. The additional content allows the user to make a more informed choice about which result to choose, and may even allow the user to directly perform an action without having to navigate to the app state. If the action the user wants to take is to obtain information, in some circumstances the DVC itself may provide the necessary information to accomplish such action.

As shown, the quick-access menu330includes three actions332a,332b,332c. The user device200may be any computing device capable of displaying on a display202one or more quick-access menus330where each quick-access menu330includes menu items332associated with a selected application icon220of an application210executable on the user device200. In addition, the user device200is capable of executing the one or more installed applications210. User devices200include, but are not limited to, mobile computing devices, such as laptops200a, tablets200b, smart phones200c, and wearable computing devices200d(e.g., headsets and/or watches). User devices200may also include other computing devices having other form factors, such as computing devices included in desktop computers200e, vehicles, gaming devices, televisions, or other appliances (e.g., networked home automation devices and home appliances).

The user devices200may use a variety of different operating systems216. In examples where a user device200is a mobile device, the user device200may run an operating system including, but not limited to, ANDROID® developed by Google Inc., IOS® developed by Apple Inc., or WINDOWS PHONE® developed by Microsoft Corporation. Accordingly, the operating system216running on the user device200may include, but is not limited to, one of ANDROID®, IOS®, or WINDOWS PHONE®. In an example where a user device200is a laptop or desktop computing device, the user device200may run an operating system including, but not limited to, MICROSOFT WINDOWS® by Microsoft Corporation, MAC OS® by Apple, Inc., or Linux. User devices200may also access the overlay system300while running operating systems216other than those operating systems216described above, whether presently available or developed in the future.

A software application210may refer to computer software that, when executed by a computing device, causes the computing device to perform a task. In some examples, a software application210may be referred to as an “application”, an “app”, or a “program”. Example software applications210include, but are not limited to, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and games.

Applications210may be executed on a variety of different user devices200. In some examples, a native application210amay be installed on a user device200prior to a user10purchasing the user device200. In other examples, the user10may download and install native applications210aon the user device200.

The functionality of an application210may be accessed on the computing device200on which the application210is installed. Additionally or alternatively, the functionality of an application210may be accessed via a remote computing device112. In some examples, all of an application's functionality is included on the computing device200on which the application210is installed. These applications210may function without communication with other computing devices200(e.g., via the Internet). In other examples, an application210installed on a computing device200may access information from other remote computing devices112during operation. For example, a weather application installed on a computing device200may access the latest weather information via the Internet and display the accessed weather information to the user10through the installed weather application. In still other examples, a web-based application210b(also referred to herein as a web application) may be partially executed by the user's computing device200and partially executed by a remote computing device112. For example, a web application210bmay be an application210that is executed, at least in part, by a web server and accessed by a web browser (e.g., a native application210a) of the user's computing device200. Example web applications210bmay include, but are not limited to, web-based email, online auctions, and online retail sites. In some examples, one or more applications210provides a quick-access menus330allowing the user10to access one or more quick-access menu items332, where each quick-access menu items332is associated with an action or functionality of the application210.

In general, the user device200may communicate with the overlay system300using any software application210, such as an overlay launcher214, which overlays an overlay over the displayed application icons220. The overlay (as shown inFIG. 3B) is configured to identify a user input, such as a selection of an application icon220associated with an application210and perform an action based on the received input. For example, a short selection of an application icon220may be defined as the user's interaction with the screen202for a period of time less than a selection time (e.g., 1200 milliseconds).

The short selection may trigger the execution of the application access mechanism associated with the application210of the selected icon220. However, a long selection defined as the user's interaction with the screen202for a period of time greater or equal than the selection time, results in the overlay system300displaying the quick access menu330.

In other implementations, a touchscreen may register the force of a user's touchscreen press. A more forceful press may result in the quick access menu330while a less forceful press may result in launching the app.

Therefore, the overlay system300uses the overlay (FIG. 3B) to determine an action, such as executing the selected application or displaying the quick-access menu330associated with the selected application210. In other words, the overlay, which is not visible to the user10, allows the display of the quick access menu330.

The overlay launcher214may be an application user interface program that is installed on the user device200that replaces a native home screen with a more customizable interface allowing the GUI204to display the quick-access menu330upon a user's selection of an application icon220. The overlay launcher214allows the user10or the user device200(e.g., the overlay system300), to personalize a home screen (or any other screen of the GUI204of the user device200. In some implementations, the overlay launcher214also manages a home screen, wallpaper(s), a launch bar, an application installation, shortcuts, and widgets.

In some examples, the user device200executes the overlay launcher214that is dedicated to interfacing with the overlay system300(executing on the user device or remotely). The user device200may communicate with the overlay system300using a more general application210, such as a web-browser application210baccessed using a web browser native application210a. Although the user device200may communicate with the overlay system300using the overlay launcher214and/or a web-browser application210b, the user device200may be described hereinafter as using the overlay launcher214to communicate with the overlay system300. In some implementations, the functionality attributed to the overlay launcher214is included as an overlay component of a larger application210that has additional functionality. For example, the functionality attributed to the overlay launcher214may be included as part of a native application210aor a web application210bas a feature that provides quick-access menus330associated with application icon220displayed on the display202by way of the GUI204.

In some implementations, as shown inFIG. 2, the overlay system300includes an overlay module310in communication with an overlay data store320. The overlay system300receives usage data associated with a user10. The overlay module310analyzes the received data and stores the analyzed data in the overlay data store320. The usage data includes data relating to the interactions of the user10with one or more applications210installed on the user device200. In some examples, the overlay system300receives usage data of multiple users10. Usage data may include, but is not limited to, a user's interaction with an application210, a time/date that a user10opens an application210, duration of using an application210, a time/date of closing the application210, frequency of using the application210, or any other relevant data indicative of the user's interaction with the application210via the GUI204. The overlay module310uses usage data associated with a user10to determine the quick-access menu items332associated with a selected application icon220of an application210. For example, the quick-access menu items332may include items being the most user-selectable links associated with the applications. For example, if a user10has a habit of using specific navigation/traffic application to go to his house, then the quick-access menu330includes a first quick-access menu item332that allows the user10to select the first quick-access menu item332associated with an access mechanism that executes the navigation/traffic application and provides the user10with directions and traffic information to his house. Therefore, the first quick-access menu item332allows the user10to by-pass selecting the application210then entering a destination; instead, the user10selects the first quick-access menu item332that directly executes the navigation/traffic application210. In some implementations, the user10may set the quick-access menu items332to application states associated with application access mechanisms of the applications. Therefore, the quick-access menu330may be customized by the user10to include application states most useful to the user10.

The data sources130may include a variety of different data providers. The data sources130may include data from application developers130a, such as application developers' websites and data feeds provided by developers. The data sources130may include operators of digital distribution platforms130bconfigured to distribute native applications210ato user devices200. Example digital distribution platforms130binclude, but are not limited to, the GOOGLE PLAY® digital distribution platform by Google, Inc., the APP STORE® digital distribution platform by Apple, Inc., and WINDOWS PHONE® Store developed by Microsoft Corporation.

The data sources130may also include other websites, such as websites that include web logs130c(i.e., blogs), application review websites130d, or other websites including data related to applications210. Additionally, the data sources130may include social networking sites130e, such as “FACEBOOK®” by Facebook, Inc. (e.g., Facebook posts) and “TWITTER®” by Twitter Inc. (e.g., text from tweets). Data sources130may also include online databases130fthat include, but are not limited to, data related to movies, television programs, music, and restaurants. Data sources130may also include additional types of data sources in addition to the data sources described above. Different data sources130may have their own content and update rate.

Interface Examples

FIG. 3Ashows a user device200having a display202displaying a GUI204that includes application icons220, where each application icon220is associated with an application210. The application icons220may have different size. For example, application icons220of APP1-APP12are smaller than application icon220for APP13. For one or more application icons220, the overlay system300displays an overlay222. For example, the first application icon220for APP1has a corresponding overlay222OVERLAY1, a second application icon220for APP2has a corresponding overlay222OVERLAY2, . . . . As previously discussed, the overlay222associated with each application icon220determines a selection method of the application icon220and determines if the selection method results in an execution of the associated application or the display of the quick-access menu330. When a user10views the display202displaying the GUI204, the user10only sees the application icons220as shown inFIG. 3A. The user10does not see the overlay222associated with the application icons220. However, in some examples, the GUI204may display an indication informing the user10that he/she has the option to view the quick-access menu330by performing a user selection/action.

Referring toFIGS. 4A and 4B, in some implementations, when a user10selects an application icon220, for example, APP3, the overlay module310displays by way of the GUI204, the quick-access menu330associated with the application210(e.g., APP3).FIG. 4Ashows the user10making a selection of the APP3application icon220, whileFIG. 4Bshows the overlay system300associating the user selection of the APP3application icon220with OVERLAY3overlay222that determines the type of selection of the user (e.g., long press or short press) and determines the action to take based on the user selection. As shown, the overlay system300identified the user selection as a selection triggering the display of the quick-access menu330. As shown, the quick-access menu330includes three quick-access menu items332a,332b,332c.

FIGS. 5A-5Dare similar toFIGS. 4A and 4B.FIGS. 5A and 5Cdisplay the GUI204as seen by the user10, whileFIGS. 5B and 5Dshow the overlay222associated with the icons220. The quick-access menu330includes an arrow340that allows the user to navigate between a first page of quick-access menu item332a,332b,332c(FIGS. 5A and 5B) and a second page of quick-access menu item332d,332e,332f(FIGS. 5C and 5D). In various implementations, avoiding multiple pages of quick-access items may be a priority. With too many options, the user may not perceive the quick-access menu as being any quicker or easier to use than the app itself.

In some implementations, the first page of quick-access menu330(FIGS. 5A and 5B) includes quick-access menu item332a,332b,332cassociated with a first function of the selected application210, while the second page of quick-access menu (FIGS. 5C and 5D) includes quick-access menu item332d,332e,332fassociated with a second function of the selected application210.

For example, an application “REVIEW/RESERVATION APP” provides restaurant reviews and allows a user to make reservation at restaurants. In this case, the first function of the application may be the reservation function where the quick-access menu items332include action application access mechanisms associated with making reservations, for example, each action associated with a specific restaurant. Moreover, the second function of the application may be the review function where the quick-access menu items332include action application access mechanisms associated reviewing a restaurant, for example, each action associated with a specific restaurant.

FIGS. 6A-6Dshow a quick-access menu330that includes the quick-access menu items332in a way that allows the user10to make a selection of one of the quick-access menu items332without removing his/her finger from the display202. In this case, the user10selects APP8application icon220. The overlay system300displays the quick-access menu330as a circular menu allowing the user10to make a circular motion as shown inFIGS. 6B and 6C. More specifically, the user10moves his/her finger from position P1on the selected application icon220(APP8) to P2, P3, P4and back to P1. To make a selection of any one of the quick-access menu items332, the user10stops at the position of the associated quick-access menu items332. To select action A, the user10stops at position P2.

FIGS. 7A and 7Bshow another method of displaying the quick-access menu330. As shown, upon a user selection of an application icon220that triggers the display of the quick-access menu330, the quick-access menu items332are shown at the bottom of the display202. In this case, the quick-access menu items332are shown as a list that the user10may scroll through using a scrolling arrow340. As shown, the quick-access menu330is positioned at the bottom of the display202, but the quick-access menu330may be positions in any other location configured to display the quick-access menu330as a list and allow the user10to scroll through the quick-access menu330.

Access mechanisms may include at least one of a native application access mechanism (hereinafter “application access mechanism”), a web access mechanism, and an application download mechanism. The user device200may use the access mechanisms to access functionality of applications210. For example, the user10may select an application icon220associated with an application access mechanism or an quick-access menu item332associated with an action access mechanism.

An application access mechanism may be a string that includes a reference to a native application and indicates one or more operations for the user device200to perform. If the user10selects a user selectable link332including an action access mechanism, the user device200may launch the native application210referenced in the application access mechanism and perform the one or more operations indicated in the application access mechanism. If the user10selects an application icon220including an application access mechanism, the user device200may launch the native application210referenced in the application access mechanism and perform the one or more operations indicated in the application access mechanism.

An application access mechanism includes data that the user device200can use to access functionality provided by a native application. For example, an application access mechanism can include data that causes the user device200to launch a native application and perform a function associated with the native application. Performance of a function according to the access mechanism may set the native application into a specified state. Accordingly, the process of launching a native application and performing a function according to an application access mechanism may be referred to herein as launching the native application and setting the native application into a state that is specified by the application access mechanism. In some examples, an application access mechanism for a restaurant reservation application can include data that causes the user device200to launch the restaurant reservation application and assist in making a reservation at a restaurant. In such examples, the restaurant reservation application may be set in a state that displays reservation information to the user10, such as a reservation time, a description of the restaurant, and user reviews. In additional examples, an application access mechanism for an internet media player application can include data that causes the user device200to launch the internet media player application and stream media from the Internet. In such examples, the internet media player application may be set in a state that displays information regarding the media (e.g., music) being streamed, such as a song name, an artist, or an album name.

Application access mechanisms may have various different formats and content. The format and content of an application access mechanism may depend on the native application210with which the application access mechanism is associated and the operations that are to be performed by the native application210in response to selection of the application access mechanism. For example, an application access mechanism for an internet music player application may differ from an application access mechanism for a shopping application. An application access mechanism for an internet music player application may include references to musical artists, songs, and albums, for example. The application access mechanism for an internet music player application may also reference operations, such as randomizing a list of songs and playing a song or album. An application access mechanism for a shopping application may include references to different products that are for sale. The application access mechanism for the shopping application may also include references to one or more operations, such as adding products to a shopping cart and proceeding to a checkout.

Block Diagram

FIG. 8illustrates example components of a user device200. In the illustrated example, the user device200includes a data processing hardware240, a storage device/hardware250, a network interface device252, and a user interface device254. The user device200may include other components not explicitly depicted.

The processing device240includes memory (e.g., RAM and/or ROM) that stores computer readable instructions and one or more processors that execute the computer readable instructions. In implementations where the processing device240includes two or more processors, the processors can execute in a distributed or individual manner. The processing device240may execute a native application210athat includes a native application module, any number of other native applications210a, a web browser210b, and an operating system216, all of which may be embodied as computer readable instructions. The operating system216acts as an interface between the processing device240and the applications210.

The processing device240executes the overlay launcher214configured with a launcher module217. The launcher module217is a set of computer readable instructions embedded (e.g., by the developer) in the native application210a, i.e., overlay launcher214. In some implementations, the developers use a software developer kit (SDK) provided by the provider of the overlay system300to implement the launcher module217. The developers may also use third party SDKs to implement part or all of the launcher module217.

In some examples, the processing device240includes the overlay module310and the storage device250includes the overlay data store320. In other examples, the processing device240includes the overlay module310, which is in communication with the overlay data store320that is located remotely from the user device200.

The storage device250includes one or more computer readable mediums (e.g., hard disk drive and/or flash memory). The storage device250can store the computer readable instructions that make up the native applications210a, the web browser210b, and the operating system216. The network interface252includes one or more devices that are configured to communicate with the network. The network interface252can include one or more transceivers for performing wired or wireless communication. Examples of the network interface252may include, but are not limited to, a transceiver configured to perform communications using the IEEE 802.11 wireless standard, an Ethernet port, a wireless transmitter, and a universal serial bus (USB) port. The user interface254includes one or more devices that receive input from and/or provide output to a user10. The user interface254can include, but is not limited to, a touchscreen, a display, a QWERTY keyboard, a numeric keypad, a touchpad, a microphone, and/or speakers.

FIG. 9provides an example arrangement of operations for a method of displaying a quick-access menu330in response to receiving an indication of a user selection of an application icon220displayed on a display202of a user device200. The method is described with respect to the user device200and the overlay system300as illustrated inFIGS. 1-8.

At block402, the method includes receiving, at data processing hardware (of a user device240or at the overlay module310), a first indication of a first user selection of a first application210. At block404, the method includes associating, the first indication with a quick-access menu330. The quick-access menu330includes user selectable links332(e.g., quick-access menu items332). Each user selectable link332is associated with an action access mechanism having a reference to the first application and indicating a performable operation for the first application. A block406, the method includes displaying on a screen202in communication with the data processing hardware the quick-access menu330. At block408, the method includes receiving a second indication of a second user selection of a user selectable link332. At block410, the method includes executing the action access mechanism associated with the selected user selectable link332.

In some examples, the method also includes displaying on the screen202a GUI204comprising one or more application access mechanisms. Each application action mechanism having a reference to a corresponding application executable on the data processing hardware and indicating a performable operation of the corresponding application. The method may also include receiving a third indication of a third user selection of an application access mechanism having a reference to the first application, the third indication being different than the first indication and executing the selected application access mechanism having the reference to the first application. Additionally, the first indication may include one of a long selection or a short selection. The long selection includes a user interaction with the screen for a first period of time greater than or equal to a selection time, while the short selection includes a user interaction with the screen for a second period of time less than the selection time. In this case, the third indication includes the other one of the long selection or the short selection.

In some examples, displaying the quick-access menu further includes displaying a first threshold number of user selectable links, or displaying a second threshold number of user selectable links when receiving a scroll indication. In some implementations, displaying the quick-access menu further includes displaying the user selectable links as icons overlaid on the display, and receiving the second indication includes receiving a swipe indication indicating a user interaction over the displayed overlaid icons, and receiving a stop indication indicating cessation of the user interaction over or on a selected overlaid icon associated with the selected user selectable link.

In some examples, the method may further include displaying the quick-access menu on a bottom portion or a top portion of the screen202. In some implementations, displaying the quick-access menu further includes displaying a pop-up menu adjacent to the selected application, and displaying the user selectable links inside the pop-up menu. In other implementations, the quick-access menu includes a scroll bar configured to move at least one of the user-selectable links into and out of a viewing area of the screen. However, eliminating the need for the scroll bar may be a design priority so that selection of a relevant quick action is not made slower or less convenient.

InFIG. 10, control begins at504upon installation of an overlay app. Element504is shown with a dashed line to indicate that the overlay app may already have been installed on the user device by a manufacturer of the phone or a wireless operator. Further, the overlay app may actually be part of the operating system itself. The overlay app may be a standard app downloadable from the digital distribution platform, but may have special privileges, such as the ability to draw over other apps (specifically, drawing over a home screen or launcher app). In other implementations, the overlay app may be part of the home screen or launcher app. For example, the overlay feature may be one of the features available to users of an alternative launcher app.

At508, the overlay feature is activated. This activation may be via a setting in the overlay app or in the operating system. In other implementations, the overlay feature may be activated upon installation or first execution of the overlay app. At512, control implemented by the overlay feature waits until an app screen is displayed. For example, the app screen may be a home screen in which icons corresponding to installed or available applications are displayed to the user. Commonly, these app icons are displayed in a grid format and may be grouped into folders that occupy some positions in the grid.

Control remains at512until the app screen is displayed, at which point control transfers to516. At516, control of the overlay feature waits until user interaction with one of the app icons is detected. Control then transfers to520. At520, control determines whether the user interaction meets certain criteria. If so, control transfers to524; otherwise, control transfers to528.

The criteria may include a length of time of a user's contact with a touchscreen or the force with which the user contacted the touchscreen. Other criteria may be the number of fingers (or other contact points) applied by the user. For example only, the criteria may indicate that either or both of a long press (a tap that lasts more than a predetermined period such as 500 ms) or a high force (a detected force greater than a predetermined threshold) will cause a transition to524. At528, a standard app icon operation is performed. Often, the standard response to interacting (such as tapping) an app icon is to open the app, which may lead to a home state of the app (default state) or a last-used state of the app. Control then returns to wait at512.

At524, control creates an action list corresponding to the app icon selected by the user. For example, the action was may be created as shown inFIG. 11. At532, control displays the created action list. At536, control determines whether the user selects an action corresponding to a state of the app. If so, control transfers to540; otherwise, control transfers to544. At540, control opens the app to the selected state indicated by the selected entry in the action list. Control then returns to512.

Opening the app to the selected state may be performed using an access mechanism, such as the view controller actuation or user interface replay. For more information on view controller actuation, see U.S. patent application Ser. No. 15/235,650, filed Aug. 12, 2016, titled “Monitoring And Actuation Of View Controller Parameters To Reach Deep States Without Manual Developer Intervention,” with first named inventor Kalyan Desineni. For more information on user interface replay, see U.S. patent application Ser. No. 15/235,859, filed Aug. 12, 2016, titled “Deep Linking To Mobile Application States Through Programmatic Replay Of User Interface Events,” with first named inventor Kalyan Desineni. The entire disclosures of the applications referenced above are incorporated by reference.

At544, control determines whether the user selects an alternative action from the action list. If so, control transfers to548; otherwise, such as if the user closes the action list (by tapping outside of the action list or selecting an icon, such as an “x”, to close the list), control returns to wait at512. At548, control performs an alternative action, such as deletion. Control then returns to512. In various implementations, a long press followed by a release may meet the criteria of520. Meanwhile, a long press followed by a dragging motion toward the border of the screen may result in deletion. The deletion process may include at least one prompt to reduce the likelihood of unintended deletions.

InFIG. 11, control begins at604, such as when the overlay app is installed. Control sends a request for action list items to a remote overlay system. The request includes a list of app, such as all apps presently visible on the current page of the home screen. Because a user device may have more installed apps than can fit on the screen at one time, the home screen (or launcher screen) may have multiple individual pages of app icons.

All potentially visible apps may be sent in the first request so that, when the user switches to another page of the home screen, network latency and network availability will not prevent action list items from being obtained for the newly displayed apps. In other implementations of604, the request includes a list of all installed apps. In yet other implementations, the request will include apps visible on any of the pages of the home screen. In other words, apps accessible from a list of all apps (such as an app tray) but not placed on the home screen will not be sent.

At608, control receives a set of default actions corresponding to each of the apps from the remote overlay system. The remote overlay system may provide the set of default actions based on aggregated and anonym ized data from other use from other users. For example, the default actions may correspond to the most popular actions performed using the app. In some implementations, app developers may specify the default actions to the remote overlay system. Once the remote overlay system has acquired enough actual user data, the remote overlay system may override the default actions specified by the developer in favor of actions determined to be popular from empirical data.

At612, control monitors usage of apps installed on the device, including noting and recording frequently visited states. At616, control determines whether an action list is needed for a first app. If so, control transfers to620; otherwise, control transfers to624. The action list may be needed for the first app when a user interaction meeting the criteria of520ofFIG. 10is detected in relation to an icon for the first app.

At624, control determines whether the set of default actions received from the remote overlay system is stale. If so, control returns to604; otherwise, control continues at612. The set of default actions may be considered stale if the amount of time since the list had been sent is greater than some predetermined threshold, such as 24 hours. The set of default actions may also be determined to be stale in response to installation of a new app since the last request for default actions was sent. Elements604,608, and624are shown with dashed lines to indicate that a remote overlay system may not be necessary. Instead, the overlay feature may simply base the action list on actual usage of apps on the device. This may result in few or no actions for the action list of apps that have been used infrequently.

At620, control determines whether there is sufficient usage data for the first app. If so, control transfers to628; otherwise, control transfers to632. At628, control chooses states of the first app with the highest recorded usage for placement on the action list. Control then continues at636. At632, control chooses default actions for the action list according to an identifier of the first app. These default actions may be those received from the remote overlay system at608. In other implementations, default actions may be chosen by the overlay system even in the absence of user data by selecting states accessible from the home (or default) state of the first app. For example, if a restaurant app has prominently displayed links in the default state for “Restaurants near me” and for “Highest rated restaurants,” the states corresponding to those links or user interface elements may be chosen as the default actions. Control then continues at636.

At636, control ranks the action list based on a history of user selection on the present device. In other words, if a user has previously selected a certain action, that action may be ranked first and displayed more prominently such as by being first in the action list. In other implementations, the ranking may be partially or entirely based on selection data aggregated by the remote overlay system. At640, control adds an action of deletion to the action list. Element640may be omitted in implementations where, such as described above, deletion can be accomplished using a gesture that does not bring up the action list. At644, control records which item from the action list is selected by the user. This data is stored and can be used in the future for ranking at636. Control then returns to612.

Further Details

The computing device700includes a processor112,250,710, memory720, a storage device114,250,320,730, a high-speed interface/controller740connecting to the memory720and high-speed expansion ports750, and a low speed interface/controller760connecting to low speed bus770and storage device730. Each of the components710,720,730,740,750, and760, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor710can process instructions for execution within the computing device700, including instructions stored in the memory720or on the storage device730to display graphical information for a graphical user interface (GUI) on an external input/output device, such as display780coupled to high speed interface740. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices700may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).

The storage device730is capable of providing mass storage for the computing device700. In some implementations, the storage device730is a computer-readable medium. In various different implementations, the storage device730may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. In additional implementations, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory720, the storage device730, or memory on processor710.

The high speed controller740manages bandwidth-intensive operations for the computing device700, while the low speed controller760manages lower bandwidth-intensive operations. Such allocation of duties is exemplary only. In some implementations, the high-speed controller740is coupled to the memory720, the display780(e.g., through a graphics processor or accelerator), and to the high-speed expansion ports750, which may accept various expansion cards (not shown). In some implementations, the low-speed controller760is coupled to the storage device730and low-speed expansion port770. The low-speed expansion port770, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device, such as a switch or router, e.g., through a network adapter.

The computing device700may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server700aor multiple times in a group of such servers700a, as a laptop computer700b, or as part of a rack server system700c.