Patent Description:
In such computing systems, users can perform a wide variety of different types of commands on content that are created with these types of applications. For instance, when a user highlights a word in a word processing document, the user can perform a variety of different types of direct commands on that highlighted text, such as to bold the text, underline the text, italicize the text, copy the text, etc. Users sometimes perform these commands by actuating user actuated elements, corresponding to the commands, that are displayed on a commanding surface. In one example, a commanding surface is also referred to as a commanding palette. The commanding palette is a display surface (or user interface display) that has a plurality of different tabs. Each tab, when actuated, displays a corresponding set of command actuators. When the user actuates one of the command actuators, the application performs a corresponding command on a selected item, such as on highlighted text, etc..

Some computing systems are deployed on mobile devices, such as on smart phones, tablet computers, etc. Such mobile devices often have limited display real estate so the commanding palette is often collapsed, by default, in order to reserve room in the display real estate to show the user's content. Also, in such applications, the commanding palette is not displayed along with the keyboard. Therefore, when a user wishes to perform any of a wide variety of different commands, the user may normally need to first provide an input to dismiss the keyboard, and then provide another input to invoke the commanding palette. The user must then perform the different actions needed to actuate an actuator on the commanding palette, in order to perform an operation on a selected item of content. This can make even simple tasks, such as highlighting text, cropping a picture, or adding a column to a table, slow and cumbersome. This also makes tasks that interleave typing and formatting particularly inefficient.

Even after the user has provided the inputs so that the commanding palette is displayed, the user interaction is still cumbersome. The user still needs to select a desired tab and then scroll through the palette (which is often arranged vertically) in order to identify and select a command that the user wishes to perform.

<NPL>, discloses a video tutorial related to a version of the software application Pages for iOS <NUM>.

<NPL>, discloses details about the software application Pages for the iPad, including information related to how to create reports, projects, assignments, letters, flyers and posters.

<NPL>, discloses information about Office for iPhone applications.

<CIT> discloses a predictive contextual toolbar that provides an identifiable region on which predicted commands can be surfaced. This user interface can be presented in cases where an indication of a request for a command (such as a formatting tool) is received, for example, while a user is interacting with a canvas of a productivity application.

It is the object of the present invention to improve the efficiency of contextual command bars.

A user input is detected that triggers a contextual command bar to be surfaced. A commanding context, in an application that the user has open, is identified and a set of commands to be surfaced in the contextual command bar is identified, based upon the context. The identified set of commands is surfaced on the contextual command bar for user interaction.

The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the background.

<FIG> is a block diagram of one example of a mobile computing device <NUM>. Device <NUM> is shown, in the example in <FIG>, generating user interfaces <NUM> with user input mechanisms <NUM> for interaction by user <NUM>. User <NUM> illustratively interacts with user input mechanisms <NUM> in order to control and manipulate mobile computing device <NUM>.

Mobile computing device <NUM> illustratively includes one or more processors <NUM>, user interface logic <NUM>, user interface mechanisms <NUM>, data store <NUM> (which, itself, can include applications <NUM>, context-to-command mappings <NUM>, and a wide variety of other items <NUM>), application component <NUM>, one or more sensors <NUM> and one or more communication systems <NUM>. Mobile computing device <NUM> can also illustratively include contextual command bar generation system <NUM> and a wide variety of other items <NUM>. Contextual command bar generation system <NUM>, itself, can illustratively include mode identifier logic <NUM>, context identifier logic <NUM>, command set identifier logic <NUM>, interaction processing logic <NUM>, command bar generator logic <NUM>, and it can include other items <NUM>. Before describing the overall operation of mobile computing device <NUM>, a brief overview of some of the items in mobile computing device <NUM>, and their operation, will first be provided.

User interface mechanisms <NUM> can be a wide variety of different mechanisms. For instance, they can be buttons, switches, keypads, thumb pads, a touch sensitive display screen, speech recognition components, or a wide variety of other mechanisms. In one example, user interface logic generates user interfaces <NUM> with user input mechanisms <NUM> that include user actuatable display elements, such as icons, links, touch sensitive buttons, etc. Logic <NUM> illustratively detects user inputs through mechanisms <NUM> and <NUM> and performs various actions or generates other user interfaces, based upon those interactions.

Application component <NUM> illustratively runs applications <NUM>. Applications <NUM> can include a wide variety of applications, such as word processing applications, slide presentation applications, spreadsheet applications, or other applications in which user <NUM> can generate or modify content.

Sensors <NUM> can include a wide variety of different sensors. For instance, sensors <NUM> can include one or more accelerometers, geographic location sensors (such as a global positioning system receiver), proximity sensors, or a wide variety of other sensors. The sensors illustratively generate sensor signals indicative of sensed variables and provide those signals to various other items, logic or components in computing device <NUM> or elsewhere.

Communication systems <NUM> can include any of a wide variety of different communication systems. For instance, they can include cellular communication systems, near field communication systems, Internet communication systems, or a variety of other wired or wireless communication systems.

Contextual command bar generation system <NUM> illustratively detects a context that user <NUM> is operating in, in one or more of the applications <NUM>, and determines whether to surface a contextual command bar. The contextual command bar illustratively has actuators that can be actuated by user <NUM> in order to perform commands that are relevant to the identified context. In one example, system <NUM> only surfaces the contextual command bar when the user is operating an application in one or more of a variety of different operational modes. For instance, when the user <NUM> is in a mode where the user is primarily consuming information (such as reading a document, presenting a slide presentation, etc.), then it may be that system <NUM> does not surface the contextual command bar so that the display screen on mobile device <NUM> has more display real estate to show the content being consumed. However, when the user is in, for instance, an editing mode, then system <NUM> may surface the contextual command bar. Thus, mode identifier logic <NUM> identifies the mode in which the user is using the application, and context identifier logic <NUM> identifies the context of the application. Command set identifier logic <NUM> identifies a set of commands that are to be displayed on the contextual command bar, given the identified context. Command bar generator logic <NUM> generates a representation of the command bar and provides it to user interface logic <NUM> for surfacing, for user interaction. Interaction processing logic <NUM> detects user interactions with the contextual command bar (which may be directly detected, or indicated by logic <NUM>, or otherwise) and performs various actions based on those interactions.

<FIG> and <FIG> (collectively referred to herein as <FIG>) show a flow diagram illustrating one example of the operation of mobile computing device <NUM> in generating a contextual command bar, and processing user interactions with the contextual command bar. It is first assumed that user <NUM> provides inputs to mobile computing device <NUM> so that application <NUM> loads and begins running an application <NUM>. In one example, the application is one which displays content that user <NUM> can interact with, consume, create, etc. Launching and running the application is indicated by block <NUM> in <FIG>. The application can be, for instance, a word processing application <NUM>, a spreadsheet application <NUM>, a slide presentation application <NUM>, or any of a wide variety of other applications <NUM>.

At some point, mode identifier logic <NUM> identifies the particular mode in which user <NUM> is using the application. This is indicated by block <NUM>. The mode may be, for instance, a mode in which user <NUM> is primarily consuming content, such as reading a document, presenting a slide presentation, etc. In another instance, the mode may be one in which user <NUM> is authoring or otherwise modifying content, such as authoring or revising a word processing document, generating a slide presentation, etc..

Mode identifier logic <NUM> can identify the mode in a wide variety of different ways. For instance, it can monitor the user inputs detected by user interface logic <NUM> to determine the mode that user <NUM> is in. If the user, for instance, is providing a high level of authoring inputs on a word processing document, the mode identifier logic <NUM> may identify the mode as an authoring mode, as opposed to a consumption mode. Identifying the mode based on user inputs is indicated by block <NUM>.

The mode can be based on detected application settings as well. For instance, if a slide presentation application is set to presentation mode, this may be detected by mode identifier <NUM> as a consumption mode.

Mode identifier logic <NUM> can also identify the mode based on other inputs, such as inputs from sensors <NUM>. For instance, where sensor <NUM> is an accelerometer that provides a sensor signal indicating that user <NUM> is walking, mode identifier logic <NUM> may interpret this as indicating that it is very unlikely that user <NUM> is in an authoring mode, but is instead in a consumption mode. This is only one example and a wide variety of other sensor inputs from sensors <NUM> can be used to identify the mode as well. Identifying the mode based on other sensor inputs is indicated by block <NUM> in the flow diagram of <FIG>.

If logic <NUM> determines that user <NUM> is in a consumption mode then the contextual command bar is not displayed, in order to maximize display real estate for the content being consumed. However, if user <NUM> is in an authoring mode, then context identifier logic <NUM> detects user inputs that would trigger the contextual command bar to be surfaced. Determining whether the mode is a consumption mode or an authoring mode is indicated by block <NUM> in <FIG> and detecting user inputs indicating that the command bar is to be surfaced is indicated by block <NUM>.

The user inputs that trigger surfacing of the contextual command bar can be a wide variety of different user inputs. For instance, in one example, if the user selects a content item on the user interface display, this may trigger surfacing of the contextual command bar. Detecting selection of a content item is indicated by block <NUM>. However, a wide variety of other user inputs may be detected and interpreted as an indication that the contextual command bar is to be surfaced. This is indicated by block <NUM>.

Context identifier logic <NUM> then identifies an application commanding context. This is indicated by block <NUM> in <FIG>. The commanding context is a context in which the application is currently running, and it is indicative of which commands a user is most likely to invoke. For instance, if the user is in an authoring mode, with a word processing document, and the user has selected a word in the document, it may be that the user is most likely to invoke one of a first set of commands. However, if the user is in a word processing document and has selected a table, it may be that the user is most likely to invoke one of a second set of commands.

In order to identify the application commanding context, context identifier logic <NUM> may first identify the particular application, or the type of application that the user is running. This is indicated by block <NUM>. By way of example, it may identify the application as a word processing application, a spreadsheet application, a slide presentation application, etc. Context identifier logic <NUM> may then identify the type of the selected content item. This is indicated by block <NUM>. For instance, it may identify the content item as a word, a paragraph, a subtitle, a table, an image, a column in a spreadsheet application, and object on a slide, etc..

Context identifier logic <NUM> may also identify the context based on the user's current activity, or history of activity. This is indicated by block <NUM>. For instance, if the user has historically performed a formatting command after the user selects a paragraph in a word processing document, then context identifier logic <NUM> may identify the context based on that history. In addition, if the user is currently authoring a document, and has performed editing inputs, such as selecting and moving text within the document, then context identifier logic <NUM> may identify the context based on that recent activity. These are examples only and context identifier logic <NUM> can identify the context that the application is in based on a wide variety of other items as well. This is indicated by block <NUM>.

Once context identifier logic <NUM> has identified the particular context of the application, then command set identifier logic <NUM> identifies a set of commands that user <NUM> is most likely to invoke, given that context. This is the set of commands that will be surfaced on the contextual command bar. Identifying the set of commands to be surfaced is indicated by block <NUM> in <FIG>. Command identifier logic <NUM> can access pre-computed context-to-command mappings <NUM>, based on the identified context. Mappings <NUM> illustratively map from an identified context to a set of commands that are to be displayed on the contextual command bar. Identifying the set of commands to be surfaced based on the context-to-command mappings <NUM> is indicated by block <NUM> in <FIG>.

<FIG> show one example of context-to-command mappings <NUM>. <FIG> shows mappings <NUM> for a word processing application. The left most column in <FIG> indicates a particular display item that is selected in a word processing document, and indicates the context of the word processing document. Following that, to the right of each identified context, is a set of display elements (or visual command identifiers) that correspond to the identified set of commands to be surfaced on the corresponding contextual command bar. By way of example, in <FIG>, if an item of text is selected, this corresponds to a "text" context. Therefore, the set of commands that will be identified on the contextual command bar include the "bold" command, the "italics" command, the "underline" command, the "highlight" command, the "font color" command, the "bullets" command, the "numbering" command, the "indent" command, etc. The final command is illustratively the "undo" command.

If a table is selected in the word processing document, then this corresponds to the "table" context. The set of commands that will be surfaced on the contextual command bar, when in the "table" context, include the "insert" commands, the "delete" command, the "clear" command, the "styles" command, and the "undo" command.

The Example in <FIG> also shows the set of commands that will be surfaced on the contextual command bar when a text object is selected, when a shape is selected, and when a picture is selected. It will be noted that these are example contexts only, and the corresponding set of commands surfaced for those contexts are example sets as well. Other contexts can be identified in a word processing application, and the contextual commands associated with each context can vary as well. Those illustrated in <FIG> are illustrated for the purpose of example only.

<FIG> shows the contexts and corresponding sets of contextual commands that are surfaced on the contextual command bar, when in a slide presentation application. Thus, the contexts correspond to user selection of a "slide", "notes", a "content placeholder", a "specific placeholder", a "table", a "text object", a "shape", a "picture", a "chart", or "smart art".

<FIG> shows an example set of contexts and corresponding contextual commands that will be surfaced on the contextual command bar, when in a spreadsheet application. The contexts identify a particular item that has been selected by the user, and the commands to the right of each context are the display elements (or visual command identifiers) and corresponding commands that will be displayed when on the contextual command bar, for each context. Thus, the contexts correspond to user selection of a "text object", a "shape", a "picture" and a "chart". Again, these are only examples of different contexts that can be identified in a spreadsheet application, and they are also only examples of the contextual commands that will be surfaced, when in each of those contexts.

In another example, command set identifier logic <NUM> can compute the set commands to be surfaced dynamically instead of, or in addition to, using mappings <NUM>. This may be based on the user's activity, or other current or changing information that tends to identify the likely commands that are to be invoked by the user. Computing the commands dynamically is indicated by block <NUM>.

Logic <NUM> can identify the set of commands, based upon the identified context, in other ways as well. This is indicated by block <NUM>.

Command bar generator logic <NUM> then generates a representation of the contextual command bar and provides it to user interface logic <NUM>, which surfaces visual command identifiers for the identified set of commands on the contextual command bar, for user interaction. This is indicated by block <NUM> in <FIG>. In one example, the visual command identifiers are arranged in order of their relevance. The relevance may be determined, for instance, based upon the likelihood that a given command is to be invoked by user <NUM> in the identified context. Thus, those that are most likely to be invoked are considered to be most relevant to the context, and the command identifiers for the most relevant commands are displayed first on the contextual command bar. Arranging the visual command identifiers on the contextual command bar, in order of relevance, is indicated by block <NUM>.

The contextual command bar can take a wide variety of different forms. In one example, it is a horizontally scrollable display that includes a visual command identifier corresponding to each of the commands in the identified set of commands. Displaying the command bar as a horizontally scrollable display is indicated by block <NUM>.

Also, in one example, the contextual command bar has a set of visual command identifiers that correspond to fixed commands. By fixed, it is meant that the visual command identifiers for those commands are displayed, regardless of the particular context identified. In one example, the set of fixed commands includes a link to the entire commanding palette, and it can include other fixed commands as well. Displaying the contextual command bar with fixed commands is indicated by block <NUM> in <FIG>. The identified set of commands can be surfaced on the contextual command bar in other ways as well, and this is indicated by block <NUM>.

In one example, the set of commands to be surfaced are drawn from the commanding palette, for the running application. <FIG> is a block diagram that schematically illustrates one example of a commanding palette <NUM>. In the example shown in <FIG>, commanding palette <NUM> is hierarchically arranged with a set of tabs <NUM>-<NUM>. Each tab has a corresponding set of commands <NUM>-<NUM>, and <NUM>-<NUM>. It can also include other items <NUM>. Therefore, assume that the user actuates a tab (such as tab <NUM>) on the commanding palette <NUM>. In that case, display elements (or visual command identifiers) corresponding to each of the commands <NUM>-<NUM> hierarchically arranged under the actuated tab <NUM> are displayed on the commanding palette display.

The identified set of commands to be surfaced on the contextual command bar can, in one example, be the most likely commands from the commanding palette <NUM>, but they need not be confined to commands under a single tab. For instance, if one tab <NUM> corresponds to formatting commands, and another tab <NUM> corresponds to inserting comments or links in the document, the identified set of commands to be surfaced on the contextual command bar, in a given context, may be commands that would be arranged under two or more different tabs <NUM>-<NUM> in commanding palette <NUM>. They need not all come from the set of commands under a single tab. Instead, they may simply be the most relevant commands taken from the entire commanding palette.

<FIG> is a block diagram that schematically illustrates one example of a contextual command bar <NUM>. It can be seen that command bar <NUM> includes a horizontally scrollable command portion <NUM> and a fixed command portion <NUM>. It can of course include other items <NUM> as well. The horizontally scrollable command portion <NUM> includes a plurality of command display elements (or visual command identifiers) <NUM>-<NUM> and a visual peek display element <NUM>. Each of the command display elements <NUM>-<NUM> is illustratively a user actuatable display element that, when actuated by user <NUM>, invokes a corresponding command represented by the actuated command display element.

For example, and referring also to <FIG>, assume that the user is in a word processing document and has selected an item of text in that document. Then, the context for contextual command bar <NUM> will be the "text" context shown in <FIG>. Each of the command display elements <NUM>-<NUM> will correspond to one of the visual command identifiers associated with the "text" context in <FIG>. For instance, command display element <NUM> may correspond to the "bold" command. Therefore, when the user actuates command display element <NUM>, the "bold" command will be invoked so that the selected text is bolded. This is just one example.

The visual peek element <NUM> is illustratively a visual display element that gives a visual cue that there are more contextual commands to the right of display element <NUM>, so that when the user scrolls the horizontally scrollable command portion <NUM> to the left, the command display elements corresponding to those commands will be visually displayed in the horizontally scrollable command portion <NUM>.

<FIG> also shows that fixed command portion <NUM> illustratively includes one or more command display elements (or visual command identifiers) <NUM>-<NUM>. Each command display element <NUM>-<NUM> corresponds to a command that can be invoked by the user, when the user actuates the command display element <NUM>-<NUM>. The fixed command portion <NUM> illustratively displays command display elements <NUM>-<NUM> for a fixed set of commands, that do not vary with command display elements displayed in the horizontally scrollable command portion <NUM>. In one example, as discussed above with respect to <FIG>, the command display elements <NUM>-<NUM> in the fixed command portion always correspond to a link to the entire commanding palette from which the contextual commands are obtained, and they can include an "undo" command as well. These are only examples of fixed command display elements displayed in the fixed command portion <NUM>.

<FIG> show various examples of contextual command bars, in different scenarios. <FIG> shows one example in which user <NUM> is authoring a text document displayed in display portion <NUM> of a mobile device <NUM>. It can be seen that the user has selected a textual word in display portion <NUM>. This indicates to context identifier logic <NUM> that the contextual command bar should be displayed. Therefore, logic <NUM> identifies the set of contextual commands to be displayed on the contextual command bar and displays contextual command bar <NUM>. Command bar <NUM> is shown with the horizontally scrollable command portion <NUM> and the fixed command portion <NUM>. It can be seen that each visual command identifier on contextual command bar <NUM> corresponds to a command that can be invoked by user <NUM> when user <NUM> actuates the corresponding visual command identifier. Contextual command bar <NUM> also includes the peek display element <NUM>. In the example shown in <FIG>, the peek element shows part of a visual command identifier corresponding to a command. Peek display element <NUM> provides a visual cue to the user that the horizontally scrollable display portion <NUM> can be scrolled to the left, so that more visual command identifiers, corresponding to more contextual commands, will scroll out from underneath the fixed command portion <NUM>, which does not move when portion <NUM> is scrolled.

In the example shown in <FIG>, the fixed command portion <NUM> has one visual command identifier. The single visual command identifier, when actuated by the user, serves as an entry point to the overall commanding palette <NUM>. Therefore, when it is actuated, commanding palette <NUM> for the running application is displayed with the most relevant tab selected.

<FIG> is similar to <FIG> and similar items are similarly numbered. However, in the example shown in <FIG>, the "cut, copy, paste" commands are displayed in a separate display section <NUM>, adjacent the highlighted text. Also, the fixed command portion <NUM> has two command display elements. One is similar to that shown in <FIG> and serves as an entry point to the overall commanding palette <NUM>. The other is a display element that, when actuated by the user, dismisses the keyboard so that the keyboard is no longer displayed on the user interface display.

<FIG> is similar to that shown in <FIG>, and similar items are similarly numbered. However, in <FIG>, the "cut, copy, paste" commands are displayed in a slightly different display element <NUM>, from that shown in <FIG>.

<FIG> and <FIG> show how the contextual command bar <NUM> can be displayed in different ways, based upon the orientation of the device displaying it. It can be seen that contextual command bar <NUM> has horizontally scrollable command portion <NUM> and fixed command portion <NUM>, as in the previous figures. <FIG> shows a smart phone in which the user has selected or highlighted an image <NUM>. Thus, the contextual command bar <NUM> is displayed, and shows visual command identifiers for commands corresponding to a context in which an image has been selected.

<FIG>, on the other hand, shows that the mobile device of <FIG> is now rotated so that the display is in landscape mode. Because the display is in landscape mode, there is sufficient display real estate to show all visual command identifiers in the contextual command bar <NUM>, without the need to hide any so that the user can see all visual command identifiers without scrolling the contextual command bar <NUM>. In such an example, user interface logic <NUM> (shown in <FIG>) detects rotation of the mobile device (such as by receiving a signal from an orientation sensor) and switches the user interface display to landscape mode. It thus displays the contextual command bar provided by system <NUM> in the way illustrated in <FIG>.

Referring again to the flow diagram of <FIG>, once the contextual command bar is surfaced for user interaction, then the user can interact with the contextual command bar in a variety of different ways. Interaction processing logic <NUM> illustratively detects user interaction with the contextual command bar. This is indicated by block <NUM> in <FIG>. User interaction processing logic <NUM> then performs any desired actions, based upon the detected user interaction. This is indicated by block <NUM>. This can take a wide variety of different forms as well.

For example, when the user interacts with the contextual command bar by providing a horizontal scroll input to scroll the contextual command bar, then interaction processing logic <NUM> controls user interface logic <NUM> to scroll the command bar accordingly. This is indicated by block <NUM>.

It may also be that the user actuates one of the visual command identifiers on the contextual command bar. For instance, it may be that the user actuates one of the visual command identifiers in the fixed command portion <NUM> (shown in <FIG>) of the contextual command bar that links to the commanding palette. In that case, interface processing logic <NUM> identifies a most relevant tab in the commanding palette (based on the detected context) and controls user interface logic <NUM> to display the commanding palette, with the most relevant tab selected. This is indicated by block <NUM>. As an example, it may be that certain contexts have corresponding contextual commands that are arranged under different tabs in the commanding palette. However, it may also be that the detected context is most highly related to one of the tabs in the commanding palette. Thus, when the user actuates the link to the commanding palette on the fixed command portion of the contextual command bar, the commanding palette will be displayed with the most relevant tab highlighted.

The visual command identifiers <NUM>-<NUM> in the horizontally scrollable command portion <NUM> may also correspond to commands that perform direct action, in response to the user actuating the corresponding visual command identifier. By way of example, where the commands are formatting commands, such as "bold", "italics", "underline", etc., the selected text may be directly bolded, italicized, underlined, etc. in response to the user actuating the corresponding visual command identifier. Performing such a direct action based on the user actuation is indicated by block <NUM> in <FIG>.

In another example, the command display elements may be entry points into a gallery or flyout menu or other similar display interface. In that case, in response to user actuation of the corresponding visual command identifier, interaction processing logic <NUM> will control user interface logic <NUM> to display the corresponding gallery or flyout menu. This is indicated by block <NUM>. <FIG> show examples of this.

It can be seen in <FIG> that the user has selected text shown generally at <NUM> on the user interface display of the mobile device. The selected text is on a slide in a slide presentation document displayed by a slide presentation application running on the mobile device. Thus, the contextual command bar <NUM> shows visual command identifiers corresponding to commands that can be performed on the selected text. <FIG> shows that the user has actuated visual command identifier <NUM> that corresponds to the command which allows the user to change the text color of the highlighted text. In response, interaction processing logic <NUM> controls user interface logic <NUM> to display the gallery <NUM> of different colors that the user can select from in order to perform the command. In the example shown in <FIG>, the visual command identifier <NUM> corresponding to changing text color is displayed, along with the remainder of contextual command bar <NUM>, even when gallery <NUM> is displayed. <FIG>, on the other hand, shows another example. <FIG> is similar to <FIG>, and similar items are similarly numbered. In <FIG>, however, once the user has selected the command display element <NUM>, then the contextual command bar disappears, and only the gallery <NUM> is displayed. Of course, these are only examples.

Referring again to the flow diagram in <FIG>, it may be also that the command associated with one or more of the visual command identifiers on the contextual command bar invokes an external experience, when it is actuated by the user. For instance, the command display element may represent a hyperlink or a comment. Therefore, when the user actuates the command display element, the hyperlink is navigated, or a comment user experience is generated by which the user can enter, view, or otherwise interact with a comment. Invoking an external experience based on user actuation of a visual command identifier in the contextual command bar is indicated by block <NUM> in <FIG>.

It will be noted that the interactions described above are examples only. Interaction processing logic <NUM> can perform actions or commands based on user interactions in other ways as well, and this is indicated by block <NUM>.

It can thus be seen that the present system greatly enhances the operation of the mobile device, itself. It extracts contextual command display elements, based upon a context of an application, and surfaces those for the user. The commands represented by those elements are a subset of the commands on the commanding palette for the application, and can be drawn from different tabs of the commanding palette. They are the commands that are most relevant to the detected context, in that they are those most likely to be invoked by the user. This saves significant processing overhead on the mobile device itself. Instead of having to generate various display screens in response to the user dismissing the keyboard, invoking the commanding palette, and navigating through different levels of the commanding palette to find a desired command, a contextual command bar with the most relevant commands is displayed and the commands can be directly invoked by the user from that command bar. This not only saves processing overhead in rendering displays and navigation needed to perform commands, but it also greatly enhances the user experience and efficiency.

The present discussion has mentioned processors and servers. In one example, the processors and servers include computer processors with associated memory and timing circuitry, not separately shown. They are functional parts of the systems or devices to which they belong and are activated by, and facilitate the functionality of the other components or items in those systems.

<FIG> is a block diagram of mobile computing device <NUM>, shown in <FIG>, except that some elements are disposed in a cloud computing architecture <NUM>. Cloud computing provides computation, software, data access, and storage services that do not require end-user knowledge of the physical location or configuration of the system that delivers the services. In various embodiments, cloud computing delivers the services over a wide area network, such as the internet, using appropriate protocols. For instance, cloud computing providers deliver applications over a wide area network and they can be accessed through a web browser or any other computing component. Software or components of device <NUM> as well as the corresponding data, can be stored on servers at a remote location. The computing resources in a cloud computing environment can be consolidated at a remote data center location or they can be dispersed. Cloud computing infrastructures can deliver services through shared data centers, even though they appear as a single point of access for the user. Thus, the components and functions described herein can be provided from a service provider at a remote location using a cloud computing architecture. Alternatively, they can be provided from a conventional server, or they can be installed on client devices directly, or in other ways.

The description is intended to include both public cloud computing and private cloud computing. Cloud computing (both public and private) provides substantially seamless pooling of resources, as well as a reduced need to manage and configure underlying hardware infrastructure.

A public cloud is managed by a vendor and typically supports multiple consumers using the same infrastructure. Also, a public cloud, as opposed to a private cloud, can free up the end users from managing the hardware. A private cloud may be managed by the organization itself and the infrastructure is typically not shared with other organizations. The organization still maintains the hardware to some extent, such as installations and repairs, etc..

In the example shown in <FIG>, some items are similar to those shown in <FIG> and they are similarly numbered. <FIG> specifically shows that mobile device <NUM> can communicate with an application component <NUM> that runs the application in the cloud and transmits pages for display on device <NUM>. In such a scenario, contextual command bar generation system <NUM> can also be disposed in the cloud (which can be public, private, or a combination where portions are public while others are private). Therefore, user <NUM> uses mobile device <NUM> to access those systems through cloud <NUM>.

<FIG> also depicts another example of a cloud architecture. <FIG> shows that it is also contemplated that some elements of mobile device <NUM> (shown in <FIG>) can also be disposed in cloud <NUM> while others are not. By way of example, data store <NUM> can be disposed outside of cloud <NUM>, and accessed through cloud <NUM>. In another example, application component <NUM> can also be outside of cloud <NUM>. Regardless of where they are located, they can be accessed directly by device <NUM>, through a network (either a wide area network or a local area network), they can be hosted at a remote site by a service, or they can be provided as a service through a cloud or accessed by a connection service that resides in the cloud. All of these architectures are contemplated herein.

It will also be noted that items shown in device <NUM> in <FIG>, can instead be disposed on a wide variety of different devices. Some of those devices include servers, desktop computers, laptop computers, tablet computers, or other mobile devices, such as palm top computers, cell phones, smart phones, multimedia players, personal digital assistants, etc..

<FIG> is a simplified block diagram of one illustrative example of a handheld or mobile computing device <NUM> that can be used as a user's or client's hand held device <NUM> in the previous Figures. <FIG> are examples of handheld or mobile devices.

<FIG> provides a general block diagram of some the components of a client device <NUM> in addition to those shown in <FIG>. In the device <NUM>, a communications link <NUM> is provided and can be part of communication systems <NUM> shown in <FIG>. Link <NUM> allows the handheld device to communicate with other computing devices and under some examples provides a channel for receiving information automatically, such as by scanning. Examples of communications link <NUM> include an infrared port, a serial/USB port, a cable network port such as an Ethernet port, and a wireless network port allowing communication though one or more communication protocols including General Packet Radio Service (GPRS), LTE, HSPA, HSPA+ and other <NUM> and <NUM> radio protocols, 1Xrtt, and Short Message Service, which are wireless services used to provide cellular access to a network, as well as Wi-Fi protocols, and Bluetooth protocol, which provide local wireless connections to networks.

In other examples, applications or systems are received on a removable Secure Digital (SD) card that is connected to a SD card interface <NUM>. SD card interface <NUM> and communication links <NUM> communicate with a processor <NUM> (which can also embody processors <NUM> from <FIG>) along a bus <NUM> that is also connected to memory <NUM> and input/output (I/O) components <NUM>, as well as clock <NUM> and location system <NUM>.

I/O components <NUM>, in one example, can be part of user interface mechanism <NUM> (in <FIG>) and can be provided to facilitate input and output operations. I/O components <NUM> for various embodiments of the device <NUM> can include input components such as buttons, touch sensors, multi-touch sensors, optical or video sensors, voice sensors, touch screens, proximity sensors, microphones, tilt sensors, and gravity switches and output components such as a display device, a speaker, and or a printer port. Other I/O components <NUM> can be used as well.

Location system <NUM> (which can be one of sensors <NUM> in <FIG>) illustratively includes a component that outputs a current geographical location of device <NUM>.

Memory <NUM> stores operating system <NUM>, network settings <NUM>, applications <NUM>, application configuration settings <NUM>, data store <NUM>, communication drivers <NUM>, and communication configuration settings <NUM>. Memory <NUM> can include all types of tangible volatile and non-volatile computer-readable memory devices. It can also include computer storage media (described below). Memory <NUM> stores computer readable instructions that, when executed by processor <NUM>, cause the processor to perform computer-implemented steps or functions according to the instructions. Similarly, device <NUM> can have a client system <NUM> which can run various applications. Processor <NUM> can be activated by other components to facilitate their functionality as well.

Examples of the network settings <NUM> include things such as proxy information, Internet connection information, and mappings. Application configuration settings <NUM> include settings that tailor the application for a specific enterprise or user. Communication configuration settings <NUM> provide parameters for communicating with other computers and include items such as GPRS parameters, SMS parameters, connection user names and passwords.

Applications <NUM> can be applications that have previously been stored on the device <NUM> or applications that are installed during use, although these can be part of operating system <NUM>, or hosted external to device <NUM>, as well.

<FIG> shows one example in which device <NUM> is a tablet computer <NUM>. In <FIG>, computer <NUM> is shown with user interface display screen <NUM>. Screen <NUM> can be a touch screen (so touch gestures from a user's finger can be used to interact with the application) or a pen-enabled interface that receives inputs from a pen or stylus. It can also use an on-screen virtual keyboard. Of course, it might also be attached to a keyboard or other user input device through a suitable attachment mechanism, such as a wireless link or USB port, for instance. Computer <NUM> can also illustratively receive voice inputs as well.

<FIG> is one example of a computing environment in which architecture device <NUM>, or parts of it, (for example) can be deployed. With reference to <FIG>, an example system for implementing some embodiments includes a general-purpose computing device in the form of a computer <NUM>. Components of computer <NUM> may include, but are not limited to, a processing unit <NUM> (which can comprise processor <NUM>), a system memory <NUM>, and a system bus <NUM> that couples various system components including the system memory to the processing unit <NUM>. The system bus <NUM> may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus also known as Mezzanine bus. Memory and programs described with respect to <FIG> can be deployed in corresponding portions of <FIG>.

Communication media typically embodies computer readable instructions, data structures, program modules or other data in a transport mechanism and includes any information delivery media. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.

By way of example only, <FIG> illustrates a hard disk drive <NUM> that reads from or writes to non-removable, nonvolatile magnetic media, a and an optical disk drive <NUM> that reads from or writes to a removable, nonvolatile optical disk <NUM> such as a CD ROM or other optical media. Other removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like.

Operating system <NUM>, application programs <NUM>, other program modules <NUM>, and program data <NUM> are given different numbers here to illustrate that, at a minimum, they are different copies.

These and other input devices are often connected to the processing unit <NUM> through a user input interface <NUM> that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB).

The computer <NUM> is operated in a networked environment using logical connections to one or more remote computers, such as a remote computer <NUM>. The remote computer <NUM> may be a personal computer, a hand-held device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer <NUM>. The logical connections depicted in <FIG> include a local area network (LAN) <NUM> and a wide area network (WAN) <NUM>, but may also include other networks. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.

The modem <NUM>, which may be internal or external, may be connected to the system bus <NUM> via the user input interface <NUM>, or other appropriate mechanism. In a networked environment, program modules depicted relative to the computer <NUM>, or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation, <FIG> illustrates remote application programs <NUM> as residing on remote computer <NUM>. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.

It should also be noted that the different examples described herein can be combined in different ways. That is, parts of one or more examples can be combined with parts of one or more other examples. All of this is contemplated herein.

Claim 1:
A mobile computing device (<NUM>), comprising:
an application component (<NUM>) that runs an application (<NUM>) that generates content;
command set identifier logic (<NUM>) that identifies a subset of commands, from a set of commands represented on a commanding surface (<NUM>) of the application, based on a context of the application;
a contextual command bar generation system (<NUM>) that generates a representation of a contextual command bar with a plurality of command actuators (<NUM>-<NUM>), each command actuator corresponding to a command in the subset of commands and being actuatable to execute the corresponding command, the representation of the contextual command bar including a palette link actuator (<NUM>-<NUM>) that is actuated to navigate to the commanding surface;
a display device; and
user interface logic (<NUM>) that controls the display device to display the contextual command bar based on the representation of the contextual command bar,
wherein the contextual command bar generation system comprises context identifier logic (<NUM>) that identifies the context of the application, and
wherein the contextual command bar generation system generates the representation of the contextual command bar including:
a horizontally scrollable command portion (<NUM>) that includes the plurality of command actuators (<NUM>-<NUM>), wherein the plurality of command actuators are arranged in order of relevance based on the likelihood that their corresponding commands are to be invoked by a user in the identified context of the application; and
a fixed command portion that includes the palette link actuator.