Associating ancillary information with an application user interface

Associating user interface elements with information ancillary to said user interface elements includes creating one or more items of user interface element ancillary information, receiving a request for user interface element ancillary information, the request identifying the context in which the information is to be used, searching, using a processor, said one or more items of user interface element ancillary information for items matching the context in which the information is to be used, and returning one or more items of user interface element ancillary information matching the context in which the information is to be used.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of United Kingdom Application Number 1214375.6 filed on Aug. 13, 2012, which is fully incorporated herein by reference.

BACKGROUND

A computer user typically communicates with applications running on the computer via a user interface (UI). Referring toFIG. 1, a window100on a computer screen is shown. The window100typically has a title bar102where the application name is often displayed. The window100typically also has a number of user interface elements such as menu items104or push buttons (not shown inFIG. 1). A user may select a menu item104and a number of options associated with that menu item104may appear as a drop down menu106.

Referring toFIG. 2, user interfaces200generally use various types of ancillary information204associated with elements202of the user interface in order to improve the user experience. Examples of such ancillary information include:

(a) Text or graphics206used to label and identify the user interface element. These text or graphics often include:(i) Translations212into a range of languages or dialects; and(ii) Alternative versions214of graphics suitable for different displays;

(c) Accessibility and assistive resources210.

A known method of associating the user interface ancillary information with user interface elements is by using a fixed identifier internal to the application that is used to link a particular user interface element (e.g., menu item, push button, voice menu, etc.)202with a piece of user interface ancillary information (e.g., an alternative translation for the text of the user interface element, alternative graphics, context help for the item, etc.)206. These fixed identifiers are typically allocated during product development and assigned to the user interface elements202. User interface ancillary information206is then tagged with the relevant fixed identifier to associate it with the corresponding user interface elements202.

The problems associated with this known approach include that the user interface ancillary information206has to be developed in close cooperation with the team developing the application, and any changes to the user interface structure require corresponding changes to the user interface ancillary information206, that is, the user interface ancillary information206is liable to change, even from version to version. Another problem is that it is difficult to make additions to or to modify the user interface ancillary information206once an application has been distributed and installed by an end user.

The use of fixed identifiers also means that information of each type must generally be provided for every single user interface element202, or provide only a basic and naive level of fall-back in cases where no specific match is found. For example, every item must be given help text, however trivial, or a simple fallback to help for the form or panel or the like takes place.

There is also generally no structured or integrated mechanism for user participation in modifying, updating or extending the ancillary information206provided for an application, and for providing feedback or requesting assistance. Similarly, there is no straightforward way for the application developers to monitor the use of the ancillary information206that was provided and detect problems or shortcomings.

BRIEF SUMMARY

A method of associating user interface elements with information ancillary to the user interface elements includes creating one or more items of user interface element ancillary information, receiving a request for user interface element ancillary information, the request identifying the context in which the information is to be used, and searching, using a processor, the one or more items of user interface element ancillary information for items matching the context in which the information is to be used. The method also includes returning one or more items of user interface element ancillary information matching the context in which the information is to be used.

A system includes a processor programmed to initiate executable operations. The executable operations include creating one or more items of user interface element ancillary information, receiving a request for user interface element ancillary information, the request identifying the context in which the information is to be used, and searching the one or more items of user interface element ancillary information for items matching the context in which the information is to be used. The executable operations further include returning one or more items of user interface element ancillary information matching the context in which the information is to be used.

A computer program product includes a computer readable storage medium having program code stored thereon. The program code is executable by a processor to perform a method. The method includes creating, using the processor, one or more items of user interface element ancillary information, receiving, using the processor, a request for user interface element ancillary information, the request identifying the context in which the information is to be used, and searching, using the processor, the one or more items of user interface element ancillary information for items matching the context in which the information is to be used. The method further includes returning, using the processor, one or more items of user interface element ancillary information matching the context in which the information is to be used.

DETAILED DESCRIPTION

The present invention relates to the field of computer application user interfaces, and more particularly to associating ancillary information with computer application user interfaces.

Embodiments of the present invention provide a method of associating user interface elements with information ancillary to said user interface elements, the method comprising the steps of: creating one or more items of user interface element ancillary information; receiving a request for user interface element ancillary information, the request identifying the context in which the information is to be used; searching said one or more items of user interface element ancillary information for items matching the context in which the information is to be used; and returning one or more items of user interface element ancillary information matching the context in which the information is to be used. The method provides the advantage that the returned user interface ancillary information is not just that which exactly matches the request, but is additionally information that is associated contextually with the requested information.

Referring toFIG. 3, a user interface element302is shown containing structured contextual information to describe a particular user interface feature. In the example ofFIG. 3that information includes the application name, a version identifier, the type of the user interface element, the label of the element and an ID for the element. It may contain other information, such as any current content/value or any other information which may be useful. It may omit any of the application name, a version identifier, the type of the user interface element, the label of the element and an ID for the element if desired. In particular, the ID for the element may be omitted.

One possible implementation of the user interface element302ofFIG. 3is an XML fragment. For a menu item to open a file, the XML fragrant may be:

In the example XML fragment, the application name is “Acme Editor”, the version identifier is “1.5.3”, the type of the user interface element is “menuitem”, the label of the element is “Open” and the ID for the element is “17945”. Note that the provision of an explicit ID for the user interface element is optional and is just one item of identifying information along with the other pieces of information.

Referring toFIG. 4, the method of associating user interface elements with information ancillary to the user interface elements starts at step402. At step404, the search service304receives a query based on the information provided in the user interface element302. At step406, the search service304searches the repository306for ancillary information204which matches the information in the received query. Typically, a semantic search may be used by the search service304. The input to the semantic search can include the structured contextual information described with reference toFIG. 3. At step408, if matches are not found, then at step412, an error is returned to the requesting application. The error may, in practice, be merely a null response. At step408, if matches are found, then at step410, the matches are returned to the requesting application. The returned matches are ancillary information204from a repository306,308that matches the supplied query. The search service304can be requested to provide either the best match or a set of matches ranked by relevance. At step414, the method ends.

An example implementation of the search service304might be a web service. The application of which the user interface element302is a part of POSTs the contextual information to a web service over the internet using the HTTP protocol and receives the matching ancillary information204from a repository308as a response. An alternative (or complementary) implementation of the search service304is as a local service, as part of the application or as part of the operating system or other environment. Such a local service receives the matching ancillary information204from a repository306as a response. Hybrid solutions are also possible, in which a local service returns matches from a local repository306or cache and additionally calls for results from a remote service and/or repository308if and when available.

The search service304described above is accessed via a specific communication mechanism using a particular protocol, encoding and grammar to encapsulate the contextual information described above with reference toFIG. 3about a user interface element of the application. The search service304may be created and operated by the application development team, a third party commercial organization, or may be community based.

An advantage of the use of a search service304is that the search service304is able to understand and exploit the contextual information that applications provide in their requests. A search service304typically include one or more repositories306of material tagged appropriately. However, it might also access external resources308and secondary search services.

When a user interacts with the user interface in such a way that ancillary information204is required (e.g., by using ‘F1’ to request help on an input field, or selecting an alternative language for the user interface) then the application assembles a set of contextual information to describe the user interface feature(s) that information is required for, expressed using the specific encoding and grammar that is accepted and understood by the search service304that is being consulted, and sends it to the search service304using the appropriate protocol.

When a response is received from the search service304, the results can be used by the application as appropriate. For example, if a suitable translation of a menu item were being requested, the best matching text could be used to replace the current or default menu item text. Similarly, if contextual help were being requested, the best matching text, or possibly a list of candidate entries, can be displayed.

The approach ofFIG. 3means that using internal fixed identifiers to relate ancillary information204to user interface elements202,302is replaced with a more flexible and powerful approach which includes using a semantic search to perform the association between the user interface element and the ancillary information.

An advantage of this approach is that the search service304is not limited to finding exact matches or naive fallbacks. The search service304is able to apply algorithms to exploit the richer set of contextual information that is supplied. For example, if help information for a menu item were requested, the service is able to carry out any or all of the following.

It can provide help information that is specifically related to the menu item, if the menu item is found. For example, if the menu item is “Open” in a drop down menu from a “File” menu item, then help information specifically related to a “File Open” menu item could be provided.

It can provide help information for closely related user interface elements, such as the “File” menu item or the form used for input of data when the “File Open” menu item is displayed.

It can provide help information for functionally related user interface elements, such as a drop down menu elsewhere in the user interface that has a similar function. For example a menu item of “Import” under the “Text” menu item used for importing text into the current file might be regarded as having a similar function.

It can provide help information corresponding to previous versions of the application. These may be versions that a user has more familiarity with or the help information may be information that has not yet been validated as correct for the current version, but which may, nevertheless, be useful to the end user.

It can provide non-contextual information for tasks that relate to the user interface element such as how to contact the developer or supplier of the application.

It can search an index of all available help information for the application. In the example of “File Open” a list of the file formats that can be opened by the application may be provided.

It can call out to external search services such as Google, Bing and the like for free text search results and the like.

Another advantage of this approach is that the ancillary information204can be developed independently of the application. For example, items of help text can be added and updated without any changes to the application. This makes it practical for help text or translated text, for example, to be developed by teams that are completely independent of the team that developed the application.

The ability to add or update items of help text without any changes to the application allows for crowd-sourcing to be used to generate help text and for translations for applications. For example, wiki technology can be exploited enabling users of an application to contribute and tag their contributions in such a way that the search service can match them to user interface elements for the application.

Ancillary information204does not have to be provided for every single user interface element. The search engine304will locate the most relevant ancillary information204. For example, if an application provides contextual help using a conventional system then help information has, in practice, to be provided for all user interface elements even if the help information is trivial. In the embodiment ofFIG. 3, help need only be provided for user interface elements204that warrant it, and the search engine304returns the next most generalized or relevant help for other user interface elements302that did not explicitly have their own help.

The embodiment ofFIG. 3can be used to associate many types of information that are not currently contextually associated with user interface elements, for example: user tips, comments, business-specific practice guidelines, how-to videos, newsgroup threads, etc.

In an alternative embodiment, additional information can be included as part of the set of structural contextual information that is provided to the search service304. This may include elements of the content that a user is working on, and/or elements of the recent history of a user's actions. This additional information can be used by the search service304to more accurately identify relevant task help and information that would be helpful for the user.

In another alternative embodiment, a user is provided with means to easily contribute to the ancillary information204content. For example, if a request for contextual help finds no good match, the user may read various other pieces of help text and information from other sources to answer their question. Once they have done this, the application can use the same contextual information to enable the user to generate new content and tag it appropriately so that it will then be available for future users to access via the search service304. Similarly, the application can provide the means for the user to post a request for the missing information, or to provide feedback on the information found.

By analyzing requests made to the search service304, it becomes possible to identify “hot spots” in an application which users often need help with. It is also possible to identify missing topics and areas of information. In a community-based search service these become opportunities for other expert users to contribute valuable content. It is also possible to identify areas of information most frequently used, languages most frequently selected, or other parameters, enabling targeting of investment for future information development.

One or more embodiments disclosed herein can be implemented using a data processing system (system). The system can include at least one processor (e.g., a central processing unit) coupled to memory elements through a system bus or other suitable circuitry. The system can store program code within the memory elements. The processor can execute the program code accessed from the memory elements via a system bus or the other suitable circuitry. For example, the system can be implemented as a computer or other programmable data processing apparatus that is suitable for storing and/or executing program code. It should be appreciated, however, that the system can be implemented in the form of any system including a processor and memory that is capable of performing and/or initiating the functions and/or operations described within this specification.

The memory elements include one or more physical memory devices such as, for example, local memory and one or more bulk storage devices. The local memory refers to RAM or other non-persistent memory device(s) generally used during actual execution of the program code. Bulk storage device(s) can be implemented as a hard disk drive (HDD), solid state drive (SSD), or other persistent data storage device. The system also can include one or more cache memories that provide temporary storage of at least some program code in order to reduce the number of times program code must be retrieved from bulk storage device during execution.

Input/output (I/O) devices such as a keyboard, a display, and a pointing device optionally can be coupled to the system. The I/O devices can be coupled to the system either directly or through intervening I/O controllers. One or more network adapters also can be coupled to the system to enable the system to become coupled to other systems, computer systems, remote printers, and/or remote storage devices through intervening private or public networks. Modems, cable modems, wireless transceivers, and Ethernet cards are examples of different types of network adapters that can be used with the system.