Trackball for a mobile device

A computer-implemented system and method for navigating through a plurality of graphical elements with a trackball. For example, a method according to one embodiment of the invention comprises: detecting a substantially vertical motion on the trackball when a first element is selected; and responsively selecting either (a) a second element which is closest vertically to the first element or (b) a third element in the same horizontal line as the first element if a leftmost horizontal coordinate of the third element is closer to a leftmost horizontal coordinate of the first element than then leftmost horizontal coordinate of the second element.

TECHNICAL FIELD

This application relates generally to the field of data processing devices, and more particularly to versatile input/output and display configurations for a data processing device.

BACKGROUND

Portable data processing devices such as Personal Digital Assistants (“PDAs”) and programmable wireless telephones are becoming more powerful every day, providing users with a wide range of applications previously only available on personal computers. At the same time, due to advances in silicon processing technology and battery technology, these devices may be manufactured using smaller and smaller form factors. Accordingly, users no longer need to sacrifice processing power for portability when selecting a personal data processing device.

Although processing devices with small form factors tend to be more portable, users may find it increasingly difficult to interact with them. For example, entering data may be difficult due to the absence of a full-sized keyboard and reading information may be difficult due to a small, potentially dim Liquid Crystal Display (“LCD”).

To solve some of these problems, the assignee of the present application developed two data processing devices known commercially as the “Sidekick” and the “Sidekick II,” respectively. The Sidekick is illustrated inFIGS. 1a-c. This data processing device100includes a keyboard101, a control knob/wheel102(e.g., for scrolling between menu items and/or data), and a set of control buttons105(e.g., for selecting menu items and/or data).

The display103is pivotally coupled to the data processing device100and pivots around a pivot point109, located within a pivot area104, from a first position illustrated inFIG. 1ato a second position illustrated inFIGS. 1b-c. When in the first position the display103covers the keyboard101, thereby decreasing the size of the device100and protecting the keyboard101. Even when the display is in the first position, however, the control knob102and control buttons105are exposed and therefore accessible by the user. The motion of the display103from the first position to a second position is indicated by motion arrow106illustrated inFIGS. 1a-b. As illustrated, when in the second position, the keyboard101is fully exposed. Accordingly, the display is viewable, and data is accessible by the user in both a the first position and the second position (although access to the keyboard is only provided in the first position).

In one embodiment, the data processing device100is also provided with audio telephony (e.g., cellular) capabilities. To support audio telephony functions, the embodiment illustrated inFIGS. 1a-cincludes a speaker120for listening and a microphone121for speaking during a telephone conversation. Notably, the speaker120and microphone121are positioned at opposite ends of the data processing device100and are accessible when the screen103is in a closed position and an open position.

The Sidekick II is illustrated inFIGS. 2a-b. This data processing device200includes many of the same features as the Sidekick such as, for example, a pivoting display205which reveals an alphanumeric keyboard305and a control wheel230. In addition, the Sidekick II includes a directional pad245for performing cursor control operations and an integrated speaker246and LED (not shown).

As mentioned above, the control wheels102and230in these and other devices may be used to scroll through lists of items such as menu items and/or data entry fields. However, one limitation of the scroll wheel is that it is only capable of navigating within a single dimension. For example, by manipulating the control wheel112, a user may move a graphical selection element either “up” or “down” in relation to the orientation of the screen (or “left” or “right” depending on how the user interface is programmed). However, the control wheel does not allow the user to navigate in any desired direction on the screen. Accordingly, a more flexible user interface solution for navigating within a graphical interface of a mobile device is needed.

SUMMARY

A computer-implemented system and method are described for navigating through a plurality of graphical elements with a trackball. For example, a method according to one embodiment comprises: detecting a substantially vertical motion on the trackball when a first element is selected; and responsively selecting either (a) a second element which is closest vertically to the first element or (b) a third element in the same horizontal line as the second element if a leftmost horizontal coordinate of the third element is closer to a leftmost horizontal coordinate of the first element than then leftmost horizontal coordinate of the second element.

DETAILED DESCRIPTION

Several different techniques for implementing a trackball on a mobile data processing device are described below. As will be apparent from the following description, many of these configurations are particularly beneficial when employed on a dual-purpose data processing device such as a personal digital assistant (“PDA”) or other mobile computing device having integrated wireless telephony capabilities (e.g., a combination PDA and cell phone). However, it should be noted that the underlying principles of the invention are not limited to a wireless telephony configuration.

A data processing device300according to one embodiment of the invention is illustrated inFIGS. 3a-b. Unlike the prior data processing devices described above, this data processing device300includes a trackball301for navigating through graphical images and text displayed on the data processing device display305. In addition, in one embodiment, the trackball301may be clicked down into the data processing device to generate a selection function (i.e., similar to the functions provided by a standard mouse button).

In one embodiment, the data processing device300display adjusts as in the prior data processing devices described above. For example, in one embodiment, the display305is pivotally coupled to the data processing device300and pivots around a pivot point309from a first position illustrated inFIG. 3ato a second position illustrated inFIG. 3b.

When in the first position the display305covers the keyboard306, thereby decreasing the size of the device300and protecting the keyboard306. Even when the display is in the first position, however, the trackball301and control buttons, such as directional pad345, are exposed and therefore accessible by the user. As illustrated inFIG. 3b, when in the second position, the keyboard306is fully exposed. Accordingly, the display is viewable, and data is accessible by the user in both the first position and the second position (although access to the keyboard is only provided in the first position).

In one embodiment, the data processing device300is also provided with audio telephony (e.g., cellular) capabilities. To support audio telephony functions, the embodiment illustrated inFIGS. 3a-bincludes a speaker346(embedded within the directional pad) for listening and a microphone321for speaking during a telephone conversation. Notably, the speaker346and microphone321are positioned at opposite ends of the data processing device300and are accessible when the screen305is in a closed position and an open position.

Implementing a trackball301on a portable data processing devices presents several unique challenges. For example, the relatively small display screen may provide inadequate space for a true “point-and-click” environment (e.g., such as a Windows environment in which is user is provided the ability to move a pointer and click on any graphic on the screen). That is, the graphics displayed on the data processing device display may be so small and close together that moving a cursor to the correct position to identify an icon may be difficult. In addition, with mobile devices, users often do not have the same focus as with a standard personal computer (e.g., users are not sitting at a desk when using a mobile device). Accordingly, simplified navigation techniques for use with a trackball are needed.

In one embodiment of the invention, to simplify navigation, in response to user manipulation of the trackball, a “selection element” or “selection graphic” traverses through a limited number of selectable user interface elements displayed within the user interface (e.g., data fields, menu items, text, etc). This approach eliminates unnecessary trackball movement by the user, prevents user errors, and makes the graphical navigation as efficient as possible.

General Trackball Navigation

One embodiment of a method for selecting elements using a trackball input device is illustrated inFIGS. 4a-b. The term “element” is used broadly in this context to refer to any type of image on a display screen including, for example, data input fields, menu items, text and graphical icons. Following the description ofFIGS. 4a-b, a series of specific examples within a graphical user interface will be provided inFIGS. 5a-f.

Turning toFIG. 4a, at401, the data processing device registers a trackball input from a user. At402a determination is made as to whether the user input is primarily a horizontal input (e.g., left or right) or primarily a vertical input (e.g., up or down). Given that the user may scroll the trackball in virtually any direction, the input may inadvertently include both horizontal and vertical components. One embodiment of the data processing device300determines whether the input is a “horizontal” input or a “vertical” input based on the relative horizontal and vertical components of the input (i.e., the input is a vector with both horizontal and vertical components). If the input has a relatively larger horizontal component, then subsequent trackball movements will be biased to continue horizontally; conversely, if the input has a relatively larger vertical component, then subsequent trackball movements will be biased vertically.

If the input is identified as vertical, then the process flows toFIG. 4b(described below). If the input is identified as horizontal, then the process flows to403where a determination is made is to whether a manual override exists for the requested motion403. For example, in some cases, the core techniques described below for selecting an element within a user interface may not be desirable. In these cases, the user interface programmer may manually identify the desired operation for the user interface. For example, if the techniques described below would cause an element to be selected which is unlikely to be the element that a user would wants to select, then the manual override will cause a different (more likely desired) element to be selected.

If no manual override exists, then at405a determination is made as to whether elements exist within the same horizontal line as that occupied by the current element—that is, whether another element exists which includes at least some overlapping vertical coordinates with the current element. For example, inFIG. 4c, elements453and454are in the same horizontal “line” as element452because at least some of their vertical coordinates overlap.

If no elements are in the same horizontal line as the current element, then no new element is selected at406. For example, if the user scrolls the trackball301to the left/right and if no elements exist to the left/right (within the range of vertical coordinates occupied by the current element), then no new element will be selected.

If, however, elements exist within the range of vertical coordinates defined by the current element, then at407a determination is made as to whether more than one element exists with overlapping vertical coordinates. If so, then at406, the one element is selected. If not, then at409the element in the same horizontal line which is closest to the direction of the trackball motion will be selected. For example, inFIG. 4c, if element452is currently selected and the user scrolls right and slightly upward, then element453will be selected. Conversely, if element452is currently selected and the user scrolls right and slightly downward, then element454will be selected. Alternatively, in one embodiment, the element with a relatively larger vertical coordinate is automatically selected (i.e., the element relatively closer to the top edge of the display—element453in the example).

FIG. 4billustrates a process for selecting an element in response to a vertical motion (i.e., moving the trackball up or down in relation to the display). At410, a determination is made as to whether a manual override exists. If so, then the new element is selected in accordance with the manual override at411. If not, then at412, the element X which is closest vertically to the current element is identified. For example, inFIG. 4c, the element closest vertically to element450is element451. At413, the element in the same horizontal line as element X (or element X if there are no other elements in the same horizontal line) with the closest left-most coordinate to the current element is selected. For example, inFIG. 4c, element452is selected in response to a downward movement from element450because the leftmost coordinate460of element452is closest to the leftmost coordinate461of element450(e.g., it is closer on the horizontal axis than the leftmost element462of element453).

Various alternate and/or additional techniques may be used to select the next element in response to a trackball signal. For example, in one embodiment, in response to a vertical or horizontal movement signal, the element having the largest horizontal range or vertical range, respectively overlapping with the current element may be selected. Various other techniques for selecting the next element may be employed while still complying with the underlying principles of the invention. Moreover, although the examples provided above focus on rightward and downward motions, the same principles are applicable to leftward and upward motions, respectively.

FIGS. 5a-fillustrate specific examples of the foregoing processes (assuming that no manual overrides exist).FIG. 5aillustrates a data entry box501with an icon picker element502for selecting an icon and data entry fields503-506for entering the name and nickname of a user. If the icon picker element502is initially selected (as shown) and the user scrolls horizontally to the right, then a determination is made that two data entry fields503-504have overlapping vertical coordinates with the icon picker element502(i.e., they are both generally to the right of the icon picker element502). As such, the data entry field which has relatively higher vertical coordinates is selected—in this example, the “first name” field503. If the user now the scrolls vertically downward, data fields504-506will be selected in succession. If the user scrolls to the left from either data entry field505or506, no new element will be highlighted because no element to the left of these fields has vertical coordinates which overlap with these fields. By contrast, if the user scrolls to the left from either data entry field503or504, the icon picker element will be selected because it shares overlapping vertical coordinates with these fields.

FIG. 5billustrates another data entry box including data entry fields for Job Title510, Company511and Birthdate512as well as a Month Picker element513. If the currently selected element is the Company field511and the user scrolls down using the trackball, then the Birthdate field512will be selected because the leftmost coordinate of the Company element511is closest to the leftmost coordinate of the Birthdate element.

FIG. 5cillustrates an Address data entry box including data entry fields for Label520, Street521, City522, State523, Zip524, and Country525. If the currently selected element is the State field523and the user scrolls down using the trackball, then the Country field525will be selected because the leftmost coordinate of the Country field525is closest to the leftmost coordinate of the State field523. Similarly, if the currently selected element is the Country field525and the user scrolls up using the trackball, then the State field523will be selected because the leftmost coordinate of the State field523is closest to the leftmost coordinate of the State field525.

FIG. 5dillustrates a data entry box for specifying calendar event information including data entry fields for an event name530, a location531, a start date field532, a month picker box for the start date533a time picker box for the start date534, an end date field535, a month picker box for the end date536, and a time picker box for the end date537. In this example, if the month picker box for the end date536is currently selected and the user scrolls up, the month picker box for the start date533will be selected because the leftmost coordinate of month picker box536is closest to the leftmost coordinate of month picker box533. Thus, month picker box533is selected even though the start date field532and time picker box534are closer vertically by one pixel. One embodiment of the invention initially groups start date field532, the month picker box533and the time picker box534together because they are in the same horizontal line and then picks the one with the closest left coordinate.

FIG. 5eillustrates another example that includes (among other elements) an Until check box540and a Reminder check box541. In this example, if the currently selected element is the Reminder check box541and the user scrolls up using the trackball, then the Until check box540will be selected because the leftmost coordinate of the Until check box540is closest to the leftmost coordinate of the Reminder check box541.

Similarly,FIG. 5fillustrates an example that includes a default reminder data entry field550, an hour data entry field551, and a minute data entry field552. In this case, if the currently selected element is the default reminder data entry field550and the user scrolls up using the trackball, then the minute data entry field552will be selected because the leftmost coordinate of the minute data entry field552is closest to the leftmost coordinate of the default reminder data entry field550.

Quick Edit Text Fields

One embodiment of the invention employs techniques for navigating and editing text entry fields using the trackball301. Specifically, in this embodiment, scrolling the trackball300causes each of the text entry fields to be selected according to the techniques described above (e.g., based on leftmost data field coordinate when moving vertically and based on overlapping vertical coordinates when moving horizontally). When a particular text entry field is initially selected, the field is “focused” and a highlight is provided to indicate that the whole text field is selected. At this stage, typing on the keyboard306cases the current text within the field to be replaced by the newly entered text. By contrast, clicking the trackball301into the data processing device300or the directional pad345causes a cursor to enter the text entry field, thereby allowing the user to edit the text. Scrolling the trackball301with the cursor in the text and/or at the end of the text causes a new field to be selected and focused. By contrast, clicking on the directional pad345at this stage causes the cursor to move through the text of the selected field.

FIGS. 6a-cprovide an example of the foregoing operation. InFIG. 6aa text entry field for entering a street602is initially selected. The entire text entry field is highlighted to provide an indication that text entry will replace the current contents of the field602. In addition, unlike prior user interfaces, the label for the text entry field (e.g., “State”) is provided as text within the field itself. In response to the user scrolling left on the trackball, the city field is once again highlighted. In the illustrated example, the user types a street number and then scrolls down on the trackball to cause a text entry field for entering a city603to become highlighted. As the user begins typing the city, the city label is replaced, as indicated inFIG. 6b. In response to a rightward scroll after the city is entered, a text entry field for entering a state604is highlighted. If the user does not enter text but instead scrolls left, all of the text for the previously-entered city is highlighted, as indicated inFIG. 6b. If the user then selects the delete button or types any text, the contents of the city text entry field are deleted or replaced, respectively, as indicated inFIG. 6c.

In one embodiment, a new text entry field is selected in response to a trackball motion only if the cursor is at the beginning or end of the text within the current text entry field and the cursor is scrolled to the left or right, respectively, or if the trackball is scrolled up or down regardless of the location of the cursor. For example, if the cursor is positioned to the right of the text in text entry box603as shown in the first frame ofFIG. 6band the user scrolls right, a new text entry field604is selected. However, in one embodiment, if the user scrolls left in this situation, the cursor will responsively move one character at a time through the text within the text entry box602until it reaches the beginning of the text.

Alternatively, in one embodiment, scrolling the trackball will cause another field to be highlighted regardless of the position of the cursor. In this embodiment, the cursor is moved via the directional pad rather than the trackball.

Text Selection

One embodiment of the invention provides techniques to select text using the trackball301when in a “reading” mode such as, for example, when viewing a Web page or email message. These techniques will be described with respect toFIG. 7.

In this embodiment, the user is reading a page of an email message containing text, hyperlinks and graphics. Three modes of operation are provided for navigating the page—a “standard” mode, a “targeting/selecting” mode (hereinafter “targeting” mode), and a “selection” mode. When in the standard mode shown in window700, only hyperlinks are selected in response to movement of the trackball301. That is, in response to the user scrolling the trackball, the highlighter will jump from one hyperlink to the next, bypassing all non-hyperlinked text. This mode may be used to standard Web browsing operations.

In one embodiment, the user may cause the device to enter the “targeting” mode by selecting a command from a menu, or pressing and/or holding down a specified shortcut key (e.g., the shift key from the keyboard306). When in targeting mode, both hyperlinked text, non-hyperlinked text and graphical elements are selected in response to the trackball movement. Specifically, as the user moves the trackball, the highlighter highlights each word in succession in the direction of the trackball motion (rather than just hyperlinked words). As illustrated in window701, the highlighter may change color and the formatting used for the hyperlinked words may change to indicate that the device is in the targeting mode. The user may position the start of the desired text selection point in either standard mode (if the start is a hyperlink) or targeting mode.

Once the start of the desired text selection point is identified, the user can enter into the selection mode by moving the trackball while holding down a specified shortcut key (e.g. Shift) to select a piece of contagious text. In one embodiment, while holding down the designated shortcut key, each additional word in succession will be highlighted along with the currently selected word(s). This is illustrated graphically in window702. The user may then copy the text (e.g., using a copy command from the menu or designated key combination), or return to the positioning state by releasing the shortcut key and moving the trackball so that another piece of text can be selected.

Selecting on a word-by-word basis as described above is efficient and less error-prone than selecting on a character-by-character basis. However, the underling principles of the invention may still be implemented using character-by-character selection.

Assuming a word-by-word selection is employed, in one embodiment a “word” is defined to be any of the following: (a) one or more letters, (b) one or more digits, blocks of spaces, (c) one punctuation/symbol/special characters. In this way, the majority of text selection circumstances are easily captured. The foregoing features are illustrated inFIG. 8which shows an exemplary sequence of numbers, letters and other characters: sapien123 1-2234-566!!!. In response to the targeting motion described above, the “sapien” sequence of characters is logically separated from the set of numbers which follow (“123”). This is true even though no space exists between the letters and numbers. As the user holds down the designated control key (e.g., Shift) and scrolls the wheel to the right, the following characters are selected in succession: 123, space, 1, -, 2234, -, 566, !, !, !. Thus, the system selects numbers separated by dashes in groups (e.g., 2234), and highlights special characters such as the dashes, spaces and exclamation points individually. The purpose of the foregoing algorithm is to provide a selection mechanism which is both efficient and which reflects selections which are most likely intended by the user (e.g., it is unlikely that the user would want to highlight only a portion of 2234 in the example above).

Browser Navigation

In one embodiment, special link navigation features are provided to improve Web browser navigation. For example, on certain Web pages on mobile devices, hyperlinks may be positioned arbitrarily and may have various sizes and lengths. Consequently, users may have a problem knowing which link will be highlighted after they roll the trackball up, down, left or right. For example, inFIG. 9a, the mail icon901is the currently highlighted link but it is unclear which link will become highlighted when the user scrolls down (e.g., Movies or Horoscopes).

To solve these problems, one embodiment of the invention visually identifies the links which will be selected in response to a trackball motion moving away from the current link. InFIG. 9b, the current link is “People Search”902. To provide an indication of the links which will be selected if the user scrolls the trackball, the potential links are highlighted in a transparent color. In addition, to differentiate the different directions of movement the links to the left/right are provided with different colors than the links above/below the current link902. In the illustrated example, the Map link903and the TV link904are colored purple and the News link905and the Personals link906are colored gray. In addition, in one embodiment, the further away the link is from the current element, the higher the transparency value associated with the link (e.g., the Geocities link907has a higher transparency value, and is therefore more transparent, than the Maps link903). As the user move to a new link, the highlights are regenerated accordingly.

Logical Barriers

One embodiment of the invention employs logical barriers between windows and other graphical elements to prevent the user from inadvertently navigating in the wrong direction. For example, if the user intends to scroll down a menu structure with the trackball301and inadvertently scrolls to the right or left, these embodiments of the invention do not immediately register the right and/or left input.

FIGS. 10a-billustrate one embodiment of a logical barrier between two months of a calendar program. InFIG. 10a, the month of September is shown in window1000and the month of October is shown in window1003. In this particular example, the user scrolls down on the trackball from September 13 to September 27 (block1001). Once September 27 has been highlighted, the logical barrier will initially prevent the highlight from moving to the October window1003in response to additional downward scrolling motion. In one embodiment, the logical barrier comprises a short duration of time (e.g., 4/10 second, ½ second, etc) during which the user may continue to scroll downward without any effect. Alternatively, or in addition, the logical barrier comprises a specified amount of rotation in the downward direction (e.g., a specified number of downward “clicks” on the trackball). After the short duration of time and/or amount of downward motion, an additional downward scroll on the trackball301will move the highlight to the October 4 date1002within the October window1003. In one embodiment, the logical barriers are user-configurable, allowing the user to specify whether a specified amount of time and/or a measurable downward scroll is to be used as the logical barrier.

FIG. 10billustrates the same general principles with a weekly calendar view. In this example, a Wednesday, November 15thentry1005is initially selected within the week of Nov. 12, 2006 (window1006). In response to scrolling the trackball to the right, each of the days from November 15 to November 18 are highlighted in succession. When the final day within the window1006is highlighted (November 18), a logical barrier will initially prevent the next window1008to be displayed in response to additional rightward trackball motion. After a specified amount of time (e.g., 4/10 second, ½ second, etc) and/or a specified amount of additional rightward motion on the trackball, the cursor will move to the first date entry1007in the new window1008.

FIGS. 10c-eillustrate a particular type of logical barrier referred to as a “rail guard” for preventing inadvertent trackball movements within a hierarchical menu structure according to one embodiment of the invention. Specifically,FIG. 10cshows a drop down menu1010with a plurality of selectable elements which may be selected via an up/down scrolling motion on the trackball301. Certain elements, such as element1011, have sub-elements associated with them (as indicated by the right-pointing arrow displayed on some of the elements). The user scrolls right on the trackball301when the element1011is highlighted to expose the sub-elements. For example, scrolling right when element1011bring up a set of “Airplane Mode” sub-elements, as illustrated inFIG. 10d.

In one embodiment, of the invention, the “rail guard” logical barrier is configured on each element to prevent the sub-elements from being exposed in response to an inadvertent right/left scrolling motion. More specifically, the sub-elements will be selected in response to a rightward motion only after a specified amount of time after the vertical motion has stopped (e.g., 2/10 of a second). Thus, any horizontal motion which occurs within the specified amount of time after vertical motion stops is deemed accidental and is ignored. Alternatively, or in addition, the sub-elements may be selected only after a specified amount of additional rightward motion on the trackball.

In one embodiment of the invention, the same concepts are applied to the graphical, circular menu structure such as the one illustrated inFIG. 10e. That is, certain menu elements such as element1020have sub-elements associated with them which will be selected in response to a rightward motion only after a specified amount of time and/or a specified about of additional rightward motion on the trackball.

The end result of the logical barriers described above is that inadvertent motion by a user at the edge of a window or other graphical element will not cause unwanted movements into an adjacent window and/or other graphical element, respectively, thereby improving the navigation experience for the user.

Embodiments of the invention may include various steps as set forth above. The steps may be embodied in machine-executable instructions. The instructions can be used to cause a general-purpose or special-purpose processor to perform certain steps. Alternatively, these steps may be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

Throughout the foregoing description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without some of these specific details. Accordingly, the scope and spirit of the invention should be judged in terms of the claims which follow.