Source: https://patents.google.com/patent/WO2011084861A1/en
Timestamp: 2019-04-22 06:54:13+00:00

Document:
2010-12-29 Application filed by Apple Inc. filed Critical Apple Inc.
A multifunction device with a touch-sensitive display: displays a text entry region and a soft keyboard with default keys; detects a contact at a first location on the touch-sensitive display that corresponds to an activation region associated with a respective default key; detects an immediate subsequent movement of the contact on the touch-sensitive display; in response to detecting the immediate subsequent movement, displays an array of keys including alternate keys for the respective default key; detects liftoff of the contact at a second location on the touch-sensitive display; and, in response to detecting the liftoff, when the second location corresponds to an activation region associated with a respective alternate key, ceases to display the array and inserts a character associated with the respective alternate key into the text entry region.
 The disclosed embodiments relate generally to electronic devices with touch- sensitive surfaces, and more particularly, to electronic devices with touch screen displays that receive input for activating the displaying of alternate keys associated with a default key.
 The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary touch- sensitive surfaces include touch pads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on a display.
 Exemplary manipulations include entering characters using one or more soft keyboards. A user may need to perform such manipulations on user interface objects in any program or application where character input is needed.
 But existing methods for using such soft keyboards are cumbersome and inefficient. For example, accessing alternate keys for a default key on a soft keyboard is tedious and creates a significant cognitive burden on a user. In addition, some conventional methods take longer than necessary to complete a task, thereby wasting a user's time and a device's power reserve, which can be particularly important consideration for battery-operated devices.
 Accordingly, there is a need for computing devices with faster, more efficient methods and interfaces for accessing alternate keys for a default key on a soft keyboard. Such methods and interfaces may complement or replace conventional methods for accessing alternate keys for a default key on a soft keyboard. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated computing devices, such methods and interfaces conserve power and increase the time between battery charges.  The above deficiencies and other problems associated with user interfaces for computing devices with touch-sensitive displays (also known as a "touch screen" or "touch screen display") are reduced or eliminated by the disclosed devices. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, tablet computer, or handheld device). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory and one or more modules, programs or sets of instructions stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI primarily through finger contacts and gestures on the touch-sensitive display. In some embodiments, the functions may include image editing, drawing, presenting, word processing, website creating, disk authoring, spreadsheet making, game playing, telephoning, video conferencing, e-mailing, instant messaging, workout support, digital photographing, digital videoing, web browsing, digital music playing, and/or digital video playing. Executable instructions for performing these functions may be included in a computer readable storage medium or other computer program product configured for execution by one or more processors.
 In accordance with some embodiments, a method is performed at a multifunction device with a touch-sensitive display. The method includes: displaying a text entry region and a soft keyboard having a plurality of default keys; detecting a contact at a first location on the touch-sensitive display that corresponds to an activation region associated with a respective default key, wherein the respective default key has a plurality of alternate keys, and detecting an immediate subsequent movement of the contact on the touch- sensitive display; in response to detecting the immediate subsequent movement of the contact, while continuing to display the plurality of default keys of the soft keyboard, displaying an array of keys including the plurality of alternate keys for the respective default key, each key in the array having an associated activation region; detecting liftoff of the contact at a second location on the touch-sensitive display; and in response to detecting the liftoff of the contact: when the second location corresponds to an activation region that is associated with a respective alternate key in the array, ceasing to display the array and inserting a character associated with the respective alternate key into the text entry region; and when the second location corresponds to a region of the touch-sensitive display that is not an activation region for any of the keys in the array, ceasing to display the array without inserting any character into the text entry region.  In accordance with some embodiments, a multifunction device includes a touch-sensitive display, one or more processors, memory, and one or more programs; the one or more programs are stored in the memory and configured to be executed by the one or more processors and the one or more programs include instructions for performing the operations of the method described above. In accordance with some embodiments, a graphical user interface on a multifunction device with a touch-sensitive display, a memory, and one or more processors to execute one or more programs stored in the memory includes one or more of the elements displayed in the method described above, which are updated in response to inputs, as described in the method above. In accordance with some embodiments, a computer readable storage medium has stored therein instructions which when executed by a multifunction device with a touch-sensitive display, cause the device to perform the operations of the method described above. In accordance with some embodiments, a multifunction device includes: a touch-sensitive display; and means for performing the operations of the method described above. In accordance with some embodiments, an information processing apparatus, for use in a multifunction device with a touch-sensitive display, includes means for performing the operations of the method described above.
 In accordance with some embodiments, a multifunction device includes: a touch-sensitive display unit configured to display a text entry region and a soft keyboard having a plurality of default keys and configured to collect touch-sensitive input, and a processing unit coupled to the touch-sensitive display unit to enable the touch-sensitive display unit to display and to receive the input from the touch-sensitive display unit. The processing unit is configured to: detect a contact at a first location on the touch-sensitive display unit that corresponds to an activation region associated with a respective default key, wherein the respective default key has a plurality of alternate keys, and detect an immediate subsequent movement of the contact on the touch-sensitive display unit; in response to detecting the immediate subsequent movement of the contact, while continuing to display the plurality of default keys of the soft keyboard, enable display of an array of keys including the plurality of alternate keys for the respective default key on the touch-sensitive display unit, each key in the array having an associated activation region; detect liftoff of the contact at a second location on the touch-sensitive display unit; and, in response to detecting the liftoff of the contact: when the second location corresponds to an activation region that is associated with a respective alternate key in the array, cease to display the array and insert a character associated with the respective alternate key into the text entry region; and when the second location corresponds to a region of the touch-sensitive display unit that is not an activation region for any of the keys in the array, cease to display the array without inserting any character into the text entry region.
 Thus, multifunction devices with touch screen displays are provided with faster, more efficient methods and interfaces for accessing alternate keys to a default key on a soft keyboard, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces may complement or replace conventional methods for accessing alternate keys to a default key on a soft keyboard.
 Figures 1A and IB are block diagrams illustrating portable multifunction devices with touch-sensitive displays in accordance with some embodiments.
 Figures 4 A and 4B illustrate exemplary user interfaces for a menu of applications on a portable multifunction device in accordance with some embodiments.
 Figures 5A-5I illustrate exemplary user interfaces for accessing alternate keys in accordance with some embodiments.
 Figures 6A-6C are flow diagrams illustrating a method of accessing alternate keys in accordance with some embodiments.
"in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" may be construed to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]," depending on the context.  As used herein, the term "resolution" of a display refers to the number of pixels (also called "pixel counts" or "pixel resolution") along each axis or in each dimension of the display. For example, a display may have a resolution of 320x480 pixels. Furthermore, as used herein, the term "resolution" of a multifunction device refers to the resolution of a display in the multifunction device. The term "resolution" does not imply any limitations on the size of each pixel or the spacing of pixels. For example, compared to a first display with a 1024x768-pixel resolution, a second display with a 320x480-pixel resolution has a lower resolution. However, it should be noted that the physical size of a display depends not only on the pixel resolution, but also on many other factors, including the pixel size and the spacing of pixels. Therefore, the first display may have the same, smaller, or larger physical size, compared to the second display.
 As used herein, the term "video resolution" of a display refers to the density of pixels along each axis or in each dimension of the display. The video resolution is often measured in a dots-per-inch (DPI) unit, which counts the number of pixels that can be placed in a line within the span of one inch along a respective dimension of the display.
 Embodiments of computing devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the computing device is a portable communications device, such as a mobile telephone, that also contains other functions, such as PDA and/or music player functions. Exemplary embodiments of portable multifunction devices include, without limitation, the iPhone® and iPod Touch® devices from Apple Inc. of Cupertino, California. Other portable devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and/or touch pads), may also be used. It should also be understood that, in some embodiments, the device is not a portable communications device, but is a desktop computer with a touch-sensitive surface (e.g., a touch screen display and/or a touch pad).
 The user interfaces may include one or more soft keyboard embodiments.
The soft keyboard embodiments may include standard (QWERTY) and/or non-standard configurations of symbols on the displayed icons of the keyboard, such as those described in U.S. Patent Applications 11/459,606, "Keyboards For Portable Electronic Devices," filed July 24, 2006, and 11/459,615, "Touch Screen Keyboards For Portable Electronic Devices," filed July 24, 2006, the contents of which are hereby incorporated by reference in their entireties. The keyboard embodiments may include a reduced number of icons (or soft keys) relative to the number of keys in existing physical keyboards, such as that for a typewriter. This may make it easier for users to select one or more icons in the keyboard, and thus, one or more corresponding symbols. The keyboard embodiments may be adaptive. For example, displayed icons may be modified in accordance with user actions, such as selecting one or more icons and/or one or more corresponding symbols. One or more applications on the device may utilize common and/or different keyboard embodiments. Thus, the keyboard embodiment used may be tailored to at least some of the applications. In some embodiments, one or more keyboard embodiments may be tailored to a respective user. For example, one or more keyboard embodiments may be tailored to a respective user based on a word usage history (lexicography, slang, individual usage) of the respective user. Some of the keyboard embodiments may be adjusted to reduce a probability of a user error when selecting one or more icons, and thus one or more symbols, when using the soft keyboard embodiments.
 Attention is now directed toward embodiments of portable devices with touch- sensitive displays. Figures 1A and IB are block diagrams illustrating portable multifunction devices 100 with touch-sensitive displays 112 in accordance with some embodiments. Touch-sensitive display 112 is sometimes called a "touch screen" for convenience, and may also be known as or called a touch-sensitive display system. Device 100 may include memory 102 (which may include one or more computer readable storage mediums), memory controller 122, one or more processing units (CPU's) 120, peripherals interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input/output (I/O) subsystem 106, other input or control devices 116, and external port 124. Device 100 may include one or more optical sensors 164. These components may communicate over one or more communication buses or signal lines 103.
 RF (radio frequency) circuitry 108 receives and sends RF signals, also called electromagnetic signals. RF circuitry 108 converts electrical signals to/from electromagnetic signals and communicates with communications networks and other communications devices via the electromagnetic signals. RF circuitry 108 may include well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and so forth. RF circuitry 108 may communicate with networks, such as the Internet, also referred to as the World Wide Web (WWW), an intranet and/or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and/or a metropolitan area network (MAN), and other devices by wireless communication. The wireless communication may use any of a plurality of communications standards, protocols and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), wideband code division multiple access (W- CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.1 lg and/or IEEE 802.11η), voice over Internet Protocol (VoIP), Wi-MAX, a protocol for e-mail (e.g., Internet message access protocol (IMAP) and/or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.
 Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between a user and device 100. Audio circuitry 110 receives audio data from peripherals interface 118, converts the audio data to an electrical signal, and transmits the electrical signal to speaker 111. Speaker 111 converts the electrical signal to human-audible sound waves. Audio circuitry 110 also receives electrical signals converted by microphone 113 from sound waves. Audio circuitry 110 converts the electrical signal to audio data and transmits the audio data to peripherals interface 118 for processing. Audio data may be retrieved from and/or transmitted to memory 102 and/or RF circuitry 108 by peripherals interface 118. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., 212, Figure 2). The headset jack provides an interface between audio circuitry 110 and removable audio input/output peripherals, such as output-only headphones or a headset with both output (e.g., a headphone for one or both ears) and input (e.g., a microphone).  I/O subsystem 106 couples input/output peripherals on device 100, such as touch screen 112 and other input control devices 116, to peripherals interface 118. I/O subsystem 106 may include display controller 156 and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive/send electrical signals from/to other input or control devices 116. The other input control devices 116 may include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, and so forth. In some alternate embodiments, input controller(s) 160 may be coupled to any (or none) of the following: a keyboard, infrared port, USB port, and a pointer device such as a mouse. The one or more buttons (e.g., 208, Figure 2) may include an up/down button for volume control of speaker 111 and/or microphone 113. The one or more buttons may include a push button (e.g., 206, Figure 2). A quick press of the push button may disengage a lock of touch screen 112 or begin a process that uses gestures on the touch screen to unlock the device, as described in U.S. Patent Application 11/322,549, "Unlocking a Device by Performing Gestures on an Unlock Image," filed December 23, 2005, which is hereby incorporated by reference in its entirety. A longer press of the push button (e.g., 206) may turn power to device 100 on or off. The user may be able to customize a functionality of one or more of the buttons. Touch screen 112 is used to implement virtual or soft buttons and one or more soft keyboards.
(light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies may be used in other embodiments. Touch screen 112 and display controller 156 may detect contact and any movement or breaking thereof using any of a plurality of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch screen 112. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that found in the iPhone® and iPod Touch® from Apple Inc. of Cupertino, California.
 A touch-sensitive display in some embodiments of touch screen 112 may be analogous to the multi-touch sensitive touchpads described in the following U.S. Patents: 6,323,846 (Westerman et al), 6,570,557 (Westerman et al), and/or 6,677,932 (Westerman), and/or U.S. Patent Publication 2002/0015024A1, each of which is hereby incorporated by reference in its entirety. However, touch screen 112 displays visual output from portable device 100, whereas touch sensitive touchpads do not provide visual output.
 A touch-sensitive display in some embodiments of touch screen 112 may be as described in the following applications: (1) U.S. Patent Application No. 11/381,313, "Multipoint Touch Surface Controller," filed May 2, 2006; (2) U.S. Patent Application No. 10/840,862, "Multipoint Touchscreen," filed May 6, 2004; (3) U.S. Patent Application No. 10/903,964, "Gestures For Touch Sensitive Input Devices," filed July 30, 2004; (4) U.S. Patent Application No. 11/048,264, "Gestures For Touch Sensitive Input Devices," filed January 31, 2005; (5) U.S. Patent Application No. 11/038,590, "Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices," filed January 18, 2005; (6) U.S. Patent Application No. 11/228,758, "Virtual Input Device Placement On A Touch Screen User Interface," filed September 16, 2005; (7) U.S. Patent Application No. 11/228,700, "Operation Of A Computer With A Touch Screen Interface," filed September 16, 2005; (8) U.S. Patent Application No. 11/228,737, "Activating Virtual Keys Of A Touch-Screen Virtual Keyboard," filed September 16, 2005; and (9) U.S. Patent Application No. 11/367,749, "Multi-Functional Hand-Held Device," filed March 3, 2006. All of these applications are incorporated by reference herein in their entirety.  Touch screen 112 may have a video resolution in excess of 100 dpi. In some embodiments, the touch screen has a video resolution of approximately 160 dpi. The user may make contact with touch screen 112 using any suitable object or appendage, such as a stylus, a finger, and so forth. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures, which can be less precise than stylus-based input due to the larger area of contact of a finger on the touch screen. In some embodiments, the device translates the rough finger-based input into a precise pointer/cursor position or command for performing the actions desired by the user.
(e.g., a click wheel) as input control device 116. A user may navigate among and interact with one or more graphical objects (e.g., icons) displayed in touch screen 112 by rotating the click wheel or by moving a point of contact with the click wheel (e.g., where the amount of movement of the point of contact is measured by its angular displacement with respect to a center point of the click wheel). The click wheel may also be used to select one or more of the displayed icons. For example, the user may press down on at least a portion of the click wheel or an associated button. User commands and navigation commands provided by the user via the click wheel may be processed by input controller 160 as well as one or more of the modules and/or sets of instructions in memory 102. For a virtual click wheel, the click wheel and click wheel controller may be part of touch screen 112 and display controller 156, respectively. For a virtual click wheel, the click wheel may be either an opaque or semitransparent object that appears and disappears on the touch screen display in response to user interaction with the device. In some embodiments, a virtual click wheel is displayed on the touch screen of a portable multifunction device and operated by user contact with the touch screen.
 Device 100 may also include one or more proximity sensors 166. Figures 1A and IB show proximity sensor 166 coupled to peripherals interface 118. Alternately, proximity sensor 166 may be coupled to input controller 160 in I/O subsystem 106. Proximity sensor 166 may perform as described in U.S. Patent Application Nos. 11/241,839, "Proximity Detector In Handheld Device"; 11/240,788, "Proximity Detector In Handheld Device"; 11/620,702, "Using Ambient Light Sensor To Augment Proximity Sensor Output"; 11/586,862, "Automated Response To And Sensing Of User Activity In Portable Devices"; and 11/638,251, "Methods And Systems For Automatic Configuration Of Peripherals," which are hereby incorporated by reference in their entirety. In some embodiments, the proximity sensor turns off and disables touch screen 112 when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call).
 Device 100 may also include one or more accelerometers 168. Figures 1A and IB show accelerometer 168 coupled to peripherals interface 118. Alternately, accelerometer 168 may be coupled to an input controller 160 in I/O subsystem 106. Accelerometer 168 may perform as described in U.S. Patent Publication No. 20050190059, "Acceleration-based Theft Detection System for Portable Electronic Devices," and U.S. Patent Publication No. 20060017692, "Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer," both of which are which are incorporated by reference herein in their entirety. In some embodiments, information is displayed on the touch screen display in a portrait view or a landscape view based on an analysis of data received from the one or more accelerometers. Device 100 optionally includes, in addition to accelerometer(s) 168, a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information concerning the location and orientation (e.g., portrait or landscape) of device 100.
 In some embodiments, the software components stored in memory 102 include operating system 126, communication module (or set of instructions) 128, contact/motion module (or set of instructions) 130, graphics module (or set of instructions) 132, text input module (or set of instructions) 134, Global Positioning System (GPS) module (or set of instructions) 135, and applications (or sets of instructions) 136. Furthermore, in some embodiments memory 102 stores device/global internal state 157, as shown in Figures 1A, IB and 3. Device/global internal state 157 includes one or more of: active application state, indicating which applications, if any, are currently active; display state, indicating what applications, views or other information occupy various regions of touch screen display 112; sensor state, including information obtained from the device's various sensors and input control devices 116; and location information concerning the device's location and/or attitude.
WINDOWS, or an embedded operating system such as Vx Works) includes various software components and/or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.
• online video module 155.
130, graphics module 132, and text input module 134, contacts module 137 may be used to manage an address book or contact list (e.g., stored in application internal state 192 of contacts module 137 in memory 102 or memory 370), including: adding name(s) to the address book; deleting name(s) from the address book; associating telephone number(s), e- mail address(es), physical address(es) or other information with a name; associating an image with a name; categorizing and sorting names; providing telephone numbers or e-mail addresses to initiate and/or facilitate communications by telephone 138, video conference 139, e-mail 140, or IM 141; and so forth.
164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions to capture still images or video (including a video stream) and store them into memory 102, modify characteristics of a still image or video, or delete a still image or video from memory 102.
130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions to arrange, modify (e.g., edit), or otherwise manipulate, label, delete, present (e.g., in a digital slide show or album), and store still and/or video images.
130, graphics module 132, audio circuitry 110, and speaker 111, video player module 145 includes executable instructions to display, present or otherwise play back videos (e.g., on touch screen 112 or on an external, connected display via external port 124).
130, graphics module 132, and text input module 134, notes module 153 includes executable instructions to create and manage notes, to do lists, and the like in accordance with user instructions.
 In conjunction with RF circuitry 108, touch screen 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 may be used to receive, display, modify, and store maps and data associated with maps (e.g., driving directions; data on stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.  In conjunction with touch screen 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, e-mail client module 140, and browser module 147, online video module 155 includes instructions that allow the user to access, browse, receive (e.g., by streaming and/or download), play back (e.g., on the touch screen or on an external, connected display via external port 124), send an e-mail with a link to a particular online video, and otherwise manage online videos in one or more file formats, such as H.264. In some embodiments, instant messaging module 141, rather than e-mail client module 140, is used to send a link to a particular online video. Additional description of the online video application can be found in U.S. Provisional Patent Application No. 60/936,562, "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos," filed June 20, 2007, and U.S. Patent Application No. 11/968,067, "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos," filed December 31, 2007, the content of which is hereby incorporated by reference in its entirety.
 Figure 1C is a block diagram illustrating exemplary components for event handling in accordance with some embodiments. In some embodiments, memory 102 (in Figures 1A and IB) or 370 (Figure 3) includes event sorter 170 (e.g., in operating system 126) and a respective application 136-1 (e.g., any of the aforementioned applications 137-151, 155, 380-390).
information. Event sorter 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates the current application view(s) displayed on touch sensitive display 112 when the application is active or executing. In some embodiments, device/global internal state 157 is used by event sorter 170 to determine which application(s) is(are) currently active, and application internal state 192 is used by event sorter 170 to determine application views 191 to which to deliver event information.
 Event monitor 171 receives event information from peripherals interface 118.
Event information includes information about a sub-event (e.g., a user touch on touch- sensitive display 112, as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I/O subsystem 106 or a sensor, such as proximity sensor 166, accelerometer(s) 168, and/or microphone 113 (through audio circuitry 110). Information that peripherals interface 118 receives from I/O subsystem 106 includes information from touch-sensitive display 112 or a touch-sensitive surface.  In some embodiments, event monitor 171 sends requests to the peripherals interface 118 at predetermined intervals. In response, peripherals interface 118 transmits event information. In other embodiments, peripheral interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and/or for more than a predetermined duration).
 In some embodiments, operating system 126 includes event sorter 170.
Alternatively, application 136-1 includes event sorter 170. In yet other embodiments, event sorter 170 is a stand-alone module, or a part of another module stored in memory 102, such as contact/motion module 130.
176, object updater 177 or GUI updater 178 to update the application internal state 192.
177, and GUI updater 178 are included in a respective application view 191.
179) from event sorter 170, and identifies an event from the event information. Event recognizer 180 includes event receiver 182 and event comparator 184. In some embodiments, event recognizer 180 also includes at least a subset of: metadata 183, and event delivery instructions 188 (which may include sub-event delivery instructions).
information includes corresponding information about the current orientation (also called device attitude) of the device.
 Event comparator 184 compares the event information to predefined event or sub-event definitions and, based on the comparison, determines an event or sub-event, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definitions 186. Event definitions 186 contain definitions of events (e.g., predefined sequences of sub-events), for example, event 1 (187-1), event 2 (187- 2), and others. In some embodiments, sub-events in an event 187 include, for example, touch begin, touch end, touch movement, touch cancellation, and multiple touching. In one example, the definition for event 1 (187-1) is a double tap on a displayed object. The double tap, for example, comprises a first touch (touch begin) on the displayed object for a predetermined phase, a first lift-off (touch end) for a predetermined phase, a second touch (touch begin) on the displayed object for a predetermined phase, and a second lift-off (touch end) for a predetermined phase. In another example, the definition for event 2 (187-2) is a dragging on a displayed object. The dragging, for example, comprises a touch (or contact) on the displayed object for a predetermined phase, a movement of the touch across touch- sensitive display 112, and lift-off of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.
 When a respective event recognizer 180 determines that the series of sub- events do not match any of the events in event definitions 186, the respective event recognizer 180 enters an event impossible, event failed, or event ended state, after which it disregards subsequent sub-events of the touch-based gesture. In this situation, other event recognizers, if any, that remain active for the hit view continue to track and process sub- events of an ongoing touch-based gesture.
183 with configurable properties, flags, and/or lists that indicate how the event delivery system should perform sub-event delivery to actively involved event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and/or lists that indicate how event recognizers may interact with one another. In some embodiments, metadata 183 includes configurable properties, flags, and/or lists that indicate whether sub-events are delivered to varying levels in the view or programmatic hierarchy.
corresponding to sub-events which define an event to be recognized.
112 in accordance with some embodiments. The touch screen may display one or more graphics within user interface (UI) 200. In this embodiment, as well as others described below, a user may select one or more of the graphics by making contact or touching the graphics, for example, with one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with the one or more graphics. In some embodiments, the contact may include a gesture, such as one or more taps, one or more swipes (from left to right, right to left, upward and/or downward) and/or a rolling of a finger (from right to left, left to right, upward and/or downward) that has made contact with device 100. In some embodiments, inadvertent contact with a graphic may not select the graphic. For example, a swipe gesture that sweeps over an application icon may not select the corresponding application when the gesture corresponding to selection is a tap.
• Online video module 155, also referred to as YouTube (trademark of Google Inc.) module 155.
 Figures 5A-5G illustrate exemplary user interfaces for accessing alternate keys for a default key in accordance with some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 6A-6C.
 In Figures 5A-5I, some finger contact movement sizes may be exaggerated for illustrative purposes. No depiction in the figures bearing on finger contact movements should be taken as a requirement or limitation for the purpose of understanding sizes and scale associated with the methods and devices disclosed herein.
 UI 500 A (Figure 5 A) depicts an exemplary text input user interface for a notes application (e.g., notes module 153) on the touch screen 112 of device 100. In this example, the UI 500 A includes a text entry region 502 and a QWERTY soft keyboard 504 below the text entry region 502. The text entry region 502 may include input text 506 with text 506-1 and a cursor 508 at position 508-A. The soft keyboard 504 includes one or more soft keys 510 that are default for the soft keyboard 504. Some of the soft keys 510 are associated with a respective Latin letter. For example, soft key 510-A is associated with the letter "E." One or more of the soft keys 510 may be associated with one or more respective alternate keys. For example, a key 510 associated with a letter can have alternate keys associated with that letter with respective diacritical marks.
 UI 500B (Figure 5B) illustrates a contact 512 on the touch screen 112. In this example, the contact 512 is at a location 512- A on the touch screen 112 corresponding to the location of soft key 510-A, which also corresponds to the activation region of soft key 510-A. In some embodiments, the activation region of a soft key 510 is the area indicated by the periphery of the key 510 as displayed on the touch screen 112.
512 (e.g., upward) in the UI on touch screen 112, from location 512-A to location 512-B. During the movement, contact 512 remains in contact with the touch screen 112. In this example, the movement of the contact follows an approximately straight line.
 UI 500D (Figure 5D) illustrates a pause of the movement of the contact 512 at location 512-B. During the pause, the contact 512 remains in contact with the touch screen 112. UI 500D also illustrates a plurality of alternate keys 514 for key 510-A being displayed on the touch screen 112 in response to the movement of contact 512 to location 512-B. The alternate keys 514 are displayed as an array (e.g., a horizontal array, a vertical array, or a two-dimensional array) of soft keys in the vicinity of key 510-A. The alternate keys 514 include a key 514-A for the original letter "E" associated with key 510-A and keys 514-B to 514-F for the letter "E" with respective diacritical marks.
 In some embodiments, the alternate keys 514-A thru 514-F are displayed within the array in an ordering or positioning that is determined based at least in part on the language with which the soft keyboard 504 is associated. For example, the ordering or positioning can be based at least in part on which diacritical marks (or the absence of a diacritical mark) are most commonly added to the letter "E" within the language; the respective alternate keys associated with the absence of a diacritical mark (i.e., the original unmodified letter "E" associated with key 510-A) and with the commonly added diacritical marks or for the letter "E" are displayed closer to the default key 510-A. Further, in some embodiments, the ordering or positioning can be determined based at least in further part on the usage frequencies of the letter "E" with and without diacritical marks, with the keys associated with the diacritical mark variations of the letter "E" that are entered more often by the user displayed closer to the default key 510-A.
 UI 500E (Figure 5E) illustrates movement of contact 512 across the touch-screen 112, from location 512-B to a location 512-C corresponding to the location of alternate key 514-B, which corresponds to the activation region of the alternate key 514-B.
corresponding to the letter "E" with a macron is input into input text 506 at cursor location 508-A, and the cursor 508 advances to another location.
 UI 500F (Figure 5F) illustrates a result of the breaking of the contact 512 from touch screen 112 at location 512-C. The alternate keys 514 ceases to be displayed on the touch screen 112. A character 516 corresponding to the letter "E" with a macron is input into input text 506 (text 506-1 changes to text 506-2), and cursor 508 advances to location 508-B. In some embodiments, in response to detecting a gesture that corresponds to a desired accent on a respective default key for a character - e.g., an upward gesture to the right for a rightward accent or an upward gesture to the left for a leftward accent - the character with the corresponding accent is input, independent of whether the alternate keys 514 are displayed.
 UI 500G (Figure 5G) illustrates movement of contact 512 across the touch screen 112 from location 512-B to a location 512-D that does not correspond to a location of any of the alternate keys 514. If contact 512 is broken from touch screen 112 at location 512-D, alternate keys 514 cease to be displayed and no new character is inputted into input text 506 (i.e., text 506-1 remain as is); UI 500G changes back to UI 500A.
512 downward in the UI on touch screen 112, from location 512-A to location 512-E.
 UI 5001 (Figure 51) illustrates the hiding of keyboard 504. In response to the immediate subsequent downward movement, the keyboard 504 is hidden (i.e., ceases to be displayed) while text entry region 502 remains displayed. In some embodiments, the keyboard 504 is "unhidden" in response to a contact anywhere on the touch screen 112 while the keyboard 504 is hidden.
 Figures 6A-6C are flow diagrams illustrating a method 600 of accessing alternate keys for a default key in accordance with some embodiments. The method 600 is performed at a multifunction device (e.g., device 300, Figure 3, or portable multifunction device 100, Figure 1) with a touch screen display and the touch-sensitive surface is on the display. Some operations in method 600 may be combined and/or the order of some operations may be changed.  As described below, the method 600 provides an intuitive way to access alternate keys for a default key. The method reduces the cognitive burden on a user when accessing alternate keys for a default key, thereby creating a more efficient human-machine interface. For battery-operated computing devices, enabling a user to access alternate keys for a default key faster and more efficiently conserves power and increases the time between battery charges.
 The device displays (602) a text entry region and a soft keyboard having a plurality of default keys. For example, in Figure 5A, UI 500A includes a text entry region 502 and a QWERTY soft keyboard 504 below the text entry region 502. The QWERTY soft keyboard 504 includes a plurality of keys 510 that are default for the QWERTY keyboard 504. The default keys 510 include keys associated with the Latin alphabet without any diacritical marks. For a non-QWERTY keyboard, the set of default keys may be different.
 In some embodiments, more than one soft keyboard is available on a device.
For example, a device can have different keyboards for different languages and/or input methods, each keyboard with its own set of default keys.
 In some embodiments, a keyboard is associated with multiple languages, with the available alternate keys differing by language. For example, a QWERTY keyboard can be used for multiple languages, but the set of available alternate keys for the QWERTY keyboard can differ by the language for which the QWERTY keyboard is used at the moment.
 Each key 510 has a respective activation region; in some embodiments, the periphery of the respective key 510 indicates its respective activation region.
 The device detects (604) a contact at a first location on the touch-sensitive display that corresponds to an activation region associated with a respective default key, wherein the respective default key has a plurality of alternate keys, and detects an immediate subsequent movement (e.g., upward movement) of the contact on the touch-sensitive display. For example, in Figure 5B, contact 512 is detected at location 512-A, which corresponds to the activation region of key 510-A. In Figure 5C, immediate upward movement of contact 512 from location 512-A to location 512-B is detected.
 As used in this specification, "immediate" movement of a contact from one location to another is movement of the contact that is initiated within a short predefined period of time from the initial contact, wherein the predefined period of time has a duration of at least 0.2 seconds and no more than 1.5 seconds (e.g., within 0.3, 0.4, 0.5, 0.6, 0.75 or 1.0 seconds).
 The device, in response to detecting the immediate subsequent movement of the contact, while continuing to display the plurality of default keys of the soft keyboard, displays (606) an array of keys including the plurality of alternate keys for the respective default key, where each key in the array has an associated activation region. For example, in Figure 5D, an array of alternate keys 514 for default key 510-A is displayed in response to detection of the immediate subsequent upward movement of the contact 512 from location 512-A to location 512-B, while the default keys 510 continue to be displayed. Each alternate key 514 has a respective activation region; in some embodiments, the periphery of a respective key 514 indicates its respective activation region.
 In some embodiments, the device displays (608) the plurality of alternate keys in an order that is determined based at least in part on a language associated with the soft keyboard. For example, in Figure 5D, the keyboard 504 can be associated with a language, and the alternate keys 514-A thru 514-F can be displayed in an ordering or positioning that is at least based on which diacritics (or no diacritic) are most commonly added within that language to the letter "E" associated with default key 510-A; the keys associated with no diacritic and the more commonly added diacritics are displayed closer to the default key 510- A.
 In some embodiments, the order or positions in which the alternate keys are displayed within the array is determined base at least in further part on respective usage frequencies of the alternate keys (610). For example, in Figure 5D, if the user has accumulated a history at the device of entering the letter "E" with the macron (using key 514- B) more often than entering the letter "E" with the breve (using key 514-C), then key 514-B is displayed closer to default key 510-A than key 514-C.
 The device detects a liftoff of the contact at a second location on the touch screen display (612). For example, in Figures 5E and 5G, the contact is moved from location 512-B to location 512-C and location 512-D, respectively. At locations 512-C or 512-D, a liftoff or breaking of the contact can occur (e.g., by the user moving their finger away from the touch screen 112).  In response to detecting the liftoff of the contact (613), when the second location corresponds to an activation region that is associated with a respective alternate key in the array, the devices ceases displaying (614) the array and inserts a character associated with the respective alternate key into the text entry region. For example, in Figure 5E, location 512-C is at a location that corresponds to the activation region of alternate key 514-B, which is associated with the letter "E" with a macron. In response to a liftoff of the contact at location 512-C, the array of alternate keys 514 ceases to be displayed and a character representing the letter "E" with a macron is inserted as character 516 into input text 506 in text entry region 502 (text 506-1 changes to text 506-2), as illustrated in Figure 5F.
 In response to detecting the liftoff of the contact (613), when the second location corresponds to a region of the touch-sensitive display that is not an activation region for any of the keys in the array, the device ceases displaying (616) the array without inserting any character into the text entry region. For example, in Figure 5G, location 512-D is at a location that does not correspond to the activation region of any of the alternate keys 514. In response to a liftoff of the contact at location 512-D, the array of alternate keys 514 ceases to be displayed and no letter is inserted into the input text 506, as illustrated in Figure 5A.
 In some embodiments, the soft keyboard can be hidden using a movement of the contact opposite of the movement that results in the displaying of the array of alternate keys. For example, the device detects (618) a contact at a location on the touch-sensitive display that corresponds to an activation region associated with a respective default key, and detects an immediate subsequent downward movement of the contact on the touch-sensitive display. For example, in Figure 5B, contact 512 is detected at a location 512-A corresponding to the activation region of key 510-A. In Figure 5H, an immediate subsequent movement of the contact 512 downward from location 512-A to location 512-D is detected. In Figure 51, in response to detection of the immediate subsequent downward movement, the keyboard 504 is hidden (620) (i.e., ceases to be displayed).
 In accordance with some embodiments, Figure 7 shows a functional block diagram of a multifunction device 700 configured in accordance with the principles of the invention as described above. The functional blocks of the device may be implemented by hardware, software, or a combination of hardware and software to carry out the principles of the invention. It is understood by persons of skill in the art that the functional blocks described in Figure 7 may be combined or separated into sub-blocks to implement the principles of the invention as described above. Therefore, the description herein may support any possible combination or separation or further definition of the functional blocks described herein.
 As shown in Figure 7, a multifunction device 700 includes: a touch-sensitive display unit 702 configured to display a text entry region 704 and a soft keyboard 706 having a plurality of default keys and configured to collect touch-sensitive input, and a processing unit 708 coupled to the touch-sensitive display unit 702 to enable the touch-sensitive display unit 702 to display and to receive the input from the touch-sensitive display unit 702. In some embodiments, the processing unit 708 includes a detecting unit 710, a ceasing unit 712, an inserting unit 714, and a hiding unit 716. The processing unit 708 is configured to: detect a contact at a first location on the touch-sensitive display unit 702 that corresponds to an activation region associated with a respective default key, wherein the respective default key has a plurality of alternate keys, and detect an immediate subsequent movement of the contact on the touch-sensitive display unit 702 (e.g., with the detecting unit 710); in response to detecting the immediate subsequent movement of the contact, while continuing to display the plurality of default keys of the soft keyboard 706, enable display of an array of keys including the plurality of alternate keys for the respective default key on the touch-sensitive display unit 702, each key in the array having an associated activation region; detect liftoff of the contact at a second location on the touch-sensitive display unit 702 (e.g., with the detecting unit 710; and, in response to detecting the liftoff of the contact: when the second location corresponds to an activation region that is associated with a respective alternate key in the array, cease to display the array (e.g., with the ceasing unit 712) and insert a character associated with the respective alternate key into the text entry region 704 (e.g., with the inserting unit 714); and when the second location corresponds to a region of the touch- sensitive display unit 702 that is not an activation region for any of the keys in the array, cease to display the array without inserting any character into the text entry region 704 (e.g., with the ceasing unit 712).
 In some embodiments, the processing unit 708 is further configured to: detect a contact at a location on the touch-sensitive display unit 702 that corresponds to an activation region associated with a respective default key, and detect an immediate subsequent downward movement of the contact on the touch-sensitive display unit 702 (e.g., with the detecting unit 710); and, in response to detecting the immediate subsequent downward movement of the contact, hide the soft keyboard 706 (e.g., with the hiding unit 716).
 In some embodiments, enabling display of the array of keys including the plurality of alternate keys for the respective default key comprises enabling display of the plurality of alternate keys in an order that is determined based at least in part on a language associated with the soft keyboard 706. In some embodiments, the order is determined based at least in further part on respective usage frequencies of the plurality of alternate keys.
 In some embodiments, the array is a horizontal array.
 In some embodiments, the detected immediate subsequent movement is an upward movement of the contact on the touch-sensitive display unit 702.
 The operations in the information processing methods described above may be implemented by running one or more functional modules in information processing apparatus such as general purpose processors or application specific chips. These modules, combinations of these modules, and/or their combination with general hardware (e.g., as described above with respect to Figures 1A, IB and 3) are included within the scope of protection of the invention.
 The operations described above with reference to Figures 6A-6C may be implemented by components depicted in Figures 1A-1C. For example, detection operation 604 and displaying operation 606 may be implemented by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 in event sorter 170 detects a contact on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. A respective event recognizer 180 of application 136-1 compares the event information to respective event definitions 186, and determines whether a first contact at a first location on the touch-sensitive surface (or whether rotation of the device) corresponds to a predefined event or sub-event, such as selection of an object on a user interface, or rotation of the device from one orientation to another. When a respective predefined event or sub-event is detected, event recognizer 180 activates an event handler 180 associated with the detection of the event or sub-event. Event handler 180 may utilize or call data updater 176 or object updater 177 to update the application internal state 192. In some embodiments, event handler 180 accesses a respective GUI updater 178 to update what is displayed by the application. Similarly, it would be clear to a person having ordinary skill in the art how other processes can be implemented based on the components depicted in Figures 1A-1C.
when the second location corresponds to a region of the touch-sensitive display that is not an activation region for any of the keys in the array, ceasing to display the array without inserting any character into the text entry region.
in response to detecting the immediate subsequent downward movement of the contact, hiding the soft keyboard.
3. The device of claim 1 or 2, including instructions for displaying the plurality of alternate keys in an order that is determined based at least in part on a language associated with the soft keyboard.
4. The device of claim 3, wherein the order is determined based at least in further part on respective usage frequencies of the plurality of alternate keys.
5. The device of any of claims 1 to 4, wherein the array is a horizontal array.
6. The device of any of claims 1 to 5, wherein the detected immediate subsequent movement is an upward movement of the contact on the touch-sensitive display.
displaying the plurality of alternate keys in an order that is determined based at least in part on a language associated with the soft keyboard.
10. The method of claim 9, wherein the order is determined based at least in further part on respective usage frequencies of the plurality of alternate keys.
11. The method of any of claims 7-10, wherein the array is a horizontal array.
12. The method of any of claims 7-11, wherein the detected immediate subsequent movement is an upward movement of the contact on the touch-sensitive display.
when the second location corresponds to a region of the touch-sensitive display that is not an activation region for any of the keys in the array, the array ceases to be displayed without any character being inserted into the text entry region.
when the second location corresponds to a region of the touch-sensitive display that is not an activation region for any of the keys in the array, cease displaying the array without inserting any character into the text entry region.
means for, in response to detecting the liftoff of the contact, when the second location corresponds to a region of the touch-sensitive display that is not an activation region for any of the keys in the array, ceasing to display the array without inserting any character into the text entry region.
17. A multifunction device with a touch-sensitive display, the multifunction device adapted to implement the method of any of claims 7-12.
18. A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a multifunction device with a touch-sensitive display, cause the device to perform the method of any of claims 7-12.
means for performing the method of any of claims 7-12.
when the second location corresponds to a region of the touch-sensitive display unit that is not an activation region for any of the keys in the array, cease to display the array without inserting any character into the text entry region.
detect a contact at a location on the touch-sensitive display unit that corresponds to an activation region associated with a respective default key, and detect an immediate subsequent downward movement of the contact on the touch-sensitive display unit; and, in response to detecting the immediate subsequent downward movement of the contact, hide the soft keyboard.
enabling display of the plurality of alternate keys in an order that is determined based at least in part on a language associated with the soft keyboard.
24. The device of claim 23, wherein the order is determined based at least in further part on respective usage frequencies of the plurality of alternate keys.
25. The device of any of claims 21-24, wherein the array is a horizontal array.
26. The device of any of claims 21-25, wherein the detected immediate subsequent movement is an upward movement of the contact on the touch-sensitive display unit.
NL2007722C (en) 2012-07-05 Device, method, and graphical user interface for manipulating soft keyboards.

References: Application No. 11
 Application No. 10
 Application No. 10
 Application No. 11
 Application No. 11
 Application No. 11
 Application No. 11
 Application No. 11
 Application No. 11
 Application No. 60
 Application No. 11