Method and system for using a keyboard overlay with a touch-sensitive display screen

Disclosed is a “keyboard overlay” that sits on top of a touch-sensitive display screen of a computing device. After aligning the overlay on the display screen, the user types on the overlay. When the user presses a key on the overlay, the pressure is transmitted to the display screen below. That pressure is registered by the display screen as a touch. The keyboard overlay is formed to provide tactile finger-position feedback so that a user can keep his fingers oriented properly over the keyboard. The overlay may be opaque with keycap information displayed in the key areas. The overlay may be transparent, allowing a user to see a virtual keyboard painted on the display screen below. The computing device can detect the presence and type of an overlay. Applications may respond differently to different types of overlays. Different applications may be invoked depending upon the type of overlay detected.

TECHNICAL FIELD

The present invention is related generally to touch-sensitive display screens, and, more particularly, to providing feedback to a user of a touch-sensitive display screen.

BACKGROUND OF THE INVENTION

Many computing applications require some amount of data entry. Some applications call for only a very limited number of characters, such as when a user enters a password or a PIN. Other applications, for example word processing or e-mail, require the user to enter extended amounts of data. For these latter applications, the keyboard reigns as the supreme data-entry device. Its design has been fashioned over more than a century to take advantage of people's nature manual dexterity. Today, typing on a keyboard is a common skill, and its supporting hardware and software are standardized and cheap.

Recently, small portable computing devices that support some form of data entry have become common. Such devices, typically smaller than a laptop computer, include, for example, cellular telephones, two-way pagers, and personal digital assistants. Often, these devices include a touch-sensitive display screen that serves both to display output from the computing device to its user and to receive input from the user. For some applications, the user “writes” with a stylus on the screen. The user's handwriting is decoded and becomes input to the computing device. In other applications, the user's input options are displayed as control icons on the screen. When the user selects an option by touching the icon associated with the option, the computing device detects the location of the touch and sends a message to the application or utility that presented the icon.

These devices often do not include a keyboard. To enter text, a “virtual keyboard,” typically a set of icons that look like the keycaps of a traditional keyboard, are painted on the screen. The user “types” by successively touching areas on the screen associated with specific keycap icons. This method works well for applications that require minimal data entry and where speed of entry is not a concern.

However, advancing data processing and communications technologies are enabling these small portable devices to support more sophisticated applications, specifically applications that call for extended data entry. As one interesting example, consider a recently introduced tablet-like detachable monitor supported by a host computing device, the host typically a personal computer (PC) sitting in a fixed location. The tablet has a touch-sensitive display screen. The tablet, once detached from the host, communicates wirelessly with the host and operates as a portable input/output device. A user carries the tablet around an office or home, using the tablet to gain access to applications running on the fixed-location host. Some of these applications, for example e-mail, word processing, and Web browsing, require extended text entry.

As experience with this tablet and with other increasingly capable portable devices has hinted, extensive data entry would be facilitated by a more robust data-entry mechanism than a stylus (or finger) on a virtual keyboard. Extensive typing on a virtual keyboard is a slow and tedious process, partly because a user must continually correct the position of his fingers over the keycap icons. A traditional hardware keyboard provides finger-positioning feedback via the indented surfaces of the keys. Touch-sensitive display screens are flat to allow good viewing, but their flatness does not provide such tactile feedback. As another hindrance to quick typing, these screens are also quite rigid with essentially no “give” to tell the user that a virtual key has been pressed.

Several attempts have been made to add a hardware keyboard to a small portable device, but none of these attempts has led to a satisfactory mechanism for extended data entry. One problem lies in the size of the hardware keyboard: full-size keyboards are cumbersome to carry around, detracting from the very portability that defines these devices, while smaller keyboards, useful for limited data-entry applications, do not comfortably accommodate the human hand to allow for rapid and extended typing.

What is needed is a way to make a touch-sensitive display screen into a more acceptable extended data-entry device. The utility of such a device would not be limited to portable display devices, but would enhance the experience of entering data on any touch-sensitive display screen.

SUMMARY OF THE INVENTION

In view of the foregoing, the present invention provides a “keyboard overlay” that sits on top of a touch-sensitive display screen of a computing device. After aligning the keyboard overlay on the display screen, the user types on the keyboard overlay. When the user presses a key on the keyboard overlay, the pressure is transmitted to the touch-sensitive display screen below. That pressure is registered by the display screen as a touch. The keyboard overlay is formed to provide tactile finger-position feedback so that a user can keep his fingers oriented properly over the keyboard. Some embodiments additionally supply feedback when the user presses a key hard enough to register on the display screen. The keyboard overlay, in combination with the touch-sensitive display screen, allows the user to type almost as conveniently and as quickly as on a traditional hardware keyboard.

In some embodiments, the keyboard overlay is formed from an opaque, rubbery plastic. Keycap information is displayed in the key areas of the overlay. In other embodiments, the overlay is transparent, allowing a user to see a virtual keyboard painted on the touch-sensitive display screen below. Some applications may blend the virtual keyboard display with other display information below the keyboard overlay.

When not in use, the keyboard overlay is removed from the touch-sensitive display screen. Flexible embodiments of the keyboard overlay may be rolled up, and rigid embodiments may be stored on a pocket of the computing device.

The touch-sensitive display screen can continue to operate as it always has, and the computing device need not even be aware of the presence of the keyboard overlay. If, however, the computing device becomes aware of the presence of the keyboard overlay, then it can modify its behavior accordingly. For example, an application running on the computing device can switch to a text-entry mode when a keyboard overlay is detected. As part of the switch, the application can paint a virtual keyboard under the keyboard overlay that matches the size and key positions of the overlay. The application moves other display information to parts of the screen not covered by the overlay.

In some embodiments, the computing device knows not only that a keyboard overlay is present, but also knows the type of the overlay. In one embodiment, the keyboard overlay contains active or passive electronic components (for example, wire jumpers, resistors, or even an electronic chip) that are powered by the computing device when the overlay is put in place. The computing device queries the electronic components to know the type of the overlay. Applications may respond differently to different types of overlays. Further, different applications may be invoked depending upon the type of overlay detected. For example, an overlay that looks the keypad of a calculator may bring up a calculator application designed to work with that keypad.

Some embodiments of the keyboard overlay incorporate a rigid frame in addition to soft plastic key areas. The frame serves to align the keyboard overlay with respect to the touch-sensitive display screen and physically isolates each key area from its neighbors, preventing pressure on one key area from blurring over into adjacent key areas.

DETAILED DESCRIPTION OF THE INVENTION

Turning to the drawings, wherein like reference numerals refer to like elements, the present invention is illustrated as being implemented in a suitable computing environment. The following description is based on embodiments of the invention and should not be taken as limiting the invention with regard to alternative embodiments that are not explicitly described herein.

A keyboard overlay according to the principles of the present invention may be developed for use with any touch-sensitive display screen.FIG. 1presents a specific example of a computing environment with such a display screen. InFIG. 1, a portable, interactive display device or “tablet”100communicates with a host computing device102via a wireless communications channel, here illustrated by a radio antenna104on the portable tablet100and by another antenna106on the host102. The portable tablet100has a touch-sensitive display screen by means of which the portable tablet100presents to its user a graphical user interface of the host102. The user sends input to the host102by touching the display screen with a stylus108or with a keyboard overlay110. The portable tablet100may support other input and output peripherals (not shown) including a mouse, speaker, camera, and the like. The portable tablet100is of the type disclosed in U.S. patent application Ser. No. 09/784,716, “Methods and Systems for a Portable, Interactive Display Device for Use with a Computer,” which is incorporated herein by reference in its entirety.

The host computing device102is separate from the portable tablet100and usually sits in a fixed location. The host102may support any number of peripherals, here illustrated by a hardware keyboard114and a mouse116attached to the host by a wired communications channel. The host102provides storage space, access to its own peripheral devices, and processing to run applications. The portable tablet100need only provide the amount of processing necessary to communicate with the host102, to run the client side of the hosting software, and to provide security functions.

The portable tablet100operates in two modes: untethered, as described above, and tethered. The untethered mode is limited by the bandwidth and range of the wireless communications channel. The host computing device102provides a docking station112that accommodates the portable tablet100. When in the docking station, the portable tablet100switches to tethered mode. In this mode, the portable tablet100operates as a display for the host102and communicates with the host102through connectors on the docking station112rather than through the wireless channel. This allows for a higher quality video connection. InFIG. 1, the docking station112's connection to the host102is by way of a wired communications channel. Other communications options are possible. The docking station112may provide power to run the portable tablet100and to recharge its batteries.

The portable tablet100and the host computing device102ofFIG. 1may be of any architecture.FIG. 2ais a block diagram generally illustrating an exemplary computer system that supports the present invention. The computer system ofFIG. 2ais only one example of a suitable environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the portable tablet100or the host102be interpreted as having any dependency or requirement relating to any one or combination of components illustrated inFIG. 2a. The invention is operational with numerous other general-purpose or special-purpose computing environments or configurations. Examples of well known computing systems, environments, and configurations suitable for use with the invention include, but are not limited to, personal computers, servers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments that include any of the above systems or devices. In their most basic configurations, the portable tablet100and the host102typically include at least one processing unit200and memory202. The memory202may be volatile (such as RAM), non-volatile (such as ROM or flash memory), or some combination of the two. This most basic configuration is illustrated inFIG. 2aby the dashed line204. The portable tablet100and the host102may have additional features and functionality. For example, they may include additional storage (removable and non-removable) including, but not limited to, magnetic and optical disks and tape. Such additional storage is illustrated inFIG. 2aby removable storage206and by non-removable storage208. Computer-storage media include volatile and non-volatile, removable and non-removable, media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Memory202, removable storage206, and non-removable storage208are all examples of computer-storage media. Computer-storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory, other memory technology, CD-ROM, digital versatile disks, other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, other magnetic storage devices, and any other media that can be used to store the desired information and that can be accessed by the portable tablet100or by the host102. Any such computer-storage media may be part of the portable tablet100or the host102. The portable tablet100and the host102may also contain communications channels210that allow them to communicate with other devices, including devices on a network212. Communications channels210are examples of communications media. Communications media typically embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and include any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communications media include optical media, wired media, such as wired networks and direct-wired connections, and wireless media such as acoustic, RF, infrared, and other wireless media. The term “computer-readable media” as used herein includes both storage media and communications media. The portable tablet100and the host102may also have input devices214such as a touch-sensitive display screen, a stylus108, a keyboard overlay110, a hardware keyboard114, a mouse116, a voice-input device, etc. Output devices216include the devices themselves, such as the touch-sensitive display screen, speakers, and a printer, and rendering modules (often called “adapters”) for driving these devices. All these devices are well know in the art and need not be discussed at length here. The portable tablet100and the host102each has a power supply218. On the portable tablet100, the power supply218includes a battery and may include circuitry for recharging the battery whenever the portable tablet100is in the docking station112.

When operating in untethered mode, the portable tablet100is supported by software that projects the user interface of the host computing device102to the portable tablet100. The software also accepts user input from the portable tablet100and sends it to the host102. As an example of this software,FIG. 2bis a block diagram of Microsoft's “WINDOWS TERMINAL SERVICES.” An application program220running on the host102sends its output to the operating system224intending that the output be displayed in one or more windows managed by the Desktop222. If the user of the application program220is using a portable tablet100, however, the Terminal Services software component226intercepts the display output, reformats it, and delivers it to the Networking software component228for transport to the portable tablet100. The display information is transported to the portable tablet100by a standard protocol such as Microsoft's Remote Desktop Protocol230or by the Independent Computing Architecture protocol. These protocols allocate the limited bandwidth of the wireless communications channel, an especially important consideration when a bandwidth-intensive peripheral, such as a camera, is attached to the portable tablet100. When the display information reaches the Networking component232on the portable tablet100, it is passed to the Terminal Services Client component236. That component interprets the information and displays it on the touch-sensitive display screen of the portable tablet100. This procedure is reversed for input generated on the portable tablet100, such as by use of the stylus108or of the keyboard overlay110. The user input is presented to the application program220as if it were generated locally on the host102. Note thatFIG. 2bis for illustrative purposes only, and the invention is not limited to the specific software components and protocols shown. In particular, the portable tablet100may run an operating system entirely different from that of the host102. The standard display protocol hides implementation differences. The protocols and transport methods used to carry the display information are chosen to suit particular needs. For example, protocols TCP/IP, SPX, IPX, and NetBEUI may each be appropriate in certain situations. Appropriate transport methods include infrared and short-range radio such as Bluetooth, IEEE's 802.11b, and IEEE 1394 Firewire.

Typically, touch-sensitive display screens are made up of a touch-sensor component constructed over a display component. The display component (see item302ofFIGS. 3bthrough3d) displays images in a manner similar to that of a typical monitor on a personal computer. A portable tablet100would probably use a liquid crystal display because of that display's low weight and small depth. Other display technologies are possible such as, for example, cathode ray tubes, plasma screens, electro-luminescent screens, and organic light-emitting diodes. The touch sensor (see item304ofFIGS. 3bthrough3d) sits on top of the display component302. The touch sensor304is transparent so that the display may be seen through it. Many touch-sensor technologies are known in the art, including four-, five-, and eight-wire resistive, capacitive, near field, optical, and acoustic wave. The keyboard overlay110may be used with any type of display component302and any type of touch sensor304.

When a user touches the touch-sensitive display screen, whether with the stylus108, with the keyboard overlay110, or with a finger, a touch-screen controller detects the touch, determines its location on the display screen, and sends that information to the operating system234of the portable tablet100. If the touch is on an area controlled by the operating system234(for example, on a configuration menu for the portable tablet100), then the operating system234processes the touch. If, on the other hand, the touch is on an area controlled by the host computing device102, then the location of the touch is sent to the operating system224of the host102. As appropriate, the touch is either processed by the host102's operating system224or sent for processing to an application program220running on the host102.

FIGS. 3athrough3dandFIG. 4show details of some embodiments of the keyboard overlay110.FIG. 3apresents a top view of a typical keyboard overlay110. The key areas300of this particular keyboard overlay110are arranged in the manner of the familiar “QWERTY” keyboard. Another embodiment of the keyboard overlay110mimics a numerical entry pad. More specialized keyboard overlays110may be created for use with particular application programs. For example, the key areas300of the keyboard overlay110may present the entry functions of a scientific calculator. The present invention is not limited to any particular arrangement of the key areas300, but contemplates all possible arrangements as may come within the scope of the following claims and equivalents thereof.

FIGS. 3bthrough3dshow cross-sections of a keyboard overlay110in place over a touch-sensitive display screen. As discussed above, the display screen is made up of a display component302and a touch sensor304. In some embodiments, the keyboard overlay110is made from a sheet of flexible material. Suitable materials include, for example, thermo-formed plastic, molded silicon rubber, neoprene, and other pliable plastic- or rubber-based compounds. In the embodiments ofFIGS. 3bthrough3d, beneath each key area300is attached a display actuator306. The display actuator306may be simply the bottom of a sheet of material that forms the keyboard overlay110. In other embodiments, the keyboard actuator306is made of a material harder than the sheet.

One of the functions of the keyboard overlay110is to provide a user with finger-position feedback. Two of many possible mechanisms for providing such feedback are shown inFIG. 3b. First, a key area300is surrounded by a ridge308. Second, the key areas300are raised relative to the “valleys”310between the key areas300. These features allow the user to feel whether or not his fingers are properly oriented over the key areas300. Other possible feedback mechanisms include a raised spot on one or more of the key areas300and a depressed crown of the key areas300.

FIG. 3cshows how the keyboard overlay110transmits pressure from a user's finger into a touch on the touch-sensitive display screen. When the user presses a key area300, the key area300“collapses” or deforms until the display actuator306comes into contact with the touch sensor304of the display screen. In the embodiment ofFIG. 3c, the display actuator306concentrates the pressure of the user's finger into a small, well defined area on the touch sensor304. This concentration allows a touch-screen controller to better locate the touch than is possible with a finger directly touching the touch sensor304.

FIG. 3dillustrates a characteristic of some embodiments of the keyboard overlay110and a mechanism for alleviating problems associated with that characteristic.FIG. 3dis a magnified cross-sectional view of a depressed key area300. First consider the right side of the depressed key area300. Because the keyboard overlay110is flexible, the pressure on the key area300causes the keyboard overlay110to flex downward. As expected, this flexing allows the display actuator306to touch the touch sensor304. However, in this particular embodiment, the keyboard overlay110is so flexible that an energetic push on the key area300also causes the keyboard overlay110to flex downward until it touches the touch sensor304at point312. This additional touch312may be strong enough to be detected by the touch-screen controller. The touch-screen controller becomes confused because there are two simultaneous touches. The touch-screen controller either produces invalid touch-location information or discards both touches. Either possibility confuses and slows down a user typing on the keyboard overlay110.

There are several possible mechanisms for preventing the problem illustrated on the right side of the depressed key area300ofFIG. 3d. For example, the flexible keyboard overlay110may be formed to be less flexible (possibly thicker) in the regions310between the key areas300. Another possibility is illustrated on the left side of the depressed key area300ofFIG. 3d. A rigid frame314is attached to the flexible keyboard overlay110. The rigid frame314prevents the keyboard overlay110from flexing too much and contacting the touch sensor304at multiple points. Many variations on the rigid frame314are possible. Some embodiments interpose a rigid element in every valley310between two key areas300. Depending upon the flexibility of the keyboard overlay110, other embodiments work with less extensive coverage.

A rigid frame314may also serve other purposes.FIG. 4illustrates three related aspects of using the keyboard overlay110: connecting the keyboard overlay110to the portable tablet100, aligning the keyboard overlay110over the touch-sensitive display screen of the portable tablet100, and detecting the presence and type of the keyboard overlay110by the portable tablet100.

The simplest way to connect the keyboard overlay110is simply to lay it on top of a touch-sensitive display screen. When the keyboard overlay110is used with a portable tablet100, a more secure connection may be more convenient. For example, the keyboard overlay110may be hinged to the body of the portable tablet100and swung into place when desired. In another embodiment, that ofFIG. 4, a rigid frame314surrounds the periphery of the keyboard overlay110. The rigid frame314slides into a slot400on the portable tablet100.

A secure connection mechanism may also simplify aligning the keyboard overlay110over the touch-sensitive display screen. Alignment is possible without a secure connection, as when a user “eyeballs” a keycap icons painted on the display screen and visually lines up the keyboard overlay110so that its key areas300correspond to the painted keycap icons. Alignment may also be achieved by the connection slot400: when the keyboard overlay110is slid fully into the slot400, the keyboard overlay110is properly aligned over the display screen.

Along with connection and alignment, some embodiments provide a way for the portable tablet100to detect the presence and type of the keyboard overlay110. Many known technologies are adaptable to detecting the presence of the keyboard overlay110. To mention just a few examples: a dashpot is pressed when the keyboard overlay110is put in place, an optical sensor detects a change in light caused by the keyboard overlay110, or an electronic sensor detects a change in capacitance caused by the presence of the keyboard overlay110.FIG. 4illustrates yet another possibility for electronic sensing. InFIG. 4, the contacts402on the portable tablet100and on the keyboard overlay110touch when the keyboard overlay110is placed fully into the connection slot400. These contacts provide electrical power and signaling connections, via leads404, to an electronic chip406embedded in the keyboard overlay110. The portable tablet100reads from the chip406the type of the keyboard overlay110. As discussed below in reference toFIG. 5a, the portable tablet100uses this type information to, for example, invoke an application program220associated with the particular type of keyboard overlay110. If the keyboard overlay110represents an interface to a specific type of scientific calculator, then an appropriate calculator program may be invoked to accept the user's input. In place of, or in addition to, the chip406, other embodiments include other active or passive electronic components (for example, wire jumpers or resistors).

FIGS. 5aand5bpresent a flowchart of exemplary steps performed when using a keyboard overlay110. Note that many of the steps in these FIGS. are appropriate only to certain embodiments of the present invention. Details within each step also vary from embodiment to embodiment.

The flowchart begins in step500when the keyboard overlay110is connected to a computing system. As discussed above in reference toFIG. 4, in some embodiments connecting merely involves placing the keyboard overlay110on top of a touch-sensitive display screen. In other embodiments, a hinge connects the keyboard overlay110to the computing system or the keyboard overlay110slides into a connection track400provided by the computing system.

In step502, the keyboard overlay110is properly aligned with respect to the touch-sensitive display screen. The keyboard overlay110is operable with computing systems that are not aware of its presence and that do not provide any special alignment aids. As discussed above in reference toFIG. 4, the user in this case visually aligns the key areas300of the keyboard overlay110over keycap icons painted by the computing system on its display screen. In other embodiments, alignment is achieved by physical means, possibly involving a secure connection mechanism. If the computing system is aware of the keyboard overlay110, it can paint alignment indications on the display screen.

In some embodiments, the computing system becomes aware of the presence of the keyboard overlay110in step504. Note that this step is not performed for “legacy” systems that were not designed with a keyboard overlay110in mind. Configuration software is added to some computing systems to allow a user to tell the computing system when a keyboard overlay110is in place. Some computing systems automatically sense the presence of the keyboard overlay110, as discussed above in reference to FIG.4.

Like step504, detecting the type of the keyboard overlay110in step506is optional. The type of the keyboard overlay110can include such information as its size, whether or not the keyboard overlay is transparent, the arrangement of the key areas300, keycap indications, if any, and the like. Methods for detecting the type of the keyboard overlay110parallel the methods discussed in reference to step504for detecting the presence of the keyboard overlay110. Specific embodiments range from supporting no detection at all, to allowing a user to tell the computing system that a specific type of keyboard overlay110is present, to automatically detecting the type of keyboard overlay110, such as by the use of an embedded electronic chip406, as discussed above in reference to FIG.4.

Some embodiments of the computing system take advantage, in step508, of their knowledge of the presence and type of the keyboard overlay110to rearrange information displayed on the touch-sensitive display screen. For example, system warning messages and configuration menus are placed so that they do not lie under the keyboard overlay110.

Building on its knowledge of the type of the keyboard overlay110gathered in step506, some embodiments of the computing system invoke, in step510, an application program220appropriate to this type of keyboard overlay110. (Note that if the computing system is a portable tablet100, then this step includes asking the host computing system102to run the appropriate application program220.) This step encourages the use of application-specific keyboard overlays110. For example, the keyboard overlay110is found to present menu-selection keys for a fast-food restaurant. An order-entry application program220is invoked, accepting menu orders typed in on the keyboard overlay110, sending the orders to the food-preparation staff, and presenting the total cost of a customer's bill on the touch-sensitive display screen. Step510becomes more valuable as keyboard overlays110and application programs220are specifically designed to work with one another.

In step512, the computing system tells the application program220that will accept input from the keyboard overlay110of the presence and type of the keyboard overlay110. As with previous steps, embodiments of this step range a large range, from doing nothing at all in the case of an unaware legacy application program220to providing a full disclosure to an application program220specifically invoked (in step510) to run with this type of keyboard overlay110.FIGS. 6aand6b, discussed below, present steps taken by an exemplary application program220when used with a keyboard overlay110.

Steps514through522present a loop of exemplary steps taken when a user types on the keyboard overlay110. In step514, the physical structure of the keyboard overlay110provides feedback to the user so that the user can keep his fingers positioned properly over the keyboard overlay110's key areas300. The discussion ofFIG. 3bpresents a few examples of how this feedback is provided: a depressed crown on top of the key area300, a ridge around the key area300, and a valley310between key areas300and serving to tactilely distinguish one key area300from its neighbors. In any case, confident that his fingers are properly positioned, the user presses a key area300in step516. The deformation of the key area300caused by the user's pressure provides feedback to the user in step518telling the user that the key area300has been pressed hard enough to generate a touch on the touch-sensitive display screen. The user's pressure is delivered to the display screen by a display actuator306and is detected by the touch-screen controller as a touch in step520. As is well known in the art, the touch is directed to the application program220(or operating system utility) that is accepting input from the location on the display screen where the touch is detected. The location of the detected touch is passed to the application program220in step522. The application program220processes the touch as appropriate.

FIGS. 6aand6bpresent a flowchart of exemplary steps performed by an application program220responding to input from a keyboard overlay110. Note that, as in the flowchart ofFIGS. 5aand5b, many of the steps in these FIGS. are appropriate only to certain embodiments of the present invention, and details within each step vary from embodiment to embodiment.

In step600, the application program220is informed of the presence and type of a keyboard overlay110. In response to that information, the application program220, in step602, rearranges the information that it displays on the touch-sensitive display screen, moving information to areas of the display screen not covered by this type of keyboard overlay110. This is similar to the operating system's action in step508ofFIG. 5a. Legacy application programs220are not aware of the keyboard overlay110and so do not perform steps600and602.

Some application programs220have a special keyboard-input mode. When the keyboard overlay110is first detected, the application program220can take the presence of the keyboard overlay110as an indication that the user wishes to enter this mode.

If the application program220is unaware of the keyboard overlay110, then it probably performs step606, painting keycap icons on the touch-sensitive display screen, before the user aligns the keyboard overlay110over the keycap icons. This may be in response to the user pressing a button or performing some other action to bring up the virtual keyboard. Some keyboard-overlay-aware application programs220instead paint the keycap icons in response to the presence of the keyboard overlay110. In some embodiments, the specific keycap icons painted and their arrangement depends upon the specific type of the keyboard overlay110. The keycap icons may also depend upon configuration information set by the user. For example, the currency icon could be “$” in the United States and “¥” in Japan. Of course, keyboard-overlay-aware application programs220need not paint keycap icons if the keyboard overlay110is known to be opaque. For a pleasing aesthetic effect, the keycap icons are alpha-blended with whatever display information is already present on the display screen.

Steps608through614form a loop of exemplary steps performed by the application program220as the keyboard overlay110is used. In step608, the application program220receives information about a touch detected by the touch sensor304. As described above in reference to step520ofFIG. 5b, detected touches are sent to the operating system utility or application program220responsible for processing input from the touch's location on the touch-sensitive display screen. Audible feedback can be sent to the user so that he knows that he hit a key area300hard enough to register a touch. In step610, the application program220correlates the touch location with a specific key area300to determine, for example, that the user just typed a letter “J.” In step612, the application program220takes action appropriate to the specific key area300just touched. For example, the application program220appends a letter “J” to a text string being entered.

Step614presents a possibility when the keyboard overlay110is transparent. The keycap icons are repainted in response to user input. For example, when CAPS is pressed, upper case letters are shown. More specialized changes are possible and depend upon the nature of the application program220.

In view of the many possible embodiments to which the principles of the present invention may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the invention. Those of skill in the art will recognize that some implementation details, such as arrangements of key areas and construction details, are determined by specific situations. Although the environment of the invention is described in terms of software modules or components, some processes may be equivalently performed by hardware components. Therefore, the invention as described herein contemplates all such embodiments as may come within the scope of the following claims and equivalents thereof.