Ergonomic keyboard user interface

Disclosed herein are system, method, and computer program product embodiments providing an ergonomic user interface keyboard. An embodiment operates by obtaining an accessibility measure corresponding to a finger of a user. An interface keyboard is configured based on an accessibility map corresponding, to the accessibility measure of the finger. The interface keyboard associated with a device is displayed.

BACKGROUND

Touchscreen devices, such as tablet computers, have a standard digital keyboard that tries to mimic an ordinary QWERTY keyboard that is difficult to use (relative to an ordinary keyboard). These digital keyboards mimic ordinary device keyboards in that they appear the same regardless of which user is using the touchscreen device. However different users may have different preferences with regard to how they prefer to use these digital keyboards.

DETAILED DESCRIPTION

Provided herein are system, apparatus, device, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for providing an ergonomic keyboard user interface.

FIG. 1is a block diagram100showing example operations for providing an ergonomic keyboard user interface, according to some embodiments. A user102may interact with a computing device104using a user interface keyboard (UK)106. Computing device104may include any computing device that receives alpha-numeric or other input from a keyboard (i.e., UIK106). Example computing devices104include, but are not limited to, mobile phones, laptops, tablets, touchscreen computing devices, navigational systems, monitors, and televisions.

UIK106may include any digitally rendered keyboard that is accessible to user102. In an embodiment, UIK106may be rendered on a touchscreen interface of computing device104. In another embodiment, UIK106may be a light-projected keyboard that is projected on another surface (or that is holographic in nature) but is responsive to a user's102finger and hand movements.

A keyboard mapping system (KMS)108may render or configure UIK106any different number of ways depending on the capabilities and/or preferences of user102. User102capabilities may include finger and hand mobility and dexterity in using UIK106. Depending on which user102is operating UIK106, those keys may be arranged in different orders or sequences.

In another embodiment, KMS108may also configure UIK106based on which application of computing device104(including web-based and cloud based applications) is currently interacting with the keyboard input from UIK106or is currently active on computing device104. For example, different applications or programs operating on or via computing device104may require or accept input that varies from the standard keys of a QWERTY keyboard, KMS108may configure UIK106to display any program-specific keys or keyboards (i.e., UIKs106). For example, a particular program may have its own UIK106that is configured for user102in the manner described herein. UIK106may be configured on a per-user and/or per-application basis.

In an embodiment, KMS108may determine the capabilities, limitations, and/or preferences of user102. Based on the determined or measured user102capabilities or limitations, KMS108may arrange and configure the display of UIK106accordingly. For example, KMS108may measure or test the finger flexibility, mobility and hand placement preferences of user102to configure how the keys of UIK106are to be arranged or displayed for the user102.

In an embodiment, user102may be missing a finger or may have limited mobility or flexibility in one or more fingers due to injury or another condition. KMS108may determine to what extent the injury impacts the user's ability to type or operate UIK106, and may arrange UIK106accordingly. The UIK106for the injured user102may vary from the UIK106for a user102with full functionality in all fingers.

Accessibility measure110may include one or more values that provide an indication of the physical mobility and/or flexibility in the fingers and hands of user102. In an embodiment, accessibility measure110may be an indication of how far a user can reach or bend with different fingers. KMS108may determine, track, measure or compute an accessibility measure110for one or more fingers of user102in one or both hands.

In an embodiment, KMS108may test all of the user's fingers, or user102may specify for which finger(s) user102wants accessibility measure110determined. For example, user102may know that a particular finger is injured or have limited mobility, but that the remaining fingers are fine. Then, for example, user102may specify that only the injured finger is tested and particularly configured for by KMS108.

In an embodiment, user102may specify a time period in which the particularly configured UIK106is to be used as the default UIK106. For example, if user102expects to regain full mobility in the finger within two weeks, user102may specify an expiration period for the configured UIK106for two weeks. Then after the expiration period, the next time user102logs into device104, user102may be presented with UIK106for full finger mobility. In an embodiment, KMS108may provide a visual indicator that communicates which UIK106is being displayed. The visual indicator may indicate which finger(s) are determined to have limited mobility and are being accounted for with UIK106.

KMS108may use different techniques or a combination of techniques to calculate accessibility measure110. In an embodiment, accessibility measure110of a thumb may be measured different from the accessibility measure110of the remaining fingers on account of the thumb moving differently (relative to the remaining fingers).

FIG. 2illustrates an example diagram200of how accessibility may be measured for the different fingers of a hand, according to an embodiment. In an embodiment, the measuring as described herein may be performed on a touchscreen user interface, such that KMS108can detect where a user touches in response to various input.

Initialization area205may be an indicator where a user102is to place the base of their thumb on the palm of their right hand for testing of the right hand and/or fingers by KMS108, KMS108may use initialization area205to align the user's hand properly on the measuring display/touchscreen interface before testing begins. In an embodiment, when KMS108detects that the user's hand has been placed on initialization area205, the testing may begin. In another embodiment, different or multiple initialization areas205may be used to align a user's hand and/or fingers for testing.

210illustrates how accessibility measure110relative to thumb mobility may be measured by KMS108. As shown in the example embodiment, accessibility measure110of the thumb may be horizontal mobility in a way that movement is natural for the thumb210.

In an embodiment, KMS108may determine a lateral mobility of the thumb, how far left and right the thumb of user102can touch or contacts on the interface. For example, the user102may be prompted to swipe their thumb left and right multiple times to determine what area or portion of touch screen interface are contacted by or accessible to thumb210. Accessibility measure110for thumb may then be computed as a left angle and a right angle corresponding to left and right movement, or a surface area indicator of what surface area is accessible to the thumb210.

220illustrates an example of how accessibility may be measured for the remaining fingers (other than thumb210). As shown, finger220may be measured in terms of vertical flexibility or bendability. In an embodiment, the ability of finger220to bend may be divided into three different sections220A-C. KMS108may determine the ease, difficulty, or dexterity of finger220based on its ability to bend and touch squares or other areas within each of three different sections220A-C. The different mobility sections220A-C may correspond to how much a normal finger can usually bend. However, for users with limited mobility, fewer areas220A-C may be tested. In another embodiment, horizontal mobility of fingers220may be tested as well. Accessibility measure110may include one more values indicative of the bendability and/or horizontal mobility of a user's finger(s).

Returning toFIG. 1, a new user102who is using computing device104may be asked whether the user102he wants or needs a custom UIK106. For example, user with full mobility in all his fingers may opt to use a standard, or un-customized UIK106(based on a default map114). Or, for example, as discussed in greater detail below, the user102may still configure UIK106based on particular user preferences saved as key preferences118with regard to the visual arrangement of the keys of UIK106.

In an embodiment, if user102answers ‘yes’ requesting a customized UIK106, KMS108may then determine accessibility measure110for one or more of the user's fingers. KMS108may launch a learning or testing interface (e.g., via a touchscreen device) through which KMS108may measure the mobility or accessibility of the fingers of user102in each hand (as described above with respect toFIG. 2, as an example).

For example, KMS108may prompt user102to place the bottom of their right palm in initialization area205. Upon detecting (through a touchscreen or other sensible user interface) that the user102has placed the palm of their hand on initialization area205or by receiving a user confirmation, KMS108may begin calibrating or measuring the fingers220and/or thumb210of user102on one or both hands. In an embodiment, user102may specify which fingers (including thumbs) the user102wants measured.

In measuring thumb210, KMS108may prompt user102to begin making back-and-forth or left-and-right swiping motions with the thumb and to stretch their thumb as far as possible. Using touchscreen interface, KMS108may record or measure the surface area covered through the thumb swiping motions. Or, for example, KMS108may prompt user to press various displayed interface buttons with their thumb. KMS108may store measurements corresponding to the user's performance as accessibility measure110for thumb210. The performance measures may indicate the time and/or success or accuracy with which the user102was able to perform whatever tests were administered.

In an embodiment, KMS108may provide a distinguishable user interface button, such as a red triangle, that user102may select if the user102is missing a thumb or has limited/no mobility in the thumb or other fingers. This selection may be stored as accessibility measure110for thumb. In an embodiment, the selection may cause KMS108to skip testing of the indicated digit and record that the user102cannot use the indicated finger for providing keyboard input. For example, if accessibility measure110is a Null, Void, or negative value, KMS108may determine that the user is missing or otherwise has no mobility in the given index/finger/thumb. Or, for example, if the user is missing a finger, the accessibility measure110may be relative to the remaining fingers.

KMS108may also perform tests on the remaining fingers220to see how much each finger can bend vertically and/or reach horizontally. In an embodiment, KMS108may prompt the user102to use one finger or multiple fingers at a time. For example, multiple fingers220may be simultaneously tested for mobility or bendability measures.

KMS108may include various techniques for measuring or detecting the usability, mobility, or disability of user102as indicated by accessibility measure110. One example of how to determine accessibility measure110may be displaying a black and white chessboard with medium granularity on an interactive interface (such as a touchscreen). In an embodiment, medium granularity may be displaying 1 inch×1 inch squares on a touchscreen interface. However, these measurements may vary in different embodiments and may vary based on the size of a touch screen where UIK106is to be displayed.

After the user has set his hand on initialization area205, one of the squares of the rendered chessboard may change colors and the user102may be prompted to press the indicated square without lifting the hand from initialization area205. In an embodiment, the prompt may include a prompt as to which finger is to be used to press the square, or may allow the user102to use any finger to press it. KMS108may determine whether user102was able to perform the requested action, how many tries it took, how long it took, and whether user102maintained or lost contact with initialization area205while performing the requested action(s).

As referenced above, the testing interface of KMS108may include an option that the user102may select if the user102is unable to reach the red square with any finger or the indicated finger. Or, for example, if the user102has not touched the indicated square upon the expiration of a timeout period (e.g., 3 seconds) KMS108may determine that the user102is unable access the indicated square due to finger mobility issues. In an embodiment, KMS108may track how many other locations on a touchscreen interface user102may have touched prior to touching the indicated square. For example, a number of previous touches may indicate that the user was trying and failing to touch the indicated square before finally succeeding, and may indicate finger mobility issues.

This process of changing the color of one or more of the squares and prompting a user102to touch the indicated square(s) may be repeated for different fingers in the user's hands, at different granularities, until KMS108may compute accessibility measure110.

In an embodiment, a threshold130may indicate a number of consecutive correct square touches or based on user102exceeding a threshold130percentage of correctness. If user meets or exceeds threshold130, KMS108may determine that the user102has full mobility based on the tested granularity. KMS108may then decreased the square granularity and test the user102again to determine the user's dexterity or finger mobility. In an embodiment, this process may be repeated until KMS108determines that the user has full mobility or until the user102fails at a particular granularity.

KMS108may test different granularities may be tested for a finger in different bendability areas or sections220A-C based on user102performance. In an embodiment, if user102fails to meet threshold correctness requirement130, the granularity of the squares may be increased and the user102may be tested again. This process may be repeated until user102passes threshold130for at least one granularity, or the largest granularity has been tested. In an embodiment, threshold130may indicate a number of incorrect touches or timeouts that require UIK106configuration (e.g., that fall out of an expected range of performance for user102).

In another testing technique, KMS108may prompt user to place their palm on initialization area205, and expand or extend their fingers as much as possible and/or place theirs fingers as close together as possible without overlapping. Then, for example, based on touchscreen interface, KMS108may determine the reach of the fingers of user102.

KMS108may use any combination of testing techniques to compute or calculate accessibility measures110. In an embodiment, accessibility measure110may be an area measure or value(s) corresponding to what portions of an interface (of touch screen device104) are accessible to user102given a starting hand-placement point of205. With regard to the fingers220, accessibility measure110may indicate the furthest reach from and closest bend relative to initialization area205. In another embodiment, accessibility measure110may indicate a length of fingers from the user's palm or from initialization area205.

In an embodiment, KMS108may have a default map114corresponding to default measure116. Default map114may indicate which keys of UIK106are accessible to which fingers of a user102who is determined to have full finger mobility or dexterity. Default map114may correspond to an initial arrangement or placement of keys for UIK106. In an embodiment, default map114may be based on a standard of modified QWERTY keyboard arrangement. For example, those keys which are generally accessible the right index finger on a standard QWERTY keyboard, would be accessible to the right index finger in UIK106for default map114.

Default measure116may indicate a baseline value or range of expected values for accessibility measure110for the various fingers being tested and may correspond to the test(s) that are administered. The baseline value or range of expected values may indicate an expected mobility or bendability of particular finger. In an embodiment, the extent to which accessibility measure110varies or falls outside the range of a corresponding default measure116may determine the extent to which KMS108alters accessibility snap112(and UIK106) for user102from default map114. In an embodiment, the baseline value116may indicate which keys of a keyboard are normally accessible to each finger (e.g., of a user with full mobility).

FIG. 3illustrates a diagram of an example default accessibility map114, according to an embodiment. Default map114shows which keys are accessible to which fingers based on default measure116. As may be seen from the example, the arrangement of keys accessible to each finger may correspond to a standard QWERTY keyboard map. In other embodiments, other keyboard map arrangements, other than QWERTY may be used as well.

KMS108may use accessibility map112(including default map114) to determine where on user interface to arrange various keys for UIK106. In an embodiment, accessibility map112may indicate how much of a variance120(if any) exists between accessibility measure110for fingers of user102and a default or expected measure116.

As referenced above, default measure116may indicate a range of expected values for accessibility measure110, indicating full mobility and corresponding to using default map114. KMS108may compare default measure116to accessibility measure110to calculate variance120which may indicate to what extent user102has extra mobility or limited mobility outside of the range of default measure116. Based on variance120, KMS108may determine accessibility map112. For example, a finger with limited mobility or flexibility may be assigned fewer keys than indicated by default map114. In an embodiment, variance120may be computed on a per-finger or per-hand basis for a user102.

FIG. 4illustrates a diagram of an example customized accessibility map112, according to an embodiment. In an embodiment, user102may be missing or have limited/no mobility in their left ring finger and pinky. KMS108may determine this variance120from default measure116based on the testing process described above, KMS108may then generate accessibility map112to account for variance120.

As may be seen in the example ofFIG. 4, the remaining fingers of left hand may be responsible for reaching or accessing more keys than in default accessibility map114as shown inFIG. 3. In another embodiment, key accesses normally assigned to the left hand may be shifted to fingers of the right hand based on variance120. For example, if user102only has one finger on the left hand, then KMS108may assign more keys to the fingers of the right hand, rather than making the single left hand finger responsible for all the keys normally assigned to the left hand.

In an embodiment, KMS108may customize or further configure accessibility map112and the display of UIK106based on key preferences118provided by user102. KMS108prompt user102to type various letters, words, sentences, or other alphabetic, numeric, or alpha-numeric combinations using an initially configured UIK106. Based on how accurately and/or quickly a responds to the prompts, KMS108may adjust accessibility map112or the display (size, arrangement, ordering) of keys of UIK106.

In an embodiment, KMS108may display UIK106or accessibility map112on an interface of device104. KMS108may then allow or prompt user to adjust how user102prefers accessibility map112or UIK106to appear. For example, user102may use a drag-and-drop operation to adjust the display of UK106. KMS108may then save the user's indication in key preferences118and adjust accessibility map112accordingly.

Key preferences118may include any key preferences that are outside of a default set of keys available or arrangement or display of keys particular to user102. Key preferences118may be provided by user102, and/or may be determined based on which application(s) of computing device104are active or are receiving input from UIK106. For example, a financial application may include currency key assignments which may be frequently used with the financial application, and which may be displayed in UIK106, The currency key assignments may be provided in addition to the standards QWERTY keys or in lieu of some of the QWERTY keys. Or, for example, a user who uses Greek lettering may choose to configure key preferences118to include one or more selectable Greek letters.

In an embodiment, KMS108may enable user102to configure and/or switch between multiple UIKs106which have been configured based on accessibility map112, For example, while alphabetic keys are shown in the map114ofFIG. 3. In other embodiments, different keyboard maps may be used. For example, UIK106may display keys corresponding to numbers, currency, Greek letters, or other key maps in different languages. In an embodiment, user102may indicate which key maps to configure in key preferences118, In an embodiment, KMS108may configure each UIK106for the different key preferences118based on a base accessibility map112. However, user may individually configure and adjust each associated UIK106. In a given session, user102may toggle or switch between different keyboard maps with different sets of keys.

FIG. 5illustrates a diagram of an example customized user interface keyboard (UIK)506, according to an embodiment. UIK506shows an example of a digital rendering (e.g., on a touchscreen interface or light rendering) of a UIK106that has been configured based on an accessibility map112.

In an embodiment, UIK506may not visually display initialization area205. As noted above, initialization area205indicates a position of where a user's palm is anticipated to be on or relative to the user interface. The lines extending from205are representative of accessibility map112and show which keys are accessible with which user fingers (and may not be displayed).

In another embodiment, if the user's right index finger had limited bendability or flexibility, then the “M” key may be positioned next to the “U” key instead of underneath the “J” key. Or, for example, “M” may be made accessible to the pinky. Which keys are accessible to which fingers may be customized on a per-user and/or per-application basis.

In an embodiment, the rendering of UIK506may account for an angular placement of user's palm on initialization area205. For example, based on which portion(s) of a touchscreen interface of device104is contacted relative to initialization area205, the angles at which the keys are displayed may vary across different users. For example, KMS108may provide two users102with full mobility different UIKs506based on their preference hand placement angles.

FIG. 6is a flowchart illustrating a process600for providing an ergonomic keyboard user interface, according to some embodiments. Method600can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 6, as will be understood by a person of ordinary skill in the art.

At610, an accessibility measure corresponding to a finger of a user is obtained. For example, KMS108may receive an indication from user102that the user has limited mobility in one or more fingers. KMS108may then test user102to determine the extent of the limitation and compute accessibility measure110for one or more of the fingers, including the finger with limited mobility and/or one or more of the other fingers. Accessibility measure110may indicate or correspond to how many keys of a keyboard are accessible to the respective finger(s). In an embodiment, accessibility measure110may indicate a variance120from an expected or default measure116. Variance120may indicate which keys that are normally accessible to a particular finger with full mobility are not accessible to a corresponding finger of the user102.

At620, an interface keyboard is configured based on an accessibility map corresponding to the accessibility measure of the finger. For example, KMS108may determine an accessibility map112corresponding the accessibility measures110of user102. In an embodiment, accessibility map112may indicate or account for a variance120from default map114. Default map114may indicate a predetermined set of keys is accessible to one or more of the fingers based on default measure116. Accessibility map112may indicate any variance120from default114.

At630, the interface keyboard associated with a device is displayed. For example, computing device104may display UIK106or UIK106may be projected on another surface. In an embodiment, KMS108may allow a user to test UIK106and further configure the appearance of UIK106or provide key preferences108. Then, for example, when user102logs in to use computing device104, the customized or configured UIK106may automatically be provided for user102. KMS108may also change which UIK106is displayed based on which application is active on computing device104.

Various embodiments may be implemented, for example, using one or more well-known computer systems, such as computer system700shown inFIG. 7. One or more computer systems700may be used, for example, to implement any of the embodiments discussed herein, as well as combinations and sub-combinations thereof.

Computer system700may include one or more processors (also called central processing units, or CPUs), such as a processor704. Processor704may be connected to a communication infrastructure or bus706.

Computer system700may also include user input/output device(s)703, such as monitors, keyboards, pointing devices, etc., which may communicate with communication infrastructure706through user input/output interface(s)702.

Computer system700may also include a main or primary memory708, such as random access memory (RAM). Main memory708may include one or more levels of cache. Main memory708may have stored therein control logic (i.e., computer software) and/or data.

Computer system700may also include one or more secondary storage devices or memory710. Secondary memory710may include, for example, a hard disk drive712and/or a removable storage device or drive714. Removable storage drive714may be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and/or any other storage device/drive.

Removable storage drive714may interact with a removable storage unit718. Removable storage unit718may include a computer usable or readable storage device having stored thereon computer software (control logic) and/or data. Removable storage unit718may be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, and/any other computer data storage device. Removable storage drive714may read from and/or write to removable storage unit718.

Secondary memory710may include other means, devices, components, instrumentalities or other approaches for allowing computer programs and/or other instructions and/or data to be accessed by computer system700. Such means, devices, components, instrumentalities or other approaches may include, for example, a removable storage unit722and an interface720. Examples of the removable storage unit722and the interface720may include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and/or any other removable storage unit and associated interface.

Computer system700may further include a communication or network interface724. Communication interface724may enable computer system700to communicate and interact with any combination of external devices, external networks, external entities, etc. (individually and collectively referenced by reference number728). For example, communication interface724may allow computer system700to communicate with external or remote devices728over communications path726, which may be wired and/or wireless (or a combination thereof), and which may include any combination of LANs, WANs, the Internet, etc. Control logic and/or data may be transmitted to and from computer system700via communication path726.

In some embodiments, a tangible, non-transitory apparatus or article of manufacture comprising a tangible, non-transitory computer useable or readable medium having control logic (software) stored thereon may also be referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system700, main memory708, secondary memory710, and removable storage units718and722, as well as tangible articles of manufacture embodying any combination of the foregoing. Such control logic, when executed by one or more data processing devices (such as computer system700), may cause such data processing devices to operate as described herein.