Patent ID: 12197802

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth such as examples of specific components, circuits, and processes to provide a thorough understanding of the present disclosure. The term “coupled” as used herein means connected directly to or connected through one or more intervening components or circuits. Also, in the following description and for purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the aspects of the disclosure. However, it will be apparent to one skilled in the art that these specific details may not be required to practice the example embodiments. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring the present disclosure. Some portions of the detailed descriptions which follow are presented in terms of procedures, logic blocks, processing and other symbolic representations of operations on data bits within a computer memory. The interconnection between circuit elements or software blocks may be shown as buses or as single signal lines. Each of the buses may alternatively be a single signal line, and each of the single signal lines may alternatively be buses, and a single line or bus may represent any one or more of a myriad of physical or logical mechanisms for communication between components.

Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present application, discussions utilizing the terms such as “accessing,” “receiving,” “sending,” “using,” “selecting,” “determining,” “normalizing,” “multiplying,” “averaging,” “monitoring,” “comparing,” “applying,” “updating,” “measuring,” “deriving” or the like, refer to the actions and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as being implemented in a specific manner. Any features described as modules or components may also be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a non-transitory computer-readable storage medium comprising instructions that, when executed, performs one or more of the methods described above. The non-transitory computer-readable storage medium may form part of a computer program product, which may include packaging materials.

The non-transitory processor-readable storage medium may comprise random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, other known storage media, and the like. The techniques additionally, or alternatively, may be realized at least in part by a processor-readable communication medium that carries or communicates code in the form of instructions or data structures and that can be accessed, read, and/or executed by a computer or other processor.

The various illustrative logical blocks, modules, circuits and instructions described in connection with the embodiments disclosed herein may be executed by one or more processors. The term “processor,” as used herein may refer to any general-purpose processor, conventional processor, controller, microcontroller, and/or state machine capable of executing scripts or instructions of one or more software programs stored in memory.

FIG.1shows an example input processing system100within which the present embodiments may be implemented. In some embodiments, the input processing system100may provide an on-demand electronic display for a display-less input device. The input processing system100includes an input device110and a display device120.

The input device110may be configured to receive and process user inputs101. In some embodiments, the input device110may be a complete electronic system. For example, the input device110may be configured to perform one or more functions in response to the user inputs101. Example actions may include, but are not limited to, unlocking a door, opening an automated (e.g., garage) door, changing the channel on a television, or changing an operation mode of the input device110. Further, in some embodiments, the input device110may be a display-less device. As used herein, the term “display-less” may refer to any device that does not contain, nor is permanently coupled to, an electronic display (including any hardware capable of displaying and/or projecting a rich user interface). Example suitable input devices may include, but are not limited to, composite input devices (e.g., physical keyboards, joysticks, key switches, key fobs, and the like) and data input devices (e.g., remote controls, mice, touch pads, and the like).

The input device110includes one or more input sources112, notification hardware (HW)114, and a notification selector116. The input sources112may be configured to receive the user inputs101. In some embodiments, the input sources112may include electrical or mechanical actuators (e.g., buttons, switches, toggles, capacitive sensors, and the like). For example, the actuators may be used to provide data and/or selection inputs to the input device110. In some other embodiments, the input sources112may include device sensors (e.g., accelerometers, gyroscopes, magnetometers, photosensors, temperature sensors, pressure transducers, and the like). Still further, in some embodiments, the input sources112may include biometric sensors (e.g., cameras, microphones, fingerprint scanners, and the like). For example, the biometric sensors may be used to provide biometric inputs to the input device (e.g., for authentication purposes). In some embodiments, authentication may be required to unlock certain features and/or operations of the input device110(such as unlocking a door).

The notification hardware114may be configured to output one or more notifications102about the user input101and/or the input device110. In some embodiments, the notification hardware114may include one or more light sources and/or speakers to provide a simply user interface that can be used to communicate notification information to the user. For example, a blinking green light or gentle beeping sound may be used to confirm that a user input101was successfully processed. On the other hand, a blinking red light or harsh beeping sound may be used to indicate that a user input101was not processed successfully and/or the input device110is in a low-battery or error state. The notification selector116may be configured to select the appropriate notification to be output or otherwise manifested via the notification hardware114. In some embodiments, the notification selector116may select each notification from a database of preconfigured notifications stored on the input device110(not shown for simplicity).

Because the input device110lacks an electronic display, the notification hardware114may serve as a simple or rudimentary user interface (UI) through which the input device110may communicate with the user. However, aspects of the present disclosure recognize that a richer UI (e.g., capable of providing more detailed information and/or instructions) may be desirable for certain tasks, such as communicating complex instructions to the user. For example, during the process of enrolling a biometric signature (e.g., to be used for authentication), the user may be required to provide a biometric input multiple times with slightly different variations, such as by changing the pose, orientation, and/or offset of the biometric input. Thus, it may be desirable to display a visual guide for how to provide the biometric input at each step of the enrollment process. Electronic displays are well-suited for a rich UI that can be used to guide the user to provide more complex inputs.

It is noted, however, that electronic displays are expensive and consume large amounts of processing resources and power. Moreover, the size of the display surface may directly affect the richness of the UI. For example, richer UIs tend to occupy larger display areas. Aspects of the present disclosure further recognize that the input device110may be used primarily in the authentication mode, and that enrollment operations tend to be triggered very infrequently. Thus, the input device110may not benefit from having an electronic display for a majority of its operation. Such low frequency of use may not outweigh the cost or sacrifices in performance, battery life, size and/or design needed to incorporate a permanent display into the input device110.

In some embodiments, the display device120may be configured to operate as an on-demand display for the input device110. The display device120may be any electronic system containing an electronic display (including any hardware capable of displaying and/or projecting a rich user interface). Examples of suitable electronic systems may include, but are not limited to, personal computing devices (e.g., desktop computers, laptop computers, netbook computers, tablets, web browsers, e-book readers, personal digital assistants (PDAs), and the like), data output devices (e.g., display screens and printers, and the like), remote terminals, kiosks, video game machines (e.g., video game consoles, portable gaming devices, and the like), communication devices (e.g., cellular phones such as smart phones), media devices (e.g., recorders, editors, and players such as televisions, set-top boxes, music players, digital photo frames, digital cameras, and the like), and in-car entertainment systems or controls.

In some embodiments, the input device110may use the notification hardware114to communicate information to be displayed on the display device120. At least some of the notifications102generated by the input device110may correspond to rich media content that can be rendered or displayed on the display device120. For example, each content item on the display device120may be triggered by a unique pattern or sequence of lights and/or sounds emitted by the input device110. When the display device120detects a particular notification102output by the input device110, the display device120may display an image or video associated with the detected notification. Accordingly, the display device120may provide a rich UI for guiding or instructing the user to perform complex tasks or user inputs101on the input device110.

The display device120includes a display122, detection hardware (HW)124, and a user interface (UI) application126. The display122may be any type of electronic display capable of displaying a rich UI. In some aspects, the display122may include any hardware capable of rendering digital images (e.g., fixed pixel displays, raster scan displays, and the like) and/or displaying or projecting non-digital images (e.g., seven-segment displays, fixed-icon displays, and other non-bitmapped displays). Example suitable display technologies may include, but are not limited to, light emitting diode (LED), organic LED (OLED), cathode ray tube (CRT), liquid crystal display (LCD), plasma, and electroluminescence (EL). In some aspects, the display122may be integrated as a physical part of the display device120. In some other aspects, the display122may be physically separated from the display device120. For example, the display device120may be coupled to (and communicate with) the display122using various wired and/or wireless interconnection and communication technologies, such as buses and networks. Examples technologies may include Inter-Integrated Circuit (I2C), Serial Peripheral Interface (SPI), PS/2, Universal Serial bus (USB), Bluetooth®, Infrared Data Association (IrDA), and various radio frequency (RF) communication protocols defined by the IEEE 802.11 standard.

The detection hardware124may be configured to detect one or more notifications102output by the input device110. In some aspects, the detection hardware124implemented by the display device120may depend, at least in part, on the notification hardware114implemented by the input device110. For example, if input device110outputs a notification102using one or more light sources, the display device120may detect the notification102(e.g., light) using one or more cameras or photosensors. On the other hand, if the input device110outputs a notification102using one or more speakers, the display device120may detect the notification102(e.g., sound) using one or more microphones. Still further, in some embodiments, the input device110may output a notification102using one or more radio-frequency (RF) transmitters. Accordingly, the display device120may detect the notification102(e.g., RF signal) using one or more RF receivers.

The UI application126may be configured to interpret and/or process the notifications102detected by the detection hardware124. In some embodiments, the UI application126may generate and/or update a rich UI on the display122in response to notifications102output by the input device110. More specifically, the UI application126may select the appropriate content to be rendered on the display122(e.g., as part of the rich UI) based, at least in part, on the detected notification102. For example, the UI application126may match the detected notification102to a particular content item (e.g., image, video, or other media content) stored on, or otherwise accessible by, the display device120. In some aspects, the content items to be rendered on the display122may include instructions for performing a complex task or providing a user input on the input device110. In a particular embodiment, the UI application126may display a step-by-step guide for initiating and/or completing a biometric enrollment process.

For example, when enrolling a biometric signature, the notification selector116may select a notification102that corresponds to an instruction for providing a biometric input at a given step of the enrollment process. The input device110may output the selected notification102via the notification hardware114(e.g., as a unique pattern of lights and/or sounds). The detection hardware124detects the notification102output by the input device110. The UI application126matches the detected notification102to a corresponding content item stored on the display device120and renders the corresponding content item on the display122. For example, the content item may include the visual guide for providing the next biometric input of the enrollment process. More specifically, the visual guide may include detailed (e.g., graphical) instructions or directions for how to provide the next biometric input (e.g., including the angle, pose, or offset at which the input is to be provided). This process may be repeated until the enrollment operation is completed.

Accordingly, the display device120of the present embodiments may provide an on-demand electronic display when a rich UI is desired for the input device110(e.g., when performing complex tasks or user inputs101). As a display-less device, the input device110may retain a small form factor and relatively low hardware complexity. Further, aspects of the present disclosure may leverage the existing hardware of the input device110(e.g., used for simple notifications) to communicate UI-specific information to the display device120. Accordingly, various types of display devices may be configured to detect and interpret the notifications output by the input device110. Moreover, the input device110need not have a priori knowledge of the display device120in order to communicate UI-specific information to the display device120.

The present embodiments further enable the input device110to be used independently of the display device120when a rich UI is no longer needed. Aspects of the present disclosure recognize that, after the input device110has been trained to recognize or authenticate the user's biometric signature, the input device110may rarely (if ever) need to operate in the enrollment mode again. As described above, the display device120may provide little (if any) benefit to the operation of the input device110in the authentication mode. The input processing system100enables the input device110to be operated by itself (e.g., as a self-contained electronic system), without the display device120.

In a particular embodiment, the input device110may be a car key fob and the display device120may be a smart phone. The key fob may require user authentication (e.g., via a fingerprint scanner) before allowing access to one or more vehicle controls (such as unlocking the doors or turning on the engine). The smart phone may provide a rich UI during an enrollment process, such as by detecting patterns of light and/or sound emitted by the key fob, which may guide the user in providing biometric inputs (e.g., fingerprints) on the key fob. However, once the key fob has been trained to recognize or detect the user's biometric signature, the smart phone may no longer be needed for subsequent operation of the key fob. For example, the user may authenticate herself on the key fob, and thereafter gain access to the car, without the smart phone (or any other display device) present.

In addition to displaying instructions for interacting with the input device110, the display device120may also display information about the input device110. Example information may include, but is not limited to, battery level, number of attempted authentication attempts, and number of operations (e.g., door unlocks) performed per authentication. In some embodiments, the input device110and display device120may further be configured for bi-directional communications. For example, the display device120may include notification hardware similar to the notification hardware114of the input device, and the input device110may include detection hardware similar to the detection hardware124of the display device.

FIG.2is a state diagram200depicting an example operation of the input processing system100ofFIG.1, in accordance with some embodiments. More specifically, the state diagram200shows an example interaction between the input device110and the display device120for purposes of providing an on-demand electronic display for a display-less device. It is noted, from the example ofFIG.2, that the input device110is a complete electrical system and does not depend on the display device120to operate. In other words, none of the operating states (212-216) of the input device110depend on any of the operating states (222-226) of the display device120.

The input device110may receive a user input212, process the input data214, and generate a notification as a result of the processing216. In some embodiments of state212, the input sources112may receive a biometric input while the input device is operating in an enrollment mode. For example, the biometric input may contain one or more biometric signatures of the user. In some embodiments of state214, the input device110may be trained or otherwise configured to recognize the biometric signature of the user from a given type of biometric input. For example, the input device110may generate or update a biometric profile which may be used to authenticate subsequent biometric inputs by the user. In some embodiments of state216, the input device110may output a notification corresponding to an instruction for providing another user input on the input device110. In some aspects, the notification may be output using existing notification hardware (e.g., lights, speakers, RF transmitters, and the like) on the input device110. Then, returning to state212, the input device110may receive the next user input for the next step of the enrollment process.

The display device120may detect the notification222, identify UI content associated with the notification224, and display the UI content on an electronic display226. In some embodiments of state222, the display device120may detect the notifications when the detection hardware124is positioned within a threshold range of the notification hardware114. In some aspects, the notification may be detected using existing sensors (e.g., cameras, microphones, RF receivers, and the like) on the display device120. In some embodiments of state224, the display device120may execute the UI application236to identify the pattern or sequence of notifications (e.g., lights, sounds, RF signals, and the like) output by the input device110. For example, the UI application126may match the detected notification to a particular content item stored on, or otherwise accessible by, the display device120. In some embodiments of state226, the UI application126may render the matching content item on a display of the display device120. For example, the displayed content item may provide a visual guide and/or list of instructions for providing the next user input on the input device110.

It is noted that the input device110may receive the next user input (at212) after generating the notification (at216) and does not depend on any action from the display device120. In other words, the input device110may operate entirely independent of the display device120. Thus, while the display device120may display a rich UI which may help guide the user's interactions with the input device110, the rich UI is not necessary to the operation of the input device110. For example, an experienced user who is familiar with the enrollment process may already know how to provide each subsequent user input (at212) even without the assistance of the display device120.

FIG.3is a timing diagram depicting an example operation of a biometric authentication system300, in accordance with some embodiments. The biometric authentication system300includes an input device301and a display device302. The input device301and display device302may be example embodiments of the input device110and display device120, respectively, ofFIG.1. In some embodiments, the input device301may be a display-less device and the display device302may be configured to operate as an on-demand electronic display for the input device301.

At time t1, the input device301receives a selection input311. In some embodiments, the selection input311may be provided using one or more electrical or mechanical actuators of the input device301. For example, the selection input311may correspond to one or more button presses. The input device301may change its operation mode312in response to the selection input311. For example, the input device301may switch from an authentication mode to an enrollment mode. The input device301may then generate a notification313based, at least in part, on the mode change312. For example, the input device301may select a particular notification associated with a set of instructions and/or guide for enrolling or managing (e.g., adding and deleting) biometric signatures and/or profiles. The notification313may be output via notification hardware on the input device301. In some embodiments, the notification hardware may include lights, speakers, RF transmitters, and/or other existing hardware resources of the input device301(e.g., hardware that may serve a function other than communicating UI information to a display device).

The display device302detects the notification313and renders a UI update314based, at least in part, on the detected notification313. For example, the notification313may be detected via one or more sensors of the display device302. In some embodiments, the sensors may include cameras, microphones, RF receivers, and/or other existing hardware resources of the display device302(e.g., hardware that may serve a function other than receiving UI information from an input device). A UI application executing on the display device302matches the detected notification313to a particular content item stored on, or accessible by, the display device302. The UI application further renders the matching content item on an electronic display (e.g., as part of a rich UI). For example, the displayed content item may provide a visual guide and/or set of instructions for inputting a biometric signature on the input device301.

At time t2, the input device301receives a biometric input321. In some embodiments, the biometric input321may be provided using one or more biometric sensors of the input device301. For example, the biometric input321may correspond to a scan or recording of a user's fingerprint, face, voice, or other biometric feature. The input device may update a biometric profile322for the user based at least in part on the biometric input321. For example, the input device301may operate in the enrollment mode when the biometric input321is received. Thus, an authentication application executing on the input device301may be trained (or further refined) to recognize the user's biometric signature from the biometric input321. The input device301may then generate a notification323based, at least in part, on the profile update322. For example, the input device301may select a particular notification associated with a set of instructions and/or guide for providing another biometric input (e.g., as part of the enrollment process). The notification323may again be output via the notification hardware on the input device301.

The display device302detects the notification323and renders a UI update324based, at least in part, on the detected notification323. For example, the notification323may again be detected via the one or more sensors of the display device302. The UI application matches the detected notification323to another content item stored on, or accessible by, the display device302. The UI application further renders the matching content item on the electronic display (e.g., as part of the rich UI). For example, the display content item may provide a visual guide and/or set of instructions for inputting another biometric signature on the input device301. As described above, during the enrollment process, the user may be required to provide multiple biometric inputs with slight variations (e.g., in angle, pose, and/or offset). Thus, the UI update324may display to the user the exact angle, pose, and/or offset at which the next biometric input is to be provided on the input device301.

At time t3, the input device301receives another biometric input331. The biometric input331may again be provided using the biometric sensors of the input device301. However, at this time, the input device301is no longer operating in the enrollment mode. In some embodiments, the input device301may revert back to the authentication mode (e.g., from the enrollment mode) once the enrollment process is completed. In some aspects, the input device301may be configured to automatically switch to the authentication mode after receiving (and processing) the last biometric input of the enrollment process. In some other aspects, the input device301may be manually switched to the authentication mode via one or more button presses. Accordingly, the input device301may authenticate the user332based on the biometric input331received at time t3. For example, the authentication application of the input device301may authenticate the user by comparing the biometric input331with one or more biometric profiles stored on the input device301.

It is noted that, while operating in the authentication mode, the input device301may not communicate any UI information to the display device302. In some embodiments, the input device301may not output any notifications in response to the biometric inputs331received while operating in the authentication mode. In some other embodiments, the input device301may continue to output basic notifications (such as a green light or a red light) in response to biometric inputs331for purposes of indicating whether the authentication was successful. However, such notifications may be easily interpreted by the user, and may not trigger any UI updates on the display device302. Moreover, aspects of the present disclosure recognize that the input device301may remain in the authentication mode for a majority of its operation. Thus, the present embodiments provide the input device301the freedom to operate as a standalone display-less device, untethered from the display device302.

FIG.4Ais a block diagram of a biometric authentication system400A, in accordance with some embodiments. The biometric authentication system400A includes an input device410and a display device420. The input device410and display device420may be example embodiments of the input device110and display device120, respectively, ofFIG.1. In some embodiments, the input device410may be a display-less device and the display device420may be configured to operate as an on-demand electronic display for the input device410.

The input device410includes one or more actuators412, a biometric sensor414, a processing system416, and a light source418. The actuators412are configured to receive a selection input401. Example actuators may include, but are not limited to, buttons, switches, toggles, and capacitive sensors. The selection input401may be used to make a selection, input data, change an operation mode of the input device410, and/or cause the input device410to perform a particular function. The biometric sensor414is configured to receive a biometric input402. Example biometric sensors may include, but are not limited to, cameras, microphones, and fingerprint scanners. The biometric input402may include a user's biometric signature and may be used for authentication purposes. The light source418is configured to output light404. Example light sources may include, but are not limited to, LEDs, lasers, EL materials, and any other photonic devices capable of emitting light in the visible spectrum, the infrared spectrum, and/or the ultraviolet spectrum. The light output404may provide notifications about the inputs401and402and/or a status of the input device410.

The processing system416is configured to process the inputs401and402received via the actuators412and biometric sensor414, respectively. In some embodiments, the processing system416may be implemented as a set of firmware and/or software modules. Example modules may include hardware operation modules for operating hardware such as the actuators412, biometric sensor414and light source418, and data processing modules for processing data associated with the selection input401and biometric input402. In some embodiments, the processing system416may include an authentication application417to authenticate and/or enable a user to access one or more functions or features of the input device410. For example, the authentication application417may be configured to operate the actuators412and biometric sensor414; enroll the biometric signature of a user; authenticate a user based on the his or her biometric signature; and change an operation mode of the input device410, such as between an authentication mode and an enrollment mode.

In some embodiments, the authentication application417may further control the light output404from the light source418based, at least in part, on the inputs401and/or402. For example, the authentication application417may control the light source418to output a green light to acknowledge a selection input401and/or successful authentication attempt and to output a red light to indicate an error and/or unsuccessful authentication attempt. In some aspects, the authentication application417may further control the light source418to output a unique pattern of lights corresponding to a rich UI to be displayed on the display device420. For example, the authentication application417may change the color, intensity, duration, and/or number of pulses of the light output404for each UI-specific light pattern. In some embodiments, the input device410may include a light notification store415to store each of the light patterns that can be output by the light source418. For example, while operating in the enrollment mode, the authentication application417may retrieve a different light pattern from the light notification store415at each step of the enrollment process.

The display device420includes one or more actuators422, a display424, a processing system426, and a camera428. The actuators422are configured to receive user inputs403. Example actuators412may include, but are not limited to, buttons, switches, toggles, and capacitive sensors. The user inputs403may be used to control and/or operate the display device420. The display424may be configured to provide a rich UI on the display device420. Example displays may include any hardware capable of rendering digital images and/or displaying or projecting non-digital images. Example display technologies may include, but are not limited, LED, OLED, CRT, LCD, plasma, and EL displays. The rich UI may include digital images, text, video, and/or other media content. The camera428is configured to detect the light output404by the input device410. Example cameras may include, but are not limited to, photodiodes, CMOS image sensor arrays, CCD arrays, and any other photosensors capable of detecting wavelengths of light in the visible spectrum, the infrared spectrum, and/or the ultraviolet spectrum.

The processing system426is configured to process the light patterns detected by the camera428. In some embodiments, the processing system426may be implemented as a set of firmware and/or software modules. Example modules may include hardware operation modules for operating hardware such as the actuators422, display424, and camera428, and data processing modules for processing data associated with the detected light patterns. In some embodiments, the processing system426may include a user interface (UI) application427to enable or otherwise facilitate user interactions with the display device420. For example, the UI application427may be configured to operate the actuators422, display424, and camera428; render media content on the display424; and respond to user input403by triggering one or more actions. Example actions include changing an operation mode of the display device420and/or graphical user interface (GUI) actions such as cursor movement, selection, menu navigation, and the like.

In some embodiments, the UI application427may generate and/or update a rich UI on the display424to further facilitate user interactions with the input device410. In some aspects, the UI application427may select media content to be rendered on the display424based, at least in part, on the user inputs403received via the actuators422. For example, in response to the user input403, the UI application427may display a combination of selection inputs401for placing the input device410in an enrollment mode. In some other aspects, the UI application427may select the media content to be rendered on the display424based, at least in part, on the light output404by the input device410. For example, the UI application427may match the detected light pattern to a particular content item that can be rendered on the display424. In some embodiments, the display device420may include a light notification lookup table (LUT)425that can be used to match the detected light pattern to a corresponding content item. For example, each entry in the LUT425may correspond to a respective notification (e.g., light pattern) stored in the light notification store415.

In some other embodiments, the camera428may be used to detect a selection input401and/or biometric input402on the input device410. For example, the camera418may capture one or more images of a user interacting with the actuators412or biometric sensor414of the input device410. The UI application427may determine from the captured images (e.g., using image processing techniques) that the user interaction triggered a particular operation (or the next step of an operation) on the input device410. The UI application427may then render the appropriate media content on the display424based on the operation (or step of the operation) most recently triggered on the input device410. In this manner, the display device420may generate and/or update the rich UI on the display424without receiving any notifications from the input device410.

FIG.4Bis a block diagram of a biometric authentication system400B, in accordance with some embodiments. The biometric authentication system400B includes an input device430and a display device440. The input device430and display device440may be example embodiments of the input device110and display device120, respectively, ofFIG.1. In some embodiments, the input device430may be a display-less device and the display device440may be configured to operate as an on-demand electronic display for the input device430.

The input device430includes one or more actuators432, a biometric sensor434, a processing system436, and a speaker438. The actuators432and biometric sensor434may be example embodiments of the actuators412and biometric sensor414, respectively, ofFIG.4A. Thus, the actuators432are configured to receive a selection input401and the biometric sensor434is configured to receive a biometric input402. The speaker438is configured to output audio406. Example speakers may include, but are not limited to, full-range drivers, mid-range drivers, coaxial drivers, tweeters, woofers, and any other electromechanical devices capable of producing sounds in the audible range, supersonic range, and/or subsonic range. The audio output406may provide notifications about the inputs401and402and/or a status of the input device430.

The processing system436is configured to process the inputs401and402received via the actuators432and biometric sensor434, respectively. In some embodiments, the processing system436may be implemented as a set of firmware and/or software modules. Example modules may include hardware operation modules for operating hardware such as the actuators432, biometric sensor434and speaker438, and data processing modules for processing data associated with the selection input401and biometric input402. In some embodiments, the processing system436may include an authentication application437to authenticate and/or enable a user to access one or more functions or features of the input device430. For example, the authentication application437may be configured to operate the actuators432and biometric sensor434; enroll the biometric signature of a user; authenticate a user based on the his or her biometric signature; and change an operation mode of the input device430, such as between an authentication mode and an enrollment mode.

In some embodiments, the authentication application437may further control the audio output406from the speaker438based, at least in part, on the inputs401and/or402. For example, the authentication application437may control the speaker438to output a gentle beep to acknowledge a selection input401and/or successful authentication attempt and to output a harsh beep to indicate an error and/or unsuccessful authentication attempt. In some aspects, the authentication application437may further control the speaker438to output a unique pattern of sounds corresponding to a rich UI to be displayed on the display device440. For example, the authentication application437may change the pitch, volume, duration, and/or number of pulses of the audio output406for each UI-specific light pattern. In some embodiments, the input device430may include an audio notification store435to store each of the audio patterns that can be output by the speaker438. For example, while operating in the enrollment mode, the authentication application437may retrieve a different audio pattern from the audio notification store435at each step of the enrollment process.

The display device440includes one or more actuators442, a display444, a processing system446, and a microphone448. The actuators442and display444may be example embodiments of the actuators422and display424, respectively, ofFIG.4A. Thus, the actuators442are configured to receive user inputs403and the display444is configured to provide a rich UI on the display device440. The microphone448is configured to detect the audio output406by the input device430. Example microphones may include, but are not limited to, microphone arrays, omnidirectional, unidirectional, or bi-directional microphones, and any other transducers capable of converting sound waves in the audible range, supersonic range, and/or subsonic range into electrical signals.

The processing system446is configured to process the audio patterns detected by the microphone448. In some embodiments, the processing system446may be implemented as a set of firmware and/or software modules. Example modules may include hardware operation modules for operating hardware such as the actuators442, display444, and microphone448, and data processing modules for processing data associated with the detected audio patterns. In some embodiments, the processing system446may include a user interface (UI) application447to enable or otherwise facilitate user interactions with the display device440. For example, the UI application447may be configured to operate the actuators442, display444, and microphone448; render media content on the display444; and respond to user input403by triggering one or more actions. Example actions include changing an operation mode of the display device440and/or GUI actions such as cursor movement, selection, menu navigation, and the like.

In some embodiments, the UI application447may generate and/or update a rich UI on the display444to further facilitate user interactions with the input device430. In some aspects, the UI application447may select media content to be rendered on the display444based, at least in part, on the user inputs403received via the actuators442. For example, in response to the user input403, the UI application447may display a combination of selection inputs401for placing the input device430in an enrollment mode. In some other aspects, the UI application447may select the media content to be rendered on the display444based, at least in part, on the audio output406by the input device430. For example, the UI application447may match the detected audio pattern to a particular content item that can be rendered on the display444. In some embodiments, the display device440may include an audio notification LUT445that can be used to match the detected audio pattern to a corresponding content item. For example, each entry in the LUT445may correspond to a respective notification (e.g., audio pattern) stored in the audio notification store435.

FIG.4Cis a block diagram of a biometric authentication system400C, in accordance with some embodiments. The biometric authentication system400C includes an input device450and a display device460. The input device450and display device460may be example embodiments of the input device110and display device120, respectively, ofFIG.1. In some embodiments, the input device450may be a display-less device and the display device460may be configured to operate as an on-demand electronic display for the input device450.

The input device450includes one or more actuators452, a biometric sensor454, a processing system456, and an RF transmitter458. The actuators452and biometric sensor454may be example embodiments of the actuators412and biometric sensor414, respectively, ofFIG.4A. Thus, the actuators452are configured to receive a selection input401and the biometric sensor454is configured to receive a biometric input402. The RF transmitter458is configured to output an RF signal408. Example RF transmitters may include, but are not limited to, near-field communication (NFC) radios, Bluetooth radios, Wi-Fi radios, cellular radios, and other suitable RF radios capable of transmitting wireless communication signals. The RF signal408may provide notifications about the inputs401and402and/or a status of the input device450.

The processing system456is configured to process the inputs401and402received via the actuators452and biometric sensor454, respectively. In some embodiments, the processing system456may be implemented as a set of firmware and/or software modules. Example modules may include hardware operation modules for operating hardware such as the actuators452, biometric sensor454and RF transmitter458, and data processing modules for processing data associated with the selection input401and biometric input402. In some embodiments, the processing system456may include an authentication application457to authenticate and/or enable a user to access one or more functions or features of the input device450. For example, the authentication application457may be configured to operate the actuators452and biometric sensor454; enroll the biometric signature of a user; authenticate a user based on the his or her biometric signature; and change an operation mode of the input device450, such as between an authentication mode and an enrollment mode.

In some embodiments, the authentication application457may further control the transmission of the RF signal408by the RF transmitter458based, at least in part, on the inputs401and/or402. For example, the authentication application457may control the RF transmitter458to transmit a first coded signal to acknowledge a selection input401and/or successful authentication attempt and to transmit a second coded signal to indicate an error and/or unsuccessful authentication attempt. In some aspects, the authentication application457may further control the RF transmitter458to transmit a unique codeword corresponding to a rich UI to be displayed on the display device440. For example, the authentication application457may modulate the amplitude, frequency, and/or phase of the RF signal408for each UI-specific codeword. In some embodiments, the input device450may include an RF notification store455to store each of the codewords that can be output by the RF transmitter458. For example, while operating in the enrollment mode, the authentication application457may retrieve a different codeword from the RF notification store455at each step of the enrollment process.

The display device460includes one or more actuators462, a display464, a processing system466, and an RF receiver468. The actuators462and display464may be example embodiments of the actuators422and display424, respectively, ofFIG.4A. Thus, the actuators462are configured to receive user inputs403and the display464is configured to provide a rich UI on the display device460. The RF receiver468is configured to detect the RF signals408transmitted by the input device450. Example RF receivers may include, but are not limited to, NFC radios, Bluetooth radios, Wi-Fi radios, cellular radios, and other suitable RF radios capable of receiving wireless communication signals.

The processing system466is configured to process the RF codewords detected by the RF receiver468. In some embodiments, the processing system466may be implemented as a set of firmware and/or software modules. Example modules may include hardware operation modules for operating hardware such as the actuators462, display464, and RF receiver468, and data processing modules for processing data associated with the detected codewords. In some embodiments, the processing system466may include a user interface (UI) application467to enable or otherwise facilitate user interactions with the display device460. For example, the UI application467may be configured to operate the actuators462, display464, and RF receiver468; render media content on the display464; and respond to user input403by triggering one or more actions. Example actions include changing an operation mode of the display device460and/or GUI actions such as cursor movement, selection, menu navigation, and the like.

In some embodiments, the UI application467may generate and/or update a rich UI on the display464to further facilitate user interactions with the input device450. In some aspects, the UI application467may select media content to be rendered on the display464based, at least in part, on the user inputs403received via the actuators462. For example, in response to the user input403, the UI application467may display a combination of selection inputs401for placing the input device450in an enrollment mode. In some other aspects, the UI application467may select the media content to be rendered on the display464based, at least in part, on the RF signals408transmitted by the input device450. For example, the UI application467may match the detected codeword to a particular content item that can be rendered on the display464. In some embodiments, the display device460may include an RF notification LUT465that can be used to match the detected codeword to a corresponding content item. For example, each entry in the LUT465may correspond to a respective notification (e.g., codeword) stored in the RF notification store455.

FIG.5is a block diagram of a display device500that can be used in an input processing system, in accordance with some embodiments. The display device500may be an example embodiment of the display device120ofFIG.1and/or any of the display device420,440, or460ofFIGS.4A-4C. In some embodiments, the display device500may be configured to provide an on-demand display for a display-less input device.

The display device500includes a display510, a device interface520, a processor530, and a memory540. The display510may be any type of electronic display capable of displaying a rich UI. Example suitable display technologies may include, but are not limited to, LED, OLED, CRT, LCD, plasma, and EL. In some aspects, the display510may be integrated as a physical part of the display device500. In some other aspects, the display510may be physically separated from the display device500. For example, the display510may be coupled to (and communicate with) the display device500using various wired and/or wireless interconnection and communication technologies.

The device interface520may include a camera interface524, a microphone interface526, and an RF receiver interface528. The camera interface524may be used to communicate with a camera of the display device500(such as the camera428ofFIG.4A). For example, the camera interface524may transmit signals to, and receive signals from, the camera to capture light-based notifications (e.g., light output404) from the input device. The microphone interface526may be used to communicate with a microphone of the display device500(such as the microphone448ofFIG.4B). For example, the microphone interface526may transmit signals to, and receive signals from, the microphone to acquire audio-based notifications (e.g., audio output406) from the input device. The RF receiver interface528may be used to communicate with an RF receiver of the display device (such as the RF receiver468ofFIG.4C). For example, the RF receiver interface528may transmit signals to, and receive signals from, the RF receiver to detect RF-based notifications (e.g., RF signal408) from the input device.

The memory540includes a UI content store541configured to store media content (e.g., images, videos, text, and the like) that can be rendered on the display510. In some embodiments, the UI content store541may store one or more content items associated with notifications detected from the input device (e.g., via the device interface520). In some aspects, the one or more content items may include instructions for providing user input on, or otherwise interacting with, the input device. For example, the UI content store541may include a light notification LUT542, an audio notification LUT543, and/or an RF notification LUT544.

The light notification LUT542may be an example embodiment of the light notification LUT425ofFIG.4A. For example, the light notification LUT542may be used to match light patterns detected via the camera interface524to UI-specific content items stored in the UI content store541. The audio notification LUT543may be an example embodiment of the audio notification LUT445ofFIG.4B. For example, the audio notification LUT543may be used to match audio patterns detected via the microphone interface526to UI-specific content items stored in the UI content store541. The RF notification LUT544may be an example embodiment of the RF notification LUT465ofFIG.4C. For example, the RF notification LUT465may be used to match RF codewords detected via the RF receiver interface528to UI-specific content items stored in the UI content store541.

The memory540may also include a non-transitory computer-readable medium (e.g., one or more nonvolatile memory elements, such as EPROM, EEPROM, Flash memory, a hard drive, etc.) that may store at least the following software (SW) modules:a user interface SW module545to generate and/or update a rich UI on the display510, the user interface SW module545further including:a light detection sub-module546to detect light-based notifications output by the input device and render, on the display510, a content item associated with the detect light pattern;an audio detection sub-module547to detect audio-based notifications output by the input device and render, on the display510, a content item associated with the detect audio pattern; andan RF detection sub-module548to detect RF-based notifications transmitted by the input device and render, on the display510, a content item associated with the detect RF codeword.
Each software module includes instructions that, when executed by the processor530, cause the display device500to perform the corresponding functions. The non-transitory computer-readable medium of memory540thus includes instructions for performing all or a portion of the operations described below.

The processor530may be any suitable one or more processors capable of executing scripts or instructions of one or more software programs stored in the display device500(e.g., within memory540). For example, the processor530may execute the user interface SW module545to generate and/or update a rich UI on the display510. In executing the user interface SW module545, the processor530may further execute the light detection sub-module546, the audio detection sub-module547, and/or the RF detection sub-module548.

For example, the processor530may execute the light detection sub-module546to detect light-based notifications output by the input device and render, on the display510, a content item associated with the detect light pattern. Further, the processor530may execute the audio detection sub-module547to detect audio-based notifications output by the input device and render, on the display510, a content item associated with the detect audio pattern. Still further, the processor530may execute the RF detection sub-module548to detect RF-based notifications transmitted by the input device and render, on the display510, a content item associated with the detect RF codeword.

FIG.6is an illustrative flowchart depicting an example operation600for processing user inputs, in accordance with some embodiments. With reference for example toFIG.1, the operation600may be performed by an input processing system100that includes an input device110and a display device120. In some embodiments, the example operation600may be used to provide an on-demand rich UI for a display-less input device.

The system100receives a first user input at a first device (610). For example, the first user input may be provided using one or more input sources112of the input device110. In some embodiments, the input sources112may include electrical or mechanical actuators (e.g., buttons, switches, toggles, capacitive sensors, and the like). For example, the actuators may be used to provide data and/or selection inputs to the input device110. In some other embodiments, the input sources112may include device sensors (e.g., accelerometers, gyroscopes, magnetometers, photosensors, temperature sensors, pressure transducers, and the like). Still further, in some embodiments, the input sources112may include biometric sensors (e.g., cameras, microphones, fingerprint scanners, and the like). For example, the biometric sensors may be used to provide biometric inputs to the input device (e.g., for authentication purposes).

The system100selects a notification from a plurality of notifications stored on the first device based at least in part on the first user input (620). In some embodiments, the notifications may be used to guide a user to provide additional user inputs and/or perform complex tasks on the input device110. More specifically, at least some of the notifications stored on the input device110may correspond to rich media content that can be rendered or displayed on the display device120. For example, the rich media content may include images, videos, and/or text that provides a visual guide or detailed instructions for providing a subsequent user input, or otherwise interacting with, the input device110.

The system100then outputs the selected notification using one or more hardware resources of the first device (630). In some embodiments, the selected notification may be output using existing notification hardware114of the input device110. For example, the notification hardware114may include one or more light sources, speakers, or RF transmitters that can be used to output, or otherwise communicate, the UI-specific notification information. In some aspects, the notification may be output as a unique pattern of lights (e.g., as described with respect toFIG.4A). In some other aspects, the notification may be output as a unique pattern of sounds (e.g., as described with respect toFIG.4B). Still further, in some aspects, the notification may be output as a unique RF codeword (e.g., as described with respect toFIG.4C).

The system100detects the notification at a second device (640). In some embodiments, the notification may be detected using existing detection hardware124of the display device120. For example, if the notification comprises a pattern of lights, the display device120may detect the notification using one or more cameras or photosensors (e.g., as described with respect toFIG.4A). On the other hand, if the notification comprises a pattern of sounds, the display device120may detect the notification using one or more microphones (e.g., as described with respect toFIG.4B). Still further, if the notification comprises one or more RF signals, the display device120may detect the notification102using one or more RF receivers (e.g., as described with respect toFIG.4C).

The system100further matches the notification to a content item stored on the second device (650). More specifically, the display device120may be configured to interpret and/or process the UI-specific notifications output by the input device110. In some aspects, the display device120may select the appropriate content to be rendered on its display122based, at least in part, on the detected notification. For example, the display device120may use a LUT to match the detected notification to a particular content item (e.g., image, video, or other media content) stored on, or otherwise accessible by, the display device120. In some embodiments, the content item may include instructions for providing a second (e.g., subsequent) user input on the input device110.

The system100then renders the content item on a display of the second device (660). In some embodiments, the display device120may generate and/or update a rich UI rendered on its display122. As described above, the rich UI may include images, videos, and/or text corresponding to a visual guide or detailed instructions for providing a subsequent user input, or otherwise interacting with, the input device110. For example, the rich UI may display a step-by-step guide for initiating and/or completing a biometric enrollment process (e.g., as described with respect toFIG.3).

The system100may further receive the second user input at the first device (670). For example, the second user input may be provided using the one or more input sources112of the input device110(e.g., according to the instructions displayed on the display device). In some embodiments, the second user input may include a data and/or selection input provided using electrical and/or mechanical actuators of the input device110. In some other embodiments, the second user input may include a biometric input provided using a biometric sensor of the input device110. It is noted, however, that the input device110may not depend on any interaction with the display device120to receive any of the user inputs (e.g., as described with respect toFIG.2).

Those of skill in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

Further, those of skill in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosure.

The methods, sequences or algorithms described in connection with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor.

In the foregoing specification, embodiments have been described with reference to specific examples thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader scope of the disclosure as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.