Patent Description:
One recent improvement in Ul has been the development of haptic technologies that engages a user's sense of touch when a Ul element on a touchscreen display has been tapped, slid, or otherwise pressed by a user. For example, smartphones may use piezoelectric benders, eccentric rotation mass motors, or linear resonant actuators to produce vibration as feedback to the user that a screen icon has been selected. While these haptic elements provide feedback that a screen icon has been selected, they cannot provide the tactile cues that a physical button provides before the physical button has been pushed. Accordingly, it is desirable to provide a haptic element to a touchscreen display that provides more comprehensive tactile cues to the user than conventional approaches. Haptic user interfaces are disclosed in <CIT>, <CIT> and <CIT>.

A system is provided as defined by claim <NUM>.

In some embodiments of the system, the system further includes a controller configured to output a driving signal to the plurality of linear actuator elements.

In some embodiments of the system, the display head is configured to display an image that overlaps the protrusion, wherein the image indicates a functional characteristic of the protrusion.

In some embodiments of the system, the system is coupled to a device. In some embodiments of the system, the reduction of the protrusion drives a change in at least one parameter of the device.

In some embodiments of the system, the system further includes an instrument panel. In some embodiments of the system, the haptic display is integrated within the instrument panel.

In some embodiments of the system, the instrument panel is configured within an aircraft.

In some embodiments of the system, the system further includes a speaker. In some embodiments of the system, the speaker is configured to emit a sound upon the reduction of the protrusion.

In some embodiments of the system, upon the reduction of the protrusion, the remaining pins within the set of the plurality of pins are reduced to a default height.

In some embodiments of the system, the protrusion comprises one or more sections.

In some embodiments of the system, the display head is configured to display a display element that overlaps the one of the one or more sections. In some embodiments of the system, the display element indicates a functional characteristic of the one of the one or more sections.

In some embodiments of the system, the system is coupled to a device. In some embodiments, the reduction of the protrusion corresponding to the one of the one or more sections drives a change in at least one parameter of the device.

In some embodiments of the system, a bias of the plurality of pins is adjusted based on at least one of pin position, pin translation velocity, the biasing force of the user, and a functional characteristic of the protrusion.

A method for operating a haptic display is also provided as defined by claim <NUM>.

In some embodiments of the method for operating a haptic display, the method includes resetting a position of the pin subset.

In some embodiments of the method for operating a haptic display, the method includes modifying the display element based on at least one of the display element shape, or position of the display element on the display head. In some embodiments of the method for operating a haptic display, the method includes modifying the protrusion based on a modification of the display element.

A display system with haptic control is disclosed. The display system includes a touchscreen display that is flexible and an actuating layer comprised of a series of translatable pins configured to press up against the flexible display, forming a physical button or hard key than can be pushed by a user, simulating a traditional physical button on an instrument panel. The display system is reconfigurable for different layouts or button depending on the needs of the user.

<FIG> is a drawing illustrating a system <NUM> including a haptic display <NUM>, in accordance with one or more embodiments of the disclosure. The haptic display <NUM> may be used in any environment. For example, the haptic display <NUM> may be configured as a portion of an instrument panel or integrated into the instrument panel. For instance, the haptic display <NUM> may be configured as a portion of an instrument panel in a vehicle such as an aircraft, ship, or automobile. In another instance, the system <NUM> may include the instrument panel, the vehicle, and/or other device.

In some embodiments, the haptic display <NUM> includes a display head <NUM> configure to output an image to a user <NUM>. The display head <NUM> may be any type of display technology including but not limited to a liquid crystal display (LCD), a light emitting diode (LED) display, a plasma display panel (PDP), and electroluminescent panel, an organic light-emitting diode (OLED) display, a quantum dot light-emitting diode (QLED) display, an active-matrix liquid crystal display (AMLCD), a digital light processing (DLP) display, a liquid crystal on silicon (LCOS) display, an electronic paper display (e.g., E ink display, or Gyricon display), and active-matrix organix light-emitting diode (AMOLED) display, an active-matrix electroluminescent display (ELD), an interfrometric modulator display (IMOD), a field emission display (FED), or a surface-conduction electron-emitter display (SED). In some embodiments, the display head <NUM> is touch sensitive, allowing the user to select icons, menu items, and other image elements on the display head <NUM>.

In some embodiments, the display head <NUM> is configured to be flexible. For example, the display head <NUM> may be configured to form one or more buttons 116a-b (e.g., hard keys), one or more keys 120a-c or other physical element (e.g., protrusions) commonly known to appear on an instrument panel. For instance, the display head <NUM> may be configured as a flexible/malleable LCD display that allows a force from the underside (e.g., nonviewing side) of the display head <NUM> to raise a portion of the display head <NUM> relative to the viewing plane of the display head. Once formed, the button 116a-b, key <NUM> a-c, or other physical element may be acted upon by the user. For example, the user <NUM> may push down on the button 116a, which would result in a decreased height of the button 116a (e.g., decrease the height of the protrusion) and signal to circuitry within the haptic display <NUM> that the button 116a has been pushed. Any type of physical element may be formed by the display head <NUM> in addition to buttons <NUM> a-b and keys <NUM> a-c including but not limited to switches (e.g., toggle, rotary) dials, knob, levers, and slides. It should be understood that the term button 116a-b may refer to any physical element formed by the display head <NUM>.

In some embodiments, the pressing of the button 116a-b, key <NUM> a-c, or other physical element (e.g., causing a reduction of the protrusion) results in a corresponding change in the device by which the system <NUM> is coupled. For example, if the button 116a is designated as power button to a radio communicatively coupled to the system <NUM>, pressing the button 116a may then result in a powering on of the radio. In another example, if the system <NUM> is communicatively coupled to an external display and the key 120a is designated to input the letter "A" when pressed, then pressing the key 120a may result in the letter "A" appearing on the external display.

In some embodiments, the haptic display <NUM> includes an actuation layer <NUM> configured to provide protrusion-forming force upon the display head <NUM>. In some embodiments, the actuation layer includes a pin board <NUM> composed of a plurality of openings, and a plurality of pins <NUM> that are threaded into the plurality of openings. The plurality of pins <NUM> are configured to translate within the plurality of openings. For example, the button 116b may be formed by a set of the plurality of pins <NUM> pressing against the underside of the display head <NUM>. The pin board <NUM> may be configured of any size, shape, number of openings, or pattern of openings. Likewise, the plurality of pins may be configured of any size (e.g., thickness or length) or number. The plurality of openings may be configured as any shape or size. The haptic display <NUM> may further include a backplane configured to provide a physical substrate for attaching display circuitry, actuators, and other components.

In some embodiments, the actuation layer <NUM> further includes a plurality of linear actuator elements <NUM> configured to translate the plurality of pins <NUM>. The plurality of linear actuator elements may be separate from, or integrated into the pin board <NUM>. The plurality of linear actuator elements <NUM> may include any technology capable of translating the plurality of pins <NUM> relative to the pin board <NUM>. For example, the plurality of linear actuator elements <NUM> may be configured as a plurality of electro-mechanical actuators. For instance, the electro-mechanical actuator may be configured as a screw, wheel and axle, or cam actuator powered by a small motor. In another example, the plurality of linear actuator elements <NUM> may be configured as a plurality of piezoelectric actuator. In another example, the plurality of linear actuator elements <NUM> may be configured as a plurality of microelectromechanical system (MEMS) actuators. For instance, the MEMS actuator may be configured as a MEMS magnetic actuator.

<FIG> is a block diagram illustrating the system <NUM>, in accordance with one or more embodiments of the disclosure. In some embodiments, the haptic display includes a controller <NUM> that includes one or more processors <NUM>, a memory <NUM>, and a communication interface <NUM>. The controller <NUM> is configured to provide processing functionality for at least the haptic display (e.g., the controller may provide a driving signal to the plurality of linear actuator elements) and can include the one or more processors <NUM> (e.g., micro-controllers, circuitry, field programmable gate array (FPGA), central processing units (CPU), application-specific integrated circuit (ASIC), or other processing systems), and resident or external memory <NUM> for storing data, executable code, and other information. The controller <NUM> can execute one or more software programs embodied in a non-transitory computer readable medium (e.g., memory <NUM>) that implement techniques described herein. The controller <NUM> is not limited by the materials from which it is formed or the processing mechanisms employed therein and, as such, can be implemented via semiconductor(s) and/or transistors (e.g., using electronic integrated circuit (IC) components), and so forth.

The memory <NUM> can be an example of tangible, computer-readable storage medium that provides storage functionality to store various data and/or program code associated with operation of the controller <NUM>, such as software programs and/or code segments, or other data to instruct the controller <NUM>, and possibly other components of the control module <NUM>, to perform the functionality described herein. Thus, the memory <NUM> can store data, such as a program of instructions for operating the haptic display <NUM>, including its components (e.g., controller <NUM>, communication interface <NUM>, etc.), and so forth. The memory <NUM> may also store data derived from the haptic display <NUM>. It should be noted that while a single memory <NUM> is described, a wide variety of types and combinations of memory <NUM> (e.g., tangible, non-transitory memory) can be employed. The memory <NUM> may be integral with the controller <NUM>, may comprise stand-alone memory, or may be a combination of both. Some examples of the memory <NUM> may include removable and non-removable memory components, such as random-access memory (RAM), read-only memory (ROM), flash memory (e.g., a secure digital (SD) memory card, a mini-SD memory card, and/or a micro-SD memory card), solid-state drive (SSD) memory, magnetic memory, optical memory, universal serial bus (USB) memory devices, hard disk memory, external memory, and so forth.

The communication interface <NUM> may be operatively configured to communicate with components of the haptic display <NUM> and the system <NUM>. For example, the communication interface <NUM> can be configured to retrieve data from the controller <NUM> or other components, transmit data for storage in the memory <NUM>, retrieve data from storage in the memory <NUM>, and so forth. The communication interface <NUM> can also be communicatively coupled with the controller <NUM> to facilitate data transfer between components of the haptic display <NUM> and the controller <NUM>. It should be noted that while the communication interface <NUM> is described as a component of the haptic display <NUM>, one or more components of the communication interface <NUM> can be implemented as external components communicatively coupled to the haptic display <NUM> via a wired and/or wireless connection. The haptic display <NUM> can also include and/or connect to one or more input/output (I/O) devices. In embodiments, the communication interface <NUM> includes or is coupled to a transmitter, receiver, transceiver, physical connection interface, or any combination thereof.

In some embodiments, the display head <NUM> and the actuation layer <NUM> are tandemly coordinated via the controller <NUM>. For example, a button 116a that is physically formed by the actuation layer <NUM> may also have a button icon displayed on the display head <NUM> that overlaps the position of the button 116a. In another example, a pressing of the button 116a by the user <NUM> may activate, or otherwise signal a response, to both the display head <NUM> and the actuation layer <NUM>. In another example, a change in the programming of the display head (e.g., to display a different set of button icons having different functions) would also result in a corresponding change of the actuation layer <NUM> to form buttons <NUM> a-b, or other haptic elements based on the current display of the display head.

<FIG> is a close-up side view of the display head <NUM>, pin board <NUM> and the plurality of pins <NUM>, under non-haptic activation conditions (e.g., no formation of a button <NUM> a-b or other physical element), in accordance with one or more embodiments of the disclosure. The plurality of pins <NUM> are inserted within the plurality of openings of the pin board <NUM>. In interest of clarity, the pin board is drawn as semi-transparent, with the plurality of pins showing through the pin board <NUM>. The display head <NUM> is disposed upon or immediately adjacent to the plurality pins <NUM>. For example, the display head may be disposed upon or immediately adjacent to a display interaction end <NUM> of one or more of the plurality of pins <NUM>. The one or more of the plurality or pins <NUM> may also include an actuation end <NUM> configured to interact with the one or more linear actuator elements <NUM> (not shown for clarity). The actuation end <NUM> may be separate from, or include, a pin board portion of one or more of the plurality of pins <NUM> that is inserted within the pin board <NUM>. It should be understood that the any size of the one or more of the plurality of pins, or the relative size of the one or more of the plurality of pins <NUM> as compared to the pin board <NUM> and/or display head <NUM> is possible, and that the illustrations herein are intended to merely the presence and/or interaction of components of the haptic display <NUM>. Therefore, the description herein should not be interpreted as a limitation of the present disclosure, but merely an illustration.

<FIG> is a close-up side view of the display head <NUM>, pin board <NUM> and the plurality of pins <NUM>, under haptic activation conditions (e.g., formation of a button <NUM> a-b or other physical element), in accordance with one or more embodiments of the disclosure. Upon activation of a subset of linear actuator elements <NUM>, a pin subset <NUM> of the plurality of pins <NUM> translates relative to the pin board <NUM> (e.g., via a biasing force provided by the linear actuator elements <NUM>), resulting in a stretching, flexing, bending, or other manipulation of a portion of the display head to form the button 116a. It should be understood that any overlapping of the one or more of the plurality of pins <NUM> and the pin board <NUM> are possible. For example, the actuation end <NUM> of one or more of the plurality of pins <NUM> may recess within the pin board <NUM> upon activation of the one or more linear actuator elements <NUM>. In another example, the plurality of pins <NUM> may have a length that does not permit one or more of the plurality of pins <NUM> to recess within the pin board <NUM> upon activation of the one or more linear actuator elements <NUM>. Similarly, the display interaction end <NUM> of the one or more of the plurality of pins may recess or not recess within the pin board. Therefore, the above description should not be interpreted as a limitation of the present disclosure, but merely an illustration.

<FIG> is a close-up side view of the display head <NUM>, the pin board <NUM>, and the plurality of pins <NUM>, wherein the button <NUM> is pressed upon by the user <NUM>, in accordance with one or more embodiments of the disclosure. Upon the pressing of the button 116a by the user <NUM> (e.g., the finger of the user providing a biasing force against the button 116a), the pin subset pin subset <NUM> of the plurality of pins <NUM> translates relative to the pin board <NUM>, and the portion of the display head <NUM> that was originally raised to form the button 116a is now partially or wholly reduced in height (e.g., to the nominal or default position). The haptic display <NUM> may be configured for any adjustment of the pin subset <NUM> and/or the portion of the display head forming the button 116a after the pressing of the button 116a. For example, the pressing of the button 116a may cause the pin subset <NUM> to retract fully so that the display interaction end <NUM> is nearly flush with the surface of the pin board <NUM>. In another example, the pressing of the button 116a may cause the pin subset to retract to a default position along with the rest of the plurality of pins <NUM>, as shown in <FIG>.

In embodiments, the haptic display <NUM> is configured to generate a tactile sensation for the user <NUM>. For example, the tactile sensation may be generated by the user initially touching a haptic-generated structure (e.g., the button 116a). For instance, the tactile sensation may be generated by the user touching the raised surface of the button 116a. In particular, the raised surface may allow a user <NUM> to easily find the button 116a by touch while the user <NUM> is performing another task (e.g., piloting an aircraft), and provides feedback to the user <NUM> that the button 116a has been identified. In another example, the tactile sensation may be generated by the user <NUM> causing a movement within the haptic-generated structure. For instance, the tactile sensation may be generated by the user <NUM> pushing the button 116a and detecting both the sense of movement of the button 116a as it is being pushed down, and the resistance from the button 116a as it is being pushed down. The combination of the resistance of the button 116a and the sense of movement of the finger of the user <NUM> during the pressing of the button 116a provides feedback to the user <NUM> that the user <NUM> is indeed pressing the button 116a.

In another example, the tactile sensation may be generated by the linear actuator elements <NUM> or other components within the actuation layer <NUM> based on a user input that reaches a specific threshold. For instance, the, once the user has pushed the button 116a with a predetermined force or displacement, the linear actuator elements <NUM> may alter the resistance to the button 116a such that the user detects a tactile sensation similar to a clicking sensation of a click button, providing feedback to the user that the button 116a has been successfully pushed. In another example, the haptic display <NUM> may provide a vibration or other sensation upon successful pressing of a button 116a or other haptic structure. For instance, the vibration may be generated any vibration method known and previously listed above. In particular, the vibration may be generated via a rapid back and forth cycling of one or more of the plurality of linear actuator elements <NUM> over a short distance. The haptic display <NUM> may be programmed to control the placement, speed, resistance, and any other characteristic of any one of the plurality of pins <NUM> in order to generate the tactile sensation for the user <NUM>.

In embodiments, the biasing force of the plurality of pins <NUM> (e.g., the force applied to one or more of the plurality of pins <NUM>) may be adjusted based on one or more pin parameters. The one or more pin parameters may be any characteristic of the pin or system including but not limited to pin position, pin translation velocity, the biasing force of the user, or the function characteristic of the button 116a. For example, for a button 116a in the process of being pressed, the biasing force of one or more of the plurality of pins <NUM> may lessen, providing feedback to the user <NUM> that the button 116a has been pushed. In another example, for a button 116a that is rapidly and forcefully pushed (e.g., high pin translation velocity or high biasing force by the user), the bias of one or more of the plurality of pins <NUM> may increase, preventing damage to the display head <NUM> and/or the pin board <NUM>. In another example, the haptic display <NUM> may be configured with two buttons 116a-b having differing basing forces. For instance, the haptic display <NUM> may include a button 116a having low biasing force that powers on a radio of an aircraft, and include a button 116b having high biasing force that shuts off the aircraft engine.

In some embodiments, the system <NUM> further includes a speaker communicatively coupled to the haptic display <NUM>. For example, when the user <NUM> has pushed the button 116a, the speaker may emit and audible click sound. In another example, when the user <NUM> has pushed the button 116a, the speaker may emit a voice message relating to the assigned function of the button 116a. The speaker may be configured as any type of speaker. For example, the speaker may be a speaker built into the haptic display <NUM>. In another example, the speaker may be a cockpit speaker, such as a speaker utilized by aircraft warning systems.

<FIG> illustrate a button 116a formed on a top surface <NUM> of the display head <NUM>, in accordance with one or more embodiments of the disclosure. The portion of the top surface <NUM> that covers the top of the button 116a may display a display element <NUM> that is relevant to the function of the button 116a (e.g., the "stop" icon in <FIG>). Before pressing by a user <NUM>, the button 116a may be stationed at a default/unpressed height (e.g., <FIG>). Upon the pressing of the button 116a by the user <NUM>, the button decreases in height (e.g., <FIG>) until the height of the button is at or near the planar surface of the top surface <NUM>. During the interaction of the user <NUM> with the button 116a, the display element <NUM> may change accordingly. Any change in the display element is possible. For example, pressing of the button 116a may result in a lessening of the contrast of the display element <NUM> (e.g., the greying-out of the "stop icon" in <FIG>).

In some embodiments, upon a pressing of the button 116a (e.g., reduction of the height of the protrusion by the user <NUM>), the remaining pins within the pin subset <NUM> are reduced to a default height. For example, when button 116a is successfully pushed by a user, the plurality of pins <NUM> that correspond to the area of the button 116a that was not physically touched by the user <NUM> will also be reduced in height similar to the pushed area. In this way, the button 116a retains its general physical shape regardless of the position or size of the finger pressing the button 116a. The system <NUM> may also be configured so that the linear actuator elements <NUM> and corresponding pins <NUM> are configured with a pressure sense and/or position sense, so that when a threshold is crossed on an individual pin <NUM> or a subset or pins <NUM>, that all of the pins <NUM> correlated with the button 116a,b will become momentarily unenergized and/or return to the default position, confirming to the user <NUM> that the button 116a,b has been pressed.

In some embodiments, the haptic display <NUM> is configured as a relatively flat surface wherein one or more buttons 116a-b forms a pocket when pushed, as shown in <FIG>. For example, the actuation layer <NUM> may be configured so that all, or a majority of, the plurality of pins <NUM> are raised, creating a large raised pin subset <NUM>. Correspondingly, the pin subset <NUM> raises all or a portion of the display head (e.g., as shown in <FIG>). Upon an input by the user <NUM> a section <NUM> of the pin subset <NUM> is pushed downward (e.g., as shown in <FIG>). For example, the section <NUM> may be configured with one or more of the plurality of pins <NUM> having a lower resistance to movement that the rest of the pin subset <NUM>.

<FIG> illustrates buttons 116a-b formed within the pin subset <NUM> of the display head <NUM>, in accordance with one or more embodiments of the disclosure. The buttons 116a-b may correspond to section 700a, 700b of the pin subset <NUM> and when in a default position, may have a top surface approximately the same plane as the pin subset, as shown in <FIG>. The buttons 116a,b may display functionally relevant display elements 604a, 600b (e.g., "stop" and "go"). Upon a touch or pressure upon the button 116a from a user <NUM>, the button 116a may reduce in elevation, forming a recess <NUM>, as shown in <FIG>. In some instances, the button 116a may be further pressed until the surface of the button is at or near the unraised surface of the display head <NUM>. After the button 116a has been pushed, the button 116a may stay depressed until an input from the user <NUM> (e.g., pressing the "go" button 116b), or from the system <NUM> is received.

<FIG> is a method <NUM> for operating the haptic display <NUM>, in accordance with one or more embodiments of the disclosure. In some embodiments, the method <NUM> includes a step <NUM> of generating a display element <NUM> on a display head <NUM>. For example, the controller <NUM> via the one or more processors <NUM> executing software stored on a memory <NUM>, may output a display element <NUM> (e.g., such as a "stop" button) to the display head.

In some embodiments, the method <NUM> includes a step <NUM> of translating a pin subset <NUM> of the plurality of pins <NUM> base on the display element <NUM>. For example, if the display element <NUM> is configured as a hexagonal stop sign on the top left corner of the top surface <NUM> of the display head <NUM>, then a pin subset <NUM> corresponding to the shape, size, and placement of the display element <NUM> will be translated towards, or differentially biased against, the area of the top surface <NUM> occupied by the display element <NUM>.

In some embodiments, the method <NUM> further includes a step <NUM> of forming a protrusion (e.g., button 116a-b, key 120a-c, or other haptic element), corresponding to the display element <NUM>. For example, the biasing force of the translation of the pin subset <NUM> against the flexible display head <NUM> may result in the formation of the button 116a.

In some embodiments, the method <NUM> further includes a step <NUM> of contacting the protrusion (e.g., button 116a) and display element <NUM> via the user <NUM>. For example, as when the user attempts to push the button 116a, the user will simultaneously touch both the display element <NUM> (e.g., as it appears on the top surface <NUM>) and the raised button 116a.

In some embodiments, the method <NUM> further includes a step <NUM> of biasing the protrusion (e.g., the buttons 116a-b, keys <NUM> a-c or other haptic elements) via the user <NUM>. For example, the user <NUM> may press upon the button 116a via the user's finger, resulting in the button 116a changing in elevation relative to the plane of the top surface <NUM>, as well as changes in the translation position of the pin subset <NUM> relative to other pins of the plurality of pins <NUM>.

In some embodiments, the method <NUM> further includes a step <NUM> of outputting a response based on at least one of the contacting the display element of the biasing of the protrusion (e.g., the buttons 116a-b, keys <NUM> a-c or other haptic elements). For example, if the button 116a is defined, via the controller, as a "stop" button for a radio, then the haptic display <NUM> may output a response or signal to the radio to shut off once the button 116a is pushed.

In some embodiments, the method <NUM> further includes a step <NUM> of resetting the pin subset <NUM>. For example, if the button 116a is defined as a radio "stop" button that has been pushed, the system <NUM> may later receive a signal to reset the pin subset via the activation layer. For example, the system <NUM> may receive an input that the radio, once powered off via the "stop button" 116a, has now been powered up, necessitating the need for the reformation of the button 116a.

In some embodiments, the method <NUM> further includes a step <NUM> of modifying the display element <NUM> based on at least one of the display element shape, or position of the display element <NUM> on the display head <NUM>. For example, if there is a need for the display head <NUM> to include a greater number of display elements <NUM>, the haptic display may be reset, via the controller, to both include the increased number of display elements <NUM> and/or to reduce the size of a specific display element <NUM>. In another example, the shape of a display element <NUM> may be altered due to user preference.

In some embodiments, the method <NUM> further includes a step <NUM> of modifying the protrusion (e.g., the buttons 116a-b, keys <NUM> a-c or other haptic elements) based on a modification of the display element. For example, a modification to the display element <NUM> based on user preference may result in a corresponding change in the button via the plurality of pins <NUM> as controlled by the plurality of linear actuator elements <NUM> via the controller <NUM>.

<FIG> is a block diagram of a display/hardware communication scheme <NUM> for use in operating the system <NUM>, in accordance with one or more embodiments of the disclosure. In some embodiments, the display/hardware communication scheme <NUM> includes display application software <NUM> configured to define buttons 116a,b (e.g., hard keys) by correlating individual linear actuator elements <NUM> within the actuation layer <NUM> with unique software defined keys via the actuator matrix configuration message <NUM>. For example, the actuator matrix configuration message <NUM> may include data (e.g., definitions, repeat counts, or ranges) that define the configuration of a specific group of linear actuator elements. Using a collection or pattern of correlated actuators, any general form and/or geometric button 116a,b shape is achievable including but not limited to a square, circle, line segment, or icon outline. The data application software <NUM> may supplement defined button 116a,b forms with graphical display aspects aligning with button 116a,b shape, such as border or label, onto the display head <NUM>.

In some embodiments, the display/hardware communication scheme <NUM> further includes an electrical actuator matrix hardware controller <NUM> configured to receive and process the actuator matrix configuration message <NUM>. For example, in response for receiving the actuator matrix configuration message <NUM>, the electrical actuator hardware controller will actuate one or more of the plurality of linear actuator elements <NUM> to form buttons 116a,b as defined in the actuator matrix configuration message <NUM>. The electrical actuator matrix hardware controller <NUM> may be configured to sense pressure placed upon one or more pins <NUM>. When the pressure against one or more pins exceeds a defined threshold, all similarly correlated actuators may be de-energized and/or return to the default position, which may signal a successful button 116a,b press. Upon the successful pressing of the button 116a,b, the electrical actuator matrix hardware controller asynchronously sends a hard key press confirmation message <NUM> that relays the identity of the pressed button 116a,b to the display application software <NUM>. The key layout and/or configuration may be dynamically controlled by the display application software during operation to form indefinite pages, menus, and configurations.

<FIG> is a flowchart illustrating a process <NUM> of preparing and sending a hard key press confirmation message <NUM>, in accordance with one or more embodiments of the disclosure. Once the electrical matrix hardware controller <NUM> has received the actuator matrix configuration message <NUM> (e.g., start <NUM>), the electrical matrix hardware controller <NUM> will perform a step <NUM> of processing the actuator matrix configuration message <NUM>, and send a result of the processing to an actuator correlation store <NUM>. The electrical matrix hardware controller <NUM> will also determine via a step <NUM> and whether the linear actuator elements <NUM> are correctly correlated in accordance with the system <NUM>, the actuator matrix configuration message <NUM>, and/or data from the actuator correlation store <NUM>.

Upon the determination that the linear actuator elements <NUM> are correctly correlated, the electrical matrix hardware controller <NUM> will perform a step <NUM> of determining whether pressure against the one or more pins <NUM> has reached a predefined threshold. If the pressure against the one or more pins <NUM> has reached a threshold, the electrical matrix hardware controller will perform a step <NUM> or signaling to the corresponding linear actuator elements <NUM> to return the corresponding pins <NUM> to the nominal and/or default position. One the corresponding pins <NUM> are placed back into the default position, the electrical matrix hardware controller <NUM> may perform a step <NUM> of sending the hard key press confirmation message <NUM> to the display application software <NUM>. The electrical matrix hardware controller <NUM> may also perform a step <NUM> of determining if a momentary wait time has expired. For example, a momentary wait time of one second may be set after the button 116a,b has been pushed to give the user an affirmative sign that the button 116a,b has been successfully pushed. After determining that the wait time has expired, the electrical matrix hardware controller <NUM> may perform a step <NUM> of reactivating the correlated liner actuator elements <NUM>, restarting a portion or, or the entirety of, the process <NUM>.

Claim 1:
A system, comprising:
a haptic display (<NUM>) comprising:
a display head (<NUM>) comprising a flexible surface;
an actuator layer (<NUM>) comprising:
a pin board (<NUM>) configured with a plurality of openings;
a plurality of pins (<NUM>) configured to translate within the plurality of openings, wherein one or more of the plurality of pins comprises:
a display interaction end (<NUM>) configured to physically interact with the display head; and
an actuation end (<NUM>); and
a plurality of linear actuator elements (<NUM>) physically coupled to the actuation end of the plurality of pins configured to bias a set of the plurality of pins against the display head, wherein a bias of the set of the plurality of pins against the display head generates a protrusion on the flexible surface, wherein the protrusion is configured to be reduced by a biasing force of a user pressing on the protrusion,
characterised in that
the biasing force of the set of the plurality of pins against the display head is configured to be adjusted during the process of being pressed by the user responsive to different biasing forces of the user pressing on the protrusion.