Patent Description:
An input display device in which a protrusion is provided on a touch panel disposed to be superposed on a display, and an image such as an operation icon is displayed at a position overlapping the protrusion has been disclosed (for example, <CIT>). A user performs an input by performing a touch operation on the protrusion.

<CIT> discloses a tactile user interface which can be applied over a touch screen or a display.

In a display device that allows a capacitive touch operation to be performed, a user interface that allows a tough position to be ascertained without looking, by allowing the touch position to be tactilely recognized due to a cover glass having an uneven shape (referred to below as a three-dimensional UI) has been proposed.

<FIG> illustrates an operation example of a conventional flat touch panel. A user U visually recognizes an operation icon <NUM> displayed on a display <NUM> and performs an input by performing a touch operation on a position of the operation icon <NUM> (note in the example of <FIG>).

<FIG> illustrates an operation example of a touch panel having a three-dimensional UI, and <FIG> is a schematic cross-sectional view of the three-dimensional UI. A transparent cover lens <NUM> having an uneven shape is attached onto a touch panel <NUM>, and a display <NUM> displays an operation icon <NUM> at a position overlapping the cover lens <NUM>. The user U performs an input by touching the cover lens <NUM> with a finger. A highly sensitive electrostatic sensor capable of detecting the capacitance (distance) of a finger even if a distance from the sensor is long is used for the touch detection. Thus, it is possible to determine whether or not there is a touch even on the thick cover lens <NUM>. This makes it possible to reduce touch mistakes (operation mistakes) under a situation in which it is difficult to look at an in-vehicle display during driving.

As described above, in the three-dimensional UI, in order to realize shapes imitating various physical switches (buttons, knobs, sliders, and the like) existing in a vehicle interior and operation detection thereof by the touch panel, it is required to handle various operation gestures.

As one of the operation gestures, as illustrated in <FIG>, in-vehicle switches include a switch <NUM> of a type that pulls (or lifts) an end portion of the switch upward, such as a power window switch. When a shape imitating such a pulling type switch <NUM> is realized by the touch panel, as illustrated in <FIG>, it is necessary to attach a cantilevered cover glass <NUM> (the same applies to resin or the like) that catches a finger to the surface of a touch panel <NUM> mounted on a display <NUM>.

However, there is a distance between a sensor that detects the capacitance and a touch position of the finger and the finger is lifted in a direction (arrow direction) opposite to the sensor. Thus, it is difficult to accurately detect a finger pulling operation by a configuration of the cover glass <NUM>.

As a solution, as illustrated in <FIG>, it is conceivable to mount a touch sensor <NUM> that detects the capacitance on the back surface of a cantilevered operation portion <NUM> of the cover glass <NUM>, but this is not realistic because the number of parts of the touch sensor <NUM> increases and the structure becomes complicated.

From the above description, there is a need for a structure and a method capable of more accurately detecting a pulling operation while maintaining a configuration imitating the pulling type switch <NUM> as illustrated in <FIG>.

An object of the present invention is to provide an input display device capable of detecting a pulling operation on a three-dimensional operation portion.

The invention relates to an input display device according to the appended claims. Embodiments are disclosed in the dependent claims. According to an aspect of the present invention, an input display device includes a display configured to display an image, a capacitive touch panel configured to be attached onto the display and include at least one three-dimensional operation portion on a surface of the touch panel, and a detection unit configured to detect an operation on the touch panel, characterized in that the three-dimensional operation portion includes a pulling portion imitating a shape of a pulling type switch, and the detection unit determines whether or not there is a pulling operation by detecting a change in a capacitance in response to a change in a distance to the touch panel from a finger touching the pulling portion.

In one embodiment, the operation portion is attached to be separable from the touch panel via an elastic member. In one embodiment, the elastic member is provided between a bottom surface of the operation portion and a fixing portion that supports the display. In one embodiment, the operation portion includes the pulling portion and a bottom surface portion disposed below the pulling portion, and the elastic member is provided between the pulling portion and the bottom surface portion. In one embodiment, the operation portion includes the pulling portion and a bottom surface portion disposed below the pulling portion, and the elastic member is provided between the pulling portion and a fixing portion that supports the display. In one embodiment, the pulling portion is configured by a member that is elastically deformable in a pulling direction. In one embodiment, the detection unit determines that a pulling operation has been performed when a decrease in capacitance of the pulling portion is equal to or greater than a threshold value. In one embodiment, the detection unit determines that a pressing operation has been performed when an increase in capacitance of the pulling portion is equal to or greater than a threshold value. In one embodiment, the detection unit determines whether or not there is a pulling operation by detecting a change in a capacitance due to a change in a shape of a finger touching the pulling portion. In one embodiment, the input display device further includes a display unit configured to displays an icon representing an input operation at a position corresponding to the operation portion of the display.

According to aspects of the present invention, since whether or not there is a pulling operation is determined by detecting a change in a capacitance in response to a change in a distance to the touch panel from a finger touching a pulling portion, it is possible to more accurately detect the pulling operation with a simple structure without increasing the number of parts of a touch sensor.

Next, embodiments of the invention will be described. An input display device according to aspects of the present invention provides an interface between a person and a machine. The input display device in the present invention is not particularly limited, but is applied to, for example, an electronic device equipped with a display with a touch panel. The electronic device equipped with the display with the touch panel is, for example, an in-vehicle device having a navigation function, an audio visual function, a television function, and the like.

Next, examples of the present invention will be described in detail with reference to the drawings. <FIG> is a block diagram illustrating a configuration of an input display device according to an example of the present invention. In the present example, an input display device <NUM> includes a display <NUM> that displays an image and a video, a capacitive touch panel <NUM> mounted on the display <NUM>, a three-dimensional UI portion (or an operation portion) <NUM> that has one or a plurality of three-dimensional shapes and is attached to the surface of the touch panel <NUM>, and a controller <NUM> that controls image display of the display <NUM>, touch detection of the touch panel <NUM>, and the like.

The display <NUM> is not particularly limited, but includes, for example, a liquid crystal panel or an organic EL panel. The display <NUM> displays image data provided from the controller <NUM>. For example, an icon representing an input operation of the three-dimensional UI portion <NUM> is displayed below the three-dimensional UI portion <NUM>.

The touch panel <NUM> includes, for example, a plurality of detection units (sensors) formed at positions where a plurality of X-side and Y-side electrode lines intersect. The detection unit changes a capacitance when a finger, a hand, or the like of a user approaches or comes into contact with the touch panel <NUM>. The touch panel <NUM> is mounted on the display <NUM> and provides an input interface for the user to make an input to an icon or the like displayed on the display <NUM>.

The touch panel <NUM> further includes one or a plurality of three-dimensional UI portions (operation portions) <NUM> on the surface of a transparent panel. The three-dimensional UI portion <NUM> has a shape that allows a user to perform a pulling operation with a finger, and includes, for example, a cantilevered pulling portion <NUM> that pulls (or lifts) an end portion upward as illustrated in <FIG>. The pulling portion <NUM> imitates the shape of a power window switch as illustrated in <FIG>. The number, the size, and the like of the three-dimensional UI portions <NUM> are not particularly limited. The three-dimensional UI portion <NUM> is configured such that, when the hand or the finger of the user performs an operation of pulling or lifting the three-dimensional UI portion <NUM>, the capacitance at that position changes.

<FIG> illustrates an example in which four three-dimensional UI portions <NUM> are attached below the touch panel <NUM>. The three-dimensional UI portion <NUM> is configured by using, for example, a transparent material such as acrylic, polycarbonate, or glass. An icon representing an input operation of the user is displayed at a position of the display <NUM> corresponding to the three-dimensional UI portion <NUM>. However, the entirety of the three-dimensional UI portion <NUM> does not need to be transparent, and a portion of the three-dimensional UI portion <NUM>, which does not interfere with the icon may be non-transparent. The user visually recognizes the icon displayed below the three-dimensional UI portion <NUM> and performs an input by performing a pulling operation on the three-dimensional UI portion <NUM>.

Next, an outline of the input display device <NUM> in the present example will be described with reference to <FIG> illustrates a schematic cross-section of the three-dimensional UI portion <NUM>. As illustrated in <FIG>, the display <NUM> is fixed on a product cover (or cover glass or the like) <NUM>, and the touch panel <NUM> is mounted on the product cover <NUM>. The three-dimensional UI portion <NUM> is attached to the surface of the touch panel <NUM>.

The three-dimensional UI portion <NUM> includes, for example, a bottom surface portion <NUM> and a pulling portion <NUM> extending from the bottom surface portion <NUM> in a cantilever manner. The bottom surface portion <NUM> is bonded to a determined position of the touch panel <NUM> by using, for example, a double-sided adhesive or the like. The coordinates indicating the attachment position of the bottom surface portion <NUM> and the shape of the bottom surface portion <NUM> are registered in the controller <NUM> in advance. For example, when the bottom surface portion <NUM> has a circular shape, the coordinates of the center and the radius of the bottom surface portion <NUM> are registered. When the bottom surface portion <NUM> has a rectangular shape, the coordinates of the intersection point between the diagonal lines and the coordinates of the corner of the bottom surface portion <NUM> are registered. Furthermore, the height of the pulling portion <NUM> of the three-dimensional UI portion <NUM> is also registered.

In the present example, the three-dimensional UI portion <NUM> detects a change in a distance (referred to below as a finger distance) to the detection unit (sensor) of the touch panel <NUM> from a finger U when the finger U touches the pulling portion <NUM>, and, in this manner, a pulling operation on the pulling portion <NUM> can be detected.

As illustrated in <FIG>, for example, as in an operation of the power window switch, the finger U is hooked on the end portion of the pulling portion <NUM>, the change in the finger distance D at which the finger U moves away from the touch sensor when the pulling operation is made is detected, and whether or not there is the pulling operation is determined. When the finger distance D increases by the pulling action, the value of the capacitance detected by the touch sensor decreases. Therefore, in the present example, the detection of the pulling operation is realized by implementing a detection structure using a cushion material <NUM> illustrated in <FIG> in which the change in the finger distance D is likely to occur when pulling is performed with the finger U, and a pulling detection algorithm for detecting the deformation of the finger U at the time of the pulling operation as illustrated in <FIG>.

In the detection structure using the cushion material, the shape, material, size, attachment position, and the like of the cushion material are not particularly limited as long as the finger distance D increases when the user lifts the pulling portion <NUM>. In the detection structure illustrated in <FIG>, two cushion materials <NUM> that are elastically deformable in the vertical direction are attached between both ends of the bottom surface portion <NUM> of the three-dimensional UI portion <NUM> and the product cover <NUM>. As the cushion material <NUM>, for example, an elastic member such as rubber, spring, sponge, or resin can be used. When the user performs an operation of pulling the pulling portion <NUM> with the finger U, the three-dimensional UI portion <NUM> is somewhat lifted upward by the cushion material <NUM>, and a slight distance L is generated between the bottom surface portion <NUM> and the surface of the touch panel <NUM>. This changes the finger distance from D to D1 (D < D1), which results in a change in a capacitance for determining whether or not there is a pulling operation.

In the above example, the three-dimensional UI portion <NUM> can be separated from the touch panel <NUM> in a pulling direction by using the cushion material <NUM>. For example, the pulling portion <NUM> may be displaced in the pulling direction without using the cushion material by imparting elasticity to the pulling portion <NUM> itself of the three-dimensional UI portion <NUM>. For example, the width of the cantilevered base portion of the pulling portion <NUM> may be narrowed or the thickness of the base portion may be thinned to facilitate elastic deformation, or the pulling portion <NUM> may be made of an elastic material.

Next, the pulling detection algorithm for detecting deformation of the finger will be described. <FIG> illustrates a state in which the finger U has touched the pulling portion <NUM>, and the finger distance at this time is D. When the user hooks the finger U on the pulling portion <NUM> and lifts the finger U, the shape of the finger is crushed by the stress to be compressed in the pulling direction. As a result, the finger distance changes from D to D2 (D < D2). This change in the finger distance results in the change in the capacitance as much as can detect the pulling operation, and the touch panel <NUM> has sensitivity that can detect such a change in the capacitance.

Next, details of the controller <NUM> will be described. The controller <NUM> is electrically connected to the display <NUM> and the touch panel <NUM>, and performs image control of the display <NUM> and touch control of the touch panel <NUM>. The controller <NUM> performs overall processing of the input display device, for example, stores data such as a height, a shape, a position, and a video display area of the three-dimensional UI portion <NUM>, performs a display area correction process, performs touch detection and operation determination (touch coordinates detection and detection of the magnitude of the finger distance/capacitance) from an output value of the touch panel <NUM>, and performs a process of video display/video switching according to the touch detection and operation determination. The processing of the controller <NUM> is performed by hardware and/or software, and is performed by using, for example, a microcontroller including an arithmetic processing unit and a ROM/RAM.

As illustrated in <FIG>, the controller <NUM> includes a touch detection unit <NUM>, an operation determination unit <NUM>, and a display control unit <NUM>. The touch detection unit <NUM> drives the plurality of electrode lines on the X side and/or the Y side of the touch panel <NUM>, measures the capacitance of each detection unit of the driven electrode line, and provides the measurement result to the operation determination unit <NUM>.

The operation determination unit <NUM> detects a touch operation or a pulling operation on the touch panel <NUM> based on the measurement result of the touch detection unit <NUM>. The touch mentioned here includes not only contact of the finger of the user with the touch panel <NUM> but also approach of the finger to the touch panel <NUM> at a predetermined distance. For example, when the finger of the user touches or approaches the flat surface of the touch panel <NUM>, the operation determination unit <NUM> determines that the touch operation has been made based on the change in the capacitance of the corresponding detection unit. Similarly, when the finger of the user touches or approaches the three-dimensional UI portion <NUM>, the capacitance of the detection unit corresponding to the three-dimensional UI portion <NUM> changes, and thus it is determined that the touch operation has been made on the three-dimensional UI portion <NUM> by the change in the capacitance.

Furthermore, when the touch operation on the three-dimensional UI portion <NUM> has been detected, and the capacitance at the touch position is changed within a predetermined time, the operation determination unit <NUM> determines whether or not there is the pulling operation from such a change. That is, as illustrated in <FIG>, when the capacitance has changed in response to the change in the finger distance, if the change is equal to or greater than a certain threshold value, the operation determination unit <NUM> determines that the pulling operation on the three-dimensional UI portion <NUM> has been made. When the operation determination unit <NUM> determines that the touch operation or the pulling operation has been made, the controller <NUM> provides the input to another electronic device or performs display control or the like corresponding to the input.

The display control unit <NUM> displays an image and a video on the display <NUM>, and displays an icon at the corresponding position of the three-dimensional UI portion <NUM>. The icon can be a design representing the input operation of the user. For example, an icon representing the operation of the power window is displayed below the three-dimensional UI portion <NUM>. In addition, the display control unit <NUM> switches an image to be displayed on the display <NUM> to another image in response to the determination of the operation determination unit <NUM> that the touch operation or the pulling operation has been made.

Next, details of the three-dimensional UI portion <NUM> in the present example will be described. First, a method of detecting the finger distance of the three-dimensional UI portion <NUM> by using the cushion material will be described. In the structure illustrated in <FIG>, when the cushion material <NUM> is sandwiched between the product cover <NUM> and the bottom surface portion <NUM> of the three-dimensional UI portion <NUM> and the pulling portion <NUM> is pulled by the finger U, the cushion material <NUM> stretches in the vertical direction, whereby the bottom surface portion <NUM> is separated from the surface of the touch panel <NUM>, the finger distance between the touch panel <NUM> and the finger U increases, and the capacitance detected by the touch panel <NUM> decreases. When the change in the capacitance is equal to or greater than the threshold value, this is determined as the pulling operation.

On the other hand, as illustrated in <FIG>, contrary to the pulling operation, when the end portion of the pulling portion <NUM> is pushed by the finger U, the cushion material <NUM> is compressed in the vertical direction, the bottom surface portion <NUM> becomes close to the surface of the touch panel <NUM>, and the finger distance between the touch panel <NUM> and the finger U decreases. Thus, the capacitance detected by the touch panel <NUM> increases. When the change in the capacitance is equal to or greater than the threshold value, it is determined that this is a pressing pushing operation.

Furthermore, the cushion material <NUM> is attached at an appropriate position so that the visibility of the icon to be displayed does not decrease or the degree of freedom of the structure of the three-dimensional UI portion <NUM> does not decrease. In the three-dimensional UI portion <NUM> illustrated in <FIG>, the bottom surface portion <NUM> and the pulling portion <NUM> may be integrally molded, or both may be bonded with an adhesive, a double-sided tape, or the like. When the bottom surface portion <NUM> and the pulling portion <NUM> are integrally molded or bonded, it is desirable that the cushion material <NUM> be installed or attached to the outer peripheral portion of the sensor or the icon display of the touch panel <NUM>.

When the cushion material <NUM> is transparent or does not need to be transparent, as illustrated in <FIG>, the cushion material <NUM> may be sandwiched and bonded between the bottom surface portion <NUM> and the pulling portion <NUM>. In this case, the pulling portion <NUM> changes in a pulling direction or the pressing pushing direction via the cushion material <NUM>, and the bottom surface portion <NUM> does not change.

Furthermore, in a case of a configuration in which the bottom surface portion <NUM> of the three-dimensional UI portion <NUM> and the pulling portion <NUM> are not in direct contact with each other or are not bonded to each other, as illustrated in <FIG>, the cushion material <NUM> is attached between the pulling portion <NUM> and the product cover <NUM> on the side portions of the bottom surface portion <NUM>, the touch panel <NUM>, and the display <NUM>. Also in this case, the pulling portion <NUM> changes in a pulling direction or the pressing pushing direction via the cushion material <NUM>, and the bottom surface portion <NUM> does not change.

Next, a method of detecting the finger distance of the three-dimensional UI portion <NUM> by not using the cushion material will be described. In a case of a sensor having accuracy enabling detection of a small change in the finger distance (for example, about <NUM>), the pulling operation or the pressing operation can be detected even with a structure not using the cushion material. <FIG> illustrate a state when the finger U has touched the end portion of the pulling portion <NUM>, and the finger distance from the sensor to the finger is D. <FIG> illustrate a state of deformation (crushing) of the finger U when the finger U is pulled even though the finger U is hooked on the end portion of the pulling portion <NUM>. The finger U is deformed by about <NUM>, thereby the finger distance from the sensor to the finger U increases to D2 (D2 > D).

Next, <FIG> illustrates a flow of a determination operation of the operation determination unit <NUM> when the cushion material is not used. The operation determination unit <NUM> detects whether or not there is a finger below the pulling portion <NUM> based on the measurement result of the capacitance by the touch detection unit <NUM> (S100). As illustrated in <FIG>, position coordinates (a portion of an arrow H) when the finger U is hooked on the pulling portion <NUM> and a threshold value (a portion of an arrow V) of the finger distance/capacitance when there is the finger U under the pulling portion <NUM> are registered in the operation determination unit <NUM> in advance. Although <FIG> illustrates only the coordinate range in one direction of the arrow H, the coordinate range by the arrow H may be specified as plane coordinates.

The operation determination unit <NUM> determines that there is the finger U below the pulling portion <NUM> when detecting a value that is within the position coordinates indicated by the arrow H and within the threshold value of the capacitance (within the finger distance) indicated by the arrow V based on the measurement result of the touch detection unit <NUM>.

Then, the operation determination unit <NUM> determines a pulling operation based on the change in capacitance (S110). <FIG> illustrates a state before the pulling operation, and <FIG> illustrates a state during the pulling operation. As illustrated in <FIG>, the finger tip is compressed or crushed by performing the pulling operation, to change the angle of the finger, so that the finger distance changes from D in <FIG> to D2. As a result, the value of the capacitance detected by the sensor of the touch panel <NUM> decreases. The operation determination unit <NUM> determines the pulling operation by detecting the change in the capacitance.

Then, when determining that the pulling operation has been performed, the operation determination unit <NUM> determines the end of the pulling operation based on the change in the capacitance (S120). <FIG> illustrates a state during the pulling operation, and <FIG> illustrates a state at the end of the pulling operation. As illustrated in <FIG>, in an operation to end the pulling operation, the finger is brought back to the original state as opposed to the pulling operation, so that the finger distance changes from D2 in Fig. 10A to D. As a result, the value of the capacitance detected by the sensor of the touch panel <NUM> increases. The operation determination unit <NUM> determines the end of the pulling operation by detecting the change in the capacitance.

Next, the graph of <FIG> shows the verification result of the change in the capacitance when the pulling operation to the three-dimensional UI portion, which does not use the cushion material, is performed by an actual machine. The vertical axis represents the value of the capacitance, and the horizontal axis represents the number of pieces of detected data. In the actual machine verified this time, the capacitance is detected at <NUM> fps (frames/second).

As illustrated in the graph of <FIG>, the value of the capacitance when the three-dimensional UI portion <NUM> is not pulled (when the finger distance is small) is about <NUM> or greater, and the value of the capacitance when the three-dimensional UI portion <NUM> is pulled (when the finger distance is large) decreases to about <NUM> to <NUM>. Therefore, by setting the threshold value for determining whether or not there is the pulling operation within a range of <NUM> to <NUM> (for example, the threshold value may be set to <NUM>), it is possible to accurately determine whether or not there is the pulling operation.

In addition, the graph of <FIG> shows the verification result of the change in the capacitance when the three-dimensional UI portion is pressed by the actual machine. The vertical axis represents the value of the capacitance, and the horizontal axis represents the number of pieces of detected data. In the actual machine verified this time, the capacitance is detected at <NUM> fps (frames/second).

Differing from the pulling operation, in the pressing operation, the finger distance does not change due to the crush of the finger. Thus, it is necessary to have a structure in which the finger distance is small at the time of pressing by a configuration (Fig. 12A) in which the pulling portion <NUM> itself of the three-dimensional UI portion <NUM> is made of a bending material or a configuration (Fig. 12B) in which the cushion material <NUM> is sandwiched between the three-dimensional UI portion <NUM> and the product cover.

In the detection example here, a difference in detection value between the presence and absence of the pushing operation is not as large as that during the pulling operation, but the change in the capacitance during the pushing operation can be read from the graph. For example, it is possible to determine the pressed state or the non-pressed state by setting the threshold value when the value of the capacitance is <NUM> or greater or <NUM> or smaller.

As described above, according to the present example, in the product having the three-dimensional UI, it is possible to accurately detect the pulling operation with a simple structure without increasing the number of touch sensors. As a result, the three-dimensional UI can cope with an operation gesture of the pulling type switch.

Claim 1:
An input display device (<NUM>) comprising:
a display (<NUM>) configured to display an image;
a capacitive touch panel (<NUM>) configured to be attached onto the display (<NUM>) and include at least one three-dimensional operation portion (<NUM>) on a surface of the touch panel (<NUM>); and
a detection unit configured to detect an operation on the touch panel (<NUM>), wherein
the three-dimensional operation portion (<NUM>) includes a pulling portion (<NUM>) imitating a shape of a pulling type switch (<NUM>), and
the detection unit is configured to determine whether or not there is a pulling operation by detecting a change in a capacitance in response to a change in a distance to the touch panel (<NUM>) from a finger touching the pulling portion (<NUM>).