Source: https://patents.google.com/patent/JP5532300B2/en
Timestamp: 2020-01-26 01:15:00
Document Index: 455496388

Matched Legal Cases: ['arts 61', 'arts 61', 'arts 61', 'arts 61', 'arts 61', 'arts 81', 'art 82', 'art 82', 'art 82', 'art 82', 'art, 23', 'art, 32', 'art, 33', 'art, 34']

JP5532300B2 - Touch panel device, touch panel control method, program, and recording medium - Google Patents
Touch panel device, touch panel control method, program, and recording medium Download PDF
JP5532300B2
JP5532300B2 JP2009293147A JP2009293147A JP5532300B2 JP 5532300 B2 JP5532300 B2 JP 5532300B2 JP 2009293147 A JP2009293147 A JP 2009293147A JP 2009293147 A JP2009293147 A JP 2009293147A JP 5532300 B2 JP5532300 B2 JP 5532300B2
JP2009293147A
JP2011134111A (en
浩一 樫尾
2009-12-24 Application filed by ソニー株式会社 filed Critical ソニー株式会社
2009-12-24 Priority to JP2009293147A priority Critical patent/JP5532300B2/en
2011-07-07 Publication of JP2011134111A publication Critical patent/JP2011134111A/en
2014-06-25 Publication of JP5532300B2 publication Critical patent/JP5532300B2/en
The present invention relates to a touch panel device, a touch panel control method, a program, and a recording medium. In particular, in a touch panel that can be mounted on a small device, the operability can be improved without hindering the information display function. The present invention relates to a touch panel device, a touch panel control method, a program, and a recording medium.
In recent years, the use of touch panels has been increasing as a user interface for mobile devices such as mobile phones and PDAs. Since the input device and the display device are integrated into the touch panel, the touch panel can be used as a user interface to reduce the size of the device, and the built-in software offers various ways of viewing and intuitive operability. This is because it can be realized.
Many touch panels used in mobile devices are resistive film type and capacitive type touch panels. By detecting two states of a contact state and a non-contact state with these touch panels, a user input trigger for the mobile device is set. For example, it is operated by touching a graphical user interface (GUI) component (button or the like) constructed in the display screen of the touch panel with a fingertip.
With the recent improvement in performance of mobile devices, mobile devices are also used for information processing that conventionally uses a personal computer or the like. For this reason, a software keyboard may be displayed on a small screen of a mobile device to perform character input, or complicated GUI input may be required.
In such a case, the GUI component may be smaller than the fingertip, and when the input accuracy of the touch panel is low or the fingertip is shaken, an unintended selection or input mistake may occur.
Conventionally, for example, when a small GUI component is selected, input errors are reduced by a method of determining the GUI component that was touched when the contact state is changed to the non-contact state.
Further, a touch panel is provided on the display surface, two proximity detection cameras are provided in the vicinity thereof, and based on the captured image, the relative positional relationship between the user's finger and the display surface is detected. It has also been proposed to magnify and display an icon that is close to a finger when detecting that the display surface is close to a predetermined distance or more (see, for example, Patent Document 1).
By doing in this way, the icon which the user is going to select can be predicted and enlarged and it can be made easy for a user to select an icon.
JP 2006-236143 A
However, for example, a method of confirming a GUI part that has been touched when a small GUI part is selected from a contact state to a non-contact state is uncomfortable for the user. That is, as a user's psychology, it should feel more natural when it is determined when the GUI component is touched like an actual button pressing operation.
In addition, if the GUI parts displayed on the display surface are enlarged and displayed, it may be easy to select an icon, but other information displayed on the display surface may be hidden behind the icon and become invisible. is there. That is, as long as there is no problem in operability, it is desirable that the GUI component be displayed small.
Furthermore, in recent years, with the improvement of functions of mobile devices, many functions such as mobile phone functions, mail transmission / reception functions, music playback functions, image capture display functions, etc. tend to be integrated into one device. The number of GUI parts displayed on the screen is also increasing. For this reason, it is becoming increasingly important to make the display on the display surface easier to see and to facilitate the selection of icons.
The present invention has been made in view of such a situation, and is intended to improve operability without impeding the information display function in a touch panel that can be mounted on a small device.
One aspect of the present invention is a proximity determination unit that determines whether or not an object is close to the touch screen based on a plurality of images obtained by imaging the center of the touch screen from each side of the touch screen; and the touch screen An area estimation means for estimating a contact area of the object on the touch screen , based on a three-dimensional shape obtained by analyzing the plurality of images, and the estimated When it is determined that an object is close to the touch screen and display control means for controlling the size of a part to be selected in the GUI, which is a predetermined screen displayed on the touch screen, based on the contact area The GUI component that is likely to be selected by the object touching the touch screen is identified, and the identified component And a selection component identifying means for displaying in a different manner from the other part, the selected part identification unit includes a center point of the region on the touch screen corresponding to the contact area in which the estimated, displayed on the touch screen The touch panel device identifies the GUI component that is highly likely to be selected based on the distance from the center of gravity of each of the GUI component that has been selected .
If an object on the touch screen is determined to have approached the object and the object number determination means for determining whether or not there are a plurality of, if the object is determined to be more, is displayed on the touch screen the First display setting means for setting the GUI to a first GUI different from the default GUI may be further provided.
If an object on the touch screen is determined to have approached the second display setting means for setting different second GUI and the GUI the default GUI displayed on the touch screen based on the type of the object Can be further provided.
The selected component specifying unit repeatedly specifies a GUI component that is highly likely to be selected at a preset period, and the object is located in an area overlapping with the specified GUI component on the touch screen. In the case of contact, selection confirmation means for confirming selection of the GUI component can be further provided.
When it is determined that an object is close to the touch screen, the GUI may be displayed on the touch screen.
One aspect of the present invention is a touch panel control method for a touch panel device including a proximity determination unit, an area estimation unit, a display control unit, and a selected component identification unit, wherein the proximity determination unit includes each side of the touch screen. on the basis of the center of the touch screen into a plurality of images captured from the case where the determination whether the object has approached the touch screen, the area estimation means, the object on the touch screen is determined to have approached , based on the three-dimensional shape obtained by analyzing the plurality of images, to estimate the area of contact on the touch screen of the object, said display control means, based on the contact area of the estimated, the controls component size to be selected in the GUI is a predetermined screen displayed on the touch screen, said selected part identification unit, said Tatchisu When it is determined that an object is close to the lean, the GUI part that is likely to be selected by the object touching the touch screen is identified, and the identified part is different from other parts. The selection is performed based on the distance between the center point of the area on the touch screen corresponding to the estimated contact area and the center of gravity of each of the GUI components displayed on the touch screen. It is a touch panel control method including the step which specifies the component of said GUI with high possibility of being performed .
One aspect of the present invention is a proximity determination unit that determines whether or not an object is close to the touch screen based on a plurality of images obtained by imaging the center of the touch screen from each side of the touch screen . An area estimation means for estimating a contact area of the object on the touch screen based on a three-dimensional shape obtained by analyzing the plurality of images when it is determined that the object is close to the touch screen; Based on the estimated contact area, display control means for controlling the size of a part to be selected in the GUI, which is a predetermined screen displayed on the touch screen, and determining that an object is close to the touch screen The GUI component that is likely to be selected by the object touching the touch screen, And a selection component identifying means for displaying the identified components in a different manner from the other part, the selected part identification unit includes a center point of the region on the touch screen corresponding to the contact area in which the estimated, It is a program that functions as a touch panel device that identifies the GUI component that is highly likely to be selected based on the distance from the center of gravity of each of the GUI components displayed on the touch screen .
One aspect of the present invention is a proximity determination unit that determines whether or not an object is close to the touch screen based on a plurality of images obtained by imaging the center of the touch screen from each side of the touch screen. An area estimation means for estimating a contact area of the object on the touch screen based on a three-dimensional shape obtained by analyzing the plurality of images when it is determined that the object is close to the touch screen; Based on the estimated contact area, display control means for controlling the size of a part to be selected in the GUI, which is a predetermined screen displayed on the touch screen, and determining that an object is close to the touch screen The GUI component that is likely to be selected by the object touching the touch screen, Selection part specifying means for displaying the specified part in a manner different from other parts, wherein the selected part specifying means includes a center point of an area on the touch screen corresponding to the estimated contact area; A record that records a program that functions as a touch panel device that identifies the GUI component that is likely to be selected based on the distance from the center of gravity of each of the GUI component displayed on the touch screen. It is a medium.
In one aspect of the present invention , based on a plurality of images obtained by imaging the center of the touch screen from each side of the touch screen, it is determined whether an object is close to the touch screen, and the object is on the touch screen. When it is determined that they are close to each other, based on the three-dimensional shape obtained by analyzing the plurality of images , the contact area of the object on the touch screen is estimated, and based on the estimated contact area, The size of a part to be selected in the GUI, which is a predetermined screen displayed on the touch screen, is controlled, and when it is determined that the object is close to the touch screen, the object contacts the touch screen. The GUI parts that are likely to be selected are identified, and the identified parts are displayed differently from other parts. That. Further, the selection can be made based on the distance between the center point of the area on the touch screen corresponding to the estimated contact area and the center of gravity of each of the GUI parts displayed on the touch screen. The GUI part having high performance is specified.
ADVANTAGE OF THE INVENTION According to this invention, in the touchscreen which can be mounted in a small apparatus, operativity can be improved, without inhibiting the information display function.
It is a figure which shows the example of the touchscreen apparatus to which this invention is applied. It is a block diagram which shows the example of an internal structure of the touchscreen apparatus of FIG. It is a figure explaining the example which images an image from 4 directions with the imaging circuit of FIG. It is a figure explaining the method of the proximity determination by the contact / proximity determination part of FIG. It is a figure explaining touch panel operation using a touch panel device. It is a figure explaining touch panel operation using a touch panel device. It is a figure explaining touch panel operation using a touch panel device. It is a flowchart explaining the example of a proximity detection process. It is a figure which shows the example at the time of making a finger | toe adjoin to the surface of a touch screen. It is a figure explaining the example of arrangement | positioning of a GUI component, and a finger contact area. It is a figure which shows the example of the center point of the gravity center point of the components of GUI, and the contact area of a finger | toe. It is a figure which shows the example at the time of moving the finger brought close to the surface of a touch screen. It is a figure which shows the example of the state which actually contacted the surface of the touch screen. It is a figure explaining selection of the components of GUI in the state of FIG. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.
FIG. 1 is a diagram illustrating an example of a touch panel device to which the present invention is applied.
The touch panel device 10 shown in the figure can detect that a finger, a stylus pen, or the like has touched the touch panel surface, and can detect that a finger, a stylus pen, or the like has approached the touch panel surface. Has been made. Further, the touch panel device 10 can detect, for example, that a plurality of fingers are in contact with or close to the touch panel surface, and can detect the area of the contact portion of the finger or stylus pen with the touch panel. Has been made.
A touch screen 11 is provided in the touch panel device 10 illustrated in FIG. 1, and the touch screen 11 includes, for example, an LCD that displays an image such as a GUI and a touch panel that specifies a contact position.
In addition, imaging circuits 12-1 to 12-4 are provided on four sides of the rectangular touch screen 11, respectively. Each of the imaging circuits 12-1 to 12-4 has an image sensor, and images a finger or a stylus pen close to the touch screen 11 from four directions.
FIG. 2 is a block diagram illustrating an internal configuration example of the touch panel device 10. As shown in the figure, the touch panel device 10 is provided with an imaging circuit 12-1 to imaging circuit 12-4, a touch panel 21, and an image / detection processing unit 22.
Further, for example, a microcomputer 23 is connected to the image / detection processing unit 22, and processing corresponding to the operation of the touch panel device 10 is executed by the microcomputer 23. For example, when the image / detection processing unit 22 detects that a finger or stylus pen is in contact with a predetermined GUI component displayed on the touch screen, a process for realizing a function assigned to the component is microscopic. It is executed by the computer 23.
As described above, the imaging circuits 12-1 to 12-4 output the image data of the finger or stylus pen imaged from the respective directions to the image / detection processing unit 22.
The touch panel 21 detects a contact of a finger or a stylus pen and a contact position thereof. As the touch panel 21, for example, a resistance film type that detects a voltage corresponding to a position operated using two opposing resistance films is used. Alternatively, other methods such as a capacitance method that detects a position by detecting a change in capacitance between a fingertip or the like and the conductive film may be used.
Each of the image data output from the imaging circuits 12-1 to 12-4 is supplied to the image recognition unit 31 of the image / detection processing unit 22.
The image recognition unit 31 discriminates an object close to the touch screen 11 based on supplied image data. For example, the image recognition unit 31 compares the feature amount extracted from the supplied image data with the feature amount information stored in advance, so that an object close to the touch screen 11 can be moved to the finger or stylus pen. Or other objects.
When it is detected that the finger or the stylus pen is close as a recognition result by the image recognition unit 31, each of the image data output from the imaging circuit 12-1 to the imaging circuit 12-4 is information on the recognition result. At the same time, it is output to the shape detector 32.
The shape detection unit 32 performs an operation of estimating the three-dimensional shape of the finger or the stylus pen by analyzing the images from the four directions imaged by the imaging circuits 12-1 to 12-4. Note that the shapes of the stylus pens are almost the same, and as a result of recognition by the image recognition unit 31, only when it is detected that the finger is approaching, calculation for estimating the three-dimensional shape is performed. Good.
The three-dimensional shape obtained as the calculation result of the shape detection unit 32 is supplied to the position / area detection unit 33 together with the image data output from the imaging circuit 12-1 to the imaging circuit 12-4.
The position / area detection unit 33 calculates the position of the finger or the stylus pen on the touch screen 11 by analyzing the images from the four directions imaged by the imaging circuit 12-1 to the imaging circuit 12-4. . For example, when the finger or stylus pen is moved vertically downward, the position / area detection unit 33 calculates the position on the surface of the touch screen 11 as the X-axis / Y-axis coordinates.
Further, the position / area detection unit 33 analyzes the images from the four directions picked up by the image pickup circuit 12-1 to the image pickup circuit 12-4, and thereby the distance between the finger or the stylus pen and the surface of the touch screen 11 Is calculated. The position / area detection unit 33 outputs this distance as, for example, Z-axis coordinates.
Further, the position / area detection unit 33 performs a calculation for estimating the contact area on the touch screen 11 of the finger or the stylus pen based on the three-dimensional shape obtained as the calculation result of the shape detection unit 32. For example, when the three-dimensional shape of the finger or the stylus pen is a substantially cylindrical shape, the position / area detection unit 33 estimates the area of the bottom surface of the cylinder as the above-described contact area, and calculates the area.
That is, as shown in FIG. 3, for example, images are captured from four directions by the imaging circuit 12-1 to the imaging circuit 12-4 at predetermined time intervals, and the obtained four images are analyzed (image processing or the like). ) And the value of the above-mentioned X-axis Y-axis Z-axis coordinate in each time is obtained, and area A is obtained as the above-mentioned contact area.
The contact / proximity determination unit 34 determines whether the finger or the stylus pen is in contact with or close to the touch screen 11 based on the processing result of the detection data processing unit 35 described later and the detection result of the position / area detection unit 33. Determine.
Based on the information output from the touch panel 21, the detection data processing unit 35 outputs information indicating which position on the surface of the touch screen 11 the finger or the stylus pen has touched. For example, the detection data processing unit 35 outputs the contact position as X-axis and Y-axis coordinates.
In addition to the determination result of the contact / proximity determination unit 34, information indicating the calculation result of the position / area detection unit 33 is output as the detection result of the image / detection processing unit 22. In other words, the detection result of the image / detection processing unit 22 includes information that determines the type of the object that is in close proximity and that indicates whether the finger or the stylus pen is close. The detection result of the image / detection processing unit 22 includes information indicating how close or in contact the finger or the stylus pen is to which position on the surface of the touch screen 11. Furthermore, the contact area on the touch screen 11 of the finger or the stylus pen is also included.
FIG. 4 is a diagram for explaining a proximity determination method by the contact / proximity determination unit 34. The figure shows a state in which the finger of the user who operates the touch panel device 10 is approaching the surface of the lower touch screen 11 in the figure. As shown in the figure, a threshold for determining proximity (a distance between the fingertip and the surface of the touch screen 11) is set in advance. This threshold value is compared with the value of the Z-axis coordinate (distance from the surface of the touch screen 11) output by the position / area detection unit 33. It is assumed that the Z-axis coordinate value output by the position / area detection unit 33 approaches 0 as the finger approaches the touch screen 11.
When the output value (Z-axis coordinate value) is equal to or smaller than the threshold value, the contact / proximity determining unit 34 determines that the finger has approached. When the output value exceeds the threshold value, the contact / proximity determining unit 34 approaches the finger. It will be determined that it is not.
The touch panel device 10 of the present invention displays a GUI on the touch screen 11 when the proximity of a finger or a stylus pen is detected.
For example, when the distance between the touch screen 11 and the finger is sufficiently large, the GUI is not displayed on the touch screen 11.
5 to 7 are diagrams for explaining an example of touch panel operation using the touch panel device 10 of the present invention.
FIG. 5 shows an example where the distance between the touch screen 11 and the finger is sufficiently large. In the figure, a region 51 indicated by a circle represents a region on the touch screen 11 where contact with a finger is assumed.
At this time, the position / area detection unit 33 calculates the distance between the finger and the surface of the touch screen 11 by analyzing the images from the four directions imaged by the imaging circuit 12-1 to the imaging circuit 12-4. , Output as Z-axis coordinates. In this case, since the Z-axis coordinate value exceeds the threshold value, the contact / proximity determining unit 34 determines that the finger is not in proximity.
For example, information representing the determination result is output to the microcomputer 23 as the detection result of the image / detection processing unit 22.
In FIG. 5, the GUI is not displayed on the touch screen 11.
FIG. 6 shows an example in which a finger approaches the touch screen 11.
At this time, the position / area detection unit 33 calculates the distance between the finger and the surface of the touch screen 11 by analyzing the images from the four directions imaged by the imaging circuit 12-1 to the imaging circuit 12-4. , Output as Z-axis coordinates. In this case, since the Z-axis coordinate value is equal to or smaller than the threshold value, the contact / proximity determining unit 34 determines that the finger has approached.
At this time, the image recognizing unit 31 also stores, for example, the feature amount extracted from the image data supplied from the imaging circuit 12-1 to the imaging circuit 12-4, the feature amount information stored in advance, and the like. By comparing, it is determined that there is a finger near the touch screen 11.
Further, the shape detection unit 32 performs an operation of estimating the three-dimensional shape of the finger by analyzing images from four directions imaged by the imaging circuit 12-1 to the imaging circuit 12-4. The position / area detection unit 33 performs a calculation for estimating the contact area of the finger on the touch screen 11 based on the three-dimensional shape obtained as the calculation result of the shape detection unit 32. Thereby, for example, the area of the region 51 is calculated.
For example, when the finger is moved vertically downward, the position / area detection unit 33 calculates the position on the surface of the touch screen 11 as the X-axis / Y-axis coordinates. Thereby, for example, the coordinates of the center point of the region 51 are calculated.
Information such as determination, discrimination, and calculation result is output to the microcomputer 23 as a detection result of the image / detection processing unit 22. The microcomputer 23 displays a GUI on the LCD of the touch screen 11. In FIG. 6, GUI parts 61 to 63 are displayed on the touch screen 11 when the proximity of the finger is detected.
That is, the touch panel device 10 of the present invention does not display the GUI on the touch screen 11 until the proximity of the finger (or stylus pen) is detected, and displays the GUI on the touch screen 11 when the proximity is detected. Has been made.
The parts 61 to 63 are GUI parts displayed when the proximity of a finger is detected. For example, when the proximity of a stylus pen is detected, another GUI part is displayed. Has been made. That is, the touch panel device 10 of the present invention can display different GUIs depending on the type of the object whose proximity is detected.
The parts 61 to 63 are displayed in an enlarged manner according to the contact area on the touch screen 11 of the finger. For example, when the contact area is equal to or smaller than the threshold value, the parts 61 to 63 are displayed with a normal display size, and when the contact area exceeds the threshold value, the parts 61 to 63 are enlarged and displayed.
By doing so, for example, when a person with a large hand or a person with thick fingers operates the touch panel device 10, the GUI parts can be enlarged and displayed. On the other hand, when a person with a small hand or a person with a small finger operates the touch panel device 10, the GUI parts are not enlarged and displayed.
Although an example in which the GUI parts are enlarged and displayed has been described here, it is of course possible to reduce the GUI parts and display them. In short, the size of the GUI component displayed on the touch screen 11 may be controlled as necessary.
In the example of FIG. 6, the case where one finger is close is described. However, for example, when two fingers are simultaneously close, different GUI components may be displayed.
As described above, the position / area detection unit 33 calculates a position of contact on the surface of the touch screen 11 when, for example, the finger is moved vertically downward. For example, the number of calculated positions may be specified, and different GUI parts may be displayed according to the specified number (that is, the number of fingers). Alternatively, the number of adjacent fingers may be specified based on the three-dimensional shape estimated by the shape detection unit 32.
For example, a default GUI may be displayed when one finger is detected, and display data of another GUI may be displayed when two fingers are detected. Similarly, when three fingers are detected, another GUI may be displayed. In this way, the user can display a desired GUI among a plurality of GUIs by a single operation of bringing a finger close to each other.
In addition, when the finger etc. which approached once left | separated from the surface of the touch screen 11, the display of GUI is also erase | eliminated.
FIG. 7 shows an example where a finger touches the touch screen 11. In the example of the figure, the finger is in contact with the touch screen 11 at the position where the GUI component 63 is displayed. For this reason, the region 51 and the component 63 are overlapped.
At this time, based on the information output from the touch panel 21, the detection data processing unit 35 displays information indicating that the finger has touched the center position of the area 51 on the surface of the touch screen 11, for example, the contact position X Output as Y-axis coordinates. Then, the determination result of the contact / proximity determination unit 34 is output as the detection result of the image / detection processing unit 22, and for example, the processing for realizing the function assigned to the component 63 is executed by the microcomputer 23. Has been made.
That is, in the state shown in FIG. 7, the selection of the component 63 is confirmed.
Next, an example of proximity detection processing by the touch panel device 10 of the present invention will be described with reference to the flowchart of FIG.
In step S21, the microcomputer 23 determines whether the proximity of the finger or the stylus pen has been detected, and waits until it is determined that the proximity of the finger or the stylus pen has been detected. This determination is performed based on the determination result of the contact / proximity determination unit 34 described above, for example.
If it is determined in step S21 that the proximity of the finger or the stylus pen has been detected, the process proceeds to step S22.
In step S22, the microcomputer 23 determines whether the proximity of a plurality of objects has been detected. This determination is performed based on the number of positions calculated in the calculation of the contact position by the position / area detection unit 33 described above, for example.
If it is determined in step S22 that proximity of a plurality of objects has been detected, the process proceeds to step S23. In this case, for example, it is assumed that two fingers are detected. It is not assumed that a plurality of stylus pens are detected.
In step S23, the microcomputer 23 sets a GUI corresponding to the number of detected objects (finger). For example, if the GUI displayed when one finger is detected is the GUI of the pattern A, the display data of the GUI of the pattern B is displayed on the touch screen 11 when two fingers are detected. It is set as image data to be displayed on the LCD. Further, for example, when three fingers are detected, the GUI display data of the pattern C is set as image data to be displayed on the LCD of the touch screen 11.
As default display data, for example, it is assumed that GUI display data of pattern A is set.
In step S24, the microcomputer 23 checks the area of the detected object (finger). Here, for example, the value of the contact area on the touch screen 11 of the finger calculated by the position / area detection unit 33 based on the three-dimensional shape obtained as the calculation result of the shape detection unit 32 is acquired.
On the other hand, if it is determined in step S22 that proximity of a plurality of objects has not been detected, the process proceeds to step S25.
In step S25, the microcomputer 23 checks the area of the detected object. Here, for example, the value of the contact area on the touch screen 11 of the finger or stylus pen calculated by the position / area detection unit 33 based on the three-dimensional shape obtained as the calculation result of the shape detection unit 32 is acquired. Is done.
Note that when the adjacent object is a stylus pen, the calculation of the contact area may not be performed.
In step S26, the microcomputer 23 determines whether or not the detected object is a stylus pen. Here, for example, determination using a determination result based on comparison of feature amounts by the image recognition unit 31 is performed.
If it is determined in step S26 that the detected object is a stylus pen, the process proceeds to step S27.
In step S27, the microcomputer 23 sets a GUI for the stylus pen. For example, GUI display data of the pattern X is set as image data to be displayed on the LCD of the touch screen 11.
On the other hand, after the process of step S24 or when it is determined in step S26 that the detected object is not a stylus pen, the process proceeds to step S28.
In step S28, the microcomputer 23 determines whether enlargement is necessary when displaying the GUI. Here, for example, it is determined whether or not the value of the contact area on the touch screen 11 of the finger acquired in the process of step S24 or step S25 exceeds a threshold value. If the value of the contact area exceeds the threshold value, enlargement is performed. It is determined that it is necessary.
If it is determined in step S28 that enlargement is necessary, the process proceeds to step S29.
In step S29, the microcomputer 23 enlarges and displays each component of the GUI on the LCD of the touch screen 11 based on the default GUI display data or the GUI display data set in the process of step S23.
On the other hand, after the process of step S27 or when it is determined in step S28 that enlargement is not necessary, the process proceeds to step S30.
In step S <b> 30, the microcomputer 23 displays each component of the GUI on the LCD of the touch screen 11 based on the default GUI display data or the GUI display data set in the process of step S <b> 27.
In this way, the proximity detection process is executed.
Conventionally, there is a problem that GUI parts displayed on the touch screen are too small to be operated. If the GUI parts displayed on the display surface are enlarged and displayed, it may be easy to select an icon, but other information displayed on the display surface may be hidden behind the icon and become invisible.
In recent years, with the improvement of functions of mobile devices, many devices such as mobile phone functions, mail transmission / reception functions, music playback functions, image capturing and display functions, etc. tend to be integrated into one device. The number of GUI parts displayed on the screen is also increasing. For this reason, it is becoming increasingly important to make the display on the display surface easier to see and to facilitate the selection of icons.
According to the touch panel device 10 of the present invention, the proximity detection process is executed as described above. By doing so, for example, when a person with a large hand or a person with a large finger operates the touch panel device 10, it is possible to enlarge and display the GUI parts. On the other hand, when a person with a small hand or a person with a small finger operates the touch panel device 10 or operates the touch panel device 10 using a stylus pen, the GUI parts are not enlarged and displayed.
Thereby, according to this invention, operativity can be improved, without inhibiting the display function of information. Moreover, since the touch panel device 10 of the present invention displays a GUI on the touch screen 11 when a finger or the like is brought close to the touch panel device 10, for example, power consumption of the device can be suppressed.
Next, selection of GUI components displayed on the touch screen 11 will be described.
As described above, based on the information output from the touch panel 21, the detection data processing unit 35 represents information indicating that the finger or the stylus pen has contacted the center position of the region 51 on the surface of the touch screen 11. The contact position is output as the X-axis and Y-axis coordinates. Then, the determination result of the contact / proximity determination unit 34 is output as the detection result of the image / detection processing unit 22. For example, the microcomputer 23 executes a process for realizing the function assigned to the selected component. The
That is, among the GUI parts displayed on the touch screen 11, the GUI part displayed at the position corresponding to the X-axis and Y-axis coordinates of the contact position is selected.
However, for example, when the displayed GUI component is small, it is difficult to accurately touch the position of the GUI component with a finger, and an operation error is likely to occur.
In the touch panel device 10 of the present invention, in order to suppress the occurrence of such an operation error, the display mode of the GUI component is changed in accordance with the proximity detection of a finger or a stylus pen. Thus, a part that is likely to be selected by contact with the finger is presented to the user, and the selection of the part is confirmed when the finger or the stylus pen actually touches the surface of the touch screen 11. To.
For example, as shown in FIG. 9, when the finger is brought close to the surface of the touch screen 11, the component 81 has a color (for example, red) different from other components among the GUI components displayed on the screen. Is displayed.
When the finger is brought close to the surface of the touch screen 11, as described above, the position / area detection unit 33 calculates the position of contact on the surface of the touch screen 11, for example, when the finger is moved vertically downward. To do. Further, the position / area detection unit 33 performs a calculation for estimating the contact area of the finger on the touch screen 11 based on the three-dimensional shape obtained as the calculation result of the shape detection unit 32.
FIG. 10 is an enlarged view of a part of the surface of the touch screen 11. Now, for example, as shown in FIG. 10, it is assumed that GUI parts are arranged. In the figure, each of the rectangles regularly arranged in the vertical direction and the horizontal direction in the figure represents a GUI part on the touch screen 11. Now, it is assumed that an area estimated to be touched on the touch screen 11 of a nearby finger is an area 51 indicated by a circle in the drawing.
For example, in the case of FIG. 10, a component that is likely to be selected by touching with a finger can be considered to be a component at a position overlapping with the region 51. In the example of the figure, at least a part of the rectangle corresponding to the component 81-1 to the component 81-9 is arranged at a position overlapping the circle corresponding to the region 51.
When the finger is brought close to the surface of the touch screen 11, the microcomputer 23 determines the most based on the contact position on the surface of the touch screen 11 calculated by the position / area detection unit 33 and the contact area on the touch screen 11. Identify parts that are likely to be selected. Here, the part most likely to be selected is identified based on the distance between the center point of the circle represented by the region 51 in FIG. 10 and the center point of the parts 81-1 to 81-9, for example. Is done.
FIG. 11 is an enlarged view of a part of the surface of the touch screen 11 as in FIG. 10, and shows the center point of the region 51 and the center of gravity point of the GUI component. In the figure, the center point of the region 51 is indicated by the black point shown at the center of the circle. Further, the center of gravity of each component is indicated by a black point shown at the center of each rectangle. In this example, all the parts have the same rectangular shape. However, for example, even when the parts have different sizes and different shapes, the center of gravity is obtained in the same manner.
Then, the microcomputer 23 identifies the component having the center of gravity closest to the center point of the region 51 as the component most likely to be selected. That is, the component 81-1 in FIG. 10 is the component most likely to be selected, and is displayed in red.
Further, as shown in FIG. 12 from the state shown in FIG. 9, when the finger is moved in the direction of the arrow shown in FIG. 12, the component 81 displayed in red is displayed in the original color, and a new one is displayed. The part 82 is displayed in red. That is, when the finger is brought close to the surface of the touch screen 11, the microcomputer 23 identifies the part having the center of gravity closest to the center point of the region 51 as the part most likely to be selected, The process of changing the display mode of the parts is repeatedly executed.
For example, the microcomputer 23 determines whether a finger or the like is close to the surface of the touch screen 11 based on information output as a detection result of the image / detection processing unit 22. When it is determined that a finger or the like is close to the surface of the touch screen 11, the microcomputer 23 identifies a component that is most likely to be selected, for example, every 0.5 seconds, and displays the component. The process of changing the mode is repeatedly executed.
As shown in FIG. 13 from the state shown in FIG. 12, it is assumed that the position of the component 82 on the surface of the touch screen 11 is touched (contacted) with a finger. In FIG. 13, it is assumed that the user touches the surface of the touch screen 11 by moving his / her finger substantially vertically downward from the state shown in FIG. 12.
At this time, it is assumed that GUI parts are arranged on the touch screen 11 as shown in FIG. It is assumed that the finger actually touches the surface of the touch screen 11 in the region 52 shown in FIG. FIG. 14 is an enlarged view of a part of the surface of the touch screen 11, and each of the rectangles regularly arranged in the vertical direction and the horizontal direction in the drawing represents a GUI part on the touch screen 11. .
In the state of FIG. 14, the component having the center of gravity closest to the center point of the region 52 is not the component 82, but the microcomputer 23 assumes that the component 82 has been selected and the function assigned to the component 82. The process for realizing is executed. That is, the selection of the part 82 is confirmed.
When the finger is moved from the state shown in FIG. 12 as shown in FIG. 13, the fingertip often touches a position different from the intended position. In such a case, if it is recognized that a part different from the part displayed in red is selected, it is felt that the operability is low for the user.
For this reason, in the touch panel device 10 of the present invention, the fine movement between the finger and the like approaching the surface of the touch screen 11 and the actual contact is ignored, and is most likely to be selected in the proximity state. Confirm the selection of the part identified as the part.
If the positions of the region 52 and the part 82 in FIG. 14 do not overlap, it is recognized that a part other than the part 82 has been selected.
As described above, according to the present invention, it is possible to suppress the occurrence of an operation error. Also, for example, it is not fixed to the GUI part touched when the contact state is changed to the non-contact state as in the conventional method, but when the GUI part is touched like an actual button pressing operation. The selection is confirmed. As a result, it is possible to provide a more natural operation environment as compared with, for example, a conventional method, while considering the suppression of the occurrence of an operation error.
The series of processes described above can be executed by hardware, or can be executed by software. When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium into a computer incorporated in dedicated hardware. In addition, by installing various programs, for example, a general-purpose personal computer 700 as shown in FIG. 15 capable of executing various functions is installed from a network or a recording medium.
In FIG. 15, a CPU (Central Processing Unit) 701 executes various processes according to a program stored in a ROM (Read Only Memory) 702 or a program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. To do. The RAM 703 also appropriately stores data necessary for the CPU 701 to execute various processes.
The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output interface 705 is also connected to the bus 704.
The input / output interface 705 is connected to an input unit 706 composed of a keyboard, a mouse, etc., a display composed of an LCD (Liquid Crystal display), etc., and an output unit 707 composed of a speaker. The input / output interface 705 is connected to a storage unit 708 composed of a hard disk and a communication unit 709 composed of a network interface card such as a modem and a LAN card. The communication unit 709 performs communication processing via a network including the Internet.
A drive 710 is also connected to the input / output interface 705 as necessary, and a removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted. Then, the computer program read from these removable media is installed in the storage unit 708 as necessary.
When the above-described series of processing is executed by software, a program constituting the software is installed from a network such as the Internet or a recording medium such as a removable medium 711.
The recording medium shown in FIG. 15 is a magnetic disk (including a floppy disk (registered trademark)) on which a program is recorded, which is distributed to distribute the program to the user, separately from the apparatus main body. Removable media consisting of optical disks (including CD-ROM (compact disk-read only memory), DVD (digital versatile disk)), magneto-optical disks (including MD (mini-disk) (registered trademark)), or semiconductor memory It includes not only those configured by 711 but also those configured by a ROM 702 storing a program, a hard disk included in the storage unit 708, and the like that are distributed to the user in a state of being incorporated in the apparatus main body in advance.
Note that the series of processes described above in this specification includes processes that are performed in parallel or individually even if they are not necessarily processed in time series, as well as processes that are performed in time series in the order described. Is also included.
DESCRIPTION OF SYMBOLS 10 Touch panel apparatus, 11 Touch screen, 12-1 thru | or 12-4 Image pick-up circuit, 21 Touch panel, 22 Image / detection processing part, 23 Microcomputer, 31 Image recognition part, 32 Shape detection part, 33 Position / area detection part, 34 Contact / proximity detector, 35 Detection data processor
Proximity determination means for determining whether or not an object is close to the touch screen based on a plurality of images obtained by imaging the center of the touch screen from each side of the touch screen ;
An area estimation means for estimating a contact area of the object on the touch screen based on a three-dimensional shape obtained by analyzing the plurality of images when it is determined that the object is close to the touch screen;
Display control means for controlling the size of a component to be selected in the GUI, which is a predetermined screen displayed on the touch screen, based on the estimated contact area ;
When it is determined that an object is close to the touch screen, the GUI component that is likely to be selected by the object touching the touch screen is identified, and the identified component is identified as another component. Comprises a selected component specifying means for displaying in a different manner ,
The selected part specifying means includes:
The possibility of the selection is based on the distance between the center point of the area on the touch screen corresponding to the estimated contact area and the center of gravity of each of the GUI components displayed on the touch screen. Identify high GUI parts
Touch panel device.
When it is determined that an object is close to the touch screen, an object number determination unit that determines whether there are a plurality of the objects;
If the object is determined to be more, according to the GUI displayed on the touch screen to claim 1, further comprising a first display setting unit configured to set different first GUI is the default GUI Touch panel device.
If an object on the touch screen is determined to have approached the second display setting means for setting different second GUI and the GUI the default GUI displayed on the touch screen based on the type of the object The touch panel device according to claim 2.
The GUI part that is highly likely to be selected is repeatedly identified at a preset period,
The touch panel device according to claim 1 , further comprising: a selection confirming unit that confirms selection of the GUI component when the object comes into contact with an area overlapping with the identified GUI component on the touch screen.
The touch panel device according to claim 1, wherein the GUI is displayed on the touch screen when it is determined that an object is close to the touch screen.
A touch panel control method for a touch panel device comprising proximity determination means, area estimation means, display control means, and selected component identification means,
The proximity determining means determines whether or not an object has approached the touch screen based on a plurality of images obtained by imaging the center of the touch screen from each side of the touch screen ;
When it is determined that the object is close to the touch screen, the area estimation unit estimates a contact area of the object on the touch screen based on a three-dimensional shape obtained by analyzing the plurality of images. And
The display control means controls the size of a part to be selected in a GUI that is a predetermined screen displayed on the touch screen based on the estimated contact area ,
The selected component specifying means is
When it is determined that an object is close to the touch screen, the GUI component that is likely to be selected by the object touching the touch screen is identified, and the identified component is identified as another component. Are displayed differently,
A touch panel control method including steps .
A program that functions as a touch panel device.
Proximity determination means for determining whether or not an object is close to the touch screen based on a plurality of images obtained by imaging the center of the touch screen from each side of the touch screen;
Display control means for controlling the size of a component to be selected in the GUI, which is a predetermined screen displayed on the touch screen, based on the estimated contact area;
When it is determined that an object is close to the touch screen, the GUI component that is likely to be selected by the object touching the touch screen is identified, and the identified component is identified as another component. And a selected component specifying means for displaying in a different manner
A recording medium in which a program that functions as a touch panel device is recorded.
JP2009293147A 2009-12-24 2009-12-24 Touch panel device, touch panel control method, program, and recording medium Expired - Fee Related JP5532300B2 (en)
JP2009293147A JP5532300B2 (en) 2009-12-24 2009-12-24 Touch panel device, touch panel control method, program, and recording medium
US12/941,298 US20110157040A1 (en) 2009-12-24 2010-11-08 Touchpanel device, and control method and program for the device
CN2010105934122A CN102109925A (en) 2009-12-24 2010-12-17 Touchpanel device, and control method and program for the device
JP2011134111A JP2011134111A (en) 2011-07-07
JP5532300B2 true JP5532300B2 (en) 2014-06-25
ID=44174104
JP2009293147A Expired - Fee Related JP5532300B2 (en) 2009-12-24 2009-12-24 Touch panel device, touch panel control method, program, and recording medium
US (1) US20110157040A1 (en)
JP (1) JP5532300B2 (en)
CN (1) CN102109925A (en)
JP5654118B2 (en) * 2011-03-28 2015-01-14 富士フイルム株式会社 Touch panel device, display method thereof, and display program
CN103176731A (en) * 2011-12-26 2013-06-26 联想(北京)有限公司 Method and device for determining ending of handwriting input
KR101565226B1 (en) * 2012-09-12 2015-11-02 미츠비시 쥬고우 메카토로시스테무즈 가부시키가이샤 Operation panel and mechanical parking facility including the same
JP5782420B2 (en) * 2012-10-10 2015-09-24 株式会社Ｎｔｔドコモ User interface device, user interface method and program
JP6144501B2 (en) * 2013-02-12 2017-06-07 富士通テン株式会社 Display device and display method
CN103472962B (en) * 2013-08-01 2016-12-28 珠海中慧微电子有限公司 A kind of method identifying touch type of capacitor
CN105745606B (en) * 2013-09-24 2019-07-26 惠普发展公司，有限责任合伙企业 Target touch area based on image recognition touch sensitive surface
DE102013223518A1 (en) 2013-11-19 2015-05-21 Bayerische Motoren Werke Aktiengesellschaft Display device and method for controlling a display device
CN103677568A (en) * 2013-12-10 2014-03-26 华为技术有限公司 Clicked object amplifying method and device based on floating touch
JP6098498B2 (en) * 2013-12-19 2017-03-22 ソニー株式会社 Information processing apparatus, information processing method, and program
JP6193180B2 (en) * 2014-06-05 2017-09-06 株式会社 日立産業制御ソリューションズ Presentation terminal and presentation method
JP6265839B2 (en) * 2014-06-09 2018-01-24 アルパイン株式会社 Input display device, electronic device, icon display method, and display program
FR3028968B1 (en) * 2014-11-21 2016-11-25 Renault Sa Graphical interface and method for managing the graphical interface when tactile selecting a displayed element
FR3028967B1 (en) * 2014-11-21 2017-12-15 Renault Sas Graphical interface and method for managing the graphical interface when tactile selecting a displayed element
JP2017083973A (en) * 2015-10-23 2017-05-18 富士通株式会社 Terminal display device, display control method and display control program
EP3246808A1 (en) * 2016-05-19 2017-11-22 Siemens Aktiengesellschaft Operating and observation device and method for operating same
GB2571395A (en) * 2018-02-23 2019-08-28 Cirrus Logic Int Semiconductor Ltd A method and system for an electronic device
JPH11312264A (en) * 1998-04-28 1999-11-09 Oki Electric Ind Co Ltd Operation and display device and automatic transaction machine
JP2006085218A (en) * 2004-09-14 2006-03-30 Clarion Co Ltd Touch panel operating device
JP4841359B2 (en) * 2006-08-21 2011-12-21 アルパイン株式会社 Display control device
WO2008047552A1 (en) * 2006-09-28 2008-04-24 Kyocera Corporation Portable terminal and method for controlling the same
JP2009216888A (en) * 2008-03-10 2009-09-24 Sanyo Consumer Electronics Co Ltd Screen display device
KR101061512B1 (en) * 2008-07-25 2011-09-02 삼성전자주식회사 A mobile terminal having a touch screen and a keypad setting method in the mobile terminal
JP5191321B2 (en) * 2008-09-02 2013-05-08 株式会社ジャパンディスプレイウェスト Information input device, information input method, information input / output device, and information input program
CN101539838A (en) * 2009-05-04 2009-09-23 深圳华为通信技术有限公司 Method and device for user input through touch screen
2009-12-24 JP JP2009293147A patent/JP5532300B2/en not_active Expired - Fee Related
2010-11-08 US US12/941,298 patent/US20110157040A1/en not_active Abandoned
2010-12-17 CN CN2010105934122A patent/CN102109925A/en not_active Application Discontinuation
JP2011134111A (en) 2011-07-07
US20110157040A1 (en) 2011-06-30
CN102109925A (en) 2011-06-29
JP3998376B2 (en) 2007-10-24 Input processing method and input processing apparatus for implementing the same
US9367167B2 (en) 2016-06-14 Bottom-up watershed dataflow method and region-specific segmentation based on historic data to identify patches on a touch sensor panel
CN102981667B (en) 2015-12-02 Multi-touch input discrimination
2013-09-06 A131 Notification of reasons for refusal
2014-03-28 A01 Written decision to grant a patent or to grant a registration (utility model)