Source: https://patents.google.com/patent/CN101866263B/en
Timestamp: 2020-08-15 03:15:59
Document Index: 219655298

Matched Legal Cases: ['arts 3', 'arts 3', 'arts 3', 'arts 4', 'arts 3', 'arts 4', 'arts 3', 'arts 3', 'arts 4', 'arts 3', 'arts 4', 'arts 3']

CN101866263B - Information processing apparatus, information processing method - Google Patents
CN101866263B
CN101866263B CN2010101585103A CN201010158510A CN101866263B CN 101866263 B CN101866263 B CN 101866263B CN 2010101585103 A CN2010101585103 A CN 2010101585103A CN 201010158510 A CN201010158510 A CN 201010158510A CN 101866263 B CN101866263 B CN 101866263B
CN2010101585103A
CN101866263A (en
城井学
2009-04-14 Priority to JP2009-097724 priority Critical
2009-04-14 Priority to JP2009097724A priority patent/JP5229083B2/en
2010-10-20 Publication of CN101866263A publication Critical patent/CN101866263A/en
2013-07-24 Publication of CN101866263B publication Critical patent/CN101866263B/en
The invention relates to an information processing apparatus, an information processing method and a program. The information processing apparatus includes a first touch panel, a second touch panel, and a controller. The first touch panel displays a first object and detects a first touch operation of a user on the first object. The second touch panel displays a second object and detects a second touch operation of the user on the second object. The controller links and changes the displayed first object and the displayed second object when the second touch operation is detected within a predetermined time period since the first touch operation has been detected.
Messaging device and information processing method
The present invention relates to comprise the messaging device of a plurality of touch panels, the information processing method that is used for this messaging device and program.
Known in the past have a multiple messaging device that respectively comprises a plurality of touch panels.For example, Japanese Patent Application Laid-Open No.2003-150273 (Fig. 1,4,10 etc.; Be called patent documentation 1 hereinafter) a kind of Wrist watch type PDA (personal digital assistant) is disclosed, in this Wrist watch type PDA, connect a plurality of panels by hinge portion, described a plurality of panels comprise the LCD (LCD) that forms with a plurality of touch panels.
Yet in patent documentation 1 disclosed PDA, touch panel only detects the operation to the button that shows on each LCD.Therefore, when the touch operation that detects a plurality of touch panels, PDA can not link the demonstration of control as each target of operand between a plurality of touch panels.That is to say that PDA can not carry out single merging and handle on a plurality of touch panels.
In view of the foregoing, need such messaging device, information processing method and program thereof, this messaging device, information processing method and program can be according to detected touch operation on a plurality of touch panels, the demonstration as each target of operand is controlled in link between a plurality of touch panels, and carries out single merging and handle.
According to one embodiment of present invention, provide a kind of messaging device, this messaging device comprises first touch panel, second touch panel and controller.First touch panel displays, first target also detects first touch operation of user to first target.Second touch panel displays, second target also detects second touch operation of user to second target.When detecting second touch operation in the predetermined amount of time that detects first touch operation, the controller link changes first shown target and the second shown target.
Use this structure, messaging device can be according to the detected touch operation of first and second touch panels, and link control is handled thereby carry out single merging as the demonstration of the target of operand between each touch panel.Therefore, owing to compare with the situation of using single touch panel, messaging device not only can be handled a plurality of targets individually but also can handle a plurality of targets uniformly, so can correspondingly improve the dirigibility of processing with user's use.
Here, predetermined amount of time is for example about 0 to 3 second, but is not limited thereto.In other words, this predetermined amount of time also comprises the situation that detects first touch operation and second touch operation simultaneously.First and second touch panels comprise the touch panel with non-contact type capacitive transducer, and first and second touch operation comprise the touch operation of not following the physics contact of first and second touch panels.In addition, for example, first and second targets comprise icon, window, whole image and various other image.First and second targets can be by cutting apart the target that single target obtains and is shown on first touch panel and second touch panel, perhaps can being separated and demonstration individually.
First touch operation and second touch operation can be operations in the opposite direction.
Therefore, be set in opposite directions by being used to link the touch operation that changes first and second targets, messaging device makes the user discern with the different of the normal non-touch operation that links easily and imports touch operation intuitively.As a result, messaging device can also prevent user's faulty operation.Here, the operation on the opposite directions not only is included in the operation on the rectilinear direction, also is included in the operation on direction of a curve and the sense of rotation.
When detecting second touch operation in the described predetermined amount of time that detects first touch operation, controller can show first target on second touch panel, and shows second target on first touch panel.
Therefore, the user can switch in the target that shows on first and second touch panels or the demonstration of picture by touch operation intuitively.
In first target and second target each can be shown as the part of single the 3rd target.In this case, each in first touch operation and second touch operation can be the operation on sense of rotation.In this case, when when detecting second touch operation in the described predetermined amount of time that detects first touch operation, controller can rotate the 3rd target.
Therefore, the user not only can watch a target on a plurality of touch panels, and can be by coming rolling target with easy intuitive operation like the class of operation of rotating actual object.
In first target and second target each can be shown as the part of single the 3rd target.In this case, first touch operation and second touch operation can be the operations on the direction of removing mutually.In this case, when when detecting second touch operation in the described predetermined amount of time that detects first touch operation, controller can change the 3rd target, makes the 3rd target be torn.
Therefore, the user not only can watch a target on a plurality of touch panels, and can by be easy to intuitive operation like the class of operation of tearing actual document etc. and change target.
In this case, messaging device can also comprise and be used for storage and the corresponding memory of data of the 3rd target, and is changed and when making that the 3rd target is torn, controller can be deleted this data from storer when the 3rd target.
Therefore, the user can handle the deletion processing of carrying out intuitively corresponding to the data of the 3rd target by seeing as the deletion of tearing the operation of the 3rd target.
First touch operation and second touch operation can be the operations on the direction that shifts near mutually.In this case, when when detecting second touch operation in the described predetermined amount of time that detects first touch operation, controller can change first target and second target, makes win target and second target be merged into a target.
Therefore, the user can be by making a plurality of objects be easy to intuitive operation mutually like the class of operation of concentrating them with actual, and the target that will show on first and second touch panels is merged into a target.
Messaging device can also comprise: sensor, detect the object that exists at the preset distance place from first touch panel and second touch panel.In this case, when when detecting second touch operation and sensor to object in the described predetermined amount of time that detects first touch operation, controller can link and change first shown target and the second shown target.
In this case, for example, the object that is present in the preset distance place is user's a hand, more specifically the root between two fingers.
Therefore, for example, also import under the situation of second touch operation with another finger of this hand when finger input first touch operation of a hand using the user, messaging device can have been imported touch operation by discerning reliably with the sensor hand with these two fingers.Therefore, even only import under the situation that first touch operation and other object are accidental with second touch panel contact, messaging device also can prevent because the demonstration variation of the detection of the mistake that the contact of other object causes and mistake.This is because in this case, this object self is by sensor, but is detected in the distance less than above-mentioned preset distance.
In addition, in this case, when sensor arrives object, controller can change first shown target and the second shown target with first pattern, when sensor did not detect object, controller can change first shown target and the second shown target with second pattern that is different from first pattern.
Therefore, even under the situation that detects the first and second identical operations, whether the detection to object exists according to sensor, and messaging device also can change first and second targets with different patterns.Therefore, messaging device can be carried out more multimodal processing and improve user friendly according to first and second touch operation.
Messaging device also can comprise: storer, storage is corresponding to first data of first target with corresponding to second data of second target.In this case, first touch operation can be the operation of surrounding first target on preset rotating direction, and second touch operation can be the operation of surrounding second target on described preset rotating direction.In this case, when when detecting second touch operation in the described predetermined amount of time that detects first touch operation, controller can be so that storer divides into groups to first data and second data and stores.
Therefore, the user can come dividing into groups corresponding to first and second data of first and second targets and storing by only carrying out the easy intuitive operation of surrounding first and second targets.
According to another embodiment of the present invention, provide a kind of information processing method, this information processing method comprises: first touch panel displays, first target also detects first touch operation of user to first target.In addition, this information processing method comprises: second touch panel displays, second target also detects second touch operation of user to second target.When detecting second touch operation in the predetermined amount of time that detects first touch operation, link changes first shown target and the second shown target.
According to still another embodiment of the invention, provide a kind of messaging device that comprises first touch panel and second touch panel that makes to carry out the program that the first detection step, second detects step and shows controlled step.First detects step comprises: first touch panel displays, first target also detects first touch operation of user to first target.Second detects step comprises: second touch panel displays, second target also detects second touch operation of user to second target.Show that controlled step comprises: when when detecting second touch operation in the predetermined amount of time that detects first touch operation, link changes first shown target and the second shown target.
As mentioned above, according to embodiments of the invention, can between a plurality of touch panels, link control and carry out single merging processing by according to detected touch operation on a plurality of touch panels as the demonstration of each target of operand.
According to the detailed description of following embodiment to best mode as shown in drawings of the present invention, it is clearer that these and other purposes, features and advantages of the present invention will become.
Fig. 1 is the figure that illustrates according to the outward appearance of the portable information device of the embodiment of the invention;
Fig. 2 is the figure that illustrates according to the hardware configuration of the portable information device of the embodiment of the invention;
Fig. 3 is the figure that illustrates according to the opening and closing state of the portable information device of the embodiment of the invention;
Fig. 4 illustrates and process flow diagram by the corresponding flow process of handling of the touch operation of carrying out according to the portable information device of the embodiment of the invention;
Fig. 5 illustrates the figure according to the example of the state of the rotate gesture the portable information device of the embodiment of the invention (rotation gesture) that watches from the in-plane of touch panel;
Fig. 6 is the figure that illustrates with the example of the corresponding concrete display process of rotate gesture shown in Fig. 5;
Fig. 7 is the figure that illustrates with another example of the corresponding concrete display process of rotate gesture shown in Fig. 5;
Fig. 8 illustrates the figure according to the example of the state of the gesture of handling knob the portable information device of the embodiment of the invention (pinch gesture) that watches from the in-plane of touch panel;
Fig. 9 is the figure that illustrates with the example of the corresponding concrete display process of gesture of handling knob shown in Fig. 8;
Figure 10 illustrates the figure according to the example of the state of the amplification gesture the portable information device of the embodiment of the invention (zoom-in gesture) that watches from the in-plane of touch panel;
Figure 11 is the figure that illustrates with the example of the corresponding concrete display process of amplification gesture shown in Figure 10;
Figure 12 illustrates the figure according to the example of the state that tears gesture (tear gesture) the portable information device of the embodiment of the invention that watches from the in-plane of touch panel;
Figure 13 is the figure that illustrates with the example of tearing the corresponding concrete display process of gesture shown in Figure 12;
Figure 14 illustrates the figure according to the example of the state of the grouping gesture the portable information device of the embodiment of the invention (grouping gesture) that watches from the in-plane of touch panel; And
Figure 15 is the figure that illustrates with the example of the corresponding concrete display process of grouping gesture shown in Figure 14.
(summary of portable information device)
Fig. 1 is the figure that illustrates according to the outward appearance of the portable information device of the embodiment of the invention.
As shown in the figure, portable information device 100 comprises so-called flip-type housing 1, and in this flip-type housing 1, two housing 1a and 1b rotatably (under openable state) are connected.Fig. 1 illustrates the state that housing 1a and 1b open.The user operates portable information device 100 in this state.
Housing 1a and 1b comprise touch panel 2a and 2b respectively on the surface of exposing under its open mode.Touch panel 2a and 2b are provided with integratedly with display 25a and 25b respectively.Touch panel 2a comprises the touch-surface with end limit 21 and end limit 22 relative with end limit 21.Similarly, touch panel 2b comprises the touch-surface with end limit 23 and end limit 24 relative with end limit 23.Touch panel 2a and 2b detect the touch operation that user's finger is carried out the target that is shown on display 25a and the 25b.The user uses two fingers (being generally thumb and forefinger) of a hand or a finger of a hand and a finger (being generally forefinger) of another hand to import touch operation.Usually, touch operation is included in the operation (operation of paddling finger on touch panel 2a and 2b) of dilatory finger on any direction or clicks (the simple touch) operation, but is not limited thereto.
Housing 1a comprises geared parts 3a on two sides of end limit 22 sides, and housing 1b comprises geared parts 3b on two sides of end limit 23 sides.Geared parts 3a is connected under engagement by linking parts 4 with geared parts 3b.Link parts 4 and be constructed such that all an end of two plates (or bar) rotatably connects, and other end of these two plates (or bar) is connected to the turning axle of geared parts 3a and 3b.By geared parts 3a and 3b and binding parts 4, housing 1a rotatably is connected with 1b.Under the situation of this structure, compare with the situation of using hinge for example to be connected housing 1a and 1b, can be so that the distance between the touch panel 2b of the touch panel 2a of housing 1a and housing 1b be closer proximity to each other.
Optical sensor 5 is set between the end limit 23 of the end limit 22 of touch panel 2a and touch panel 2b.Optical sensor 5 comprises 5a of optical sensor portion and the 5b of optical sensor portion, the 5a of this optical sensor portion is set on the housing 1a continuously from the end limit 22 of touch panel 2a, and the 5b of this optical sensor portion is set on the housing 1b continuously from the end limit 23 of touch panel 2b.The surface of the touch-surface of the touch-surface of touch panel 2a, touch panel 2b and optical sensor 5 is set to when housing 1a and 1b are in open mode in the same plane.
(hardware configuration of portable information device)
Fig. 2 is the figure that the hardware configuration of portable information device 100 is shown.
As shown in the figure, except touch panel 2a and 2b (display 25a and 25b) and optical sensor 5, portable information device 100 also comprises CPU (CPU (central processing unit)) 11, RAM (random access memory) 12 and flash memory (flash memory) 13.Portable information device 100 also comprises Department of Communication Force 14, exterior I/F (interface) 15, key/switch portion 16, microphone 17, loudspeaker 18, head phone 19 and camera 20.
The various piece switching signal of CPU 11 and portable information device 100 to be carrying out various operations, and control is handled and other processing with the corresponding various types of demonstrations controls of touch operation (touch gestures) (show and change) that touch panel 2a and 2b are carried out generally.
RAM 12 is as the perform region of CPU 11, and storage comprises various types of data of the various GUI (target) that will be handled by CPU 11 and such as the program that is used to carry out with the application of the corresponding various display process of touch operation that touch panel 2a and 2b are carried out provisionally.
For example, flash memory 13 is NAND types.Flash memory 13 storage comprises such as various types of data of the image of various icons and such as the control program that will be carried out by CPU 11 be used to carry out the various programs of the application of various types of display process.These application can be stored in other recording medium such as the storage card (not shown).Portable information device 100 can comprise conduct to the replacement of flash memory 13 or additional HDD.
Department of Communication Force 14 is the interfaces that are used for according to the standard of Ethernet (Ethernet, registered trademark), WLAN (LAN (Local Area Network)) etc. portable information device 100 being connected to the Internet or LAN.
Exterior I/F 15 based on the various standards of USB (USB (universal serial bus)), WLAN etc. by with wired or wireless connection of external unit exchange various types of data.Replacedly, exterior I/F 15 can be the interface that is used to be connected to such as the various storage cards of memory stick.
Key/switch portion 16 is accepted with the corresponding operation of function that is equal to following function and is transmitted input signal to CPU 11: such as to the ON/OFF (on/off) of power supply (not shown) and to the switching of various functions can not be by function to the operation execution of touch panel 2a and 2b; And can be by the function that the operation of touch panel 2a and 2b is carried out.
Be connected to by Department of Communication Force 14 under the situation of the miscellaneous equipment on the network at portable information device 100, the audio frequency of microphone 17 input such as user voices is used for verbal message.
19 outputs of loudspeaker 18 and head phone are stored in the flash memory 13 etc. or from the sound signal of Department of Communication Force 14 or microphone 17 inputs.
Camera 20 is by image pick-up device capturing still image and moving image such as CMOS (complementary metal oxide semiconductor (CMOS)) sensor and CCD (charge-coupled image sensor) sensor.The data of catching are stored in RAM 12 or the flash memory 13, perhaps are sent to miscellaneous equipment on the network by Department of Communication Force 14.
Although resistive film mode or capacitive way can instead be used the alternate manner such as way of electromagnetic induction, matrix switch mode, surface acoustic wave mode and infrared mode as the mode of operation of touch panel 2a and 2b.When capacitive way was used as mode of operation, " touch operation " in the present embodiment not only comprised the situation that user's finger physically contacts touch panel 2a and 2b, also comprised to make the user point situation enough approaching in the scope that can detect capacitance variations.
Optical sensor 5 is reflection-types for example, and comprises such as the light-emitting device of the LED (light emitting diode) of emission infrared light and such as the optical pickup apparatus of photodiode and phototransistor.Whether optical sensor 5 detects has object to be present in the predetermined distance range of optical sensor 5 tops.In the present embodiment, whether optical sensor 5 hand (finger) that detects users is present in the predetermined distance range of optical sensor 5 tops.That is to say that whether the hand that optical sensor 5 detects the user is present in optical sensor 5 tops makes two fingers of user stride across touch panel 2a and 2b.In this case, predetermined distance range is for example about 3cm to 15cm, but is not limited thereto.
Detecting the touch operation of touch panel 2a and 2b and having object to be present under the situation in this predetermined distance range, judge that the user is with two finger execution touch operation.For example, using under the situation of thumb and forefinger execution to the touch operation of touch panel 2a and 2b the user, optical sensor 5 detects the root between thumb and the forefinger.On the other hand, detect touch operation at optical sensor 5 but in above-mentioned predetermined distance range, do not detect under the situation of object, judge the user with the finger touch touch panel 2a of a hand also with the finger touch touch panel 2b of another hand.
Although not shown, portable information device 100 also comprises the cable and the flexible base, board of the electrical connection that is used for touch panel 2a and 2b.Cable and flexible base, board can be set to stride across geared parts 3a and 3b and link parts 4.
Display 25a and 25b are LCD or the OELD (display of organic electroluminescence) of for example TFT (thin film transistor (TFT)) etc., and show the GUI that is used for touch operation and other image such as icon, window and whole image.Display 25a and 25b form with touch panel 2a and 2b as mentioned above.
(operation of portable information device)
Next, will the operation of the portable information device 100 of above-mentioned structure be described.
(the opening and closing operation of portable information device)
At first, will the opening and closing operation of portable information device 100 be described.Fig. 3 is the figure that the opening and closing state of portable information device 100 is shown.
As shown in Figure 3A, for example, when the user lifts housing 1a under housing 1a and 1b closing state, thereby housing 1a is opened in geared parts 3a and 3b and 4 rotations of binding parts, shown in Fig. 3 B to 3D.Therefore, expose touch panel 2a and 2b.Then, shown in Fig. 3 E, when housing 1a and 1b open 180 when spending, housing 1a and 1b are static, make that the touch-surface of touch panel 2a and 2b and optical sensor 5 are in the same plane.Under the state shown in Fig. 3 E, input user's touch operation.By like this with the touch-surface setting at grade, the user can carry out operation intuitively, and is not subjected to the interference of the optical sensor between different touch panels and this two touch panels.
(target of portable information device shows control and treatment)
Next, will describe and show control and treatment, wherein, will with touch panel 2a and the corresponding Object linking of the detected touch operation of 2b (touch gestures) by portable information device 100.In the following description, CPU 11 is described to operating main body (agent), still carries out the operation of CPU 11 collaboratively with the program of launching in RAM 12.
Fig. 4 illustrates the process flow diagram that the target of being carried out by portable information device 100 shows the flow process of control and treatment.
As shown in the figure, at first, CPU 11 is display-object (step 41) on touch panel 2a and 2b.Then, CPU 11 detects the touch operation (step 42) that whether detects simultaneously target on touch panel 2a and 2b.In this case, " simultaneously " may not be meant proper synchronization, comprises the situation that for example detected each touch operation in 0.5 second.
Detect at the same time under the situation of touch operation ("Yes"), CPU 11 judges the whether rotation (step 43) on same sense of rotation of touch point of detected touch operation.When judging that the touch point rotates on same sense of rotation ("Yes"), CPU 11 judges whether optical sensor 5 detects object (hand) (step 44) in predetermined distance range.Under the situation that does not detect object (hand) in predetermined distance range ("No"), CPU11 judges that detected touch operation is wrong detection, and finishes the link of target is shown control and treatment.
Under the situation that detects object (hand) in predetermined distance range ("Yes"), CPU 11 judges that touch operation is rotate gesture (step 45).
Fig. 5 is the figure of example that the state of the rotate gesture of watching from the in-plane of touch panel 2a and 2b is shown.
Shown in Fig. 5 A and 5B, the user touches touch panel 2a and 2b respectively with two finger F 1 and F2 (thumb and forefinger), and with the finger root as fulcrum pivoting finger on the direction of the arrow of this figure.In this case, the touch operation that CPU 11 detects on this sense of rotation, and it is in one's hands to confirm that optical sensor 5 detects.
Return with reference to Fig. 4, when detecting rotate gesture, CPU 11 carries out with the corresponding demonstration control of rotate gesture (show and change) and handles (step 46).Fig. 6 is the figure that illustrates with the example of the corresponding concrete demonstration control and treatment of rotate gesture.State when Fig. 6 A and 6B are illustrated in the input rotate gesture, Fig. 6 C are illustrated in the state after the input rotate gesture.
As shown in Figure 6A, as target O1 and O2, for example, just be displayed on touch panel 2a and the 2b by the application A of portable information device 100 execution and the GUI picture of B.In this state, shown in Fig. 6 A and 6B, the user imports rotate gesture with two finger F 1 and F2.Then, shown in Fig. 6 C, CPU 11 carries out and change the GUI picture of using A and the processing of using the GUI picture of B mutually between touch panel 2a and 2b.
In the figure, CPU 11 carries out the processing that changes whole image according to rotate gesture, but can change the position such as other target (image) of window and icon between touch panel 2a and 2b.
Fig. 7 is the figure that illustrates with another example of the corresponding concrete demonstration control and treatment of rotate gesture.State when Fig. 7 A and 7B are illustrated in the input rotate gesture, Fig. 7 C are illustrated in the state after the input rotate gesture.
Shown in Fig. 7 A, be displayed on respectively on touch panel 2a and the 2b as upper lateral part O1 and the following side O2 of the target O of picture etc.In this state, the user is with two finger F 1 and the above-mentioned rotate gesture of F2 input, shown in Fig. 7 A and 7B.Then, CPU 11 with the Z axle of these figure as turning axle with target O Rotate 180 degree, shown in Fig. 7 C.
CPU 11 not only can be with target O Rotate 180 degree, and can be with rotation angle (amount of movement of each touch point) the corresponding angle of target O rotation with rotate gesture.
Return with reference to Fig. 4, when judging that the touch point does not rotate on same sense of rotation in step 43 ("No"), CPU 11 judges whether the touch point moves (step 47) point-blank on the direction that shifts near mutually.When the judgement touch point is mobile point-blank on the direction that shifts near mutually ("Yes"), CPU 11 judges whether optical sensor 5 detects object (hand) (step 48) in predetermined distance range.Under the situation that does not detect object (hand) in predetermined distance range ("No"), CPU 11 judges that detected touch operation is wrong detection, and finishes the link of target is shown control and treatment.
Under the situation that detects object (hand) in predetermined distance range ("Yes"), CPU 11 judges that touch operation is the gesture (step 49) of handling knob.
Fig. 8 is the figure of example that the state of the gesture of handling knob of watching from the in-plane of touch panel 2a and 2b is shown.
Shown in Fig. 8 A and 8B, the user touches touch panel 2a and 2b respectively with two finger F 1 and F2 (thumb and forefinger), and moves these fingers point-blank on the reverse direction (direction of arrow among the figure) that the touch point of these two finger F 1 and F2 shifts near mutually.That is to say that the user carries out as pinches thing or make these two finger F 1 and F2 and touch gestures together with two finger F 1 and F2.In this case, the touch operation on the straight line reverse direction that CPU 11 detected touch point shift near mutually, and it is in one's hands to confirm that optical sensor 5 detects.
Return with reference to Fig. 4, when detecting when handling knob gesture, CPU 11 carries out with the corresponding demonstration control of the gesture of handling knob (show and change) and handles (step 50).Fig. 9 is the figure of example that illustrates with the corresponding concrete demonstration control and treatment of gesture of handling knob.Fig. 9 A and 9B are illustrated in the state of importing when handling knob gesture, and Fig. 9 C is illustrated in the gesture state afterwards of handling knob of importing.
Shown in Fig. 9 A, be displayed on the touch panel 2a such as the target O1 of the picture of landscape (background), and, be displayed on the touch panel 2b such as the target O2 of people's picture.In this state, shown in Fig. 9 A and 9B, the user imports the above-mentioned gesture of handling knob with two finger F 1 and F2.Then, shown in Fig. 9 B, CPU 11 moves to optical sensor 5 sides with target O1 and O2.In addition, for example, when being equal to or less than preset distance, CPU 11 composite object O1 and target O2 also show them, shown in Fig. 9 C when the distance between these two target O1 and the O2 (two touch point between distance).That is to say that CPU 11 will be combined on the target O1 as the target O2 of people's picture and target O1 as the picture of landscape, go up at touch panel 2a (perhaps touch panel 2b) and produce and show target O3 as new picture.
Except above-mentioned combined treatment, suppose various types of processing conducts and the corresponding demonstration control and treatment of the gesture of handling knob.For example, CPU 11 can carry out and show control, and wherein, the microgranular or cellulated target that will definitely be presented on touch panel 2a and the 2b according to handling knob is merged into one.This demonstration control can be applied to recreation application etc.
In addition, according at the gesture of handling knob such as two targets of picture, CPU 11 can carry out processing, wherein, these two targets as a special edition (album) data storage in flash memory 13 grades.
Return with reference to Fig. 4, when judging the touch point in step 47 not on the direction that shifts near mutually when mobile point-blank ("No"), CPU 11 judges touch point whether on the direction of removing mutually mobile (step 51) point-blank.
When judging the touch point not on the direction of removing mutually when mobile point-blank ("No"), CPU 11 judges and does not import detectable gesture, finishes the link of target is shown control and treatment.
When the judgement touch point is mobile point-blank on the direction of removing mutually ("Yes"), CPU 11 judges whether optical sensor 5 detects object (hand) (step 52) in predetermined distance range.
Under the situation that detects object (hand) in predetermined distance range ("Yes"), CPU 11 judges whether touch operation is to amplify gesture (step 53).
Figure 10 is the figure of example that the state of the amplification gesture of watching from the in-plane of touch panel 2a and 2b is shown.
Shown in Figure 10 A and 10B, the user touches touch panel 2a and 2b respectively with two finger F 1 and F2 (thumb and forefinger), and moves these fingers point-blank on the reverse direction (direction of arrow among the figure) that the touch point of these two finger F 1 and F2 is removed mutually.
This gesture is the gesture on the reverse direction of gesture of handling knob.In this case, the touch operation on the straight line reverse direction that CPU 11 detected touch point are removed mutually, and it is in one's hands to confirm that optical sensor 5 detects.
Return with reference to Fig. 4, when detecting the amplification gesture, CPU 11 carries out with the corresponding demonstration control of amplification gesture (demonstration changes) and handles (step 53).CPU 11 illustrates and amplifies the figure of the example of the corresponding concrete demonstration control and treatment of gesture.State when Figure 11 A and 11B are illustrated in input amplification gesture, Figure 11 C are illustrated in input and amplify gesture state afterwards.
Shown in Figure 11 A, be displayed on respectively on touch panel 2a and the 2b such as upper lateral part O1 and the following side O2 of a target O of picture and window.In this state, the user is with two finger F 1 and the above-mentioned amplification gesture of F2 input, shown in Figure 11 A and 11B.Then, CPU 11 amplification target O are shown in Figure 11 C.That is to say that CPU 11 shows the amplification target O as target O1 and O2 respectively on touch panel 2a and 2b.
Return with reference to Fig. 4, under the situation that does not detect object (hand) in step 52 in predetermined distance range ("No"), CPU 11 judges that touch operation is for tearing gesture (step 55).
Figure 12 is the figure that the example of the state of watching from the in-plane of touch panel 2a and 2b that tears gesture is shown.
Shown in Figure 12 A and 12B, the user touches touch panel 2a and 2b respectively with the finger F 1 of a hand and the finger F 2 (forefinger) of another hand, and on the reverse direction (direction of arrow among the figure) that the touch point of these two finger F 1 and F2 is removed mutually moveable finger point-blank.
This touch operation is similar to the action that the user tears the situation of actual document.Here, the moving direction of touch operation is identical with above-mentioned amplification gesture.Therefore, the user carries out touch operation, makes these two finger F 1 and F2 or this two hands not be positioned at optical sensor 5 tops.That is to say,, amplify gesture and tear gesture and also whether exist and differentiate mutually based on the detection of optical sensor 5 even in the touch operation on same direction.Touch operation on the straight line reverse direction that CPU 11 detected touch point are removed mutually, and confirm that optical sensor 5 does not detect in one's hands.
Return with reference to Fig. 4, when detecting when tearing gesture, CPU 11 carries out and tears the corresponding demonstration control of gesture (show and change) and handle (step 56).Figure 13 is the figure that illustrates and tear the example of the corresponding concrete demonstration control and treatment of gesture.Figure 13 A and 13B are illustrated in the state of importing when tearing gesture, and Figure 13 C is illustrated in input and tears gesture state afterwards.
As shown in FIG. 13A, upper lateral part O1 and the following side O2 such as a target O of document files is displayed on respectively on touch panel 2a and the 2b.In this state, the user imports the above-mentioned gesture of tearing with two finger F 1 and F2, shown in Figure 13 A and 13B.Then, CPU 11 changes target O, makes target O is torn into a slice O1 and another sheet O2, shown in Figure 13 B.
In addition, shown in Figure 13 C, CPU 11 can show pop-up window W, and whether user for confirmation is from the data of deletion such as flash memory 13 grades corresponding to target O.In this case, whether the user selects deleted data by touch operation on pop-up window W.The processing that CPU 11 carries out corresponding to this selection.
Return with reference to Fig. 4, when judgement does not detect touch operation simultaneously in step 42 ("No"), whether at the fixed time CPU 11 judges poor section interior (step 57) detection time between the touch operation of touch panel 2a and 2b.
That is to say that CPU 11 judges from detecting the touch operation that whether detects in the predetermined amount of time to the touch operation of touch panel 2a (2b) touch panel 2b (2a).Here, predetermined amount of time is for example about 3 seconds, but is not limited thereto.
When judging that detection time, difference did not fall within the predetermined amount of time ("No"), CPU11 finishes the link of target is shown control and treatment.
In the time of in judging poor section at the fixed time detection time ("Yes"), CPU 11 judges that whether these two touch points are all at same sense of rotation upper drawing circle circle (step 58).When judging these two touch points not all at same sense of rotation upper drawing circle circle ("No"), CPU 11 finishes the link of target is shown control and treatment.
When judging these two touch points all at same sense of rotation upper drawing circle circle ("Yes"), CPU 11 judges that touch operation are grouping gestures (step 59).
Figure 14 is the figure of example that the state of the grouping gesture of watching from the in-plane of touch panel 2a and 2b is shown.
Shown in Figure 14 A, the user at first uses the finger F (forefinger) of a hand to touch touch panel 2a, and moveable finger F, with such as clockwise predetermined direction upper drawing circle circle.Subsequently, in the section, the user touches touch panel 2b with finger F, and moveable finger F at the fixed time, with the identical direction upper drawing circle circle of moving direction on the touch panel 2a.The a series of touch operation of CPU 11 detections as drawing circle are as the grouping gesture.
Return with reference to Fig. 4, when detecting the grouping gesture, CPU 11 carries out with the corresponding demonstration control of grouping gesture (show and change) and handles (step 60).Figure 15 is the figure of example that illustrates with the corresponding concrete demonstration control and treatment of gesture of dividing into groups.State when Figure 15 A and 15B are illustrated in input grouping gesture, Figure 15 C are illustrated in the state after the input grouping gesture.
Shown in Figure 15 A, be displayed on respectively on touch panel 2a and the 2b such as the target O1 and the O2 of file icon.In this state, the user imports above-mentioned grouping gesture with finger F in predetermined duration, shown in Figure 15 A and 15B.Then, CPU 11 shows the group frame G that surrounds target O1 and O2, shown in Figure 15 C.
In addition, CPU 11 will be stored in the flash memory 13 etc. corresponding to the packet of target O1 that is surrounded by group frame G and O2 (file icon) and with it.
And in this case, CPU 11 can show on touch panel 2a or 2b that indication carried out the text or the image of packet transaction.For example, CPU 11 shows the text or image on the touch panel 2b that shows the second target O2 that surrounds.Certainly, the text or image also can be displayed on the touch panel 2a.
Target O1 and O2 can not be icons, but the file of picture or document, and its area can be virtually any size, as long as its is less than touch-surface of touch panel 2a and 2b.
Except the situation of surrounding two targets, even under the situation that detects the touch operation of surrounding three targets or three above targets continuously, CPU 11 also can divide into groups to these targets similarly.
By above-mentioned processing, be through with to the link demonstration control and treatment of target.
(uses of two touch panels)
As the use of two touch panel 2a in the present embodiment and 2b, suppose various uses.
For example, as mentioned above, two touch panel 2a and 2b can be used as single picture.In this case, a target is shown as and strides across this two touch panel 2a and 2b, and, carry out with to the touch operation of touch panel 2a and 2b corresponding single merging handle.
In addition, (cloud) side is used a touch panel 2a (2b) beyond the clouds, uses another touch panel 2b (2a) at client-side.
Side beyond the clouds shows the image of the downloaded of the high in the clouds ISP from the network that connects by Department of Communication Force 14.At client-side, be presented in the portable information device 100 image of the file etc. of storage.
By the above-mentioned various touch gestures of each target input for high in the clouds side and client-side, the user can so that portable information device 100 carry out such as processing switching, combination or the grouping of the display position of target.
And, for the task in the multitasking, can use two touch panel 2a and 2b.
In this case, by between touch panel 2a and 2b, each target being imported above-mentioned various touch gestures mutually, the display position that the user can the task switching picture or each task merged into a task.
(summary of present embodiment)
As mentioned above, according to present embodiment, portable information device 100 can link the demonstration of change as each target of the object of touch gestures according to the touch operation (touch gestures) to touch panel 2a and 2b.
Therefore, owing to compare with the situation of using single touch panel, portable information device 100 not only can be handled a plurality of targets individually but also can handle a plurality of targets uniformly, so can correspondingly improve the dirigibility of processing with user's use.
In addition, since touch gestures be similar to rotation actual object or document operation, pinch operation, widen operation, tear operation etc., so the user can carry out the demonstration control and treatment corresponding to each gesture intuitively.
The present invention is not restricted to the above embodiments, can carry out various distortion to the present invention under the situation that does not break away from main idea of the present invention.
In the above-described embodiment, shown in Figure 12 and 13, such example has been described: wherein, on the gesture of tearing gesture vertical direction (Y direction) in the drawings.Yet, tear gesture also in the horizontal direction on (directions X) or the vergence direction.
When making in the horizontal direction under the situation of tearing gesture, although straight line and curve between exist differently, can have the similar situation of gesture direction of the rotate gesture shown in gesture direction and Fig. 5 and 6.Yet, whether existing by the detection of confirming optical sensor 5, CPU 11 can distinguish these two kinds of gestures reliably and prevent wrong detection.
In the above-described embodiments, portable information device 100 detects all touch gestures that rotate, make up, amplify, tear and divide into groups, and the demonstration control of carrying out corresponding to these gestures (show and change) is handled.Yet portable information device 100 only needs to detect at least a in these gestures and carries out demonstration control and treatment corresponding to this.
In the above-described embodiments, such example has been described: wherein, portable display device 100 is provided with two touch panel 2a and 2b.Yet the number of touch panel is not limited to two, as long as it is a plurality of, and, can between these touch panels, carry out the processing identical with above-mentioned processing.
In the above-described embodiments, with the touch operation of user's finger input, but also can use such as the instrument input of stylus touch operation to touch panel 2a and 2b to touch panel 2a and 2b.
The example that can use portable information device 100 of the present invention in the foregoing description comprises various messaging devices, for example, cell phone, PDA, smart phone, portable music/video player, electronic dictionary and electronic notebook (electronic organizer).
And the present invention is not limited to portable information device, equally applicable to various fixed messaging devices.
The application comprise with on April 14th, 2009 to Japan that Jap.P. office submits to relevant theme of disclosed theme among the patented claim JP 2009-097724 formerly, the full content of this patented claim is incorporated this paper into way of reference.
It will be understood by those skilled in the art that under the situation of the scope that does not break away from claims and equivalent thereof, can carry out various distortion, combination, sub-portfolio and replacement according to designing requirement and other factors.
First touch panel shows first target and detects first touch operation of user to first target;
Second touch panel shows second target and detects second touch operation of user to second target; With
Controller, when when in the predetermined amount of time that detects first touch operation, detecting first touch operation and second touch operation simultaneously, link changes first shown target and the second shown target, and wherein the touch point of first touch operation and second touch operation rotates on same sense of rotation or moves point-blank.
2. messaging device according to claim 1,
Wherein, when when in the described predetermined amount of time that detects first touch operation, detecting first touch operation and second touch operation simultaneously, controller shows first target on second touch panel, and shows second target on first touch panel.
3. messaging device according to claim 1,
Wherein, each in first target and second target is shown as the part of the 3rd single target,
In first touch operation and second touch operation each is the operation on sense of rotation, and
When detecting first touch operation and second touch operation simultaneously in the described predetermined amount of time that detects first touch operation, controller rotates the 3rd target.
4. messaging device according to claim 1,
First touch operation and second touch operation are the operations on the direction of removing mutually, and
When detecting first touch operation and second touch operation simultaneously in the described predetermined amount of time that detects first touch operation, controller changes the 3rd target, makes the 3rd target be torn.
5. messaging device according to claim 4 also comprises:
Storer is used for storage and the corresponding data of the 3rd target,
Wherein, when the 3rd target was changed and makes that the 3rd target is torn, controller was deleted this data from storer.
6. messaging device according to claim 1,
Wherein, first touch operation and second touch operation are the operations on the direction that shifts near mutually, and
When detecting first touch operation and second touch operation simultaneously in the described predetermined amount of time that detects first touch operation, controller changes first target and second target, makes win target and second target be merged into one.
7. messaging device according to claim 1 also comprises:
Sensor detects the object that exists at the preset distance place from first touch panel and second touch panel,
Wherein, when when detecting first touch operation and second touch operation and sensor to object simultaneously in the described predetermined amount of time that detects first touch operation, the controller link changes first shown target and the second shown target.
8. messaging device according to claim 7,
Wherein, when sensor arrives object, controller changes first shown target and the second shown target with first pattern, and when sensor did not detect object, controller changed first shown target and the second shown target with second pattern that is different from first pattern.
9. messaging device according to claim 1 also comprises:
Storer, storage be corresponding to first data of first target with corresponding to second data of second target,
Wherein, first touch operation is the operation of surrounding first target on preset rotating direction,
Second touch operation is the operation of surrounding second target on described preset rotating direction, and
When detecting first touch operation and second touch operation simultaneously in the described predetermined amount of time that detects first touch operation, controller makes storer divide into groups to first data and second data and stores.
10. information processing method comprises:
First touch panel displays, first target also detects first touch operation of user to first target;
Second touch panel displays, second target also detects second touch operation of user to second target; And
When in the predetermined amount of time that detects first touch operation, detecting first touch operation and second touch operation simultaneously, link changes first shown target and the second shown target, and wherein the touch point of first touch operation and second touch operation rotates on same sense of rotation or moves point-blank.
CN2010101585103A 2009-04-14 2010-04-07 Information processing apparatus, information processing method CN101866263B (en)
JP2009-097724 2009-04-14
JP2009097724A JP5229083B2 (en) 2009-04-14 2009-04-14 Information processing apparatus, information processing method, and program
CN101866263A CN101866263A (en) 2010-10-20
CN101866263B true CN101866263B (en) 2013-07-24
CN102147672A (en) * 2010-02-04 2011-08-10 英华达（上海）科技有限公司 Handheld electronic device
US9489078B2 (en) 2011-02-10 2016-11-08 Samsung Electronics Co., Ltd. Portable device comprising a touch-screen display, and method for controlling same
JP5777915B2 (en) * 2011-03-29 2015-09-09 京セラ株式会社 Electronic device and display method thereof
WO2013113760A1 (en) * 2012-01-30 2013-08-08 Leica Microsystems Cms Gmbh Microscope with wireless radio interface and microscope system
US10490166B2 (en) 2012-07-09 2019-11-26 Blackberry Limited System and method for determining a display orientation of a mobile device
FR3003964B1 (en) * 2013-04-02 2016-08-26 Fogale Nanotech Device for interacting, without contact, an electronic and / or computer apparatus, and apparatus provided with such a device
JP5717813B2 (en) * 2013-08-20 2015-05-13 京セラ株式会社 Mobile device
CN105407800B (en) * 2013-09-11 2019-04-26 麦克赛尔株式会社 Brain disorder evaluating apparatus and storage medium
US20170052678A1 (en) * 2014-05-15 2017-02-23 Sony Corporation Information processing device, display control method, and program
WO2015192085A2 (en) * 2014-06-12 2015-12-17 Apple Inc. Systems and methods for multitasking on an electronic device with a touch-sensitive display
JP5788068B2 (en) * 2014-08-01 2015-09-30 京セラ株式会社 Mobile device
US20170255374A1 (en) * 2014-09-16 2017-09-07 Nec Corporation Multi-screen display position exchange method, information processing apparatus, and control method and control program of information processing apparatus
CN107077314A (en) * 2016-09-12 2017-08-18 深圳前海达闼云端智能科技有限公司 A kind of electronic equipment
JP6621785B2 (en) * 2017-10-04 2019-12-18 株式会社Ｎｔｔドコモ Display device
JP2019109802A (en) * 2017-12-20 2019-07-04 コニカミノルタ株式会社 Display device, image processing device and program
CN1949905A (en) * 2005-10-14 2007-04-18 Lg电子株式会社 Method for displaying multimedia contents and mobile communications terminal capable of implementing the same
JP4650624B2 (en) * 2005-07-27 2011-03-16 日本電気株式会社 Backlight unit and liquid crystal display device
JP4650280B2 (en) 2006-01-19 2011-03-16 ソニー株式会社 Display device and method, and program
RU2007109208A (en) * 2007-03-13 2007-07-10 Владимир Анатольевич Ефремов (RU) Device for input and output of computer data
JP5229083B2 (en) 2013-07-03
US20200183577A1 (en) 2020-06-11 Devices, Methods, and Graphical User Interfaces for Messaging
JP6570758B2 (en) 2019-09-04 Device, method, and graphical user interface for providing feedback during interaction with an intensity sensitive button
EP2977850A1 (en) 2016-01-27 Mobile terminal and control method thereof
EP3101587B1 (en) 2019-03-06 Mobile terminal and control method for the mobile terminal
KR102015913B1 (en) 2019-08-29 Mobile terminal and control method thereof
US20160132118A1 (en) 2016-05-12 Method and apparatus for controlling haptic feedback of an input tool for a mobile terminal