INFORMATION PROCESSING APPARATUS INSTALLED WITH TOUCH PANEL AS USER INTERFACE

An information processing apparatus includes a detection unit capable of detecting first and second touch positions on a touch panel touched by first and second objects, respectively, a storage unit that stores the first and second touch positions and holds a final touch position as the touch position after each touch is released, a calculation unit that calculates a position obtained by a predetermined rule from the first and second touch positions stored by the storage unit, and a determination unit that determines whether an operation performed on the touch panel is an operation of moving a display content displayed on the touch panel, or an operation of rotating or changing a size of a display content displayed on the touch panel, based on whether the position calculated by the calculation unit is moved, a speed of movement, or an amount of movement.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

First Embodiment

FIG. 1is a diagram showing an example of an external configuration of an image processing apparatus1in a first embodiment of the present invention.

Image processing apparatus1is configured with an MFP (Multi-Function Peripheral) and has various functions including scan, print, copy, fax, network, and email transmission/reception functions. Image processing apparatus1executes a job designated by a user. Image processing apparatus1has a scanner2at the top of the apparatus, which operates when a scan job is executed. Scanner2is configured to include an image reading unit2afor optically reading a document image and a document conveyance unit2bfor automatically conveying a document sheet by sheet to image reading unit2a. Scanner2reads a document set by a user to generate image data. Image processing apparatus1also has a printer3at the bottom center of the apparatus body, which operates when a print job is executed. Printer3is configured to include an image Riming unit3aand a paper feed conveyance unit3b. Image forming unit3aforms an image, for example, by an electrophotographic technique based on input image data and outputs the image. Paper feed conveyance unit3bconveys a sheet material such as print paper sheet by sheet to image forming unit3a. Printer3outputs print based on image data designated by a user.

On the front side of image processing apparatus1, an operation panel4is provided, which functions as a user interface when a user uses image processing apparatus1. Operation panel4is configured to include a display unit5for displaying a variety of information to the user and an operation unit6for the user to perform operation input. Display unit5is configured with, for example, a color liquid crystal display having a predetermined screen size and can display various images. Operation unit6is configured to include a touch sensor (touch panel)6aarranged on the screen of display unit5and a plurality of push button-type operation keys6barranged around the screen of display unit5. The user performs various input operations to operation unit6while looking at a display screen displayed on display unit5and thereby performs a setting operation on image processing apparatus1for executing a job or instructing image processing apparatus1to execute a job.

Touch sensor6aarranged on the screen of display unit5can detect not only a single touch operation by the user but also a multi-touch operation. The single touch operation refers to an operation of touching one point on a display screen of display unit5and includes, for example, single-tap, double-tap, scroll, and drag operations. The multi-touch operation refers to an operation of touching a plurality of points simultaneously on a display screen of display unit5and includes, for example, pinch operations including pinch-in, pinch-out, and rotate. When at least one point on a display screen of display unit5is touched, touch sensor6acan specify the touch position and thereafter can detect a release from the touch state and a movement of the touch position. The user thus can make a job setting, for example, by performing various gesture operations on a display screen of display unit5.

Operation keys6barranged around the screen of display unit5are configured, for example, with a ten-key pad with numbers 0 to 9. Operation keys6bmerely detect a push operation by the user.

FIG. 2is a block diagram showing an example of a hardware configuration of image processing apparatus1.

Image processing apparatus1includes scanner2, printer3, and operation panel4as described above as well as a control unit10, a fax unit20, a network interface21, a wireless interface22, and a storage device23as shown inFIG. 2. Those units of image processing apparatus1can input/output data from/to each other through a data bus19.

Control unit10centrally controls operation panel4, scanner2, printer3, FAX unit20, network interface21, wireless interface22, and storage device23shown inFIG. 2. FAX unit20transmits/receives FAX data through a not-shown public telephone circuit. Network interface21is an interface for connecting image processing apparatus1to a network such as a LAN (Local Area Network). Wireless interface22is an interface for wirelessly communicating with an external device, for example, by NFC (Near Field Communication). Storage device23is nonvolatile storage means configured with, for example, a hard disk drive (HDD) or a solid state drive (SSD). Storage device23can temporarily store image data received through a network and image data generated by scanner2.

As shown inFIG. 2, control unit10is configured to include a CPU11, a ROM12, an SRAM14, an NVRAM15, and an RTC17. CPU11reads out a program13stored in ROM12for execution in response to power-on of image processing apparatus1. Control unit10then starts a control operation for each unit as described above. In particular, CPU11is a main unit that controls operation in image processing apparatus1. CPU11not only controls a job execution operation but also controls the operation of operation panel4functioning as a user interface. Specifically, CPU11performs control of changing display screens appearing on display unit5of operation panel4and, in addition, when a user's input operation is detected by touch sensor6aand operation keys6b, specifies what operation event is the input operation, and executes control corresponding to the specified operation event. The operation event is an event produced by a user's input operation. For input operations to touch sensor6a, there are a plurality of operation events, for example, including single-tap, double-tap, long-tap, scroll, drag, and pinch. The control corresponding to the operation events includes, for example, control of switching display screens, control of starting execution of a job, and control of stopping execution of a job. The operation of CPU11as described above will be described in detail later.

SRAM14is a memory that provides a working storage area for CPU11. SRAM14stores, for example, temporary data produced by execution of program13by CPU11.

NVRAM15is a battery backed-up nonvolatile memory and stores setting values and information in image processing apparatus1. Screen information16is stored in advance in NVRAM15as shown inFIG. 2. Screen information16is configured with information related to a plurality of display screens to be displayed on display unit5of operation panel4. Screen information16of each display screen includes a variety of images such as icon images and button images allowing the user to perform a tap operation. That is, a screen configuration that allows the user to perform gesture operations is defined in screen information16. A plurality of display screens to be displayed on display unit5have respective different screen configurations. Accordingly, the operation events that can be accepted when the user performs a gesture operation on touch sensor6avary.

RTC17is a real time clock that is a clock circuit keep counting time.

FIG. 3is a diagram showing a conceptual configuration of program13executed by CPU11.

Program13is configured to include a main program13aand a plurality of operation event determination routines13b,13c,13d, and13eprepared as subroutines of main program13a. Main program13ais automatically read out and activated by CPU11at power-on of image processing apparatus1. A plurality of operation event determination routines13bto13eare subroutines for specifying whether an input operation (gesture operation) by the user is single-tap, double-tap, or long-tap, or any one of scroll (flick), drag, pinch, and rotate when touch sensor6adetects the input operation. Operation event determination routines13bto13eare prepared as individual subroutines because the specific content and procedure of a specific determination process varies among operation events to be specified. In the present embodiment, when touch sensor6adetects an input operation by the user, CPU11activates only a necessary operation event determination routine from among a plurality of operation event determination routines13bto13e. An operation event corresponding to the input operation is thus specified efficiently. Specific process contents of CPU11will be described below.

FIG. 4is a diagram showing an example of functional blocks implemented by CPU11activating main program13a.

As shown inFIG. 4, CPU11executes main program13athereby to function as a setting unit31, a display control unit32, an operation event determination unit33, a control execution unit34, and a job execution unit35.

Setting unit31is a processing unit that sets an operation event to be detected based on a user's input operation, from among a plurality of operation events, in association with each display screen to be displayed on display unit5. That is, setting unit31specifies an operation event acceptable in each display screen by reading out and analyzing screen information16stored in NVRAM15. Setting unit31then associates the specified operation event with each display screen in advance. For example, setting unit31sets an operation event in association with each display screen by adding information related to the specified operation event to screen information16of each display screen. Setting unit31associates at least one of a plurality of operation events including single-tap, double-tap, long-tap, scroll, drag, and pitch with one display screen. For example, in a case of a display screen that can accept all the operation events, setting unit31associates all of the operation events.

The information that associates operation events may be added in advance at a timing when screen information16is stored into NVRAM15at a time of shipment of image processing apparatus1. Screen information16stored in NVRAM15may be updated even after the shipment of image processing apparatus1, for example, due to addition of an optional function, installation of a new application program, and customization of a display screen. When screen information16is updated, a screen configuration of each display screen is changed. When screen information16is updated, an operation event that cannot be accepted before then may become acceptable after updating of screen information16. Setting unit31therefore functions at the beginning in conjunction with activation of main program13aby CPU11. Setting unit31sets an operation event to be detected based on a user's input operation from among a plurality of operation events in association with each display screen while a startup process of image processing apparatus1is being performed.

Display control unit32reads out screen information16stored in NVRAM15and selects one display screen from among a plurality of display screens for output to display unit5, thereby to display the selected display screen on display unit5. Upon completion of the startup process of image processing apparatus1, display control unit32selects an initial screen from among a plurality of display screens and displays the initial screen on display unit5. Display control unit32thereafter successively updates display screens on display unit5based on a screen update instruction from control execution unit34.

Operation event determination unit33is a processing unit that specifies an operation event corresponding to an input operation when touch sensor6aof operation panel4detects the input operation by the user on a display screen. Operation event determination unit33is one of functions implemented by main program13a. Operation event determination unit33specifies an operation event associated in advance with a display screen currently appearing on display unit5at a timing when a user's input operation is detected by touch sensor6a. Operation event determination unit33specifies an operation event corresponding to the user's input operation by activating only the operation event determination routine that corresponds to the specified operation event. That is, when a user's input operation on a display screen is detected, only the operation event determination routine that corresponds to the operation event associated with the display screen by setting unit31is activated from among a plurality of operation event determination routines13bto13e, in order to determine only the operation event that can be accepted in the display screen. Here, a plurality of operation events may be associated with a display screen. This is the case, for example, where a display screen appearing on display unit5can accept three operation events, namely, single-tap, double-tap, and scroll. In such a case, operation event determination unit33successively activates the operation event determination routines corresponding to those operation events, thereby specifying the operation event corresponding to the user's input operation. In this manner, when some input operation is performed by the user on touch sensor6a, operation event determination unit33activates only the operation event determination routine that corresponds to the operation event acceptable by the display screen appearing on display unit5at that timing, rather than activating all the operation event determination routines13bto13eevery time. Accordingly, the operation event corresponding to the user's input operation can be specified efficiently without activating unnecessary determination routines.

When operation event determination unit33can specify an operation event corresponding to the user's input operation by activating only the necessary operation event determination routine, the specified operation event is output to control execution unit34. Even when only the necessary operation event determination routine is activated as described above, an operation event corresponding to the user's input operation cannot be specified in some cases. For example, it is assumed that the user performs an operation such as long-tap on a display screen that can accept three operation events, namely, single-tap, double-tap, and scroll. In this case, an operation event corresponding to the user's input operation cannot be specified even by activating operation event determination routines13b,13c, and13ecorresponding to three operation events of single-tap, double-tap, and scroll, respectively. In this case, operation event determination unit33does not perform an output process to control execution unit34.

Control execution unit34is a processing unit that executes control based on an operation performed by the user on operation panel4. When the user performs a gesture operation on touch sensor6a, control execution unit34inputs the operation event specified by operation event determination unit33as described above and executes control based on that operation event. By contrast, when the user performs an operation on operation key6b, control execution unit34receives an operation signal directly from that operation key6b, specifies the operation (operation event) performed by the user based on the operation signal, and executes control based on the specified operation. Examples of the control executed by control execution unit34based on the user's input operation include control of updating a display screen appearing on display unit5and control of starting or stopping execution of a job. Accordingly, control execution unit34is configured to control display control unit32and job execution unit35as shown inFIG. 4. Specifically, when a display screen is to be updated based on the input operation by the user, control execution unit34instructs display control unit32to update the screen. When execution of a job is to be started or stopped, control execution unit34instructs job execution unit35to start or stop execution of a job. Accordingly, display control unit32updates the display screen appearing on display unit5based on an instruction from control execution unit34. Job execution unit35starts execution of a job or stops a job already being executed, based on an instruction from control execution unit34. The control executed by control execution unit34may include control other than those described above.

Job execution unit35controls execution of a job specified by the user by controlling the operation of each unit in image processing apparatus1. Job execution unit35is resident in CPU11to centrally control the operation of each unit while a job is being executed in image processing apparatus1.

Specific process procedures performed in CPU11having the functional configuration as described above will now be described.

FIG. 5is a flowchart showing an example of a process procedure performed by CPU11of image processing apparatus1.

This process is started when image processing apparatus1is powered on and CPU11activates main program13aincluded in program13.

First, CPU11activates main program13a, then reads out screen information16(step S1), and associates an operation event with each display screen based on screen information16(step S2). When the association of all the operation events with each display screen is completed, CPU11displays an initial screen on display unit5of operation panel4(step S3). When a display screen appears on display unit5in this manner, CPU11sets an operation event determination routine corresponding to the operation event associated with the display screen (step S4). This brings about a state in which an operation event determination routine that corresponds to an operation event acceptable by the display screen currently appearing on display unit5is prepared.

CPU11enters the standby state until an input operation is detected by one of touch sensor6aand operation key6b(step S5). When an input operation by the user is detected (YES in step S5), CPU11determines whether the input operation is the one detected by touch sensor6a(step S6). If the input operation is the one detected by touch sensor6a(YES in step S6), CPU11executes a loop process for specifying an operation event corresponding to the user's input operation by successively activating the operation event determination routines preset in step S4(steps S7, S8, S9). In this loop process (steps S7, S8, S9), all of operation event determination routines13bto13eincluded in program13are not activated in order. In this loop process (steps S7, S8, S9), only the operation event determination routine set in step S4that corresponds to the operation event acceptable in the display screen currently appearing is activated. In a case where a plurality of operation event determination routines are successively activated in the loop process, the loop process is terminated at a timing when an operation event corresponding to the user's input operation is specified in any one of the operation event determination routines. In other words, in this loop process (steps S7, S8, S9), not all of the operation event determination routines set in step S4are always activated. In this loop process (steps S7, S8, S9), if an operation event corresponding to the user's input operation can be specified halfway before all are activated, the loop process is terminated without activating the operation event determination routines that are to be activated subsequently.

When the loop process (steps S7, S8, S9) is terminated, CPU11determines whether an operation event can be specified through the loop process (steps S7, S8, S9) (step S10). The determination in step S10is required because the user may perform a gesture operation that is not acceptable on the display screen currently appearing. If an operation event corresponding to the user's input operation cannot be specified (NO in step S10), CPU11returns to the standby state (step S5) without proceeding to the subsequent process (step S11) until an input operation by the user is detected again. By contrast, if an operation event corresponding to the user's input operation can be specified in the loop process (steps S7, S8, S9) (YES in step S10), the process by CPU11proceeds to the next step S11.

If an input operation by the user is detected (YES in step S5) and the input operation is the one detected by operation key6b(NO in step S6), the process by CPU11also proceeds to step S11. That is, when the user operates operation key6b, the operation event can be specified by the operation signal, and, therefore, the process proceeds to the process in the case where an operation event can be specified (step S11).

When an operation event corresponding to the user's input operation is specified, CPU11executes control corresponding to the input operation (step S11). Specifically, as described above, control of updating the display screen on display unit5, job execution control, or any other control is performed. CPU11then determines whether the display screen appearing on display unit5is updated through execution of the control in step S11(step S12). As a result, if it is determined that the display screen is updated (YES in step S12), the process by CPU11returns to step S4. Specifically, CPU11sets an operation event determination routine corresponding to an operation event associated with the updated display screen (step S4). By contrast, if the display screen is not updated (NO in step S12), the process by CPU11returns to step S5. Specifically, CPU11enters the standby state until an input operation by the user is detected again (step S5). CPU11then repeats the process above.

By performing the process as described above, CPU11can perform a process corresponding to the operation performed by the user on operation panel4. In particular, the process as described above may be performed concurrently during execution of a job, and when the user performs a gesture operation on the display screen, the required minimum number of operation event determination routines are activated in order to specify only the operation event that can be accepted on the display screen. Therefore, the operation event corresponding to the user's gesture operation can be specified efficiently without activating unnecessary operation event determination routines in execution of a job.

FIG. 6is a diagram showing an example of a preview image display screen G15 that previews an image.

Preview image display screen G15 is displayed on display unit5of operation panel4. Preview image display screen G15 has a screen configuration including a preview area R3 for previewing an image selected by the user. The operations that can be performed by the user on preview image display screen G15 include a pinch operation for reducing or enlarging a preview image and a rotate operation for rotating a preview image. The pinch operation includes a pinch-in operation for reducing a preview image and a pinch-out operation for enlarging a preview image. The pinch-in operation is an operation of moving two points of a preview image displayed in preview area R3 so as to reduce the distance therebetween with two fingers touching the two points, as shown by an arrow F5 inFIG. 6(a). This pinch-in operation allows the preview image displayed in preview area R3 to be displayed in a reduced size. The pinch-out operation is an operation of moving two points of a preview image displayed in preview area R3 so as to increase the distance therebetween with two fingers touching the two points, as shown by an arrow F6 inFIG. 6(b). This pinch-out operation allows the preview image displayed in preview area R3 to be displayed in an enlarged size. The rotate operation is an operation of moving two points of a preview image displayed in preview area R3 so as to rotate the position between the two points with two fingers touching the two points, as shown by an arrow F7 inFIG. 6(c). This rotation operation allows a preview image displayed in preview area R3 to be displayed in a rotated state.

In preview image display screen G15, not only when a pinch-out operation is performed but also when a double-tap operation is performed on a point in a preview image displayed in preview area R3, a process of displaying the preview image in an enlarged size is performed with the point at the center. In preview image display screen G15, when a preview image is displayed in an enlarged size and the entire image cannot be displayed in preview area R3, a drag operation can be accepted. In preview image display screen G15, when a drag operation is performed, the enlarged display portion is moved and displayed. In preview image display screen G15, a scroll (flick) operation for switching the displayed image to the next (or previous) image can be accepted.

In this manner, preview image display screen G15 shown inFIG. 6has a screen configuration that can accept four operation events, namely, scroll (flick), drag, double-tap, and pinch, and does not accept the other operation events. Accordingly, setting unit31sets four operation events of scroll (flick), drag, double-tap, and pinch in association with preview image display screen G15 shown inFIG. 6.

FIG. 7is a diagram showing the relationship between display screens and operation events acceptable in each display screen.

InFIG. 7, an operation event acceptable in each display screen is denoted by “YES”, and an operation event not acceptable is hatched. As shown inFIG. 7, there are various kinds of display screens to be displayed on display unit5of operation panel4, and acceptable operation events vary among display screens. Then, as described above, setting unit31specifies an acceptable operation event and sets an operation event to be detected based on a user's input operation in association with each display screen. That is, the operation events associated with each display screen by setting unit31are the same as shown inFIG. 7.

InFIG. 7, a drag operation is conditionally acceptable in a preview image. That is, in this display screen, a drag operation is not an operation event that is always acceptable but is acceptable when a particular condition is met. For example, as shown inFIG. 6(h) above, when a preview image is displayed in an enlarged size in preview area R3 of preview image display screen G15, a drag operation for moving the enlarged display portion is acceptable. However, it is not necessary to move the enlarged display portion when a preview image is not displayed in an enlarged size. In such a state, therefore, a drag operation for moving the enlarged display portion is not acceptable in preview image display screen G15.

FIG. 8is a diagram for explaining a touch position on the touch panel (touch sensor6a) that is stored in SRAM14.

Coordinates T1 (X1, Y1) of a touch position by a first object (for example, the fingertip of a thumb) and coordinates T2 (X2, Y2) of a touch position by a second object (for example, the fingertip of an index finger) on the touch panel (touch sensor6a) are detected every sampling period (or real-time) and recorded in SRAM14. Before touching, initial coordinate values (A, A) are stored for T1 (X1, Y1) and T2 (X2, Y2).

When the first and second objects are moved on the touch panel while being touched, coordinates T1 (X1, Y1) and coordinates T2 (X2, Y2) are changed every sampling period (or real-time).

After the touch by the first object is released (after the first object is lifted from the touch panel), the coordinates of the final touch position by the first object is held as T1 (X1, Y1). Similarly, after the touch by the second object is released (after the second object is lifted from the touch panel), the coordinates of the final touch position by the second object is held as T2 (X2, Y2).

CPU11calculates a position (coordinates) I obtained by a predetermined rule from coordinates T1 (X1, Y1) and coordinates T2 (X2, Y2). Here, a predetermined rule is to obtain a midpoint between coordinates T1 (X1, Y1) and coordinates T2 (X2, Y2). That is, coordinates I are calculated by ((X1+X2)/2, (Y1+Y2)/2).

The predetermined rule is a rule for obtaining a position from coordinates T1 (X1, Y1) and coordinates T2 (X2, Y2), and coordinates I may be obtained not by the midpoint but by the following expression:

coordinatesI=((X1+X2)×a,(Y1+Y2)×a) (where a is any given number that is not zero (weight coefficient).  (b)

Coordinates I represent a point having the following features. That is, coordinates I represent a point that is moved when a scroll operation or a drag operation is being performed. Otherwise, coordinates I represent a point where when a scroll operation or a drag operation is being performed, the speed of the movement or the amount of the movement within a predetermined time is equal to or greater than a threshold value. On the other hand, when a pinch-in operation, a pinch-out operation, or a rotate operation is being performed, coordinates I do not move theoretically (considering an error, when a pinch-in operation, a pinch-out operation, or a rotate operation is being performed, the speed of the movement of coordinates I or the amount of the movement within a predetermined time is smaller than a threshold value). InFIG. 8, the threshold value is represented by “r”. If the velocity vector of the movement of coordinates I or the amount of the movement within a predetermined time falls within the dotted circle, it can be determined that a pinch-in operation, a pinch-out operation, or a rotate operation is performed. If the velocity vector of the movement of coordinates I or the amount of the movement within a predetermined time falls on the dotted circle or out of the dotted circle, it can be determined that a scroll operation or a drag operation is performed.

Using these features of coordinates I, the information processing apparatus1in the present embodiment determines whether the operation by the user is a scroll operation or a drag operation, otherwise a pinch-in operation, a pinch-out operation, or a rotate operation, based on the movement of coordinates I.

According to the present embodiment, after the touch by the first object is released, the coordinates of the final touch position by the first object is held as T1 (X1, Y1). After the touch by the second object is released, the coordinates of the final touch position by the second object is held as T2 (X2, Y2). Accordingly, coordinates I can be calculated even in a state in which a touch is made with one finger. Therefore, it can be determined that a scroll operation or a drag operation is performed based on a state of the movement of coordinates I.

FIG. 9is a flowchart showing a process executed by CPU11of the information processing apparatus in the first embodiment.

This process is implemented by CPU11executing the program of operation event determination routine13e(determination for scroll, drag, pinch, and rotate) inFIG. 3. The process in the flowchart inFIG. 9is repeatedly executed at predetermined time intervals (for example, every 20 milliseconds). The predetermined time interval is the sampling period for touch coordinates and the calculation period for coordinates I.

Referring toFIG. 9, in step S101, it is determined whether the touch/release state of the touch panel is changed. Here, the determination is YES if

(A) a state in which no touch is made changes to a state in which one or more points are touched;

(B) a state in which one or more points are touched changes to a state in which no touch is made; or

(C) the number of touched points is changed.

If YES in step S101, the process in the present period is terminated. If NO in step S101, in step S103, the touch coordinates (position) on the touch panel are detected. When a plurality of points are touched, all of the touch coordinates are detected. The touch coordinates are stored into SRAM14. As described with reference toFIG. 8, after the touch is released, the final touch coordinates are held.

In step S105, it is determined whether there is any change in touch coordinates from the previous period. This is to determine whether any one of the touch positions is moved.

If NO in step S105, the process in the present period is terminated. If YES in step S105, in step S107, coordinates I (for example, the midpoint) are calculated.

In step S109, it is determined whether the moving speed of coordinates I is equal to or greater than a threshold value. In step S109, it may be determined whether coordinates I are moved, or whether the amount of the movement of coordinates I within a predetermined time (for example, from the previous sampling period to the present time) is equal to or greater than a threshold value.

If YES in step S109, in step S111, it is determined that the operation by the user is a scroll operation or a drag operation, and a screen imaging process in accordance with a scroll operation or a drag operation is performed. The determination as to whether the operation is a scroll operation or a drag operation can be made, for example, based on the display content of the screen, the display content at the touch position, and the time interval from when a touch is made to when the touch position is moved.

If NO in step S109, in step S113, it is determined that the operation by the user is a pinch-in operation, a pinch-out operation, or a rotate operation, and a screen image process in accordance with a pinch-in operation, a pinch-out operation, or a rotate operation is performed. The determination as to whether the operation is a rotate operation, a pinch-in operation, or a pinch-out operation is made based on the direction in which the touch position is moved. Specifically, if the touch positions at two points are rotated in a predetermined direction about the midpoint, it is determined that the operation is a rotate operation. If the touch positions at two points are moved in a direction toward the midpoint, it is determined that the operation is a pinch-in operation. If the touch positions at two points are moved in a direction away from the midpoint, it is determined that the operation is a pinch-out operation.

The effects of the present embodiment will now be described.

FIG. 10is a flowchart showing a process in a conventional technique (FIG. 24) when the touch/release state is changed.

As described with reference toFIG. 24, when the touch/release state is changed, a YES determination is made in step S201, and the process from step S207is executed. Therefore, substantially, the number of touch points is acquired (S207), and it is determined whether the number of touch points is one or two (S209), as illustrated inFIG. 10. After that, the touch coordinates are detected (S211, S215), and an image process in accordance with the number of touch points is performed (S213, S219). For a pinch operation and a rotate operation, a process of calculating the midpoint between the touch positions at two points (S217) is performed.

FIG. 11is a flowchart showing a process in the first embodiment (FIG. 9) when the touch/release state is changed.

As described with reference toFIG. 9, when the touch/release state is changed, a YES determination is made in step S101, and the process ends. It is therefore unnecessary to perform a substantial process as shown inFIG. 11. As described above, the present embodiment can significantly reduce the processing when the touch/release state is changed.

FIG. 12is a flowchart showing a process in a conventional technique (FIG. 24) when the touch/release state is not changed.

As described with reference toFIG. 24, when there is no change in the touch/release state, a NO determination is made in step S201, and the process from step S203is executed. Therefore, substantially, the touch coordinates are detected (S203), and the number of touch points is acquired (S207) if the coordinates are changed, as illustrated inFIG. 12. It is determined whether the number of touch points is one or two (S209), followed by detection of the touch coordinates (S211, S215) and an imaging process in accordance with the number of touch points (S213, S219). For a pinch operation and a rotate operation, a process of calculating the midpoint between the touch positions at two points is performed (S217).

FIG. 13is a flowchart showing a process in the first embodiment (FIG. 9) when the touch/release state is not changed.

As described with reference toFIG. 9, when there is no change in the touch/release state, a NO determination is made in step S101, and the process from step S103is executed. Specifically, the touch coordinates are detected (S103), and the midpoint (coordinates I inFIG. 8) are calculated (S107) if the touch coordinates are changed (YES in S105). Based on a state of the movement of coordinates I (S109), a screen imaging process in accordance with a scroll operation or a drag operation (S111) or a screen imaging process in accordance with a pinch-in operation, a pinch-out operation, or a rotate operation (S113) is performed.

InFIG. 13, the process of acquiring and determining the number of touch points (S207, S209inFIG. 12) can be eliminated. The determination in step S109can be performed using the value of the midpoint (coordinates I) that has to be acquired in a case of a pinch-in operation, a pinch-out operation, or a rotate operation. Accordingly, the present embodiment can significantly reduce the processing in the case where there is no change in the touch/release state.

FIG. 14is a diagram for explaining the relationship between the touch position and the midpoint in a time sequence in the first embodiment.

Referring toFIG. 14, at time t1, no touch is made on the touch panel, and the coordinates (A, A) as initial values are recorded both in coordinates T1 (X1, Y1) (address: 0 in the figure) and coordinates T2 (X2, Y2) (address: 1 in the figure). In the present embodiment, a touch at one point or two points is detected, and, therefore, only address: 0 and address: 1 are used in the figure. In a case where a touch at three or more points is detected, coordinates T3 (X3, Y3) (the touch position at the third point) and the subsequent coordinates are recorded in address: 2 and the subsequent addresses in the figure. At time t1, coordinates ((A+A)/2, (A+A)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

InFIG. 14, in the fields of address: 0 and address: 1, the first letter “0” indicates that a touch at the coordinates is not made, and the first letter “1” indicates that a touch at the coordinates is made.

At time t2, it is assumed that only one point on the touch panel is touched. Here, the coordinates (X1, Y1) at the touch position are recorded in coordinates T1 (address: 0 in the figure). Coordinates T2 (address: 1 in the figure) remain the initial values (A, A). At time t2, coordinates ((X1+A)/2, (Y1+A)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

At time t2, there is a change in touch/release from the previous time. In step S101inFIG. 9, therefore, a YES determination is made, and no substantial process in the flowchart inFIG. 9is performed. Specifically, no process for scroll, drag, pinch, or rotate is performed, and a process for tap not shown in the flowchart is performed. Accordingly, even when the midpoint coordinates are greatly varied due to a change of the initial values (A, A) to the actual touch coordinates (X1, Y1), it is not erroneously determined that the change is caused by a scroll operation or a drag operation.

At time t3, it is assumed that the touched one point is moved. Here, coordinates (X11, Y11) after the movement are recorded in coordinates T1 (address: 0 in the figure). Coordinates T2 (address: 1 in the figure) remain the initial values (A, A). At time t3, coordinates ((X11+A)/2, (Y11+A)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

At time t3, there is no change in touch/release from the previous time. Therefore, a NO determination is made in step S101inFIG. 9, and a process of determining the operation (S109to S113) is performed based on the moving speed of the midpoint between coordinates T1 and coordinates T2. In the determination of the moving speed, for example, it is determined whether coordinates I (midpoint) inFIG. 8move over a distance greater than the threshold value r from the previous detection timing. If YES, an imaging process in accordance with a scroll operation or a drag operation is performed in step S111inFIG. 9. If NO, an imaging process is performed in accordance with a pinch operation or a rotate operation in step S113. InFIG. 14, it is assumed that the moving speed of the midpoint is fast (the midpoint is moved), and an imaging process in accordance with a scroll operation is performed.

The threshold value r inFIG. 8is preferably set to a value greater than the amount of movement of coordinates I that is caused by hand shaking when the user is reducing or increasing the distance between the thumb and the index finger during a pinch operation. Accordingly, even when the midpoint is shaken while the fingers are closed or opened, the shake can be set equal to or smaller than the threshold value. Therefore, even with hand shaking, a pinch operation is not erroneously determined as a scroll operation or a drag operation. The threshold value r is preferably a distance from 5 mm to 20 mm on the touch panel.

At time t4, it is assumed that one point on the touch panel is additionally touched (that is, a state in which, in total, two points are touched). Here, coordinates (X11, Y11) at the touch position are recorded in coordinates T1 (address: 0 in the figure). Coordinates (X2, Y2) at the touch position are recorded in coordinates T2 (address: 1 in the figure). At time t4, coordinates ((X11+X2)/2, (Y11+Y2)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

At time t4, there is a change in touch/release from the previous time.

Therefore, a YES determination is made in step S101inFIG. 9, and no substantial process in the flowchart inFIG. 9is performed. Specifically, no process for scroll, drag, pinch, and rotate is performed, and a process for tap not shown in the flowchart is performed.

At time t5, it is assumed that both of the touched two points are moved. Here, coordinates (X111, Y111) after the movement are recorded in coordinates T1 (address: 0 in the figure). Coordinates (X22, Y22) after the movement are recorded in coordinates T2 (address: 1 in the figure). Coordinates ((X111+X22)/2, (Y111+Y22)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

At time t5, there is no change in touch/release from the previous time. Therefore, a NO determination is made in step S101inFIG. 9, and a process of determining the operation (S109to S113) is performed based on the moving speed of the midpoint between coordinates T1 and coordinates T2. InFIG. 14, it is assumed that the moving speed of the midpoint is slow (or the moving speed is zero), and an imaging process in accordance with pinch-out is performed.

At time t6, it is assumed that the touch at coordinates T1 on the touch panel is released (that is, a state in which, in total, one point is touched). Here, the coordinates (X111, Y111) of the final touch position are held in coordinates T1 (address: 0 in the figure). Coordinates (X22, Y22) at the touch position are recorded in coordinates T2 (address: 1 in the figure). At time t6, coordinates (X111+X22)/2, (Y111+Y22)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

At t6 inFIG. 14, the touch state at address: 0 is released, and, therefore, the first letter in the field is changed to “0”.

At time t6, there is a change in touch/release from the previous time. Therefore, a YES determination is made in step S101inFIG. 9, and no substantial process in the flowchart inFIG. 9is performed. Specifically, no process for scroll, drag, pinch, and rotate is performed, and a process for tap not shown in the flowchart is performed.

At time t7, it is assumed that touch coordinates T2 are moved. Here, coordinates (X111, Y111) of the final touch position are held in coordinates T1 (address: 0 in the figure). Coordinates (X222, Y222) after the movement are recorded in coordinates T2 (address: 1 in the figure). At time t7, coordinates ((X111+X222)/2, (Y111+Y222)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

At time t7, there is no change in touch/release from the previous time. Therefore, a NO determination is made in step S101inFIG. 9, and a process of determining the operation (S109to S113) is performed based on the moving speed of the midpoint between coordinates T1 and coordinates T2. InFIG. 14, it is assumed that the moving speed of the midpoint is fast (the midpoint is moved), and an imaging process in accordance with scroll is performed.

At time t8, it is assumed that a touch at coordinates T1 on the touch panel is made again (that is, a state in which, in total, two points are touched). Here, coordinates (X3, Y3) of the touch position are held in coordinates T1 (address: 0 in the figure). Coordinates (X222, Y222) at the touch position are recorded in coordinates T2 (address: 1 in the figure). At time t8, coordinates ((X3+X222)/2, (Y3+Y222)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

At t8 inFIG. 14, a touch at address: 0 is made, and, therefore, the first letter in the field is changed to “1”.

At time t8, there is a change in touch/release from the previous time. Therefore, a YES determination is made in step S101inFIG. 9, and no substantial process in the flowchart inFIG. 9is performed. Specifically, no process for scroll, drag, pinch, or rotate is performed, and a process for tap not shown in the flowchart is performed.

At time t9, it is assumed that both of the touched two points are moved. Here, coordinates (X33, Y33) after the movement are recorded in coordinates T1 (address: 0 in the figure). Coordinates (X2222, Y2222) after the movement are recorded in coordinates T2 (address: 1 in the figure). At time t9, coordinates ((X33+X2222)/2, (Y33+Y2222)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

At time t9, there is no change in touch/release from the previous time. Therefore, a NO determination is made in step S101inFIG. 9, and a process of determining the operation (S109to S113) is performed based on the moving speed of the midpoint between coordinates T1 and coordinates T2. InFIG. 14, it is assumed that the moving speed of the midpoint is slow (or the moving speed is zero), and an imaging process in accordance with pinch-out is performed.

At time t10, it is assumed that touch coordinates T2 are moved. Here, coordinates (X33, Y33) of the touch position are held in coordinates T1 (address: 0 in the figure). Coordinates (X22222, Y22222) after the movement are recorded in coordinates T2 (address: 1 in the figure). At time t10, coordinates ((X33+X22222)/2, (Y33+Y22222)/2) are recorded as the midpoint between coordinates T1 and coordinates T2.

At time t10, there is no change in touch/release from the previous time. Therefore, a NO determination is made in step S101inFIG. 9, and a process of determining the operation (S109to S113) is performed based on the moving speed of the midpoint between coordinates T1 and coordinates T2. InFIG. 14, it is assumed that the moving speed of the midpoint is fast (the midpoint is moved), and an imaging process in accordance with scroll is performed.

As described above, in the first embodiment, a midpoint is obtained from the touch positions, and the operation by the user is determined based on a state of movement. An imaging process is performed based on the determination result.

Second Embodiment

FIG. 15is a flowchart showing a process executed by CPU11of the information processing apparatus in a second embodiment.

The information processing apparatus in the second embodiment executes a process illustrated in the flowchart inFIG. 15in place of the process in the flowchart inFIG. 9. The information processing apparatus in the second embodiment records the touch positions at the third and subsequent points in the field of “address2” and the subsequent fields inFIG. 14, and calculates the barycenter position of a plurality of touch positions in place of the midpoint. The user's operation is determined based on a movement of the barycenter position.

The process in the flowchart inFIG. 15is repeatedly performed at predetermined time intervals (for example, every 20 milliseconds).

The process in steps S301to S305inFIG. 15is the same as the process in steps S101to S105inFIG. 9, and a description thereof is not repeated here.

If YES in step S305, in step S307, the barycenter position of a plurality of touch positions is calculated as coordinates I.

In step S309, it is determined whether the moving speed of coordinates I is equal to or greater than a threshold value. In step S309, it may be determined whether coordinates I are moved, or whether the amount of the movement within a predetermined time is equal to or greater than a threshold value.

If YES in step S309, in step S311, it is determined that the operation by the user is a scroll operation or a drag operation, and a screen imaging process in accordance with a scroll operation or a drag operation is performed. The determination as to whether the operation is a scroll operation or a drag operation is made, for example, based on the display content of the screen, the display content at the touch position, and the time interval from when a touch is made to when the touch position is moved.

If NO in step S309, in step S313, it is determined that the operation by the user is a pinch-in operation, a pinch-out operation, or a rotate operation, and a screen imaging process in accordance with a pinch-in operation, a pinch-out operation, or a rotate operation is performed. Whether the operation is a pinch-in operation, a pinch-out operation, or a rotate operation is determined based on the direction in which the touch position is moved. Specifically, when the touch positions at two or more points are rotated in a predetermined direction about the midpoint, it is determined that the operation is a rotate operation. If the touch positions at two or more points are moved in a direction toward the midpoint, it is determined that the operation is a pinch-in operation. If the touch positions at two or more points are moved in a direction away from the midpoint, it is determined that the operation is a pinch-out operation.

The second embodiment has the effect of significantly reducing the processing irrespective of whether the touch/release state is changed or not, in the same manner as in the first embodiment.

Third Embodiment

FIG. 16is a flowchart showing a process executed by CPU11of the information processing apparatus in a third embodiment.

Referring toFIG. 16, in step S401, it is determined whether a preview is being displayed on the touch panel. A preview is a reduced image of at least one page from among images (scanned images, externally received images) of a plurality of pages stored in storage device23.

If NO in step S401, the process here ends. If YES, the process from step S403is executed. In step S403, a subroutine of detecting a user's gesture operation is executed. The process in this subroutine is the same as the process in steps S101to S107inFIG. 9or in steps S301to S307inFIG. 15.

In step S405, it is determined whether the operation made by the user is a scroll operation by determining whether the moving speed of the midpoint or barycenter is equal to or greater than a threshold value. If YES, in step S407, an image of another page (a previous page or a next image in accordance with the direction of the scroll operation) is displayed on the touch panel.

FIG. 17is a diagram showing a specific example of a display content on the touch panel of the information processing apparatus in the third embodiment.

in a case where an image of the Dn-th page is previewed at the center of the screen, when the user touches the screen to move the touch position to the left, an image of the next page (D(n+1)th page) that has been grayed out is moved to the center of the screen, and the image of the D(n+1)th page is to be previewed. In a case where an image of the Dn-th page is previewed at the center of the screen, when the user touches the screen to move the touch position to the right, an image of the previous page (D(n−1)th page) that has been grayed out is moved to the center of the screen, and the image of the D(n−1)th page is to be previewed.

Fourth Embodiment

FIG. 18is a flowchart showing a process executed by CPU11of the information processing apparatus in a fourth embodiment.

The information processing apparatus in the fourth embodiment executes a process illustrated in the flowchart inFIG. 18in place of the process in the flowchart inFIG. 9.

The process in the flowchart inFIG. 18is repeatedly executed at predetermined time intervals (for example, every 20 milliseconds).

The process in steps S501to S511and S515inFIG. 18is the same as the process in steps S101to S111and S113inFIG. 9, and a description thereof is not repeated here.

InFIG. 18, if NO in step S509, in step S513, it is determined whether both of the touch positions at two points are moved. If YES in step S513, the process proceeds to step S515. If NO, the process proceeds to step S511.

In the fourth embodiment, the process for a pinch-in operation, a pinch-out operation, or a rotate operation is performed only when both of touch positions at two points are moved. This has the effect of preventing an erroneous process against the user's intention.

Fifth Embodiment

In the forgoing first to fourth embodiments, a fixed threshold value is used to determine the user's operation based on a movement of the center (or barycenter). In a fifth embodiment, however, the threshold value is varied according to situations.

FIG. 19is a flowchart showing a process executed by CPU11of the information processing apparatus in the fifth embodiment.

The flowchart inFIG. 19illustrates a process of changing the threshold value. The process shown inFIG. 19can be executed concurrently with the process in the flowchart illustrated in the first to fourth embodiments.

In step S601, when there is a change in touch position, it is determined whether only a touch position at one point is changed or both of touch positions at two points are changed. If only a touch position at one point is changed, in step S603, the threshold value is reduced, for example, to 12 dots. If both of touch positions at two points are changed, in step S605, the threshold value is increased, for example, to 50 dots.

When only a touch position at one point is changed, there is a high possibility that the user's operation is a scroll operation or a drag operation. In step S603, therefore, the threshold value is reduced to facilitate a determination that the operation is a scroll operation or a drag operation. On the other hand, when both of touch positions at two points are changed, there is a high possibility that the user's operation is a pinch-in operation, a pinch-out operation, or a rotate operation. In step S605, therefore, the threshold value is increased to facilitate a determination that the operation is a pinch-in operation, a pinch-out operation, or a rotate operation.

Sixth Embodiment

FIG. 20is a flowchart showing a process executed by CPU11of the information processing apparatus in a sixth embodiment.

The information processing apparatus in the sixth embodiment executes a process illustrated in the flowchart inFIG. 20in place of the process in the flowchart inFIG. 9.

The process in the flowchart inFIG. 20is repeatedly executed at predetermined time intervals (for example, every 20 milliseconds).

The process in steps S701to S707inFIG. 20is the same as the process in steps S101to S107inFIG. 9, and a description thereof is not repeated here.

After the process in step S707, in step S709, it is determined whether the previous determination result of the user's operation is a pinch operation or a rotate operation. If YES, in step S711, a first value is set for the threshold value. If NO, in step S713, a second value is set for the threshold value. Here, the relationship of the first value>the second value holds. The process from step S715is thereafter performed. The process in steps S715to S719inFIG. 20is the same as the process in steps S109to S113inFIG. 9, and a description thereof is not repeated here.

When the previous determination result of the user's operation is a pinch operation or a rotate operation, there is a high possibility that the user's operation at the next detection timing is also a pinch operation or a rotate operation. In step S711, therefore, the threshold value is increased to facilitate a determination that the operation is a pinch operation or a rotate operation. On the other hand, if the previous determination result of the user's operation is a scroll operation or a drag operation, there is a high possibility that the user's operation at the next detection timing is also a scroll operation or a drag operation. In step S713, therefore, the threshold value is reduced to facilitate a determination that the operation is a scroll operation or a drag operation.

Seventh Embodiment

FIG. 21is a flowchart showing a process executed by CPU11of the information processing apparatus in a seventh embodiment.

The information processing apparatus in the seventh embodiment executes a process illustrated in the flowchart inFIG. 21in place of the process in the flowchart inFIG. 9.

The process in the flowchart inFIG. 21is repeatedly executed at predetermined time intervals (for example, every 20 milliseconds).

The process in steps S801to S811inFIG. 21is the same as the process in steps S101to S111inFIG. 9, and a description thereof is not repeated here.

If NO in step S809, in step S813, it is determined whether the previous determination result of the user's operation is a pinch operation. If NO, assuming that a pinch operation is started, and, in step S815, “0” is recorded as “the amount of movement of the touch position from the start of pinch operation”. In step S817, then, an initial value of the threshold value is set. The threshold value set here may be the same as the threshold value previously used in step S809or may be greater. If a greater threshold value is set, a NO determination is facilitated in the determination in step S809in the next period. Specifically, if a NO determination is once made in step S809(if it is determined that the operation is pinch), a determination that the operation is a pinch operation is facilitated in the determination in the next period.

In step S819, an imaging process in accordance with a pinch operation is performed. Here, the determination of a rotate process is omitted.

If a YES determination is made in step S813, in step S821, the amount of movement from the previous touch position is added to the “amount of movement of the touch position from the start of pinch operation”. In step S823, a threshold value is set based on the value of the “amount of movement of the touch position from the start of pinch operation”. Here, the greater is the “amount of movement of the touch position from the start of pinch operation”, the larger threshold value is set.

When the previous determination result of the user's operation is a pinch operation, there is a high possibility that the user's operation at the next detection timing is also a pinch operation. In step S823, therefore, the threshold value is increased to facilitate a determination that the operation is a pinch operation, also in the next determination. Here, as the pinch operation continues, the threshold value is increased.

Eighth Embodiment

FIG. 22is a flowchart showing a process executed by CPU11of the information processing apparatus in an eighth embodiment.

The information processing apparatus in the eighth embodiment executes a process illustrated in the flowchart inFIG. 22in place of the process in steps S813to S823in the flowchart inFIG. 21.

Specifically, if NO in step S809(FIG. 21), in step S901(FIG. 22), it is determined whether the previous determination result of the user's operation is a rotate operation. If NO, assuming that a rotate operation is started, in step S903, the angle at the start of rotation operation (the angle formed by a straight line between touch positions at two points at the start of rotate operation) is recorded. In step S905, then, an initial value of the threshold value is set. The threshold value set here may be the same as the threshold value previously used in step S809or may be greater. If a greater threshold value is set, a NO determination is facilitated in the determination in step S809in the next period. That is, if a NO determination is once made in step S809(if it is determined that the operation is rotate), a determination that the operation is a rotate operation is facilitated also in the determination in the next period.

In step S907, an imaging process in accordance with a rotate operation is performed. Here, the determination of a pinch process is omitted.

If a YES determination is made in step S901, in step S909, the angle formed by a straight line between the touch positions at two points at present is compared with the angle at the start of rotate operation that is recorded in step S903. In step S911, it is determined whether the result of comparison is equal to or greater than a predetermined angle (for example, 30°). If YES, in step S913, the threshold value is set to a value smaller than the initial value, and the process proceeds to step S907. If NO, the process proceeds to step S907.

There is a high possibility that a rotate operation ends approximately at 30°. Therefore, if the rotation from the initial angle is 30° or greater in step S911, in step S913, the threshold value is reduced. This facilitates a determination that the operation is a scroll operation or a drag operation, in the next determination.

Ninth Embodiment

FIG. 23is a flowchart showing a process executed by CPU11of the information processing apparatus in a ninth embodiment.

The information processing apparatus in the ninth embodiment executes a process illustrated in the flowchart inFIG. 23in place of the process in the flowchart inFIG. 9.

The process in the flowchart inFIG. 23is repeatedly executed at predetermined time intervals (for example, every 20 milliseconds).

The process in steps S1001to S1009inFIG. 23is the same as the process in steps S101to S109inFIG. 9, and a description thereof is not repeated here.

If YES in step S1009, in step S1011, it is determined whether the previous determination result of the user's operation is a scroll operation. If NO, assuming that a scroll operation is started, in step S1015, an initial value is set as a threshold value. The threshold value set here may be the same as the threshold value previously used in step S1009or may be smaller. If a smaller threshold value is set, a YES determination is facilitated in the determination in step S1009in the next period. That is, if a YES determination is once made in step S1009(if it is determined that the operation is a scroll operation), a determination that the operation is a scroll operation is facilitated also in the determination in the next period.

If YES in step S1011, in step S1013, the threshold value is changed to a smaller value. If a smaller threshold value is set, a YES determination is facilitated in the determination in step S1009in the next period. In step S1017, an imaging process in accordance with a scroll operation is performed. Here, the determination of a drag process is omitted.

If NO in step S1009, in step S1019, it is determined whether the previous determination result of the user's operation is a pinch operation. If NO, assuming that a pinch operation is started, in step S1021, an initial value is set as a threshold value. Here, the threshold value set here may be the same as the threshold value previously used in step S1009or may be greater. If a greater threshold value is set, a NO determination is facilitated in the determination in step S1009in the next period. That is, if a NO determination is once made in step S1009(if it is determined that the operation is a pinch operation), a determination that the operation is a pinch operation is facilitated also in the determination in the next period.

If YES in step S1019, in step S1023, the threshold value is changed to a greater value. If a greater threshold value is set, a NO determination is facilitated in the determination in step S1009in the next period. In step S1025, an imaging process in accordance with a pinch operation is performed. Here, the determination of a rotate process is omitted.

Effect of Embodiments

According to the embodiments above, in the information processing apparatus installed with a touch panel capable of detecting two or more points, the coordinates of two or more points are always detected irrespective of a touch state or a release state. The coordinates include actual values (the actual touch position at present) and stored values (the final touch position). Based on these coordinates of two or more points, a position (for example, midpoint) obtained by a predetermined rule is calculated. The user's operation is determined based on a variation in the obtained position.

The process in the present embodiment only requires processing in a CPU, for example, shift processing. For example, the midpoint of coordinates that requires less processing time is always detected, so that the user's operation can be determined from the detected midpoint using the characteristic that the midpoint greatly varies during scroll (flick) and the midpoint is hardly moved during pinch. That is, the process of determining a gesture operation can be implemented with a simple process.

According to the foregoing embodiments, even when two or more points on the touch panel are touched, when the touch position is moved quickly and the coordinates of the midpoint (or barycenter) are thereby moved quickly, the process in accordance with a scroll operation or a drag operation is performed. This has the effect of good operability for users.

OTHERS

In the forgoing embodiments, an information processing apparatus installed in an image forming apparatus (or image processing apparatus) has been described by way of example. The present invention, however, is applicable to an information processing apparatus installed as a user interface in smart phones, tablet terminals, PCs (Personal Computers), home appliances, office appliances, and controllers.

The image forming apparatus may be any of a monochrome/color copier, a printer, a facsimile machine, or an MFP (Multi-Functional Peripheral). The image forming apparatus may be the one that forms an image by an electrophotographic technique or the one that forms an image by an ink-jet technique.

The process in the forgoing embodiments may be performed either by software or by a hardware circuit.

A program for executing the process in the foregoing embodiments may be provided. A recording medium, such as a CD-ROM, a flexible-disk, a hard disk, a ROM, a RAM, or a memory card, encoded with the program may be provided to users. The program may be downloaded to the apparatus through a communication circuit such as the Internet. The process described in written form in the flowchart is executed by a CPU in accordance with the program.

The embodiments above provide an information processing apparatus that can make processing easy, a method of controlling the information processing apparatus, and a control program for the information processing apparatus. An information processing apparatus with good operability for users is also provided.