ELECTRONIC DEVICE, METHOD, AND COMPUTER-READABLE STORAGE MEDIUM FOR CHANGING TRAJECTORY OF GESTURE

An example electronic device may include a touch sensor and a processor. Based on a touch input at a plurality of contact points contacted on the touch sensor, the processor can identify the distance between the plurality of contact points and the center point of the plurality of contact points. Based on a speed of the center point due to the movement of the contact points, the processor can obtain a probability that the touch input corresponds to a first designated gesture associated with the distance and a probability that the touch input corresponds to a second designated gesture associated with the movement of the center point. The processor can change at least one of the distance or the center point based on the probabilities.

BACKGROUND

Field

The disclosure relates to an electronic device, a method, and a computer-readable storage medium for changing a trajectory of a gesture.

Description of Related Art

Recently, the spread of various types of a portable electronic devices, such as smartphones, a tablet PCs, wireless earphones, and smart watches, is expanding. Such portable electronic devices may receive one or more touch inputs using a touch sensor. The portable electronic device may identify a gesture using the received one or more touch inputs based on a specified time interval. The portable electronic device may identify various gestures based on the number of times, a trajectory, and/or the count of the received one or more touch inputs.

SUMMARY

An electronic device according to an example embodiment may include a touch sensor and at least one processor (including, e.g., processing circuitry). The processor may be configured to, based on a touch input at a plurality of contact points contacted on the touch sensor, identify a distance between the plurality of contact points, and a center point of the plurality of contact points; based on a speed of the center point by a movement of the contact points, obtain probabilities in which the touch input may correspond to each of a first specified gesture related to the distance and a second specified gesture related to a movement of the center point; and based on the probabilities, change at least one of the distance or the center point.

A method of an electronic device according to an example embodiment may include, based on a touch input at a plurality of contact points contacted on a touch sensor, identifying a distance between the plurality of contact points, and a center point of the plurality of contact points; based on a speed of the center point by a movement of the contact points, obtaining probabilities in which the touch input corresponds to each of a first specified gesture related to the distance and a second specified gesture related to a movement of the center point; and based on the probabilities, changing at least one of the distance or the center point.

One or more programs of one or more computer-readable storage media storing the one or more programs according to an example embodiment may be configured to, when executed by at least one processor (including, e.g., processing circuitry) of an electronic device perform operations including, based on a touch input at a plurality of contact points contacted on a touch sensor, identifying a distance between the plurality of contact points, and a center point of the plurality of contact points; based on speed of the center point by a movement of the contact points, obtain probabilities in which the touch input corresponds to each of a first specified gesture related to the distance and a second specified gesture related to a movement of the center point; and based on the probabilities, change at least one of the distance or the center point.

DETAILED DESCRIPTION

The various example embodiments of the present disclosure and terms used herein are not intended to limit the technology described in the present disclosure to specific embodiments, and should be understood to include various modifications, equivalents, or substitutes of the corresponding example embodiments. In relation to the description of the drawings, a same reference numeral may be used for a similar component. A singular expression may include a plural expression unless it is clearly meant differently in the context. In the present disclosure, an expression such as “A or B”, “at least one of A and/or B”, “A, B or C”, or “at least one of A, B and/or C”, and the like may include all possible combinations of items listed together. Expressions such as “1st”, “2nd”, “first” or “second”, and the like may modify corresponding components regardless of order or importance and are only used to distinguish one component from another component, not to limit the corresponding components. When a (e.g., first) component is referred to as “connected (functionally or communicatively)” or “accessed” to another (e.g., second) component, the component may be directly connected to the other component or may be connected through another component (e.g., a third component).

The term “module” used in the present document may include a unit configured with hardware, software, or firmware, or any combination thereof, and may be used interchangeably with terms such as logic, logic block, component, or circuit, and the like. The module may be an integrally configured component or a minimum unit or part thereof that performs one or more functions. For example, a module may be configured with an application-specific integrated circuit (ASIC).

Hereinafter, various example embodiments of the present disclosure will be described with reference to the accompanying drawings.

The input module 150 (including, e.g., input circuitry) may receive a command or data to be used by another component (e.g., the processor 120) of the electronic device 101, from the outside (e.g., a user) of the electronic device 101. The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

A connecting terminal 178 may include a connector via which the electronic device 101 may be physically connected with the external electronic device (e.g., the electronic device 102). According to an embodiment, the connecting terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

The camera module 180 (including, e.g., a camera) may capture a still image or moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

FIGS. 2A and 2B are views illustrating an example gesture corresponding to an input, according to various embodiments. Referring to FIGS. 2A to 2B, an electronic device 101 for identifying the gesture corresponding to the input at a plurality of contact points according to an embodiment is illustrated. The electronic device 101 may include or correspond to the electronic device of FIG. 1. A display 240 may include o correspond to the display module 160 of FIG. 1. Referring to FIG. 2A, in a state 200, the electronic device 101 according to an

embodiment may receive touch inputs at the plurality of contact points on the display 240. For example, the electronic device 101 may identify a gesture 203 corresponding to the received touch inputs. The gesture 203 may be identified, for example, based on touch inputs following a specified trajectory. The trajectory may be referred to as a path. The trajectory identified by the electronic device 101 may be one or more. For example, the electronic device 101 may identify the specified trajectory, by identifying the path of the touch inputs. For example, the specified trajectory may include the touch inputs at the plurality of contact points. For example, the electronic device 101 may identify a type of a gesture based on the touch inputs following the specified trajectory. The type of gesture will be described below with reference to FIG. 3.

The electronic device 101 according to an embodiment may identify a touch input according to the specified trajectory (or pattern) based on a specified time interval (e.g., 8 ms or 16 ms). For example, the electronic device 101 may identify a touch input corresponding to contact points 210. For example, in response to identifying the touch input at the contact points 210, the electronic device 101 may perform an operation for identifying the touch input based on the specified time interval. As an example, the touch input at the contact points 210 may be referred to as contact points indicating a contact with an external object.

For example, the electronic device 101 may identify touch inputs corresponding to a position of each of the contact points 210, based on the specified time interval. For example, the electronic device 101 may identify a touch input corresponding to a contact point 210-1 and/or a contact point 210-2 at a first time point. The electronic device 101 may identify a touch input corresponding to a contact point 220-1 and/or a contact point 220-2 at a second time point. The electronic device 101 may identify a touch input corresponding to a contact point 230-1 and/or a contact point 230-2 at a third time point.

For example, based on a distance between the contact point 210-1 and the contact point 220-1, and the specified time interval, the electronic device 101 may identify the touch input corresponding to the contact point 210-1, the touch input corresponding to the contact point 220-1, and/or the touch input corresponding to the contact point 230-1 as a first index. Based on identifying each of the touch inputs as the first index, the electronic device 101 may identify a first specified trajectory using the touch input corresponding to the contact point 210-1, the contact point 220-1, and/or the contact point 230-1.

For example, the electronic device 101 may identify each of the touch inputs by the contact point 210-2, the contact point 220-2, and/or the contact point 230-2 as a second index. For example, the electronic device 101 may identify a second specified trajectory using the touch input by the contact point 210-2, the contact point 220-2, and/or the contact point 230-2.

The electronic device 101 according to an embodiment may identify a center point (pivot) 205 and/or a distance 211 between contact points 210 based on the identified touch input. For example, the center point 205 may correspond to a position between the contact points 210. As an example, the center point 205 may be positioned at a half point between the contact points 210.

The electronic device 101 according to an embodiment may identify a change in the distance 211 based on identifying the touch input at the contact points 210. For example, the electronic device 101 may identify the touch input corresponding to contact points 220 and/or contact points 230, based on the specified time interval.

For example, the electronic device 101 may identify a distance 221 between the contact points 220 and/or a distance 222 between the contact points 230. For example, the electronic device 101 may identify center points corresponding to each of the distance 221 and the distance 222 while identifying the distance 221 and/or the distance 222. As an example, the center points may correspond to the center point 205. For example, the specified time interval may be 8 ms and/or 16 ms. However, it is not limited thereto.

The electronic device 101 according to an embodiment may identify the gesture 203 using each of the touch inputs by the contact points 210, 220, and 230. For example, the gesture 203 may be identified based on identifying the first specified trajectory including the touch inputs by the contact points 210-1, 220-1, and 230-1, and/or the second specified trajectory including the touch inputs by the contact points 210-2, 220-2, and 230-2. The gesture 203 may be referred to as a pinch-zoom gesture. The electronic device 101 may perform an event corresponding to the gesture 203 based on identifying the gesture 203. The event may be an example of an event indicating an enlargement of a screen in which the electronic device 101 displays on the display 240. For example, the event may be an example of an event indicating an enlargement of content (e.g., an image) in which the electronic device 101 displays on the display 240.

For example, the electronic device 101 may identify touch inputs based on the specified trajectory, from the contact points 230 to the contact points 210, with respect to the center point 205. The electronic device 101 may identify a specified gesture (e.g., a zoom-in gesture) based on the identified touch inputs (or the specified trajectory). Based on the identified specified gesture, the electronic device 101 may perform an event indicating a reduction of the screen or a reduction of the content (e.g., the image), displayed on the display 240. However, it is not limited thereto.

The electronic device 101 according to an embodiment may identify a movement of the center point 205 while identifying the touch inputs according to the specified trajectory. As an example, based on identifying the movement of the center point 205 to be below a specified distance, the electronic device 101 may identify a pinch-zoom gesture. As an example, the electronic device 101 may identify a scroll gesture different from the pinch-zoom gesture, based on identifying the movement of the center point 205 to be greater than or equal to the specified distance. For example, the electronic device 101 may identify an intention of a user, based on the movement of the center point 205 and/or an amount of change in a distance between contact points. The electronic device 101 may use data indicating a probability, in order to identify the intention of the user. An operation in which the electronic device 101 identifies a gesture corresponding to the user intention using the data will be described below with reference to FIG. 4.

Referring to FIG. 2B, in a state 250, the electronic device 101 according to an embodiment may identify touch inputs by contact points 260. For example, in response to identifying the touch inputs by the contact points 260, the electronic device 101 may identify touch inputs by contact points 270 and 280 different from the contact points 260 based on the specified time interval. For example, the electronic device 101 may identify a touch input by a contact point 260-1 at a first time point. The electronic device 101 may identify a touch input by a contact point 270-1 at a second time point. The electronic device 101 may identify a touch input by a contact point 280-1 at a third time point. For example, the electronic device 101 may identify a movement of the contact points 260, 270, and 280 based on identifying the touch inputs.

For example, the electronic device 101 may identify the touch inputs by each of the contact points 260-1, 270-1, and 280-1 as a first index, based on the specified time interval and/or a distance between the contact points 260-1, 270-1, and 280-1. For example, the electronic device 101 may identify a first specified trajectory using positions of the contact points 260-1, 270-1, and 280-1, based on identifying as the first index.

For example, the electronic device 101 may identify each of touch inputs by contact points 260-2, 270-2, and 280-2 as a second index. The electronic device 101 may identify a second specified trajectory using positions of the contact points 260-2, 270-2, and 280-2, based on identifying as the second index. For example, based on identifying the first specified trajectory and/or the second specified trajectory, the electronic device 101 may identify a gesture 253.

The electronic device 101 according to an embodiment may identify center points 255 and/or distances 261, 271, and 281, corresponding to each of the contact points 260, 270, and 280. The electronic device 101 may identify a distance of each of the contact points 260, 270, and 280 while identifying positions of the contact points 260, 270, and 280. For example, the electronic device 101 may identify the distance 261, the distance 271, and/or the distance 281, respectively, based on the specified time interval. For example, each of the distances 261, 271, and 281 may be substantially similar.

For example, the electronic device 101 may identify the center points 255 while identifying each of the contact points 260, 270, and 280. For example, the center points 255 may be obtained based on each of the contact points 260, 270, and 280. For example, a center point 255-1 may correspond to a position between the contact points 260. For example, a center point 255-2 may correspond to a position between the contact points 270. For example, a center point 255-3 may correspond to a position between the contact points 280. For example, each of the center points 255 may be disposed at a center of a distance between the contact points 260, 270, and 280. However, it is not limited thereto.

For example, the electronic device 101 may identify a movement of the center point 255 based on the specified trajectory. Based on the identified movement of the center point 255, the electronic device 101 may identify speed at which the center point 255 is moved. Based on the identified speed, the electronic device 101 may infer a user intention. An operation in which the electronic device 101 identifies the user intention will be described below with reference to FIG. 4.

The electronic device 101 according to an embodiment may identify the gesture 253 based on identifying the specified trajectory. For example, the specified trajectory may include the first specified trajectory based on the touch inputs by the contact points 260-1, 270-1, and 280-1, and/or the second specified trajectory based on the touch inputs by the contact points 260-2, 270-2, and 280-2. The gesture 253 may be, for example, a two-hand scroll gesture. As an example, the electronic device 101 may perform an operation indicating a scroll of a screen displayed on the display 240 based on identifying the two-hand scroll gesture. However, it is not limited thereto.

Referring to FIGS. 2A and 2B, the electronic device 101 according to an embodiment may identify touch inputs during the specified time interval. The electronic device 101 may identify a specified trajectory using the touch inputs. The electronic device 101 may identify a specified gesture (e.g., the gesture 203 or the gesture 253) based on the specified trajectory. For example, the specified gesture that the electronic device 101 identifies based on the specified trajectory may be one or more. For example, the electronic device 101 may identify a contact point based on identifying a touch input at one or more contact points (e.g., the contact points 210 of FIG. 2A and/or the contact points 260 of FIG. 2B) at the first time point. Based on identifying the contact point, the electronic device 101 may infer one or more specified gestures.

For example, the electronic device 101 may perform an operation corresponding to a first gesture among the one or more specified gestures. The electronic device 101 may use data indicating a probability related to the one or more specified gestures in order to perform an operation corresponding to the one gesture. Using the data, the electronic device 101 may select the first gesture from among the one or more specified gestures. For example, the electronic device 101 may identify the first gesture using a movement of the center point 205 or 255, speed at which the center point 205 or 255 is moved, and/or a change in a distance between contact points. As the electronic device 101 selects the one gesture, the electronic device 101 may change at least a portion of the center point and the distance obtained based on the touch input according to the specified trajectory. An operation in which the electronic device 101 changes the at least a portion will be described below with reference to FIG. 6.

As described above, the electronic device 101 may identify one or more touch inputs corresponding to a plurality of contact points using at least one sensor. For example, the electronic device 101 may identify the identified one or more touch inputs according to the specified trajectory. For example, based on identifying the one or more of the touch inputs, a user intention to input the specified gesture may be inferred. For example, the electronic device 101 may use probability data to infer the user intention. The probability data may refer, for example, to data indicating a probability that the specified gesture matches the user intention. The electronic device 101 may change at least a portion of the center point and/or the distance corresponding to each of the plurality of contact points using the probability data. Based on changing the at least a portion, the electronic device 101 may perform an operation corresponding to the specified gesture to be suitable to the user intention. Hereinafter, in FIG. 3, a simplified block diagram of the electronic device 101 will be described below.

FIG. 3 is a block diagram of an example electronic device according to various embodiments. Referring to FIG. 3, an electronic device 101 may be an example of the electronic device 101 of FIG. 1 and/or the electronic device 101 of FIG. 2. Referring to FIG. 3, the electronic device 101 according to an embodiment may include at least one of at least one processor 120, memory 130, a touch sensor 310, and/or a display 320. The processor 120, the memory 130, the touch sensor 310, and the display 320 may be electronically and/or operably coupled with each other by an electronic component such as a communication bus. A type and/or a number of hardware components included in the electronic device 101 is not limited to the arrangement illustrated in FIG. 3. For example, the electronic device 101 may include only a portion of the hardware components illustrated in FIG. 3.

The at least one processor 120 (including, e.g., processing circuitry) of the electronic device 101 according to an embodiment may include a hardware component for processing data based on one or more instructions. For example, the hardware component for processing the data may include an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), an application processor (AP), a communication processor (CP), a graphics processing unit (GPU), and/or a central processing unit (CPU). The number of processors 120 may be one or more. For example, the processor 120 may have a structure of a multi-core processor such as a dual core, a quad core, or a hexa core.

The memory 130 of the electronic device 101 according to an embodiment may include a hardware component for storing data and/or instructions inputted and/or outputted to the processor 120. The memory 130 may include, for example, a volatile memory such as random-access memory (RAM) and/or a non-volatile memory such as a read-only memory (ROM). The volatile memory may include, for example, at least one of a dynamic RAM (DRAM), a static RAM (SRAM), a cache RAM, or a pseudo SRAM (PSRAM). The non-volatile memory may include, for example, at least one of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a hard disk, a compact disk, or an embedded multimedia card (eMMC).

In the memory 130 of the electronic device 101 according to an embodiment, one or more instructions indicating an operation to be performed on data by the processor 120 may be stored. A set of instructions may be referred to as firmware, an operating system, a process, a routine, a sub-routine and/or an application. For example, the processor 120 of the electronic device 101 and/or the electronic device 101 may perform an operation of FIG. 7 by executing a set of a plurality of instructions distributed in a form of an application. Hereinafter, an application being installed in the electronic device 101 may refer, for example, to one or more instructions provided in the form of the application which are stored in the memory 130 of the electronic device 101, and the one or more applications may be stored in an executable format (e.g., a file with an extension specified by the operating system of the electronic device 101). The electronic device 101 according to an embodiment may provide an operation based on a specified gesture to a user, using at least one of the one or more applications.

The display 320 according to an embodiment may output visualized information to the user by being controlled by a controller such as the processor 120. The display 320 may include a flat panel display (FPD) and/or an electronic paper. The FPD may include a liquid crystal display (LCD), a plasma display panel (PDP), and/or one or more light emitting diodes (LEDs). The LED may include an organic LED (OLED).

According to an embodiment, the display 320 may include the touch sensor 310 (e.g., a touch sensor panel (TSP)) for detecting an external object (e.g., a finger of the user) on the display 320. However, it is not limited thereto. For example, the touch sensor 310 may be included in the electronic device 101 independently of the display 320. For example, based on the touch sensor 310, the electronic device 101 may detect an external object (e.g., a stylus) contacting on the display 320 or floating on the display 320. The processor 120 may identify at least one touch input based on the detected external object. For example, based on the at least one touch input identified using touch sensor 310, the processor 120 may identify at least one of information on extent where the at least one touch input is performed, information on speed at which the at least one touch input is performed, and information on a direction in which the at least one touch input is performed. As an example, the processor 120 may identify at least one specified trajectory based on the at least one touch input using the touch sensor 310. The processor 120 may identify at least one gesture based on the identified at least one specified trajectory.

For example, the touch sensor 310 of the electronic device 101 may be configured with a plurality of layers. For example, the touch sensor 310 may include a drive electrode, a dielectric, and/or a sense electrode. For example, the touch sensor 310 may identify a changed capacitance value in a plurality of areas (e.g., the contact points 210 of FIG. 2A) where the drive electrode and the sense electrode cross each other. As an example, the touch sensor 310 may identify a capacitance value between the drive electrode and the sense electrode. The touch sensor 310 may identify the capacitance value between the drive electrode and the sense electrode that is changed based on a touch input.

According to an embodiment, the touch sensor 310 may transmit data on the identified capacitance value to the processor 120. The processor 120 may receive the data from the touch sensor 310. The processor 120 may identify information on a user input (or trajectory) based on the data.

According to an embodiment, the user input may be configured with at least one touch input. For example, the processor 120 may identify touch inputs identified from a time point when the user input starts to a time point when the user input is released as the at least one touch input. For example, the processor 120 may identify the touch inputs based on a specified time interval from the time point when the user input starts to the time point when the user input is released. The processor 120 may identify the identified touch inputs as at least one gesture. As an example, the processor 120 may identify the at least one gesture (e.g., the gesture 203 of FIG. 2A or the gesture 253 of FIG. 2B) by identifying the user input according to a predefined period.

A gesture identified by the electronic device 101 according to an embodiment may be one or more. For example, the gesture may include at least one of a tap gesture, a drag gesture, a swipe gesture, a pinch-zoom gesture, and/or a scroll gesture. As an example, the electronic device 101 may initiate execution of content based on identifying the tap gesture for the content in a screen displayed on the display 320. As an example, the electronic device 101 may move the content in the screen based on identifying the drag gesture for the content included in the screen. As an example, the electronic device 101 may reduce and/or enlarge the screen displayed in the display 320 based on identifying a pinch gesture. As an example, the electronic device 101 may scroll the screen displayed in the display 320 based on identifying the scroll gesture.

The electronic device 101 according to an embodiment may identify touch inputs at a plurality of contact points based on the specified trajectory using the touch sensor 310. The electronic device 101 may identify a plurality of gestures in response to identifying the touch inputs. The electronic device 101 may identify that one of the plurality of gestures matches a user intention. The electronic device 101 may use data related to the one gesture to identify that the one gesture matches the user intention. The data may include a probability for the gesture corresponding to the specified trajectory. The electronic device 101 may perform an operation suitable for the user intention based on identifying the one gesture matching the user intention by using the data.

As described above, the electronic device 101 according to an embodiment may identify a plurality of external objects contacted on the display 320 by using the touch sensor 310. The electronic device 101 may infer motion of the plurality of external objects based on identifying the plurality of external objects. As an example, the electronic device 101 may identify that the plurality of external objects respectively move in different directions with respect to a point. As an example, the electronic device 101 may identify that the plurality of external objects move based on the same direction. For example, the electronic device 101 may identify a gesture based on a direction in which the plurality of external objects move.

For example, the electronic device 101 may change a portion of a trajectory corresponding to the identified gesture to perform an operation based on the identified gesture. As an example, a distance between contact points in contact with the plurality of external objects may be changed. As an example, a center point positioned at a point between the contact points may be changed. For example, the electronic device 101 may prevent a performance of an operation that the user does not intend by changing the portion of the trajectory. For example, the electronic device 101 may perform, in a visually natural way, an operation corresponding to the gesture by changing the portion of the trajectory. The electronic device 101 may use probability data for inferring the motion of the plurality of external objects, in order to perform an operation matching the user intention. Hereinafter, in FIG. 4, an operation in which the electronic device 101 identifies a gesture using the probability data will be described below.

FIG. 4 is a graph illustrating a probability with respect to a gesture identified by an example electronic device according to various embodiments. Referring to FIG. 4, a graph 400 indicating a probability that an electronic device 101 according to an embodiment identifies a gesture corresponding to a touch input at a plurality of contact points is illustrated. The graph 400 may indicate a probability (e.g., p or p′ of FIG. 4) in which the gesture corresponding to the input at the plurality of contact points matches a user intention based on moving speed (e.g., ν of FIG. 4) of a center point (e.g., the center point 205 of FIG. 2A). For example, the probability may correspond to one of values 0 to 1. The electronic device 101 corresponding to FIG. 4 may be an example of the electronic device 101 of FIG. 1 and/or the electronic device 101 of FIG. 2A.

Referring to the graph 400 according to an embodiment, based on the touch input at the plurality of contact points contacted on a touch sensor (e.g., the touch sensor 310 of FIG. 3), the electronic device 101 may identify a distance (e.g., the distance 211 of FIG. 2A) between the plurality of contact points and a center point (e.g., the center point 205 of FIG. 2A) of the plurality of contact points. For example, the electronic device 101 may obtain a probability corresponding to a plurality of gestures based on identifying the touch input at the plurality of contact points. For example, based on the speed of the center point by a movement of the contact points,

the electronic device 101 may obtain probabilities in which the touch input corresponds to each of a first gesture related to a movement of the center point and/or a second gesture related to the distance. However, it is not limited thereto. As an example, the electronic device 101 may obtain the probabilities by identifying the distance between the plurality of contact points. For example, the electronic device 101 may identify the speed of the center point by

identifying the movement of the contact points based on a specified time interval. For example, a probability corresponding to the first gesture may be indicated by a first graph 410. For example, a probability corresponding to the second gesture may be indicated by a second graph 420. As an example, the first gesture may be referred to as a two-hand scroll gesture (e.g., the gesture 253 of FIG. 2B). As an example, the second gesture may be referred to as a pinch-zoom gesture (e.g., the gesture 203 of FIG. 2A).

For example, in a case that the moving speed of the center point is 0, the electronic device 101 may identify the probabilities in which the touch inputs at the plurality of contact points correspond to each of the first gesture and the second gesture, with the same probability (e.g., 0.5). However, it is not limited thereto.

The first graph 410 according to an embodiment may be an example of a graph that is proportional to the moving speed (e.g., the ν of FIG. 4) of the center point. For example, referring to the first graph 410, as the moving speed of the center point increases, the probability in which the touch input by the plurality of contact points is identified as the first gesture may increase. For example, based on identifying that the moving speed of the center point increases, the electronic device 101 may infer the touch input at the plurality of contact points as the first gesture, based on a higher probability. For example, the first graph 410 may be indicated based on Equation 1.

Referring to the Equation 1, P may refer to the probability corresponding to the first gesture. They may refer to the moving speed of the center point. The electronic device 101 according to an embodiment may obtain the probability matching the first gesture based on identifying the moving speed of the center point by using the Equation 1. The electronic device 101 according to an embodiment may change at least a portion of the distance (e.g., the distances 261, 271, and 281 of FIG. 2B) between the plurality of contact points in a state of identifying that the touch input by the plurality of contact points is related to the first gesture. An operation of at least partially changing the distance by the electronic device 101 will be described below with reference to FIG. 6.

The second graph 420 according to an embodiment may be an example of a graph that is inversely proportional to the moving speed of the center point. For example, referring to the second graph 420, as the moving speed of the center point increases, the probability of being identified as the second gesture may be lower. For example, based on identifying that the moving speed of the center point is increased, the electronic device 101 may infer the touch input at the plurality of contact points as the second gesture based on the probability lower than the probability of identifying as the first gesture. For example, the second graph 420 may be indicated based on Equation 2.

Referring to the Equation 2, the P′ may refer to the probability corresponding to the second gesture. The ν may refer to the moving speed of the center point. The electronic device 101 according to an embodiment may obtain the probability that the touch input at the plurality of contact points matches the second gesture based on identifying the moving speed of the center point by using the Equation 2. The electronic device 101 according to an embodiment may change at least a portion of the center points corresponding to the plurality of contact points in a state of identifying that the touch input by the plurality of contact points is related to the second gesture. An operation in which the electronic device 101 changes the at least a portion of the center points will be described below with reference to FIG. 6.

The electronic device 101 according to an embodiment may perform an operation related to at least one of the first gesture and the second gesture based on changing at least one of the distance and the center point related to the plurality of contact points. For example, the electronic device 101 may obtain a parameter for performing the operation based on probabilities respectively corresponding to the first gesture or the second gesture. The parameter may refer, for example, to a degree to which at least one of the first gesture and/or the second gesture affects the operation. An operation in which the electronic device 101 obtains the parameter based on the obtained probabilities will be described below with reference to FIG. 5.

As described above, the electronic device 101 according to an embodiment may identify the touch input by the plurality of contact points using the touch sensor. Based on identifying the touch input at the plurality of contact points, a specified trajectory may be identified. The electronic device 101 may identify one or more gestures based on identifying the specified trajectory during a specified time. The electronic device 101 may identify the first gesture matching the user intention among the one or more gestures and the second gesture different from the first gesture, using probability data included in the graph 400. The electronic device 101 may perform at least one event using the parameter corresponding to the first gesture and the second gesture.

FIG. 5 is a diagram illustrating an example electronic device that performs an operation with respect to a gesture based on an influence level, according to various embodiments. An electronic device 101-1 of FIG. 5 and/or an electronic device 101-2 of FIG. 5 may be an example of the electronic device 101 of FIG. 1. For example, a size of a display 531 of the electronic device 101-2 may be different from a size of the display 521 of the electronic device 101-1. As an example, the size of the display 531 may be larger than the size of the display 521.

Referring to FIG. 5, the electronic device 101 according to an embodiment may identify at least one gesture using a touch sensor. The at least one gesture 510 may be referred to as the gesture 203 of FIG. 2A. For example, a distance 514 between contact points 512-1 and 512-2 may include a value different from a distance 513. For example, the distance 514 may include a value greater than or equal to the distance 513. As an example, the distance 514 may include a value that is twice the distance 513.

The electronic device 101 according to an embodiment may perform an event corresponding to the gesture 510 identified by the touch sensor (e.g., the touch sensor 310 of FIG. 3) according to a size and/or a touch sensing area of a display (e.g., the display 320 of FIG. 3). For example, the electronic device 101 may perform the event based on a parameter for the gesture 510. As an example, the parameter may, for example, indicate a degree to which the identified gesture affects the event performed by the electronic device 101. As an example, the parameter may include at least one value between 0 and 1. The electronic device 101 may obtain the parameter by using Equation 3.

Referring to the Equation 3, the α may refer to the degree that affects the event performed by the electronic device 101 based on the gesture identified by the touch sensor. The e(t) may refer to information on time. For example, the information on the time may refer to a time interval for the electronic device 101 to identify contact points included in the at least one gesture. As an example, the information on the time may be a value obtained by dividing a value (e.g., 8.3 ms) specified at the time interval. As an example, the e(t) may correspond to at least one value among 1 to 2. The P′ may be referred to in the P′ of FIG. 4. The s(i) may refer to information related to the size of the display. As an example, the s(i) may include at least one value among 0.5 to 1. For example, based on obtaining the parameter (e.g., 1) for the gesture 510, the electronic device 101 may expand (e.g., double) a screen on the display based on the parameter by identifying the distance 514, which is twice the distance 513. However, it is not limited thereto.

The electronic device 101-1 and the electronic device 101-2 according to an embodiment may identify the gesture 510 based on the same time interval. A screen displayed on the display 521 of the electronic device 101-1 may include information substantially similar to a screen displayed on the display 531 of the electronic device 101-2. For example, the parameter for the gesture 510 (e.g., the α of the Equation 3) may differ based on the size of each of the displays 521 and 531.

For example, in a state 520, the electronic device 101-1 may adjust a size of an area of the screen displayed on the display 521 based on identifying the gesture 510. For example, the electronic device 101-1 may enlarge a size of a screen displayed in an area 523 to be displayed in an area 525. A size of the areas 523 and 525 may be adjusted based on the distances 513 and 514 and/or the parameter (e.g., the α of the Equation 3).

For example, in a state 530, the electronic device 101-2 may adjust a size of an area of the screen displayed on the display 531 based on identifying the gesture 510. For example, the electronic device 101-2 may enlarge a size of a screen displayed in an area 533 to display in an area 535. A size of the areas 533 and 535 may be adjusted based on the distances 513 and 514 and/or the parameter (e.g., the α of the Equation 3).

For example, the size of the areas 523 and 533 based on contact points 511 may be substantially similar. For example, the size of the areas 525 and 535 based on contact points 512 may differ based on the parameter. However, it is not limited to the above-described embodiment.

The electronic device 101 according to an embodiment may at least partially correct a performance of the event corresponding to the at least one gesture based on the probability (e.g., the P′ of the Equation 2) obtained through the graph 400 of FIG. 4 by using the Equation 2. An operation in which the electronic device 101 at least partially corrects the performance of the event based on the Equation 3 will be described below with reference to FIG. 6.

As described above, the electronic device 101 may control a performance of an operation with respect to the gesture based on at least one parameter. For example, the parameter may be obtained based on the probability indicating the identification of the gesture based on the touch input by the plurality of contact points, a specified time interval, and/or the size of the display of the electronic device 101. The electronic device 101 may provide a more convenient user interface (UI) to the user by adjusting the performance of the operation with respect to the gesture based on obtaining the parameter.

FIG. 6 is a diagram illustrating an operation in which an example electronic device corrects at least a portion of a gesture based on an influence level, according to various embodiments. An electronic device 101 of FIG. 6 may be an example of the electronic device 101 of FIG. 1 and/or the electronic device 101 of FIG. 2A.

Referring to FIG. 6, the electronic device 101 according to an embodiment may obtain a touch input by contact points 601, 602, and 603, based on a specified time interval by using a touch sensor (e.g., the touch sensor 310 of FIG. 3). For example, the electronic device 101 may identify a first input with respect to contact points 601-1 and 601-2 at a first time point of the specified time interval. For example, the electronic device 101 may identify a second input by the contact points 602-1 and 602-2 at a second time point of the specified time interval. For example, the electronic device 101 may identify an n-th input by contact points 603-1 and 603-2 at an n-th time point of the specified time interval.

For example, the electronic device 101 may obtain center points 610 and/or distances 661, 662, and 663 corresponding to each of the contact points 601, 602, and 603 based on identifying the first input, the second input, and/or the n-th input. For example, the electronic device 101 may identify a change in a position of the center points 610-1, 610-2, and 610-3, and/or the distances 661, 662, and 663, obtained based on the specified time interval. As an example, the distances 662 and 663 may be different from the distance 661.

For example, the electronic device 101 may identify one or more gestures (e.g., the gesture 203 of FIG. 2A or the gesture 253 of FIG. 2B) based on identifying the change in the position of the center points and/or the distance.

The electronic device 101 according to an embodiment may obtain a probability corresponding to the one or more gestures using probability data (e.g., the probability data included in the graph 400 of FIG. 4). For example, the electronic device 101 may identify a gesture 640 based on a probability of a first gesture (e.g., the gesture 253 of FIG. 2B) corresponding to the first graph 410 of FIG. 4 and/or a probability of a second gesture (e.g., the gesture 203 of FIG. 2A) corresponding to the second graph 420 of FIG. 4. As an example, the gesture 640 may be at least partially similar to the gesture 253 of FIG. 2B.

The electronic device 101 according to an embodiment may change a specified trajectory corresponding to the gesture 640 by using a parameter with respect to the second gesture based on a relatively low probability. For example, the electronic device 101 may use Equation 4 to change the specified trajectory.

Referring to the Equation 4, for example, the g′ (n), may refer to a corrected data value. The g(n) may refer to a data value identified by the electronic device 101. The α may be referred to the α of the Equation 3. The n may refer to information on a time interval. The 9′(n−1) may refer to a corrected data value prior to the g′(n)

The electronic device 101 according to an embodiment may change a portion of the trajectory corresponding to the gesture 640 using the Equation 4. For example, the electronic device 101 may change each of the distances to be less than the distances 661, 662, and 663 between the contact points 601, 602, and 603. For example, the electronic device 101 may respectively change a position of the contact points 602-1 and 602-2 to be positioned at different points 620-1 and 620-2. For example, a distance between the different points 620-1 and 620-2 may correspond to a value less than the distance 662. For example, the distance between the different points 620-1 and 620-2 may be similar to the distance 661.

For example, the electronic device 101 may respectively compensate for a position of the contact points 603 to be positioned at different points 630-1 and 630-2. For example, a distance between the different points 630-1 and 630-2 may be less than the distance 663. For example, the distance between the different points 630-1 and 630-2 may be similar to the distance 661.

Although not illustrated, in a case that the probability with respect to the second graph 420 of FIG. 4 is relatively higher than the probability with respect to the first graph 410 of FIG. 4, the electronic device 101 according to an embodiment may correct a specified trajectory of the second gesture different from the first gesture, based on a parameter with respect to the first gesture (e.g., the gesture 253 of FIG. 2B) corresponding to the first graph. For example, the electronic device 101 may identify a movement of a center point by a movement of contact points included in the second gesture. For example, the electronic device 101 may change a position of the center point to be less than a distance from which the center point is moved based on the movement of the identified center point. However, it is not limited to the above-described embodiment.

The electronic device 101 according to an embodiment may store the data value corrected using the Equation 4, as a resource file based on an operating system included in the electronic device 101. For example, the data value may be indicated by Table 1.

Corrected Distance

Time
Distance between
Second Graph 420
Parameter (e.g. α of
between Contact

Interval
Contact Points
of FIG. 4)
Equation 3)
Points

Point

Point

Point

Point

Referring to the Table 1, the electronic device 101 according to an embodiment may obtain contact points based on the specified time interval. For example, the electronic device 101 may identify a distance (e.g., the distance 663) between the obtained contact points, a probability, and/or a parameter at the n-th time point. The electronic device 101 may change the distance between the contact points using the Equation 4. For example, the electronic device 101 may identify a distance (e.g., 744.4255) between the contact points at a first time point. For example, at a second time point, based on a distance between the identified contact points (e.g., 578.6548942), a probability (e.g., 0.11), and/or a parameter (e.g., 0.55), the electronic device 101 may obtain a distance (e.g., 735.3081167) in which the distance is corrected. The corrected distance may be similar to the distance between the identified contact points at the first time point, other than the distance between the identified contact points. However, it is not limited to the data values included in the Table 1 described above. The electronic device 101 according to an embodiment may perform an event corresponding to the gesture 640 by changing the trajectory of the gesture 640. For example, based on changing at least one of a distance or a center point between contact points, the electronic device 101 may execute an event related to at least one of the first gesture (e.g., the gesture 253 of FIG. 2B) or the second gesture (e.g., the gesture 203 of FIG. 2A) based on an application executed by a processor (e.g., the processor 120 of FIG. 3). For example, the electronic device 101 may control a screen displayed through a display based on the executed event. As an example, the event may be referred to as screen scrolling. For example, the electronic device 101 may perform the screen scrolling more naturally by changing the trajectory of the gesture 640.

As described above, the electronic device 101 according to an embodiment may obtain contact points based on the specified time interval. The electronic device 101 may identify a plurality of gestures based on a position of the obtained contact points. The electronic device 101 may identify the first gesture (e.g., the gesture 253 of FIG. 2B) based on a higher probability using probability data, among the plurality of gestures. The electronic device 101 may obtain an influence level (or parameter) with respect to the second gesture (e.g., the gesture 203 of FIG. 2A) different from the first gesture among the plurality of gestures, based on the Equation 3. For example, the electronic device 101 may change the position of the contact points based on the obtained influence level with respect to the second gesture. The electronic device 101 may perform an event corresponding to the first gesture to be suitable to a user intention by changing the position of the contact points.

FIG. 7 is a flowchart illustrating an example operation in which an example electronic device corrects at least a portion of a plurality of gestures based on an input, according to various embodiments. At least one of operations of FIG. 7 may be performed by the electronic device 101 of FIG. 1 and/or the processor 120 of FIG. 1.

Referring to FIG. 7, in operation 700, a processor according to an embodiment may identify an input using a touch sensor. For example, the input may be one or more. For example, the processor may identify one or more inputs based on a specified time interval.

Referring to FIG. 7, in operation 710, the processor according to an embodiment may identify a plurality of gestures based on the identified input. For example, the plurality of gestures may include at least one of the gesture 203 of FIG. 2A and/or the gesture 253 of FIG. 2B. For example, the plurality of gestures may be identified based on the number and/or a position of one or more contact points.

Referring to FIG. 7, in operation 720, the processor according to an embodiment may identify a probability corresponding to each of the identified plurality of gestures. For example, the processor may identify the probability corresponding to each of the plurality of gestures using the probability data included in the graph 400 of FIG. 4. The probability may be different for each of the plurality of gestures.

Referring to FIG. 7, in operation 730, the processor according to an embodiment may obtain an influence level corresponding to each of the plurality of gestures based on the identified probability. The influence level may refer, for example, to the α of the Equation 3 of FIG. 5. For example, the processor may obtain an influence level with respect to a gesture based on a relatively low probability among the plurality of gestures. However, it is not limited thereto.

Referring to FIG. 7, in operation 740, the processor according to an embodiment may identify whether a probability corresponding to a first specified gesture among the plurality of gestures is greater than or equal to a probability corresponding to a second specified gesture. For example, the first specified gesture may be a gesture corresponding to the second graph 420 of FIG. 4. For example, the first specified gesture may be referred to the gesture 203 of FIG. 2A. For example, the second specified gesture may be a gesture corresponding to the first graph 410 of FIG. 4. For example, the second specified gesture may be referred to the gesture 253 of FIG. 2B. For example, the processor may perform operation 740 using the probability data included in the graph 400 of FIG. 4.

Referring to FIG. 7, in a case that the probability corresponding to the first specified gesture is greater than or equal to the probability corresponding to the second specified gesture (the operation 740—YES), in operation 750, the processor according to an embodiment may correct the first specified gesture based on an influence level corresponding to the second specified gesture. For example, the processor may change a position of center points between contact points configuring the first specified gesture. For example, the processor may at least partially refrain from performing an operation corresponding to the second specified gesture by changing the position of the center points. For example, the processor may provide a more accurate operation corresponding to the first specified gesture to a user who intends to perform the operation corresponding to the first specified gesture, by at least partially refraining from performing the operation corresponding to the second specified gesture.

Referring to FIG. 7, in a case that the probability corresponding to the first specified gesture is less than the probability corresponding to the second specified gesture (the operation 740—NO), in operation 760, the processor according to an embodiment may correct the second specified gesture based on an influence level corresponding to the first specified gesture. For example, the processor may change a trajectory of the second specified gesture to perform the correction. For example, a position of at least a portion of a plurality of contact points corresponding to the second specified gesture may be adjusted. The trajectory of the second specified gesture may be partially indicated based on a straight line by adjusting the position of the at least a portion. The processor may execute an event corresponding to the second specified gesture by correcting the second specified gesture. For example, the processor may at least partially refrain from performing the operation based on the first specified gesture by changing the trajectory of the second specified gesture.

The electronic device according to an embodiment may identify one or more gestures using the touch sensor. The electronic device may identify a gesture matching a user intention based on using a probability based on a movement of contact points among the identified one or more gestures.

The electronic device according to an embodiment may identify the one or more gestures using the touch sensor. A method for the electronic device to identify the gesture that satisfies a user intention among the identified one or more gestures is required.

As described above, an electronic device 101 according to an example embodiment may include a touch sensor 310 and a processor 120. The processor may be configured to, based on a touch input at a plurality of contact points 210, 220, 230, 260, 270, 280, 511, 512, 601, 602, or 603 contacted on the touch sensor, identify a distance 211, 221, 222, 261, 271, 281, 513, 514, 661, 662, or 663 between the plurality of contact points, and a center point 205, 255, 515, or 510 of the plurality of contact points. The processor may be configured to, based on a speed of the center point by a movement of the contact points, obtain probabilities 410 and 420 in which the touch input may correspond to each of a first specified gesture 203 related to the distance and a second specified gesture 253 related to a movement of the center point. The processor may be configured to, based on the probabilities, change at least one of the distance or the center point.

For example, the processor may be configured to change the center point of the distance or the center point in a state in which the touch input being related to the first specified gesture is identified based on the probabilities.

For example, the processor may be configured to, based on the probabilities that are inversely proportional to the speed of the center point, identify that the touch input is related to the first specified gesture.

For example, the processor may be configured to change the distance of the distance or the center point in another state in which the touch input being related to the second specified gesture is identified based on the probabilities.

For example, the processor may be configured to, based on the probabilities that are proportional to the speed of the center point, identify that the touch input is related to the specified gesture.

For example, the processor may be configured to change the distance to be below a distance between the contact points.

For example, the processor may be configured to identify the movement of the center point by the movement of the contact points. The processor may be configured to, based on the identified movement of the center point, change the center point to be below a distance in which the center point is moved.

For example, the processor may be configured to, based on a specified time interval, identify the movement of the contact points.

For example, the processor may be configured to, based on changing at least one of the distance or the center point, execute an event related to at least one of the first specified gesture or the second specified gesture, based on an application executed by the processor.

For example, the processor may be configured to, based on a size of a touch sensing area based on the touch sensor, change at least one of the distance or the center point.

As described above, a method of an electronic device 101 according to an embodiment may include, based on a touch input at a plurality of contact points 210, 220, 230, 260, 270, 280, 511, 512, 601, 602, or 603 contacted on a touch sensor 310, identifying a distance 211, 221, 222, 261, 271, 281, 513, 514, 661, 662, or 663 between the plurality of contact points, and a center point 205, 255, 515, or 510 of the plurality of contact points. The method may include, based on speed of the center point by a movement of the contact points, obtaining probabilities 410 and 420 in which the touch input corresponds to each of a first specified gesture 203 related to the distance and a second specified gesture 253 related to a movement of the center point. The method may include, based on the probabilities, changing at least one of the distance or the center point.

For example, the method may include changing the center point of the distance or the center point in a state where the touch input being related to the first specified gesture is identified based on the probabilities.

For example, the method may include, based on the probabilities that are inversely proportional to the speed of the center point, identifying that the touch input is related to the first specified gesture.

For example, the method may include changing the distance of the distance or the center point in another state in which the touch input being related to the second specified gesture is identified based on the probabilities.

For example, the method may include, based on the probabilities that are proportional to the speed of the center point, identifying that the touch input is related to the second specified gesture.

For example, the method may include changing the distance to be below a distance between the contact points.

For example, the method may include identifying the movement of the center point by the movement of the contact points. The method may include, based on the identified movement of the center point, changing the center point to be below a distance in which the center point is moved.

For example, the method may include, based on a specified time interval, identifying the movement of the contact points.

For example, the method may include, based on changing at least one of the distance or the center point, executing an event related to at least one of the first specified gesture or the second specified gesture, based on an application executed by a processor.

For example, the method may include, based on a size of a touch sensing area based on the touch sensor, changing at least one of the distance or the center point.

As described above, one or more programs of one or more non-transitory computer-readable storage media storing the one or more programs according to an embodiment may be configured to, when executed by a processor 120 of an electronic device 101, based on a touch input at a plurality of contact points 210, 220, 230, 260, 270, 280, 511, 512, 601, 602, or 603 contacted on a touch sensor 310, identify a distance 211, 221, 222, 261, 271, 281, 513, 514, 661, 662, or 663 between the plurality of contact points, and a center point 205, 255, 515, or 510 of the plurality of contact points. The one or more programs may be configured to, based on speed of the center point by a movement of the contact points, obtain probabilities 410 and 420 in which the touch input corresponds to each of a first specified gesture 203 related to the distance and a second specified gesture 253 related to a movement of the center point. The one or more programs may be configured to, based on the probabilities, change at least one of the distance or the center point.

For example, the one or more programs may be configured to, when executed by the processor, change the center point of the distance or the center point in a state in which the touch input being related to the first specified gesture is identified based on the probabilities.

For example, the one or more programs may be configured to, when executed by the processor, based on the probabilities that are inversely proportional to the speed of the center point, identify that the touch input is related to the first specified gesture.

For example, the one or more programs may be configured to, when executed by the processor, change the distance of the distance or the center point in another state in which the touch input being related to the second specified gesture is identified based on the probabilities.

For example, the one or more programs may be configured to, when executed by the processor, based on the probabilities that are proportional to the speed of the center point, identify that the touch input is related to the second specified gesture.

For example, the one or more programs may be configured to, when executed by the processor, change the distance to be below a distance between the plurality of contact points.

For example, the one or more programs may be configured to, when executed by the processor, identify the movement of the center point by the movement of the contact points. The one or more programs may be configured to, when executed by the processor, based on the identified movement of the center point, change the center point to be below a distance in which the center point is moved.

For example, the one or more programs may be configured to, when executed by the processor, based on a specified time interval, identify the movement of the contact points.

For example, the one or more programs may be configured to, when executed by the processor, based on changing at least one of the distance or the center point, execute an event related to at least one of the first specified gesture or the second specified gesture, based on an application executed by the processor.

For example, the one or more programs may be configured to, when executed by the processor, based on a size of a touch sensing area based on the touch sensor, change at least one of the distance or the center point.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, the device and the component described in embodiments may be implemented by using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to an instruction. A processing device (including, e.g., processing circuitry) may perform an operating system (OS), and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process and generate data in response to the execution of the software. For the convenience of understanding, one processing device is sometimes described as being used, but a person who has ordinary knowledge in the relevant technical field will understand that the processing device may include a plurality of processing elements and/or a plurality of types of processing elements. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, another processing configuration, such as a parallel processor, is also possible.

Software may include a computer program, code, an instruction, or a combination of one or more thereof, and may configure a processing device to operate as desired or command the processing device independently or collectively. The software and/or data may be embodied in any type of a machine, a component, a physical device, a computer storage medium or a device, in order to be interpreted by the processing device, or to provide the instruction or the data to the processing device. The software may be distributed on a network-connected computer system, and may be stored or executed in a distributed manner. The software and the data may be stored in one or more computer-readable recording media.

A method according to an embodiment recorded on a computer-readable medium by being implemented in a form of a program command that may be performed through various computer devices. In this case, the medium may continuously store a computer-executable program or temporarily store it for execution or download. In addition, the medium may be various recording media or storage in a form of a single or several hardware coupled, but is not limited to media directly connected to a certain computer system, and may exist distributed on a network. An example of the media may include a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, magneto-optical medium such as a floptical disk, and ROM, RAM, flash memory, and the like, which are configured to store program instructions. In addition, as an example of other media, there are also recording media or storage media managed by an app store that distributes an application, a site that supplies or distributes various software, a server, and the like.

Although the embodiments have been described according to limited embodiments and drawings as described above, a person who has ordinary knowledge in the relevant technical field is capable of various modifications and transform from the above description. For example, an appropriate result may be achieved even if the described technologies are performed in a different order from the described method, and/or components of the described system, structure, device, circuit, and the like coupled or combined in a form different from the described method, or substituted or replaced by another component or an equivalent. It will also be understood that any of the embodiment(s) described herein may be used in conjunction with any other embodiment(s) described herein.

Therefore, other implementations, other embodiments, and equivalents to the scope of the claims are in the scope of the claims.