Display apparatus and computer-readable medium

In a display apparatus, a processor performs an information storing process of storing the displayed information to be correlated with the locus of the handwriting recognized by the recognition process in the memory, when it is determined by the handwriting position determining process that the locus of the handwriting position is superposed on the information which is displayed on the display unit, and an information output process of reading and outputting the information, which is stored in the memory to be correlated with the handwritten locus recognized by the recognition process by the information storing process, when it is determined by the handwriting position determining process that the locus of the handwriting position is not superposed on the information which is displayed on the display unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based noon and claims the benefit of priority from prior Japanese Patent Applications No. 2010-189687, filed Aug. 26, 2010; and No. 2011-072965, filed Mar. 29, 2011, the entire contents of all of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a display apparatus which has a function of recognizing handwritten characters, and a computer-readable medium.

2. Description of the Related Art

With the spread of electronic apparatuses equipped with a touch panel, a technique of recognizing handwritten characters is making progress day by day. In addition, various types of characters can be recognized by a function of recognizing handwritten characters. Recently, a technique especially for recognition of handwritten numerical expressions has been developed (for example, see Jpn. Pat. Appln. KOKAI Publication No. 2009-080615).

In the meantime, operation of these electronic apparatuses has become complicated, making them difficult to understand due to the increase in functions of such apparatuses. For example, there is a function of substituting a character string for a character, and calling the character string substituted for the character (substitution and calling function). However, to execute such a function, a plurality of key operations are required for calling the function, thus usability is not good.

In addition, for example, word processors have a function (character converting function) of registering a frequently-used noun or a word of a spelling including a number of characters, which is substituted by a character desired by the user, and calling and displaying the registered word by inputting the substitute character. However, to register a word (character string) to be registered in association with a desired character by the character converting function, it is necessary to perform troublesome operations such as an operation of setting the registering function, operation of inputting the word to be registered, operation of inputting the substitute character, and operation of finishing the registering function; the usability of which is also not good.

BRIEF SUMMARY OF THE INVENTION

Accordingly, it is an object of the present invention to provide a display apparatus and a computer-readable medium, which can easily substitute desired display information by a desired character, and simply call and display the desired display information, by using a handwritten character recognizing function.

According to one aspect of the present invention, there is provided a display apparatus including: a display unit that has a position detection function of detecting a position where the user performs handwriting; a memory; and a processor that performs: an information display control process of causing the display unit to display information in response to user operation; a recognition process of recognizing a locus of a handwriting position which is detected by the position detection function of the display unit; a handwriting position determining process of determining whether the locus of the handwriting position detected by the position detection function is superposed on the information which is displayed on the display unit by the information display control process; an information storing process of storing the displayed information to be correlated with the locus of the handwriting recognized by the recognition process in the memory, when it is determined by the handwriting position determining process that the locus of the handwriting position is superposed on the information which is displayed on the display unit; and an information output process of reading and outputting the information, which is stored in the memory to be correlated with the handwritten locus recognized by the recognition process by the information storing process, when it is determined by the handwriting position determining process that the locus of the handwriting position is not superposed on the information which is displayed on the display unit.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1is a front view illustrating an external appearance of a graph function calculator10according to an embodiment of a display apparatus of the present invention.

The function calculator10has a small size such that the user can sufficiently hold and operate the calculator10by one hand due to the need for portability. A front surface of a main body of the function calculator10is provided with a key input unit11and a color display unit16.

The key input unit11includes a numeric and operation symbol key group12, a function key group13, mode setting key group14configured to display menu pictures of various operation modes and command setting of an operation mode, a cursor key15configured to perform operation of moving a cursor displayed on a color display unit16and selecting a data item, and function keys F1 to F6 configured to selectively designate one of various functions displayed as a menu along a lower end of the color display unit16.

As the numeral and operation symbol key group12, keys “0” to “9” (numeric) and keys “+”, “−”, “×”, and “÷” (operation symbols), a key “AC” (all clear), and key “EXE” (execution)12aare arranged.

As the function key group13, a key “log” (logarithm), a key “sin” (sine), and a key “ab/c” (fraction) are arranged.

As the mode setting key group14, a key “Menu”, a key “SHIFT”, and a key “OPTN” (option) are arranged.

The keys arranged in the numeral and operation symbol key group12, and the keys arranged in the function key group13also function as keys configured to input various characters (symbols) and operators according to respective inscriptions on the main body surface above the respective keys.

The color display unit16is formed of a color liquid crystal display unit of the dot-matrix type. A display screen of the color display unit16is provided with a touch panel16T of a transparent tablet for detecting the position where the screen is touched, called the touching position. A locus accompanying a handwritten input to the touch panel16T is displayed on the color display unit16by echo back.

FIG. 2is a block diagram illustrating a configuration of an electronic apparatus of the graph function calculator10.

The function calculator10includes a processor (CPU)21.

The processor (CPU)21controls operations of various units of the circuit, while causing a RAM26to store various data or reading various data from the RAM26, in accordance with a system program which is stored in advance in a ROM22, a calculator control program which is read from an external storage medium23such as a memory card to the ROM22by a storage medium reader/writer24, or a calculator control program which is downloaded and read from a Web server (program server)30on a communication network N such as the Internet to the ROM22through a communication controller25.

The system program or the calculator control program stored in the ROM22is started in response to a key input signal from the key input unit11and a touching position detection signal from the touch panel16T.

As described above, the processor (CPU)21is connected with the color display unit (LCD)16, in addition to the ROM22, the storage medium reader/writer24, the communication controller25, the RAM26, the key input unit11, and the touch panel16T.

As the calculator control program stored in the ROM22, various control programs22aare stored, such as an operation processing program to execute operation processing in accordance with a desired numerical expression input by the user through the key input unit11or the touch panel16T, and a graph drawing processing program to execute graph drawing processing in accordance with the desired numerical expression input by the user.

In addition, as the calculator control program, stored are a character recognition program22b1to recognize characters (including symbols) and operators which are input by handwriting to the touch panel161, and a numerical expression and character string unit extraction program22b2to extract numerical expressions and character strings which are displayed in accordance with the handwritten positions. For example, the extraction unit of the numerical expressions by the program22b2is an equation, and the extraction unit of the character strings by the program22b2is a word or an idiomatic phrase.

The ROM22is provided with a defined function table22c, which stores a character or a character string desired by the user for calling a function (defined function) that is defined by combining functions executable in accordance with the various control programs22aby the user.

For example, a character string “factor” is stored in association with a defined function “output a result of factorization”, a character string “d/dx” is stored in association with a defined function “differentiate a graph expression for X”, and a character string “Solve” is stored in association with a defined function “output a result obtained by solving ‘numerical expression=0’ for X”.

In addition, for example, a character string “Search” is stored in association with a defined function “search dictionary”, a character string “Read” is stored in association with a defined function “display pronunciation”, and a character string “Zoom” is stored in association with d defined function “enlarge”.

The RAM26includes a display data storing unit26awhich functions as a frame buffer of the color display unit16, a recognized character storing unit26bwhich stores characters that are recognized in accordance with the character recognition program22b1, and a character and display information correlation storing unit26c.

The character and display information correlation storing unit26cstores, when handwritten characters are input on information displayed on the color display unit16, the displayed information or a numerical expression relating to the information in association with the handwritten characters input and recognized.

In this process, when the displayed information is a numerical expression and the characters which are input by handwriting and recognized are characters with an operator, an original numerical expression, which has an operation result that is obtained in accordance with the operator and corresponds to the displayed numerical expression, is subjected to an inverse operation, and the numerical expression of the inverse operation result is stored in association with the recognized characters excluding the operator.

In addition, when the displayed information is a graph and the characters which are input by handwriting and recognized are characters with an operator, an original numerical expression which has an operation result that is obtained in accordance with the operator and corresponds to the displayed numerical expression, is subjected to an inverse operation, and the numerical expression of the inverse operation result is stored in association with the recognized characters excluding the operator.

Next, the following is an explanation of an information display operation process using the handwritten character recognizing function of the graph function calculator according to the above structure.

FIG. 3is a flowchart illustrating an information display operation process using the handwritten character recognizing function of the graph function calculator10.

In a display state where an object desired by the user is input and a process of the input object can be performed (Step S1), first, it is determined whether a handwritten character is input or not on the display screen of the color display unit16, based on an input signal from the touch panel16T (Step S2).

At Step S2, when it is determined that a handwritten character is input based on the fact that a predetermined time has passed since the input signal from the touch panel16T or the “EXE” (execution) key12ais input (Step S2(Yes)), the character recognition program22b1is started, and the handwritten character (including a symbol) is subjected to recognition and stored in the recognized character storing unit26b(Step S3).

Then, it is determined whether the recognized character (or character string) stored in the recognized character storing unit26bis a character, character stored in the defined function table22c, or a character with an operator (Step S4).

At Step S4, when it is determined that the recognized character is not a character stored in the defined function table22cor a character with an operator, the calculator goes to a character substitution calling process illustrated inFIG. 4(Step SA).

When it is determined that the recognized character is a character stored in the defined function table22c, the calculator goes to a function operation process illustrated inFIG. 10(Step SB).

When it is determined that the recognized character is a character with an operator, the calculator goes to an operator-included character substitution calling process illustrated inFIG. 13(Step SC).

FIG. 4is a flowchart illustrating a character substitution calling process accompanying the information display operation process of the graph function calculator10.

FIGS. 5A,5B, and5C are diagrams illustrating first substitution display operation accompanying the character substitution calling process of the graph function calculator10.

For example, as illustrated inFIG. 5A, in a state where a numerical expression “X2+5X+4” desired by the user is input and displayed on the display screen of the color display unit16(Step S1), suppose that a handwritten character “f” is input and recognized to be superposed on the numerical expression as illustrated inFIG. 5B(Steps S2and S3). Since the character “f” is neither a character stored in the defined function table22cnor a character with an operator, a recognition result thereof is determined as an ordinary character, and the calculator goes to the character substitution calling process illustrated inFIG. 4(Step S4->SA).

Then, it is determined that a position of the recognized handwritten character “f” is superposed on the displayed numerical expression “X2+5X+4” (Step A1(Yes)).

Then, it is determined that the displayed numerical expression “X2+5X+4” is neither a graph (Step A1a(No)) nor an image ((Step A1b(No)).

Thereby, the recognized character “f” and the displayed numerical expression “X2+5X+4” are correlated with each other, and stored in the character and display information correlation storing unit26c(Step A2a).

At this step, as illustrated inFIG. 5C, a message “substitution of f=X2+5X+4 is performed” M1, which indicates that the displayed numerical expression is substituted for the recognized handwritten character f, is displayed on the display screen (16) which displays the numerical expression “X2+5X+4”.

FIG. 6is a diagram illustrating a second substitution display operation accompanying the character substitution calling process of the graph function calculator10.

For example, as illustrated inFIG. 6, in a state where a text(Rashomon-Akutagawa Ryunosuke) . . . ” desired by the user is input or read from the storing unit or the exterior and displayed on a text display screen Gt of the color display unit16(Step S1), suppose that a handwritten character “α” is input and recognized to be superposed on a part “(Rashomon)” of the text (Steps S2and S3). Since the character “α” is neither a character stored in the defined function table22cnor a character with an operator, a recognition result thereof is determined as an ordinary character, and the calculator goes to the character substitution calling process illustrated inFIG. 4(Step S4->SA).

Then, it is determined that a position of the recognized handwritten character “α” is superposed on the displayed text “(Rashomon-Akutagawa Ryunosuke) . . . ” (Step A1(Yes)).

Thereby, the recognized character “α” and the displayed word “(Rashomon)”, which is extracted according to the numerical expression and character string unit extraction program2b2based on the text position on which the recognized character “α” is superposed, are correlated with each other and stored in the character and display information correlation storing unit26c(Step A2a).

FIG. 7is a diagram illustrating a third substitution display operation accompanying the character substitution calling process of the graph function calculator10.

For example, illustrated inFIG. 7, in a state where an image desired by the user is input or read out from the storing unit or the exterior and displayed on an image display screen Gp of the color display unit16(Step S1), suppose that a handwritten character “y” is input and recognized in a state of being superposed on a part of a numerical expression in the image (Steps S2, S3). Since the character “y” is neither a character stored in the defined function table22cnor a character with an operator, a recognition result thereof is determined as an ordinary character, and the calculator goes to the character substitution calling process illustrated inFIG. 4(Step S4->SA).

Then, it is determined that a position of the recognized handwritten character “y” is superposed on the displayed image (Step A1(Yes)).

Then, it is determined that the displayed image is not a graph (Step A1a(No)), but an image (Step A1b(Yes)), and the displayed image is subjected to OCR (optical character recognition), and stored as a character string (Step A1c).

Thereby, the recognized character “y” and successive numerical expression (“D=√{square root over (. . . )})2), which is extracted in accordance with the numerical expression and character string unit extraction program22b2(or extracted in accordance with range designation by user operation) based on the character position on which the recognized character “y” is superposed, are correlated with each other and stored in the character and display information correlation storing unit26c(Step A2a).

FIGS. 8A,8B, and8C are diagrams illustrating a fourth substitution display operation accompanying the character substitution calling process of the graph function calculator10.

For example, as illustrated inFIG. 8A, in a state where a graph Y1according to a graph expression “Y1-X2+2X−3” desired by the user is input or read out from the storing unit or the exterior and displayed on a graph display screen Gf of the color display unit16(Step S1), suppose that a handwritten character “h” is input and recognized in a state of being superposed on the graph Y1(Steps S2, S3), as illustrated inFIG. 8B. Since the character “h” is neither a character stored in the defined function table22cnor a character with an operator, a recognition result thereof is determined as an ordinary character, and the calculator goes to the character substitution calling process illustrated inFIG. 4(Step S4->SA).

Then, it is determined that a position of the recognized handwritten character “h” is superposed on the displayed graph Y1(Step A1(Yes)), and it is determined that the displayed object is a graph (Step A1a(Yes)).

Thereby, the recognized character “h” and the expression “X2+2X−3” of the graph Y1, on which the recognized character “h” is superposed, are correlated with each other, and stored in the character and display information correlation storing unit26c(Step A2b).

In this process, as illustrated inFIG. 85, a message M2“substitution of “h=X2+2X−3” is performed”, which indicates that the expression of the displayed graph Y1is substituted for the recognized handwritten character “h”, is displayed on the display screen (16) on which the graph Y1is displayed.

Also, when a desired character is input by handwriting on the graph Y displayed as an image on the color display unit16, a graph expression “Y=F(x)” of the graph Y may be analyzed from the image, and thereby the recognized handwritten character and the analyzing graph expression may be stored in correlation (Steps A1to A2a).

FIGS. 9A,9B, and9C are diagrams illustrating a calling display operation accompanying the character substitution calling process of the graph function calculator10.

For example, after the various substitution display operations illustrated inFIG. 5AtoFIG. 8C, suppose that a handwritten character “f” is input and recognized (Steps S2, S3) as illustrated inFIG. 9B, in a display state of the color display unit16(Step S1) in which no characters are input as illustrated inFIG. 9A. a recognition result is determined as an ordinary character, and the calculator goes to the character substitution calling process as illustrated inFIG. 4(Step S4->SA).

Then, it is determined that the recognized handwritten character “f” is not superposed on any display object (Step A1(No)), and it is also determined that a position of the recognized handwritten character “f” is not on a graph display screen Gf (seeFIGS. 8A,8B, and8C) (Step A3(No)).

In this case, since the numerical expression “X2+5X+4” correlated with the recognized character “f” is stored in the character and display information correlation storing unit26c, the numerical expression “X2+5X+4” which correlated with the recognized character “f” is output and displayed on the display screen of the color display unit16, as illustrated inFIG. 9C(Step A4).

When a handwritten character “α” is input and recognized in the display screen of the color display unit16, to which no characters are input as illustrated inFIG. 9A(Steps S1to SA), the word “(Rashomon)” (seeFIG. 6) which is correlated with the recognized character “α” and stored in the character and display information correlation storing unit26cis read out and displayed (Steps A1, A3, A4).

In the same manner, when a handwritten character “y” is input and recognized (Steps S1to SA) in the display screen of the color display unit16to which no characters are input, the numerical expression “D=√{square root over (. . . )})2” (seeFIG. 7) which is stored in the character and display information correlation storing unit26cand correlated with the recognized character “y” is read out and displayed (Steps A1, A3, AA).

In addition, for example, a handwritten character “h” is input and recognized (Steps S2, S3) in a state where the graph display screen Gf which displays coordinate axes is displayed on the color display unit16(Step S1), a recognition result thereof is determined as an ordinary character, and the calculator goes to the character substitution calling process illustrated inFIG. 4(Step S4->SA).

Then, it is determined that a position of the recognized handwritten character “h” is not superposed on the displayed object (Step A1(No)), and it is determined that the position of the recognized handwritten character “h” is on the graph display screen Gf (Step A3(Yes)).

In this case, since the expression “X2+2X−3” which correlated with the recognized character “h” is stored in the character and display information correlation storing unit26c, the graph Y1which corresponds to the expression “X2+5X+4” correlated with the recognized character “h” is output and displayed on the display screen of the color display unit16, as illustrated inFIG. 8B(Step A5).

On the other hand, at Steps A4and A5, when it is determined that a character string or a graph expression which corresponds to the recognized handwritten character is not stored in the character and display information correlation storing unit26c, the process of the above steps is not performed.

As described above, when a handwritten character (symbol) is input to be superposed on displayed information such as a desired character string or graph, the displayed information is stored in correlation with the recognized handwritten character. Thereafter, when the same handwritten character is input to an area of the display screen where information is not input, the information correlated with the character is called and displayed. Therefore, it is possible to very easily perform a task of substituting various information for a character, and calling and displaying the information substituted for the character.

FIG. 10is a flowchart illustrating a function operation process accompanying the information display operation process of the graph function calculator10.

FIGS. 11A,11B, and11C are diagrams illustrating a first display operation accompanying the function operation process of the graph function calculator10.

For example, in a state where an expression “X2+X−6” desired by the user is input and displayed on the display screen of the color display unit16as illustrated inFIG. 11A(Step S1), suppose that a handwritten character string “factor” is input and recognized to be superposed on the expression as illustrated inFIG. 11B(Steps S2, S3). It is determined that a recognition result thereof is character string “factor” which is stored in the defined function table22c, and the calculator goes to the function operation process illustrated inFIG. 10(Step S4->SB).

Then, it is determined that a position of the recognized handwritten character string “factor” is superposed on the displayed numerical expression “X2+X−6” (Step B1(Yes)).

Then, it is determined that the displayed numerical expression is neither a graph (Step B1a(No)) nor an image (Step B1b(No)).

Thereby, in accordance with the defined function “output a result of factorization” which is stored in the defined function table22cand correlated with the recognized character ring “factor”, the operation process of factorizing the displayed numerical expression “X2+X−6” is performed (Step B2a), and an operation result “(X−2) (X−3)” is output and displayed on the display screen of the color display unit16together with the numerical expression, as illustrated inFIG. 11C(Step S3).

In addition, when the character string “Solve”, which is stored in the defined function table22c, is input by handwriting to be superposed on the displayed numerical expression “X2+X−6”, it is determined that the position of the recognized handwritten character string “Solve” is superposed on the displayed numerical expression “X2+X−6” (Step B1(Yes)).

Then, it is determined that the displayed numerical expression is neither a graph (Step B1a(No)) nor an image (Step B1b(No)).

Thereby, in accordance with the defined function “output a result of solving “numerical expression=0” with respect to X” which is stored in the defined function table22cand correlated with the recognized character string “Solve”, the operation process of determining a solution X which satisfies the displayed numerical expression “X2+X−6”=0 is performed (Step B2a). Then, an operation result “X=2, X=−3” is output and displayed on the display screen of the color display unit16together with the numerical expression “X2+X−6” (Step B3).

In addition, in a state where an image desired by the user is input or read out from the storing unit or the exterior and displayed on the image display screen Gp (seeFIG. 7) of the color display unit16(Step S1), suppose that a handwritten character string “Zoom” is input and recognized to be superposed on part of a numerical expression in the image (Steps S2, S3). Then, a recognition result thereof is the character string “Zoom” stored in the defined function table22c, and the calculator goes to the function operation process illustrated inFIG. 10(Step34->SB).

Then, it is determined that a position of the recognized handwritten character string “Zoom” is superposed on the displayed image (Step B1(Yes)).

Then, it is determined that the displayed image is not a graph (Step B1a(No)), but an image (Step B1b(Yes)), and the character is subjected to OCR (optical character recognition) and taken as a character string (Step B1c).

Thereby, in accordance with the defined function “enlarge” which is stored in the defined function table22cto be correlated with the recognized character string “Zoom”, performed is the process of enlarging a successive numerical expression part “D=√{square root over (. . . )})2” which is extracted in accordance with the numerical expression and character string unit extraction program22b2(or extracted in accordance with range designation by the user operation) based on the character position on which the recognized character string “Zoom” is superposed (Step B2a). Then, the enlarged numerical expression part “D=√{square root over (. . . )})2” is displayed on the color display unit16(Step B3).

FIGS. 12A,12B, and12C are diagrams illustrating a second display operation accompanying the function operation process of the graph function calculator10.

For example, as illustrated inFIG. 12A, in a state where a graph Y2according to a graph expression “Y2=X2−2” desired by the user is displayed on the graph display screen Gf of the color display unit16(Step S1), suppose that a handwritten character string “d/dx” is input and recognized to be superposed on the numerical expression as illustrated inFIG. 12B(Steps S2, S3). It is determined that a recognition result thereof is the character string “d/dx.” stored in the defined function table22c, and the calculator goes to the function operation process illustrated inFIG. 10(Step S4->SB).

Then, it is determined that a position of the recognized handwritten character string “d/dx” is superposed on the displayed graph Y1(Step B1(Yes)), and it is determined that the displayed object is a graph (Step B1a(Yes)).

Thereby, in accordance with a defined function “differentiate the graph expression with respect to X” which is stored in the defined function table22cto be correlated with the recognized character string “d/dx”, performed is the operation process of differentiating the expression “Y1=X2−2” of the displayed graph Y1with respect to X. Then, as illustrated inFIG. 12C, a graph Y3in accordance with an operation result “Y3=2X” is output and displayed on the graph display screen Gf the color display unit16(Step B2b).

On the other hand, when it is determined at Step B1that the position of the recognized handwritten characters are not superposed on the displayed object (Step B1(No)), an error message is displayed (Step B4).

As described above, when a character which is correlated with a preset defined function is input by handwriting to be superposed on displayed information such as a desired character string and graph, the displayed information is subjected to the operation process in accordance with the defined function correlated with the recognized character, and an operation result thereof is displayed. Therefore, it is possible to very easily perform the work of operating on desired information with a predefined function and displaying a result thereof.

FIG. 13is a flowchart illustrating an operator-included character substitution calling process accompanying the information display operation process of the graph function calculator10.

FIGS. 14A,14B, and14C are diagrams illustrating a first display operation accompanying the information display operation process of the graph function calculator10.

For example, as illustrated inFIG. 14A, in a state where a numerical expression “3X2+1” desired by the user is input and displayed on the display screen of the color display unit16(Step S1), suppose that a handwritten operator-included character “g′” is input and recognized in a state of being superposed on the numerical expression (Steps S2, S3). It is determined that a recognition result thereof is an operator-included character, and the calculator goes to the operator-included character substitution calling process illustrated inFIG. 13(Step S4->SC).

Then, it is determined that a position of the recognized operator-included handwritten character “g′” is superposed on the displayed numerical expression “3X2+1” (Step C1(Yes)).

Then, it is determined that the displayed image is neither a graph (Step C1a(No)) nor an image (Step C1b(No)).

Thereby, an original numerical expression, which has an operation result which is obtained by operating the recognized character part “g” in accordance with the recognized operator “′” (differentiation symbol) and corresponds to the displayed numerical expression, is subjected to an inverse operation (that is, the numerical expression “3X2−1” is integrated), and the numerical expression “X3+X+C” (C is a constant of integration) of the operation result is correlated with the recognized character “g” excluding the recognized operator “′” and stored in the character and display information correlation storing unit26c(Step C2a).

In this process as illustrated inFIG. 14C, a message M3“substitution of “g′=3X2+1, g=X3+X+C” is performed”, which indicates that the displayed numerical expression is substituted for the recognized handwritten operator-included character “g′”, is displayed on the display screen (16) on which the numerical expression “3X2+1” is displayed.

For example, when the numerical expression “3X2+1” illustrated inFIG. 14AandFIG. 145is displayed as part of an image, it is determined that the displayed image is not a graph (Step C1a(No)), but an image (Step C1b(Yes)), and the image is subjected to OCR and taken as a character string (Step C1c).

Thereby, in the same manner as above, an original numerical expression, which has an operation result which is obtained by operating the recognized character part “g” in accordance with the recognized operator “′” (differentiation symbol) and corresponds to the displayed numerical expression, is subjected to an inverse operation (that is, the numerical expression “3X2+1” is integrated), and the numerical expression “X3+X+C” (C is a constant of integration) of the operation result is correlated with the recognized character “g” excluding the recognized operator “′” and stored in the character and display information correlation storing unit26c(Step C2a).

FIGS. 15A,15B, and15C are diagrams illustrating a second display operation accompanying the operator-included character substitution calling process of the graph function calculator10.

For example, as illustrated inFIG. 15A, in a state where the graph Y2according to a numerical expression “Y2=X2−2” desired by the user is input or read out from the storing unit or the exterior and displayed on the graph display screen Gf of the color display unit16(Step S1), suppose that a handwritten operator-included character “y′” is input and recognized in a state of being superposed on the graph Y2(Steps S2, S3). It is determined that a recognition result thereof is an operator-included character, and the calculator goes to the operator-included character substitution calling process illustrated inFIG. 13(Step S4->SC).

Then, it is determined that a position of the recognized operator-included handwritten character “y′” is superposed on the displayed graph Y2(Step C1(Yes)), and it is determined that the displayed object is a graph (Step C1a(yes)).

Thereby, an original numerical expression, which has an operation result which is obtained by operating the recognized character part “y” in accordance with the recognized operator “′” (differentiation symbol) and corresponds to the numerical expression “X2−2” of the displayed graph Y2, is subjected to an inverse operation (that is, the numerical expression “X2−2” is integrated), and the numerical expression “(1/3)X3−2X+C” (C is a constant of integration) of the operation result is correlated with the recognized character part “y” excluding the recognized operator “′” and stored in the character and display information correlation storing unit26c(Step C2b).

In this process, as illustrated inFIG. 15C, a message M4“substitution of “y′=X2−2, y=(1/3)X3−2X+C” is performed”, which indicates that the displayed graph is substituted for the recognized handwritten operator-included character “y′”, is displayed on the display screen (16) on which the graph Y2is displayed.

FIGS. 16A,16B, and16C are diagrams illustrating first calling display operation accompanying the operator-included character substitution calling process of the graph function calculator10.

As illustrated inFIG. 16A, in a display state of the color display unit16in which no characters are input (Step S1), suppose that an operator-included character “∫fdx” is input by handwriting and recognized as illustrated inFIG. 16B(Steps S2, S3). It is determined that a recognition result thereof is determined as an operator-included character, and the calculator goes to the operator-included character substitution calling process illustrated inFIG. 13(Step S4->SC).

Then, it is determined that a position of the recognized handwritten operator-included character “∫fdx” is not superposed on any displayed object (Step C1(No)), and it is determined that the position of the recognized handwritten character “∫fdx” is not on a graph display picture Gf (seeFIGS. 15A,15B, and15C) (Step C3(No)).

In this case, since the numerical expression “X2+5X+4” which corresponds to the recognized character “f” is stored in the character and display information correlation storing unit26c, an operation result “(1/3)X3+(5/2)X2+4X+C” (C is a constant of integration), which is obtained by operating (integrating) the numerical expression “X2+5X+4” according to the recognized operator “∫dx” (integral symbol), is output and displayed on the display screen of the color display unit16as illustrated inFIG. 16C(Step C4).

FIGS. 17A,17B, and17C are diagrams illustrating a second calling display operation accompanying the operator-included character substitution calling process of the graph function calculator10.

for example, as illustrated inFIG. 17A, in a display state of the graph display picture Gf in which coordinate axes and a graph Y3are input to the color display unit16(Step S1), suppose that an operator-included character “y′” is input by handwriting and recognized as illustrated inFIG. 17B(Steps S2, S3). It is determined that a recognition result thereof is determined as an operator-included character, and the calculator goes to the operator-included character substitution calling process illustrated inFIG. 13(Step S4->SC).

Then, it is determined that a position of the recognized handwritten operator-included character “y′” is not superposed on any displayed object (Step C1(No)), and it is determined that the position of the recognized handwritten character “y′” is on the graph display picture Gf (Step C3(Yes)).

In this case, since the numerical expression “(1/3) X3+2X+C” which corresponds to the recognized character “y” is stored in the character and display information correlation storing unit26c, the graph Y2of an operation result “X2−2” which is obtained by operating (differentiating) the numerical expression “(1/3) X3+2X+C” according to the recognized operator “′” (differentiation symbol) is output and displayed on the graph display picture Gf of the color display unit16as illustrated inFIG. 17C(Step C5).

On the other hand, at Steps C4and C5, when it is determined that the character string or expression which corresponds to the character part of the recognized handwritten operator-included character is not stored in the character and display information correlation storing unit26c, the process of the related step is not performed.

As described above, when an operator-included character is input by handwriting to be superposed on displayed information such as a desired character string or graph, an original numerical expression which has an operation result that obtained in accordance with the recognized handwritten operator and corresponds to the displayed character string (numerical expression) or an expression of the displayed graph, is subjected to an inverse operation, and operation result thereof is stored to be correlated with the recognized operator-included character excluding the operator. Then, when the same operator-included character is input by handwriting to a non-input area of the display screen, the numerical expression which is correlated with the character without the operator is called, and an operation result of the numerical expression obtained in accordance with the operator is displayed. Therefore, for example, it is possible to very easily perform the work of returning a numerical expression or graph expression subjected to a certain operation to a numerical expression before the operation substituting the numerical expression for a desired character without an operator corresponding to the operation, calling the numerical expression substituted for the character without the operator by inputting the operator-included character by handwriting, and performing an operation according to the operator and displaying an operation result thereof.

Therefore, even in the graph function calculator10equipped with various functions and having the above structure, it is possible to easily designate and execute a function desired by the user by gesture operation for the touch-panel color display unit16, without complicated and troublesome operations.

The operation methods of the graph function calculator10described in the above embodiment, that is, methods such as the information display operation process illustrated in the flowchart ofFIG. 3, the character substitution calling process accompanying the information display operation process and illustrated in the flowchart ofFIG. 4, the function operation process accompanying the information display operation process and illustrated in the flowchart ofFIG. 10, and the operator-included character substitution calling process accompanying the information display operation process and illustrated in the flowchart ofFIG. 13may be stored and distributed, as a program executable by a computer, in an external storage medium (23) such as a memory card (such as a ROM card and a RAM card), a magnetic disk (such as a floppy disk and a hard disk), an optical disk (such as a CD-ROM and a DVD), and a semiconductor memory. In addition, the computer (21) of the electronic computer (10) equipped with a handwriting input unit (16T) can realize the information display operation process using the handwritten character recognizing function explained in the above embodiment, and execute a similar process by the above method, by reading the program stored in the external storage medium (23) into the storing device (22), and controlling the operation thereof by the read program.

In addition, data of the program to realize the above method can be transmitted in a form of a program code through a communication network (public line) N. It is possible to realize the information display operation process using the handwritten character recognizing function, by taking the program data into a computer (21) of an electronic computer (10) equipped with a handwriting input unit (16) through a communication device (25) connected to the communication network N.

The present invention is not limited to the above embodiment, but can be variously modified within a range not departing from the gist of the invention. In addition, the above embodiment includes inventions of various stages, and various inventions can be extracted by proper combinations of a plurality of disclosed constituent elements. For example, even when some constituent elements are removed from all the constituent elements disclosed in each embodiment or some of the constituent elements are combined in a different form, the structure in which the constituent elements are removed or combined can be extracted as an invention when it can solve the problem and can obtain effects.