Abstract:
A handwriting recognition method and a handwriting recognition device are provided to recognize a character sequence continuously inputted by a user for convenience. The present method comprises steps of calculating various features of the inputted character sequence which include single character recognition accuracy features and space geometry features of different stroke combinations in the inputted character sequence, calculating segmentation reliabilities of respective stroke combinations in different segmented patterns by using a probabilistic model in which coefficients of the probabilistic model are estimated by a parameter estimation method through sample trainings, recognizing characters in different writing patterns by using a multiple-template matching method when performing single character recognition of the stroke combinations, searching for the best segmentation path and conducting post-processing to optimize the recognition results. The present method and device have advantages of simple structure, low hardware requirement, fast recognition speed and high recognition accuracy and can be implemented in an embedded system.

Description:
TECHNICAL FIELD 
       [0001]    The present invention relates generally to character input. More specifically, the present invention relates to a handwriting recognition method and corresponding device that may recognize writing-box-free character sequence inputted continuously by user with improved input efficiency. 
       BACKGROUND ART 
       [0002]    At present, handwriting recognition modules have been widely used in all kinds of electronic devices such as mobile phones. It is convenient for user to interact with the electronic devices. With the handwriting recognition modules, user needn&#39;t to learn other character input method by pressing keyboard. 
         [0003]    Non Patent Literature 1 (see below) discloses a handwriting recognition method which designs physical feature (off-stroke features) of segmented patterns to recognize a writing-box-free character sequence. In this method, off-stroke information could be obtained from the last sampling point of the previous stroke and the first sampling point of the next stroke, which is represented as the dotted line shown in  FIG. 1 . The physical information further includes information such as width/height of segmented patterns and handwriting time of the corresponding segmented patterns. In this method, the physical information includes shape features, position features and gap features of the segmented patterns; lengths of strokes; an average distance of off-strokes; an average time of off-strokes; distances of off-strokes; sine and cosine of angles of the off-strokes and off-stroke gaps. This method focuses on off-stroke process from the end point of the previous stroke to the start point of the current stroke and thus recognizes handwriting input. 
         [0004]    This handwriting recognition method assumes that even joined-up handwriting occurs between different characters, the distance and time period of off-strokes between characters shall both be larger than those of the off-strokes within the characters. This method also assumes that each stroke distribution fits a normal distribution. Based on such assumptions, this handwriting recognition method calculates segmented-pattern likelihood based on means and variances of the features by using a probabilistic model. Finally, this method determines a best segmentation path by using dynamic programming (DP). 
         [0005]    One problem existing in the above Non Patent Literature 1 is that the segmentation of the handwriting character sequence relies upon handwriting time of each stroke. The time period of off-strokes is a very important feature in this method. This method assumes that the larger the time period of off-strokes between segmented patterns is, the higher the segmentation accuracy is. The above assumption is reasonable when user writes at a relatively constant speed. However, during the utilizations, user usually writes at different speeds, for example, writing fast for a while and slowly for a subsequent while. Therefore, if user changes writing speed during handwriting process, it will be very difficult for the method disclosed in Non Patent Literature 1 to accurately segment the handwritings. 
         [0006]    Another problem existing in the above Non Patent Literature 1 is that this method only uses geometry features and time features to determine if the segmentation is correct. This method assumes that the distance of off-strokes between characters is larger than the distance of off-stroke between strokes within the characters. However, such an assumption is not always correct. The Non Patent Literature 1 lists several typical examples of segmentation errors as shown in  FIG. 2 . It can be seen from  FIG. 2  that the distance of off-strokes between certain characters is smaller than that between strokes within characters. As it is shown in the first example in  FIG. 2 , ‘ 5 ’ is over segmented due to excessively large gap between strokes within the character. But as it is shown in the second and third examples, when the distance between characters of an inputted character sequence changes dramatically and sizes of the characters are different remarkably, segmentation errors occur. 
       CITATION LIST 
     Non Patent Literature 1 
       [0000]    
       
         “Online Character Segmentation Method for Unconstrained Handwriting Strings Using Off-stroke Features” (Source: Hitachi Ltd. in the Tenth International Workshop on Frontiers in Handwriting Recognition, La Baule, France, 2006) 
       
     
       SUMMARY OF INVENTION 
       [0008]    The technical object of the present invention is to provide a handwriting recognition method and device which are able to recognize a character sequence continuously inputted by user in irrespective of writing speed changes. 
         [0009]    According to one aspect of the present invention, a handwriting recognition method is proposed to recognize a writing-box free character sequence continuously inputted by user. The method comprises: calculating features relative to single character recognition accuracies of different stroke combinations in the inputted character sequence, which is based on single character recognition results of different stroke combinations and sub-stroke combinations formed by segmenting strokes in the stroke combinations; determining space geometry features of the different stroke combinations according to space geometry relationships of the sub-stroke combinations formed by segmenting strokes in the stroke combinations; determining segmentation reliabilities of respective stroke combinations of the inputted character sequence in different segmented patterns based on the features relative to single character recognition accuracies and the space geometry features; determining segmentation paths based on the segmentation reliabilities, and presenting to user the character sequence recognition results according to the determined segmentation paths. 
         [0010]    According to the other aspect of the present invention, a handwriting recognition device is proposed to recognize a writing-box free character sequence continuously inputted by user. The handwriting recognition device comprises: a handwriting input unit configured to collect the character sequence continuously inputted by user; a single character recognition unit configured to recognize different stroke combinations in the character sequence and to obtain single character recognition results; a segmentation unit configured to calculate features relative to single character recognition accuracies of different stroke combinations in the inputted character sequence based on the single character recognition results of different stroke combinations and sub-stroke combinations formed by segmenting strokes in the stroke combinations and determine space geometry features of the different stroke combinations according to space geometry relationships of the sub-stroke combinations, to determine segmentation reliabilities of respective stroke combinations of the inputted character sequence in different segmented patterns based on the features relative to single character recognition accuracies and the space geometry features, and to determine segmentation paths based on the segmentation reliabilities; and a display control unit configured to control a display screen to present user the character sequence recognition results according to the determined segmentation paths. 
         [0011]    Because of adopting writing-box free manner, user can continuously input a character sequence so as to improve handwriting input efficiency. As to the input method which requires the user to write each character within each writing-box, intermission between handwriting characters often interrupts the user&#39;s thinking to decrease the input speed. The method requiring each character to be written within the prescribed writing-boxes (for example, the commonly two-box input method in current mobile phone requires user to switch between two writing-boxes frequently) also changes handwriting habit of the user and reduces handwriting input efficiency. However, without changing handwriting habit, the method and device according to an embodiment of the present invention allow continuous character sequence input and allow recognition results&#39; output separately or overall. 
         [0012]    During calculating the segmentation reliabilities of the character sequence, the method and device of the present embodiment consider that not only the commonly used space geometry features but also the single character accuracy of merged stroke combination and that of sub-stroke combination, as a result, it can achieve correct segmentation in cases that the correct segmentation is difficult to be performed by traditional technology, for example, strokes in different characters are partially overlapping in space, or the stroke gaps in a character is too big. 
         [0013]    Moreover, the method and device of the present embodiment do not rely on the input time of each stroke when performing the character sequence segmentation, so it can adapt to different input habits of users. Even a user inputs the character sometimes fast and sometimes slow, the segmentation accuracy will not be decreased according to the method and device of the present embodiment. 
         [0014]    In addition, the space geometry features of the stroke combination adopted in the method and device of the present embodiment are normalized features based on the estimated average width or height of characters, so the device of present embodiment can adapt to a character sequence with any size. Since multiple-template training and multiple-template matching methods are adopted in the single character recognition unit, the characters in different writing patterns by different users (e.g., simplified characters of Kanji by Chinese) can be accurately recognized by the method and device of the present embodiment. Furthermore, the method and device of the present embodiment utilize the language model and dictionary matching so that the device has the functions of spell check and word correction. 
         [0015]    Finally, the recognition objects of the method and device of the present embodiment can be English word, Japanese kana combination, Chinese sentence, Korean character combination, and etc. The timing of performing handwriting recognition can be designated arbitrarily. The recognition result can be continually updated while the user inputs the character sequence, or the recognition results can be displayed after the user finishes the whole character sequence input. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0016]    The foregoing and other objectives, features, and advantages of the invention will be more readily understood upon consideration of the following detailed description of the invention, taken in conjunction with the accompanying drawings. 
           [0017]      FIG. 1  illustrates a conventional character recognition method based on off-stroke features. 
           [0018]      FIG. 2  illustrates problems occurring when recognizing characters based on the off-stroke features in prior art. 
           [0019]      FIG. 3  is a structure schematic diagram illustrating a handwriting recognition device according to an embodiment of the present invention. 
           [0020]      FIG. 4  is a flowchart illustrating a sample training process of the handwriting recognition device according to an embodiment of the present invention. 
           [0021]      FIG. 5A  is a schematic diagram illustrating stroke combinations and their sub-stroke combinations in the handwriting recognition device according to an embodiment of the present invention. 
           [0022]      FIG. 5B  is a schematic diagram illustrating stroke combinations and their sub-stroke combinations in the handwriting recognition device according to an embodiment of the present invention. 
           [0023]      FIG. 5C  is a schematic diagram illustrating stroke combinations and their sub-stroke combinations in the handwriting recognition device according to an embodiment of the present invention. 
           [0024]      FIG. 5D  is a schematic diagram illustrating stroke combinations and their sub-stroke combinations in the handwriting recognition device according to an embodiment of the present invention. 
           [0025]      FIG. 6A  is a schematic diagram explaining space geometry features of the stroke combinations in the handwriting recognition device according to an embodiment of the present invention. 
           [0026]      FIG. 6B  is a schematic diagram explaining space geometry features of the stroke combinations in the handwriting recognition device according to an embodiment of the present invention. 
           [0027]      FIG. 6C  is a schematic diagram explaining space geometry features of the stroke combinations in the handwriting recognition device according to an embodiment of the present invention. 
           [0028]      FIG. 6D  is a schematic diagram explaining space geometry features of the stroke combinations in the handwriting recognition device according to an embodiment of the present invention. 
           [0029]      FIG. 7  is a schematic diagram illustrating different writing patterns for the same character according to an embodiment of the present invention. 
           [0030]      FIG. 8  is another schematic diagram illustrating different writing patterns for the same character according to an embodiment of the present invention. 
           [0031]      FIG. 9A  is a schematic diagram illustrating multiple-template training and multiple-template matching according to an embodiment of the present invention. 
           [0032]      FIG. 9B  is a schematic diagram illustrating multiple-template training and multiple-template matching according to an embodiment of the present invention. 
           [0033]      FIG. 9C  is a schematic diagram illustrating multiple-template training and multiple-template matching according to an embodiment of the present invention. 
           [0034]      FIG. 10  is a function curve diagram illustrating a Logistic Regression Model according to an embodiment of the present invention. 
           [0035]      FIG. 11  is a flowchart illustrating a handwriting recognition procedure according to an embodiment of the present invention. 
           [0036]      FIG. 12A  is a schematic diagram illustrating segmentations through different segmentation paths according to an embodiment of the present invention. 
           [0037]      FIG. 12B  is a schematic diagram illustrating segmentations through different segmentation paths according to an embodiment of the present invention. 
           [0038]      FIG. 12C  is a schematic diagram illustrating segmentations through different segmentation paths according to an embodiment of the present invention. 
           [0039]      FIG. 13A  is a schematic diagram illustrating handwriting recognition results of the handwriting recognition device according to an embodiment of the present invention. 
           [0040]      FIG. 13B  is a schematic diagram illustrating handwriting recognition results of the handwriting recognition device according to an embodiment of the present invention. 
           [0041]      FIG. 13C  is a schematic diagram illustrating handwriting recognition results of the handwriting recognition device according to an embodiment of the present invention. 
           [0042]      FIG. 13D  is a schematic diagram illustrating handwriting recognition results of the handwriting recognition device according to an embodiment of the present invention. 
           [0043]      FIG. 14  is a schematic diagram illustrating an application of the handwriting recognition method according to an embodiment of the present invention on an electronic dictionary. 
           [0044]      FIG. 15  is a schematic diagram illustrating candidates of at least a part of recognition results provided to the user for selection and error correction according to an embodiment of the present invention. 
           [0045]      FIG. 16A  is a schematic diagram illustrating applications of the handwriting recognition method according to an embodiment of the present invention on a notebook computer. 
           [0046]      FIG. 16B  is a schematic diagram illustrating applications of the handwriting recognition method according to an embodiment of the present invention on a mobile phone. 
       
    
    
     DESCRIPTION OF EMBODIMENTS 
       [0047]    Preferred embodiments will be explained by referring to the accompanying drawings. In the drawings, same reference numerals will be used for indicating same or similar components, although illustrated in different figures. Unnecessary parts and functions for the present invention will be omitted for brevity so as to avoid confusion in understanding. 
         [0048]      FIG. 3  is a structure schematic diagram illustrating a handwriting recognition device according to an embodiment of the present invention. 
         [0049]    As shown in  FIG. 3 , the handwriting recognition device according to an embodiment of the present invention is used to recognize a writing-box-free character sequence continuously inputted by user. The handwriting recognition device consists of a handwriting input unit  110  for collecting scripts of the user and digitizing it as an input script signal; a handwriting script storage unit  120  for saving the input script signal generated by the handwriting input unit  110  and a character sequence recognition unit  130  for recognizing the inputted character sequence. The character sequence recognition unit  130  consists of three sub-units, segmentation unit  132 , single character recognition unit  131  and post-processing unit  133 . 
         [0050]    Since adopting writing-box-free input, the user can continuously input a character sequence so as to improve handwriting input efficiency. A recognition result will be real-time displayed during the user input procedure. Alternatively, the overall recognition result will be provided after the user inputs the completed sentence. In traditional input methods that require the user to write characters within the writing-box, intermission between handwriting characters often interrupts the user&#39;s thinking and decrease the input speed. The method requiring each character to be written within the prescribed writing-boxes (for example the two-box input method commonly used in current mobile phones requires user to switch between two writing-boxes frequently) also changes handwriting habit of user and reduces handwriting input efficiency. However, without changing the handwriting habit, the method and device according to an embodiment of the present invention allow continuous character sequence input and allow recognition results&#39; output separately or overall. 
         [0051]    The segmentation unit  132  extracts various space geometry features of respective stroke combinations in the inputted character sequence from the input script signal, obtains single character recognition results and single character recognition accuracies of respective stroke combinations by calling the single character recognition unit  131 , then calculates “segmentation reliabilities” based on a Logistic Regression Model and obtains the best N segmented patterns by using an N-best algorithm, which will be described detailedly in the later part. 
         [0052]    The post-processing unit  133  corrects the character sequence recognition results of the segmentation unit  132  by utilizing language model and matching dictionary database. 
         [0053]    As shown in  FIG. 3 , the handwriting recognition device according to an embodiment of the present invention further includes a display control unit  150  and a candidate selection unit  140 . On the one hand, the display control unit  150  controls the system to display the scripts and present to user on a display screen when the user inputs strokes in the handwriting input unit  110 , and on the other hand, the display control unit  150  displays recognition candidates generated by the character sequence recognition unit  130  on the display screen for user selection. The candidate selection unit  140  selects, under the user operation, the character sequence or single character from the corresponding candidates and provides the recognition results to user or provides to other applications, for example, the application of dictionary to explain the recognition results. 
         [0054]    According to an embodiment of the present invention, the intercept and the regression coefficients of the Logistic Regression Model utilized in the character sequence recognition unit  130  are estimated by data trainings of the samples. 
         [0055]      FIG. 4  is a flowchart illustrating a training process of the handwriting recognition device according to an embodiment of the present invention. 
         [0056]    According to an embodiment of the present invention, samples in the data training includes not only single character samples but also each strokes in the characters and a combination of several strokes within a character or a combination of strokes within two different characters. Each of the above samples is defined as one kind of stroke combination. 
         [0057]    As shown in  FIG. 4 , in step S 10 , handwriting scripts are collected. In Step S 11 , the collected data are added to a corresponding stroke combination class. Then pre-processing is conducted in Step S 12  and stroke combination features are calculated in Step S 13 . 
         [0058]    The features for sample training are the m-dimensional feature (x 1 , x 2 , . . . , x M ) in the Logistic Regression Model. The stroke combination features include a gap between the bounding boxes of the sub-stroke combination, a width of merged sub-stroke combination, a vector and distance between sub-stroke combinations, a single character recognition accuracy of merged sub-stroke combination, a difference between merged recognition accuracy and recognition accuracies of the sub-stroke combinations, a ratio of the first candidate&#39;s single character accuracy to other candidate&#39;s single character accuracy of the merged sub-stroke combination, and so on. 
         [0059]    Before the feature calculation in Step S 13 , a pre-processing should be performed in Step S 12 , which estimates a character&#39;s average height H avg  and character&#39;s average width W avg  according to heights and widths of the inputted character sequence as a normalization preparation for the space geometry features of the stroke combinations so that the handwriting recognition device according to an embodiment of the present invention could be applied to a character sequence with any size. 
         [0060]    The concept of sub-stroke combination (“sub-stroke” for short hereinafter) according to an embodiment of the present invention will be explained by taking an example of segmentation from the kth stroke to the k+3th stroke in a character sequence. From the kth stroke, there are four possible segmented patterns as shown in  FIGS. 5A ,  5 B,  5 C and  5 D. 
         [0061]    1) one-stroke combination only includes the kth stroke and does not have sub-strokes. 
         [0062]    2) two-stroke combination includes the kth and k+1th sub-strokes. 
         [0063]    3) three-stroke combination has two sub-stroke classification modes. 
         [0064]    Mode 1: the previous sub-stroke is the kth stroke and the next sub-stroke is the stroke combination of the k+1th and k+2th strokes. 
         [0065]    Mode 2: the previous sub-stroke is the stroke combination of the kth and k+1th strokes and the next sub-stroke is the k+2th stroke. 
         [0066]    4) four-stroke combination has three sub-stroke classification modes. 
         [0067]    Mode 1: the previous sub-stroke is the kth stroke and the next sub-stroke is the stroke combination of the k+1th, k+2th and k+3th strokes. 
         [0068]    Mode 2: the previous sub-stroke is the stroke combination of the kth and k+1th strokes and the next sub-stroke is the stroke combination of the k+2th and k+3th strokes. 
         [0069]    Mode 3: the previous sub-stroke is the stroke combination of the kth, k+1th and k+2th strokes and the next sub-stroke is the k+3th stroke. 
         [0070]    It can be seen from the embodiment of the present invention that the sub-stroke combination could be different combinations formed by sequentially segmenting strokes in a certain “stroke combination”. For example, for a stroke combination in a writing order of “k, k+1, k+2”, its sub-stroke combination could be the “Sub-stroke Class 1” generated by segmenting between the strokes “k” and “k+1” or the “Sub-stroke Class 2” generated by segmenting between the strokes “k+1” and “k+2”, as shown in  FIG. 5C . 
         [0071]    In the device according to an embodiment of the present invention, various features of the stroke combination, including single character recognition accuracy features and space geometry features of the sub-stroke combination, are calculated for all possible stroke combinations in the character sequence. The various detailed features are listed as follows: 
         [0072]    (a) a single character recognition accuracy, C merge , of merged sub-strokes: the larger it is, the larger the possibility of merging into a single character is; 
         [0073]    (b) a difference, (2*C merge −C str1 −C str2 ), between merge recognition accuracy C merge  and single character recognition accuracies, C str1  and C str2 , of two sub-strokes. If the difference is larger than 0, it means that a possibility of merging into a single character from the two strokes is larger than a possibility of two sub-strokes being single characters respectively. The larger the difference is, the larger the possibility of merging into a single character is; 
         [0074]    (c) a ratio of the first candidate&#39;s single character recognition accuracy of the merged sub-strokes (C merge ) to other candidate&#39;s single character recognition accuracy of the merged sub-strokes (C mergeT ) (T represents the Tth candidate of the single character recognition and the value of T can be set): if the ratio is relatively large, it means that a matching distance between the merged stroke combination and the first candidate of the single character recognition is quite near and matching distances between the merged stroke combination and other candidates are far, which indicates that the possibility of merging into a single character is relatively large; 
         [0075]    (d) a gap between two bounding boxes of sub-strokes, gap/W avg  (or gap/H avg ): the smaller the gap of the sub-strokes is, the larger the possibility of forming a single character after merge is. If the gap is a negative value, the possibility of forming a single character after merge is much larger; 
         [0076]    (e) a merged sub-stroke width, W merge /W avg  (or W merge /H avg ): the smaller the merged width is, the larger the possibility of forming a single character is; 
         [0077]    (f) a vector, V s2-e1 /W avg  (or V s2-e1 /H avg ), between the end sampling point of the previous sub-stroke and the start sampling point of the next sub-stroke; 
         [0078]    (g) a distance, d s2-e1 /W avg  (or d s2-e1 /H avg ), between the end sampling point of the previous sub-stroke and the start sampling point of the next sub-stroke; 
         [0079]    (h) a distance, d s2-s1 /W avg  (or d s2-s1 /H avg ), between the start sampling point of the previous sub-stroke and the start sampling point of the next sub-stroke. 
         [0080]    In the above features, “/” represents a division sign, and W avg  and H avg  represent the estimated character average width and character average height during the pre-processing procedure. The space geometry features of (d)-(h) refer to  FIG. 6A-6D  and dots in the figures represent a start point of each stroke. 
         [0081]    For the above features (a), (b) and (c), the single character recognition accuracy C merge  and other candidate accuracy C mergeT  of the merged sub-strokes, and single character recognition accuracies, C str1  and C str2 , of two sub-strokes are obtained by calling the single character recognition unit in Step S 14 . 
         [0082]    The single character recognition unit according to an embodiment of the present invention adopts a template matching method to recognize the single character. The single character recognition accuracy is determined by the distance of the template matching. The smaller the distance is, the larger the accuracy is. In the sample training of the single character recognition, machine learning algorithms (for example, GLVQ) are adopted to generate feature templates. The single character feature vector includes “stroke direction distribution features”, “grid stroke features” and “peripheral direction features”. Before the feature extraction, pre-processing is conducted, which includes operations such as “isometric smooth”, “centroid normalization” and “nonlinear normalization” so as to regulate the features of the samples. In the template matching, a “multi-stage cascade matching” method is adopted to filter candidates out stages by stages so as to improve matching speed. The above single character recognition method is disclosed in Chinese patent application publication No. CN101354749A and all contents in this application are incorporated into the present invention for reference. 
         [0083]    During practical writing procedure, different users may usually write the same character in different writing patterns. For example, an English letter “A” may have a plurality of writing patterns as shown in  FIG. 7 . 
         [0084]    A Japanese kanji “         ” may have three writing patterns as shown in  FIG. 8 , in which the latter two writing patterns are simplified characters. 
         [0085]    Therefore, in order to improve robustness of the handwriting recognition, a “multiple-template training” method is adopted in the device according to an embodiment of the present invention so as to perform individual training for different writing patterns of the same character so that the “multiple-template matching” method could be used for recognizing characters in various writing patterns. In order to perform the “multiple-template training”, the collected samples are firstly classified according to their different writing patterns. For example, for the above mentioned Kanji “         ”, the present embodiment adopts three formats of samples shown in  FIGS. 9A ,  9 B and  9 C to form the multiple-template training during the sample training. 
         [0086]    As shown in  FIG. 4 , in Step S 15 , coefficients of the Logistic Regression Model are calculated. The key of realizing handwriting character sequence&#39;s recognition is correctly segmenting the character sequence. The device and method of an embodiment of the present invention calculate segmentation reliabilities of respective stroke combinations of the inputted character sequence in various kinds of segmented patterns according to various features of the inputted character sequence. A segmentation reliability formula of the present embodiment adopts the Logistic Regression Model (LRM) which is: 
         [0000]    
       
         
           
             
               
                 
                   
                     f 
                      
                     
                       ( 
                       Y 
                       ) 
                     
                   
                   = 
                   
                     
                       1 
                       
                         1 
                         + 
                         
                            
                           
                             - 
                             Y 
                           
                         
                       
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0087]    A function curve diagram of the Logistic Regression Model is shown in  FIG. 10 . When Y changes in a range of −∞˜+∞, a value of f(Y) ranges from 0 to 1, which means that the segmentation reliability ranges from 0% to 100%. When Y=0, f(Y)=0.5, which indicates that the segmentation reliability is 50%. 
         [0088]    In the above Logistic Regression Model, 
         [0000]        Y=g ( X )=β 0 +β 1   x   1 +β 2   x   2 + . . . +β m   x   m   (2).
 
         [0089]    X=(x 1 , x 2 , . . . , x m ) is a risk factor of the Logistic Regression Model. When the device and method of the present embodiment calculate the segmentation reliabilities, X=(x 1 , x 2 , . . . , x m ) represents as an m-dimensional feature of the stroke combination. (β 0 , β 1 , β 2 , . . . , β m ) represents an intercept and regression coefficients of the Logistic Regression Model. 
         [0090]    After calculating m-dimensional features of all possible stroke combinations in the character sequence, the device and method of the present embodiment adopt a maximum likelihood estimation method (or other parameter estimation methods such as least square estimation method) to estimate the intercept β 0  and regression coefficients (β 1 , β 2 , . . . , β m ) of the Logistic Regression Model for the segmentation reliabilities. 
         [0091]    Assuming that there are n stroke combination samples and observation values are (Y 1 , Y 2 , . . . , Y n ) respectively. For the ith stroke combination, the m-dimensional feature is X i =(x i1 , x i2 , . . . , x im ) and the observation value is Y i . N regression relationships may be expressed as: 
         [0000]    
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           
                             Y 
                             1 
                           
                           = 
                           
                             
                               β 
                               0 
                             
                             + 
                             
                               
                                 β 
                                 1 
                               
                                
                               
                                 X 
                                 11 
                               
                             
                             + 
                             
                               
                                 β 
                                 2 
                               
                                
                               
                                 X 
                                 12 
                               
                             
                             + 
                             … 
                             + 
                             
                               
                                 β 
                                 m 
                               
                                
                               
                                 X 
                                 
                                   1 
                                    
                                   m 
                                 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             Y 
                             2 
                           
                           = 
                           
                             
                               β 
                               0 
                             
                             + 
                             
                               
                                 β 
                                 1 
                               
                                
                               
                                 X 
                                 21 
                               
                             
                             + 
                             
                               
                                 β 
                                 2 
                               
                                
                               
                                 X 
                                 22 
                               
                             
                             + 
                             … 
                             + 
                             
                               
                                 β 
                                 m 
                               
                                
                               
                                 X 
                                 
                                   2 
                                    
                                   m 
                                 
                               
                             
                           
                         
                       
                     
                     
                       
                         … 
                       
                     
                     
                       
                         
                           
                             Y 
                             n 
                           
                           = 
                           
                             
                               β 
                               0 
                             
                             + 
                             
                               
                                 β 
                                 1 
                               
                                
                               
                                 X 
                                 
                                   n 
                                    
                                   
                                       
                                   
                                    
                                   1 
                                 
                               
                             
                             + 
                             
                               
                                 β 
                                 2 
                               
                                
                               
                                 X 
                                 
                                   n 
                                    
                                   
                                       
                                   
                                    
                                   2 
                                 
                               
                             
                             + 
                             … 
                             + 
                             
                               
                                 β 
                                 m 
                               
                                
                               
                                 
                                   X 
                                   nm 
                                 
                                 . 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0092]    During the sample training, for the ith stroke combination, if the stroke combination is reliable, let 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       f 
                       i 
                     
                     = 
                     
                       
                         f 
                          
                         
                           ( 
                           
                             Y 
                             i 
                           
                           ) 
                         
                       
                       = 
                       
                         
                           1 
                           
                             1 
                             + 
                             
                                
                               
                                 - 
                                 
                                   Y 
                                   i 
                                 
                               
                             
                           
                         
                         -&gt; 
                         1 
                       
                     
                   
                   , 
                   
                     
                       f 
                        
                       
                         ( 
                         
                           Y 
                           i 
                         
                         ) 
                       
                     
                     &gt; 
                     0.5 
                   
                   , 
                   
                     i 
                     . 
                     e 
                     . 
                   
                   , 
                   
                     
                       
                         Y 
                         i 
                       
                       &gt; 
                       0 
                     
                     ; 
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0000]    if the stroke combination is not reliable (i.e., this stroke combination pattern is not correct), let 
         [0000]                f   i     =       f        (     Y   i     )       =       1     1   +          -     Y   i             -&gt;   0         ,           f ( Y   i )&lt;0.5, i.e., Y i &lt;0  (5).
 
         [0093]    Substituting Y=g(X)=β 0 +β 1 x 1 +β 2 x 2 + . . . +β m x m  into the Logistic Regression Model formula, then 
         [0000]    
       
         
           
             
               
                 
                   
                     f 
                      
                     
                       ( 
                       Y 
                       ) 
                     
                   
                   = 
                   
                     
                       1 
                       
                         1 
                         + 
                         
                            
                           
                             - 
                             Y 
                           
                         
                       
                     
                     = 
                     
                       
                         1 
                         
                           1 
                           + 
                           
                              
                             
                               - 
                               
                                 g 
                                  
                                 
                                   ( 
                                   X 
                                   ) 
                                 
                               
                             
                           
                         
                       
                       = 
                       
                         π 
                          
                         
                           ( 
                           X 
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
         [0000]    is obtained. 
         [0094]    Setting p i =P(f i =1|X i ) as a probability of f i =1, then a conditional probability of f i =0 is P(f i =0|X i )=1−p i . Thus a probability of one observation value is P(f i )=p i   f     i   (1−p i ) (1-f     i     ) . 
         [0095]    Since respective observations are independent, their joint distribution can be represented as a product of respective marginal distributions, which is 
         [0000]    
       
         
           
             
               
                 
                   
                     1 
                      
                     
                       ( 
                       β 
                       ) 
                     
                   
                   = 
                   
                     
                       ∏ 
                       
                         i 
                         = 
                         1 
                       
                       n 
                     
                      
                     
                       
                         
                           
                             
                               π 
                                
                               
                                 ( 
                                 
                                   X 
                                   i 
                                 
                                 ) 
                               
                             
                             
                               f 
                               i 
                             
                           
                            
                           
                             [ 
                             
                               1 
                               - 
                               
                                 π 
                                  
                                 
                                   ( 
                                   
                                     X 
                                     i 
                                   
                                   ) 
                                 
                               
                             
                             ] 
                           
                         
                         
                           1 
                           - 
                           
                             f 
                             i 
                           
                         
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
         [0096]    The above equation is called as a likelihood function for n observations. The object is to estimate the parameters which maximize this function value. Therefore, the key of the maximum likelihood estimation is to estimate the most suitable parameters (β 0 , β 1 , β 2 , . . . , β m ) which maximize the above likelihood function. Taking logarithm to the above likelihood function, then a log-likelihood function is obtained. A derivative of the log-likelihood function is then calculated to get m+1 likelihood equations. Finally, Newton-Raphson method is applied to iteratively calculate these m+1 likelihood equations and thus coefficients (β 0 , β 1 , β 2 , . . . , β m ) in the Logistic Regression Model can be obtained and can be saved in the device of present embodiment for using in the recognition procedure. 
         [0097]    According to another embodiment of the present invention, segmentation reliabilities of the inputted character sequence in respective segmented patterns can also be calculated with a normal distribution model. 
         [0098]      FIG. 11  is a flowchart illustrating a handwriting recognition procedure according to an embodiment of the present invention. As shown in  FIG. 11 , in Step S 20 , the user inputs handwriting and the strokes of the character sequence are collected in the handwriting input unit  110 . Then in Step S 21 , collected scripts are saved in the handwriting script storage unit  120  and are displayed in the user interface by the display control unit  150  in Step S 22 . 
         [0099]    Then, for the strokes saved in the script storage unit, the character sequence recognition unit  130  performs operations of “pre-processing”, “stroke combination feature calculation”, “single character recognition”, “segmentation reliability calculation”, “segmentation optimum path selection” and “recognition post-processing” in the Steps S 23 , S 24 , S 25 , S 26 , S 27  and S 28  respectively. 
         [0100]    In details, execution procedures in Steps S 23 , S 24  and S 25  are similar to those steps in the above Logistic Regression Model coefficients estimation by the sample training. In Step S 23 , a pre-processing is performed to estimate the character&#39;s average height H avg  and character&#39;s average width W avg  according to heights and widths of the character sequence as a normalization preparation for the space geometry features of the stroke combination so that the handwriting recognition device according to an embodiment of the present invention could be applied to the character sequence with any size. 
         [0101]    In Step S 24 , various features, including single character recognition accuracy features and space geometry features of the sub-stroke combination, of the stroke combination are calculated for all possible stroke combinations in the character sequence. 
         [0102]    In Step S 25 , the single character recognition unit is called to obtain the single character recognition accuracy C merge  and other candidate accuracy C mergeT  of the merged sub-strokes, and single character recognition accuracies C str1  and C str2  of two sub-strokes. 
         [0103]    In Step S 26 , by utilizing above formulas (1) and (2) of the Logistic Regression Model, the method according to the present embodiment calculates the segmentation reliabilities f(Y) of respective stroke combinations for the inputted character sequence in various segmented patterns based on the respective features (X=(x 1 , x 2 , . . . , x m )) of the inputted character sequence and coefficients (β 0 , β 1 , β 2 , . . . , β m ) obtained in the sample training. 
         [0104]    In Step S 27 , the method according to the present embodiment calculates the most possible N segmentation paths using the N-Best method. A start point of each stroke is defined as an element-node and a path consisting of the element-node or an element-node combination is a corresponding stroke combination. A cost function for each partial path is C(Y)=1−f(Y), in other words, the higher the segmentation reliability is, the smaller the value of the cost function for the partial path is. The N-Best method is used to select the best N paths which make the sum of the values of the cost function for all passed paths to be the least, second least, . . . . Nth least. 
         [0105]    The N-Best method can be implemented by various means, for example, multiple candidates can be generated by combining dynamic programming (DP) method and stack algorithms. In the present embodiment, the N-Best method includes two steps: forward search and backward search. The forward search adopts an improved Viterbi algorithm (Viterbi algorithm is a dynamic programming method for searching the most possible implicit state sequence) for recording states of the best N partial paths transferred to each element-node (i.e., a sum of cost function values of passed paths) and the state of the kth element-node is only relative to the state of the k-1th element-node. The backward search is a stack algorithm based on the A* algorithm. A heuristic function for each node k is a sum of two functions, a “path cost function” which represents the sum of the cost function value for the shortest path from the start point to the kth node and a “heuristic estimation function” which represents the estimation of the path cost from the kth node to the target node. In the backward search, a path score in the stack is a full-path score and the optimal path always locates in the stack top. Thus, this algorithm is a global optimum algorithm. 
         [0106]    Assuming that the user has inputted a handwriting character sequence “define” as shown in  FIG. 6A ,  FIG. 12A  illustrates a segmentation result for the handwriting character sequence according to an embodiment of the present invention. Three most possible segmented patterns by the N-Best method are illustrated in  FIG. 12A ,  FIG. 12B  and  FIG. 12C  respectively. The first candidate of single character recognition result for each character in the first segmented pattern is “define (i.e., correct answer)”, the first candidate in the second segmented pattern is “ccefine” and the first candidate in the third segmented pattern is “deftine”. 
         [0107]    In Step S 28 , finally the method of the present embodiment performs post-processing and corrects errors (e.g., spelling mistake of the English word) for the recognition results by matching with the dictionary (English word dictionary) or using language model (for example, bigram model). 
         [0108]    In Step S 29 , the display control unit  150  controls the display screen to present the handwriting recognition results and the relative candidates to user so that user can select or confirm the displayed recognition results in the candidate selection unit  140  (default recognition result is the first candidate of single character recognition for each character in the first segmented pattern). The user can select the correct segmented pattern from candidate segmented patterns of the character sequence or can select the correct recognition results from candidates of respective characters to manually correct a part of recognition result in the character sequence, for example, clicking a single character or a phrase to select the recognition result from their corresponding candidates.  FIG. 15  is a schematic diagram illustrating the candidates of the clicked single character which is provided to user for selecting and correcting according to an embodiment of the present invention. 
         [0109]    Step S 30  detects whether the user has confirmed or selected a certain candidate. If the user continues writing without confirming or selecting any candidate, the process goes to Step S 20  and continues the above recognition processing. If it has detected that a certain candidate has been selected, Step  31  selects the recognition result from the candidates and displays the recognition result or provides to other applications. At the same time, the recognition result of the handwriting input is updated in Step S 32 . 
         [0110]    During calculating the segmentation reliability of the character sequence, the method and device of the present embodiment consider, not only the commonly used space geometry features but also the single character recognition accuracy of the merged stroke combination and the single character recognition accuracies of the sub-stroke combinations, as a result, it can achieve correct segmentation and recognition result in cases that the correct segmentation is difficult to be performed by traditional technology, for example, strokes in different characters are partially overlapping in space, or the stroke gaps in a character is too big. 
         [0111]    Moreover, the method and device of the present embodiment do not rely on the input time of each stroke when performing the character sequence segmentation, so it can adapt to different input habits of users. Even a user inputs the character sometimes fast and sometimes slow, the segmentation accuracy will not be decreased according to the method and device of the present embodiment. 
         [0112]    In addition, the space geometry features of the stroke combination adopted in the method and device of the present embodiment are normalized features based on the estimated average width or height of characters, so the device of present embodiment can adapt to a character sequence with any size. Since the multiple-template training and multiple-template matching methods are adopted in the single character recognition, the characters in different writing patterns by different users (e.g., simplified characters of Kanji by Chinese) can be accurately recognized by the method and device of the present embodiment. Furthermore, the method and device of the present embodiment utilize the language model and dictionary matching so that the device has the functions of spell check and word correction. 
         [0113]    Finally, the recognition objects of the method and device of the present embodiment can be English word, Japanese kana combination, Chinese sentence, Korean character combination, and etc. The timing of performing handwriting recognition can be designated arbitrarily. The recognition result can be continually updated while the user inputs the character sequence, or the recognition results can be displayed after the user finishes the whole character sequence input. 
         [0114]      FIGS. 13A ,  13 B,  13 C and  13 D are schematic diagrams illustrating handwriting recognition results of the handwriting recognition device according to an embodiment of the present invention. Not only the space geometry features of the stroke combination but also the single character recognition accuracies are considered during the recognition process, as a result, the method of the present embodiment can achieve correct recognition in cases that the traditional technology is difficult to perform correct segmentation, for example, strokes in different characters are partially overlapping in space, or the distance between characters is smaller than the distance between strokes in a character, or font sizes are being different during the handwriting input. For example, as shown in  FIG. 13D , the strokes of “d” and “e” and the strokes of “f” and “i” partially overlap in space. As shown in  FIG. 13A  and  FIG. 13C , the gap between “         ” and “         ” is smaller than the inter-stroke distance within “         ” and the gap between “         ” and “         ” is smaller than the inter-stroke distance within “         ”. As shown in  FIGS. 13B and 13D , font sizes of characters in “                   ” and “define” are different from each other. The method according to the embodiment of present invention can perform correct recognition in the above cases. 
         [0115]      FIG. 14  illustrates an electronic dictionary according to an embodiment of the present invention. As shown in  FIG. 14 , a series of English handwriting characters are recognized and the recognition results are displayed. Japanese translation of the inputted handwriting is presented to user by looking up the recognized English word in an English-Japanese dictionary. As shown in  FIG. 15 , when user clicks a certain single character from the recognition result, candidates of this single character will be provided to the user for correction. 
         [0116]    Briefly speaking the present embodiment can allow user to perform overall correction for the recognition result of the whole character sequence, and also can allow user to correct any single character recognition result. 
         [0117]    According to another embodiment of the present invention, the display area and the handwriting input area can be configured on different planes or on the same plane as shown in  FIGS. 16A and 16B . For example, the handwriting area for the notebook computer can be configured on the plane where the keyboard locates. 
         [0118]    As described above, the method and device of the present invention can be applied to or be incorporated into any terminal product which is able to adopt handwriting as input or control manner, for example, personal computer, laptop, PDA, electronic dictionary, MFP, mobile phone, handwriting device with large touching screen, and etc. 
         [0119]    The description and drawings only illustrate the principle of the present invention. It shall be noted that those skills in the art could achieve different structures, although these different structures are not clearly described and indicated but these structures embody the principle of the present invention and shall be included within the spirit and scope of the present invention. In the above descriptions, multiple examples are described aiming at respective steps. Although the inventor exerts himself to explain relative examples, it does not mean that these examples should have corresponding relationship according to the representing numerals. As long as there is no contradiction between conditions limited in the selected examples, examples with un-corresponding representing numerals may constitute a technical solution and such technical solution shall be considered as being encompassed by the present invention. 
         [0120]    It is to be understood that the claims are not limited to the precise configuration and components illustrated above. Various modifications, changes and variations may be made in the arrangement, operation and details of the systems, methods, and devices described herein without departing from the scope of the claims.