PATTERN RECOGNITION DEVICE

Abstract: A pattern recognition device in accordance with the pre­ sent invention is arranged to have learning of a reference pattern vector carried out in a recognition unit (12) by mak­ ing use of the pause periods in the recognition processing, without particularly providing a learning section for learn­ ing the reference pattern vector. Thus, a part or the entirety of the arithmetic processing unit (30, 36,38) where the rec­ ognition result is obtained in the recognition unit (12) by col­ lating the input pattern with the recognition dictionary, can be utilized as the learning portion of the reference pattern vector. In concrete terms, the operation of multiplication-ac­ cumulation (inner product) which represents the main oper­ ation in the recognition processing and the learning pro­ cessing, can be carried out by means of similar processes of handling. Therefore, by utilizing the processing section for sum of the products operation, of the recognition unit, which is in the idle state for pattern recognition, it becomes possible to carry out learning of reference pattern vector ef­ ficiently in time, without forcibly interrupting the pattern rec­ ognition processing by formally setting a learning condition.

Description: BACKGROUND OF THE INVENTION Field of the Invention The present invetion relates to a pattern recognition device which enables high accuracy recognition for the input patterns through effective learning of the reference pattern vector. Description of the Prior Art The recent advancement in the pattern recognition techniques, such as the character recognition, speech recognition, and drawing reading, is remarkable with the development of various pattern recognition devices like voice activated work processor (dictation machine) and the handwritten character reading and inputting device. In a pattern recognition device of this kind, the recognition for the input pattern is obtained generally by carrying out the processing of matching between a reference pattern vector which is prepared in advance and the input pattern. For this reason, in order to make high accuracy recognition for the input patterns, it becomes necessary to prepare a highly substantial reference pattern vector through learning of the reference pattern vector by collecting a large number of patterns that are used in practice and input patterns when the pattern recognition device is used. However, the fact that it is necessary to collect a large number of patterns in advance and that it requires enormous processing time for the learning processing of the reference pattern vector that utilizes the collected patterns, have been posed as the major task to be taken care of. For instance, when learning is carried out by the K-L expansion of the syllable recognition carried out by the K-L expansion of the syllable reference pattern vector that consists of 101 categories represented as 256-dimensional vectors, for speech recognition of syllables, it requires about 5 hours of processing time even when used is a large high-speed computer with operation processing speed of 1 MIPS. A conventional pattern recognition device is shown in EP-A-0 085 545 and includes an input means through which patterns to be recognized are inputted to said device; an operation means including a first arithmetic processing unit for obtaining products and sums; reference pattern vector memory; a recognition means for recognizing input patterns with the aid of said operation means by matching the input patterns with reference patterns, each reference pattern being prepared in the form of a pattern vector and stored in the reference pattern vector memory; and a learning means for carrying out learning of the reference patterns, whereby reference pattern vectors are formed by KL expansion of characteristic kernels. Now conventionally, the pattern collection and the learning of the reference pattern vector are carried out generally in a different state from that for the recognition of input patterns. Namely, it is ordinarily being done by switching the operational mode from the recognition condition to the learning condition. Thus, for example, in the on-line handwritten character reader, the user executes the learning of the reference pattern vector by interrupting the pattern recognition processing and by inputting the pattern for learning. For this reason, there was an inconvenience that the recognition processing for an input pattern cannot be executed during the learning processing of the reference pattern vector. In addition, when recognition is made for the patterns of intermittently inputted human-spoken words, like in the voice activated word processor (dictation machine), the processing section for the sum of products operation for the recognition processing of the input pattern is active over only a certain fixed duration in which the patterns are inputted in the recognition condition. For the remaining times, in spite of its being in the recognition condition, it actually remains in the waiting condition for the patterns to be inputted, so that the arithmetic processing unit is in a paused state in effect. Therefore, there are generated time losses, and the arithmetic units of the recognition device are of lower efficiency. On the other hand, to achieve recognition with high accuracy of the patterns inputted by an unspecified speaker or an unspecified writer, it is conceivable to collect the input patterns to be used for the recognition processing as the patterns for learning, and to carry out the learning of the reference pattern vector by these patterns. By so doing, it becomes possible to carry out an effective collection of the patterns for learning the reference pattern vector, as well as to achieve a substantiation of the reference pattern vector in an easy manner. However, in order to learn the numerous patterns that are collected in this manner by setting the learning condition, there will be many difficulties, for example, it requires a more colossal amount of processing time. Providing a background/foreground system to permit two programs to execute is known for general purpose computer systems from the Encyclopaedia of Computer Science edited by Ralson & Meek, published by Van Nostrand Reinhold, New York, U.S.A. 1978, pages 953 to 957. SUMMARY OF THE INVENTION An object of the present invention is to provide a pattern recognition device which makes it possible to carry out the learning of a reference pattern vector without interrupting the pattern recognition processing. Another object of the present invention is to provide a pattern recognition device which enables to improve the recognition factor of input patterns. Another object of the present invention is to provide a pattern recognition device which enables to learn a reference pattern vector for pattern recognition, in an efficient manner without loss of time. Another object of the present invention is to provide a pattern recognition device which enables to prepare a highly reliable reference pattern vector in an efficient manner without putting a burden on the user. Another object of the present invention is to provide a pattern recognition device which enables to make an effective use of the arithmetic processing unit for the operation of multiplication - accumulation, in a short duration. A pattern recognition device in accordance with the present invention is characterized in that a characteristic kernel memory is provided for storing said characteristic kernels; said learning means is arranged to perform learning of reference patterns by modifying elements of each characteristic kernel by said operation means with reference to the input patterns; said operation means is arranged to begin the operation for gaining products and sums that are required for the modification of said characteristic kernels when the recognition process is halted and said operation means need not carry out the operation necessary for the recognition, the halt ensuing from an intermittent absence of pattern input while an operator is inputting a series of patterns to be recognized; said operation means is arranged to halt said modification when the recognition is resumed, and to store the intermediate result of the modification operation in a memory in order to enable said operation means to resume the operation for the modification from said intermediate result, when the recognition process is halted again; a second arithmetic processing unit is provided for carrying out operations other than the operation of multiplication-accumulation (inner product) in the recognition processing; and a third arithmetic processing unit is provided for carrying out operations other than the operation of multiplication-accumulation (inner product) in the learning processing. These and other objects and advantages of the present invention will be more apparent from the following description of the preferred embodiments, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a simplified block diagram for a pattern recognition device embodying the present invention; Fig. 2 is a block diagram for illustrating a more detailed construction of the recognition unit of the pattern recognition device shown in Fig. 1; Fig. 3 is a timing chart for illustrating the timing of the operation of the pattern recognition device shown in Fig. 1; Figs. 4a to 4c are flow charts for illustrating the processing operation of the pattern recognition device shown in Fig. 1; Fig. 5 is a simplified block diagram for another embodiment of the pattern recognition device in accordance with the present invention; Fig. 6 is a block diagram for illustrating a more detailed construction of the pattern recognition device shown in Fig. 5; Figs. 7a and 7b are timing charts for illustrating the timing of the second embodiment shown in Fig. 5; Fig. 8 is a block diagram for a third embodiment of the pattern recognition device in accordance with the present invention; and Fig. 9(a),(b) illustrates one example of Chinese characters to be recognized erroneously. DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring to Fig. 1, a pattern recognition device embodying the present invention is shown with reference numeral 10. The pattern recognition device 10 includes a recognition unit 12 having the functions of recognition processing and learning processing. As shown in Fig. 3 the learning of the reference pattern vector is executed by the processing section 12 in the pause period of recognition processing at the processing section 12, without a particular learning section for learning the reference pattern vector. That is, a part or the entirety of the arithmetic processing unit where recognition is achieved at the recognition unit 12 by collating the input pattern and the reference pattern vector, can be utilized as is for the learning of the reference pattern vector. In concrete terms, the operation of multiplication-accumulation (inner product), which constitutes the principal operation in the recognition processing and the learning processing is carried out for the two processes according to similar processes of handling. Accordingly, by using the processing section for sum of the products in the recognition unit 12, which is in an idle state of the pattern recognition processing, it becomes possible to carry out learning of the reference pattern vector effectively in time, without formally setting the learning condition by forcibly interrupting the pattern recognition processing. The device of the present embodiment will generate a substantial effect when it is applied, for example, to the case of recognizing syllables whose speech input is intermittently input on-line according to the dialogic form. The case of speech input will be described in what follows. The pattern recognition device 10 comprises a preprocessing section 14 which converts the input speech to a pattern, a dictionary memory 16 for memorizing the dictionary for recognition, a recognition unit 12 which obtains the result of recognition for the input speech pattern by matching the input speech pattern from the preprocessing section 14 and the reference pattern vector in the dictionary memory 16, as well as carries out learning of the reference pattern vector in the dictionary memory 16 duruing the pausing duration of the recognition processing, and a display 18 and a keyboard 20 which carries out correction processing for the result of the pattern recognition. In the recognition unit 12, there are arranged a correlation matrix (characteristic kernel) memory section 22, a memory section for interim state in learning 24, and an error/reject memory section 26. The error/reject memory section 26 is for recording a newly input speech pattern and the result of the matching processing between the input speech pattern and the reference pattern vector, namely, the learning information such as the degree of similarity, when a recognition error or a recognition rejection for the input speech pattern is occurred in the midst of executing the recognition processing or learning processing in the recognition unit 12 in order to prepare for the learning of the reference pattern vector. The correlation matrix (characteristic kernel) memory section 22 is for recording the correlation matrix (characteristic kernel) of the correct category and the correlation matrix (characteristic kernel) of the erroneous category, or either one of the two kernels, when there was generated a recognition error or a recognition rejection. In addition, the interim state of learning memory section 24 is for recording the interim state in the learning of the reference pattern vector at the recognition unit 12. It should be noted that the error/reject memory section 26 is sometimes constructed to form a part of the memory region of the correlation matrix (characteristic kernel) memory section 22 or the interim state learning memory section 24. As shown in Fig. 2, the recognition unit 12 with the recognition function and the learning function icludes a first arithmetic processing unit 30 which carries out the operation of multiplication-accumulation (inner product) that represents the principal operation in the recognition processing or the learning processing to be done upon receipt of the input speech pattern from the preprocessing section 14, a recognition operation control section 32 which controls the preprocessing section 14, the first operation control section 30, and the dictionary memory 16 in order to carry out, for the purpose of recognition processing, the operation of multiplication-accumulation (inner product) for recognition at the first arithmetic processing unit 30, and a learning operation control section 34 which controls the first arithmetic processing unit 30 and the dictionary memory 16 in order to carry out, for learning processing, the operation of multiplication-accumulation (inner product) for learnaing at the first arithmetic processing unit 30. In the recognition operation control section 32 there is installed a second arithmetic processing unit 36 for carrying out operations other than the operation of multiplication-accumulation (inner product) for recognition processing, for example, the sorting processing operation. Similarly, in the learning operation control section 34 there is installed a third arithmetic processing unit 38 for carrying out operations other than the operation of multiplication-accumulation (inner product) for the learning processing, such as the division for normalization. the recognition unit 12 further includes a control section 40 which judges, based upon the signals from the recognition operation control section 32 and the learning operation control section 34, whether or not it is possible to carry out learning of the reference pattern vector at the first arithmetic processing unit 30, and sorts out the information for correction processing which is sent from the keyboard 20 to the recognition operation control section 32 or the learning operation control section 34. Referring to Fig. 4a, the processing operation for the case of carrying out the recognition of a syllabic speech pattern in the pattern recognition device as shown in Figs. 1 and 2 will be described. In such a case, there is furnished in general a reference pattern vector for unspecified speakers. To begin, when the speaker inputs a syllable by voicing it in step 100, there is obtained an input speech pattern through the preprocessing section in step 102. Next, the input speech pattern is forwarded to the first arithmetic processing unit 30, and is processed for recognition operation by being collated with the reference pattern vector at the recognition operation control section 32 (step 104). The pattern recognition processing is carried out, for example, by the use of the well known Multiple Similarity Method or the partial space method. Then, in step 106, assume for example, that the syllable was erroneously recognized as "a" in spite of actual voicing of "pa", and whether or not the difference in the degrees of similarity between the input speech pattern and the respective reference pattern vectors for "a" and "pa" is slight, that is whether or not the input speech pattern is usable as a learning data, is recognized by the learning control section 34. If it is recognized in step 108 that it is usable as a learning data, then the input speech pattern and the result for its recognition will be memorized in the error/reject memory section 26 as a learning information. Next, in step 110, whether or not the input speech pattern to be recognition-processed is being input continuously through the preprocessing section 14 is recognized by the control section 40. If the input speech pattern is recognized to be input continuously, the processings for the steps 102 to 108 are repeated. If, for example, when the speaker is contemplating what is to be said next or when the inputting of the speech pattern is interrupted by the so-called pausing or the like, in other words, when learning is possible, then whether or not the learning condition for the reference pattern vector is in good order is recognized, based on the information for modification processing that comes from the control section 40, by the learning operation control section 34 (step 112). For instance, when it is known from the previously executed learning of the reference pattern vector that the errors made in recognizing "a", "pa", and "ta" are more than 10 times each, learning of the reference pattern vector will be carried out by judging that the learning conditions for "a", "pa", "ka", "ka" and "ta" are in good order. On the other hand, if the learning conditions are found to be not in good order, the processings in the steps 102 to 108 will be repeated. When the learning conditions are in good order and the learning of the reference pattern vector is to be carried out in step 114, the learning information for pattern "a", for example, is read from the error/reject memory section 26 by the learning operation control section 34, to carry out the learning of the reference pattern vector (step 116). Here, learning of the reference pattern vector is to be executed, for example, by varying the rate of contribution of the input speech pattern to the correlation matrix (characteristic kernel) or the correlation matrix for the reference pattern vector, by the use of the K-L expansion, based on the input speech pattern and the correct name of the category that is to be input for correcting the recognition result. For example, when learning is to be excited by means of the result of recognition for the input speech pattern, the information given by the speaker indicating acceptance or rejection of the recognition result, the input speech pattern, other input speech patterns already memorized, and the compound similarity (multiple similarity), learning and updating of the reference pattern vector registered in the dictionary memory 16 are carried out by the use of the correlation matrix (characteristic kernel) and the confusion matrix that shows the error trend during the recognition. The process of learning will be described in detail later. The learning of the reference pattern vector will be continued until all of the input speech patterns in the categories that form the learning object are exhausted. During the learning processing of the reference pattern vector, the reference pattern vector that is registered in the dictionary memory 16, is kept in the state as it is. If a speech is input during the learning processing, the control section 40 judges in step 118 that learning is not possible. Based on the result of judgment, the learning operation control section 34 saves the interim state in learning of the reference pattern vector at the first arithmetic processing unit 30 in the memory section for interim state in learning 24 (step 120). Then, it goes back to step 104 in which the recognition operation is processed by the recognition operation control section 32 through matching of the input speech pattern and the reference pattern vector in the first arithmetic processing unit 30. When it is recognized in step 110 that learning is possible, the interim state in learning of the reference pattern vector is reloaded by the learning operation control section 34 into the first arithmetic processing unit 30 from the interim state memory section 24, to resume the learning processing of the reference pattern vector. Then whether the learning is completed or not is decided in step 122 by the learning operation control section 34, and at the point in time where a new reference pattern vector is obtained by the completion of the learning, the reference pattern vector that is registered in the dictionary memory 16 is updated (step 124). Accordingly, after that point in time where the updating took place, recognition processing of an input pattern will be carried out by the use of the new modified reference pattern vector. The processings in the above are executed repeatedly and the reference pattern vector is updated successively to substantiate the dictionary. Therefore, as the learning of the reference pattern vector goes on, there will be expected an improvement in the recogntion factor of the input patterns. With the operation as described in the above, the pattern recognition device 10 carries out speech recogntion continuously as seen from the side of the user who inputs the speech. Next, the learning processing at the first arithmetic processing unit 30 of the pattern recognition device 10 will be described in more detail. The learning operation control section 34 receives an information on the degree of similarity determined for each of the category of such an input pattern, from the error/reject memory section 26, and reads the correlation matrix (characteristic kernel) for the correct category and the correlation matrix (characteristic kernel) for the erroneous category, or one of the correlation matrix (characteristic kernel) of the two, from the correlation matrix (characteristic kernel) memory section 22, to update the correlation matrix (characteristic kernel) in the following manner. Now, if it is assumed that the number of the categories to be recognized is L, and that each of the categories are designated by the numbers 1, 2, ..., ℓ, ..., and L, then the correlation matrix (characteristic kernel) K (ℓ) for the category (ℓ) is represented by the following equation. Here, fα is the speech pattern vector belonging to the category (ℓ), f T is the transposed of the vector f α , and w α (ℓ) is a proper coefficient. Next, if the correlation matrix (characteristic kernel) modified for n times is represented by the (n+1)-th modification in the present embodiment will be given by K n+1 (ℓ) = (1-w (ℓ) )K n (ℓ) +w (ℓ) f αo f αo T . (2) In the above equation, f αo is the speech pattern for category (ℓ) which is input at the time of the (n+1)-th modification, and w (ℓ) is a predetermined coefficient. By substituting Eq.(1) into Eq.(2) one obtains Therefore, by the (n+1)-th modification the coefficient corresponding to the input speech pattern f αo is modified according to w αo,n (ℓ) → w αo,n (ℓ) + (1-w αo,n (ℓ) ) w (ℓ) , (4) and the coefficient corresponding to other speech pattern in category (ℓ) is changed by w α,n (ℓ) → (1-w (ℓ) ) w α,n (ℓ) . (5) Namely, if it is set w (ℓ) >0, then the input speech pattern f αo will be assigned more importance than other speech patterns by these modifications, if it is set w (ℓ) <0, the input speech pattern f αo will be given less significance than other speech patterns, and if it is set w (ℓ) =0, the situation will be maintained the same as before. With the contents of the correlation matrix (characteristic kernel) memory section 22 modified as in the above, the eigenvalve problem for the modified correlation matrix (characteristic kernel) will now be solved. The eigenvalue problem can be solved by incorporating a well-known solution program for the eigenvalue problem, such as "Jacobi method", in the microcomputer or the minicomputer. As described in the above, it becomes possible to obtain an updated reference pattern vector by solving the eigenvalue problem for the correlation matrix (characteristic kernel). Here, it may be arranged at this point in time to send a signal that indicates the completion of learning to the modification terminal to request a confirmation from the user of the reference pattern vector learned as in the above. By so doing, it becomes possible for the user to check the validity of the learning of the reference pattern vector, prior to the updating of the reference pattern vector that is registered in the dictionary memory 16. When the reference pattern vector may be updated, the learning processing completes by the storing of the new reference pattern vector in the dictionary memory 16. By referring to Figs. 4b and 4c, recognition of the handwritten character patterns by the first embodiment of the pattern recognition device will now be described. In Fig. 4b is illustrated the learning processing that operates as a background processing while in Fig. 4c is illustrated the interruption processing of the recognition processing that is generated for the recognition unit 12 for every time at the completion of the preprocessing for the input character pattern. Namely, in the case of inputting handwritten characters, due to small speed in which the inputting is made, the recognition processing for the input pattern is arranged to be actuated as an interruption for the recogntion section 12 at the point in time where the pattern inputting (the preprocessing) is completed. When the preprocessing for the input character pattern is completed, the interruption processing is actuated as shown in Fig. 4c, and the input pattern is recognized by collating it against the reference pattern vector. Here, if in spite of handwriting, for example, of Chinese character "shiki", shown in Fig. 9a, it was recognized erroneously as Chinese character "shoku", shown in Fig. 9b, and if the difference in the degrees of similarity between the reference pattern vector for respective category and the input pattern is slight, then the input pattern, the information on the degrees of similarity, and others are memorized as the learning data. When the recognition processing of the input pattern by the interruption processing is completed, the background processing, that is, the learning processing, will be carried out, as shown in Fig. 4b. The learning processing that is executed as the background processing is basically similar to the learning processing as shonw in Fig. 4a. Although the recognition processing is being arranged to be enabled through appropriate interruption while the learning of the reference pattern vector is executed in the allowed state of interruption, in updating the reference pattern vector, the processing loop is constructed in such a way as to forbid the interruption of recognition processing which utilizes the reference pattern vector, until the completion of updating of the reference pattern vector. In this manner, it is arranged that the learning of the reference pattern vector and the recognition processing of the input by the use of the reference pattern vector can be carried out by making use of a common recognition unit 12. As has been described in the foregoing, by the use of the device of the present embodiment, the learning processing of the reference pattern vector is carried out by utilizing the pausing period for the recognition processing of the recognition unit 12. Therefore, in comparison to the conventional method in which the learning processing is carried out en bloc, it becomes possible to carry out dictionry updating without being conscious of the duration for learning processing, not only for the user who uses the device constantly but also for the user who uses it temporarily. In addition, it becomes possible to achieve a substantial improvement in the recognition factor through updating of the reference pattern vector by an effective use of the first arithmetic processing unit 30 without incurring losses in time. It is noted that the effective learning systems that are available presently carry out learning processing repeatedly on the numerous data that are collected based principally on the statistical processing, with the sum of the products as its basis. It then requires a powerful operational capability. In contrast, the present device which carries out learning of the reference pattern vector, as described in the foregoing, is extremely effective in the case of realizing, in particular, a pattern recognition device with high precision. Moreover, taking the advancement in the operational devices into account, the present invention will be extremely effective in realizing a high precision pattern recognition device with more functions. Referring to Fig. 5, a second embodiment of the pattern recognition device in accordance with the present invention is shown with reference numeral 50. The pattern recognition device 50 comprises a preprocessing section 51 which converts the input into a pattern, a dictionary memory for recognition 52, which memorizes the reference pattern vector, a recognition unit 54 which determines the recognition result for the input pattern by collating the input pattern and the reference pattern vector in the dictionary memory for recognition 52, a learning section 56 which causes to learn the reference pattern vector which is in the dictionary memory for recognition 52, a display 58 which carries out modification processing for the result of pattern recognition, and a keyboard 60. The recognition unit 54 and the learning section 56 are provided with an arithmetic processing unit with identical construction and are arranged to operate in parallel fashion, as shown by Figs. 7a and 7b, when it becomes necessary to carry out learning of the reference pattern vector. Analogous to the case of the first embodiment described earlier, it becomes possible to achieve the construction effect in which learning of the reference pattern vector can be carried out without forcibly interrupting the processing of pattern recognition, permitting to achieve an improvement in the recognition capabilities of the reference pattern vector. The learning section 56 includes an error/reject memory section 62 which memorizes, when there is generated a recognition reject or a recognition error for a newly input pattern during execution of learning processing in preparation of learning of the reference pattern vector, the input pattern and the result of matching processing between the input pattern and the reference pattern vector, a correlation matrix (characteristic kernel) memory section 64 which memorizes the correlation matrix (characteristic kernel) for the correct category and the correlation matrix (characteristic kernel) for the erroneous category, or either one of these correlation matrices (characteristic kernels), that are to be used at the time of occurrence of a recognition reject or a recognition error, and a memory section for interim state in learning 66 which memorizes the interim state in the learning of the recogntion dictionary at the recognition unit 54. The error/reject memory section 62 may be constructed as a part of the memory region for the correlation matrix (characteristic kernel) memory section 64 or the memory section for interim state in learning 66. As shown in Fig. 6, the recogntion section 54 includes a first recognition arithmetic processing unit 68 which carries out the operation of multiplication-accumulation (inner product) that represents the principal operation in the recogntion processing, by receiving an input pattern from the preprocessing section 51, and a recognition operation control section 70 that controls the preprocessing section 51, the first recognition arithmetic processing unit 68, and the dictionary memory 52, in order to carry out the operation of multiplication-accumulation (inner product) for recogition at the first recognition arithmetic processing unit 68. In the recognition operation control section 70 there is provided a second recognition arithmetic processing unit 72 which carries out operations, such as the sorting processing operation, other than the operation of multiplication-accumulation (inner product) in the recognition operation. The learning section 56 includes a first learning arithmetic processing unit 74 which carries out the operation of multiplication-accumulation (inner product) that represents the principal operation in the learning processing, by receiving an input pattern from the preprocessing section 51, and a learning operation control section 76 which controls the preprocessing section 51, the first learnaing arithmetic processing unit 74, and the dictionary memory 52, in order to carry out the sum of the products operation for learning at the first learning arithmetic processing unit 74. In the learning operation control section 76 there is provided a second learning arithmetic processing unit 78 which carries out operations, such as the division for normalization, other than the operation of multiplication-accumulation (inner product) in the learning processing. The recognition unit 54 further includes a control section 80 which detects the occurrence of a recognition error or reject for the input pattern by means of a signal from the recognition operation control section 70, and supplies an information for the modification processing from the keyboard 60 to both of the recognition operation control section 70 and the learning operation control section 76, in order to be able to carry out the learning processing in parallel with the recognition processing in the recognition unit. Next, the operation of the second embodiment will be discribed. First, when a handwritten character, for example, is input, an input character pattern is found through the preprocessing section 51. Then, the input character pattern is forwarded to the recognition arithmetic processing unit 68 and is processed for recognition operation by being collated with the reference pattern vector at the recognition operation control section 70. If a recogntion rejection or a recognition error takes place for the input character pattern in this stage, then the control section 80 detects the occurrence of such a recognition reject or error, and supplies an information on the correct category for the input character pattern coming from the keyboard 60 to the recognition operation control section 70 and the learning operation control section 76. Here, the recognition operation control section 70 extracts respectively the degree of similarity between the input character pattern and the erroneous category and the degree of similarity between the input character pattern and the reference pattern vector pattern with correct category. The learning operation control section 76 receives an information on the degrees of similarity determined for the input character pattern with each of the categories, reads the correlation matrix (characteristic kernel) for the correct category and the correlation matrix (characteristic kernel) for the erroneous category, or one of the two correlation matrices (characteristic kernels), from the correlation matrix (characteristic kernel) memory section 64, and updates the correlation matrix (characteristic kernel) as was described earlier. By solving the eigenvalue problem for the correlation matrix (characteristic kernel), subsequent to the above, it becomes possible to obtain a recogntion dictionary which is updated. At this stage, it may be arranged to let the user to confirm the reference pattern vector learned as in the foregoing, by sending a signal that indicates the completion of learning to the modification terminal. By so arranging, it becomes possible for the user to check the validity of the learning of the reference pattern vector before updating the reference pattern vector which is registered in the dictionary memory 52. When the updating of the reference pattern vector is warranted, learning processing completes with the storing of the new reference pattern vector in the dictionary memory 52. Now, however, the process of learning the reference pattern vector takes a long time in general. Moreover, since the above process is executed independently of the recognition processing for the input pattern, there is a possibility that the learning of the reference pattern vector is not completed yet at the time when the power supply for the device is shut off accompanying the completion of the recognition processing. If such a situation occurs, the interim state in learning of the reference pattern vector, that is being done in the first learning arithmetic processing unit 74, is saved in the memory section for the interim state 66 by the learning operation control section 76. The simplest method for saving such an information is to save all of the program region for learning of the reference pattern vector, the data region, the execution address, and its status flag. Of these, there exists no absolute need for saving the program region provided that there is a guarantee that it will not be changed during the execution of the recogntion processing. Next, when the pattern recognition processing resumes under the turning-on of the power supply for the device, the information on the kernls that has been saved in the memory section for interim state in learning 66 is reloaded in the first learning arithmetic processing unit 74. Using the execution address in the information, learning of the reference pattern vector is resumed. In Figs. 7a and 7b are illustrated examples of the time flow of such a process. In this manner, by the use of the present device it becomes possible to achieve a construction effect for the reference pattern vector by carrying out learning of the reference pattern vector without interrupting the processing for the pattern recognition and to aim at an improvement in the recognition capabilities of the dictionary. In additon, there will be obtained such effects as being able to carry out with extreme ease the development and substantiation of the reference pattern vector which used to require in the past much labor and an enormous amount of time. Referring to Fig. 8, a third embodiment of the pattern recognition device in accordance with the present invention is shown with reference numeral 80. The pattern recognition device 80 includes, similarly to the first embodiment shown in Fig. 2, a recognition unit that has the learning function and the recognition function, with a construction which enables parallel processings of learning and recognition at the recognition units 82 and 84, respectively. Accordingly, the concrete construction for the recognition units 82 and 84 is similar to that of the recognition unit 12 of the first embodiment, so that no further explanation about it will be given. It is to be noted that the present invention should not be limited to the embodiments described in the foregoing. Thus, for example, as to the kinds of the characteristic vectors that represent the input patterns and the pattern recognition methods that make use of the characteristic vectors, various systems that have been known hitherto may appropriately be applied depending upon their purposes of use. Moreover, although the description about these embodiment has been made in conjunction with the speech recognition for syllables and the recognition of handwritten characters, they may also be applied to the recognition of other types of patterns. Moreover, the present invention can be applied to the device irrespective of their use for specified users or for unspecified users. Furthermore, the present invention is applicable not only to the syllable speech recognition and the handwritten character recognition but also to the pattern inputting device for recognizing the patterns input by means of a dictionary for recognition, such as the devices for word speech recognition, continued numeral recognition, and inputting handwritten drawings.

Claims: A pattern recognition device including: an input means through which patterns to be recognized are inputted to said device; an operation means (30) including a first arithmetic processing unit for obtaining products and sums for the operation of multiplication-accumulation i.e. inner product; reference pattern vector memory (16); a recognition means (32) for recognizing input patterns with the aid of said operation means by matching the input patterns with reference patterns, each reference pattern being prepared in the form of a pattern vector and stored in the reference pattern vector memory (16); and a learning means (34) for carrying out learning of the reference patterns, whereby reference pattern vectors are formed by Karhunen-Loève or KL expansion of characteristic kernels; characterized in that: a characteristic kernel memory (22) is provided for storing said characteristic kernels; said learning means (34) is arranged to perform learning of reference patterns by modifying elements of each characteristic kernel by said operation means (30) with reference to the input patterns; said operation means (30) is arranged to begin the operation for gaining products and sums that are required for the modification of said characteristic kernels when the recognition process is halted and said operation means need not carry out the operation necessary for the recognition, the halt ensuing from an intermittent absence of pattern input while an operator is inputting a series of patterns to be recognized; said operation means (30) is arranged to halt said modification when the recognition is resumed, and to store the intermediate result of the modification operation in a memory (24) in order to enable said operation means (30) to resume the operation for the modification from said intermediate result, when the recognition process is halted again; a second arithmetic processing unit (36) is provided for carrying out operations other than the operation of multiplication-accumulation i.e. inner product in the recognition processing; and a third arithmetic processing unit (38) is provided for carrying out operations other than the operation of multiplication-accumulation i.e. inner product in the learning processing. A pattern recognition device as claimed in claim 1, in which the first arithmetic processing unit comprises a processing section for adding operation. A pattern recognition device as claimed in claim 1 or 2, wherein: terminal equipment (18,20) is provided for carrying out modification processing for the result of pattern recognition; an error/reject memory section (26,62) is provided for recording, for the purpose of learning processing, a newly input pattern and the result of matching processing, between the input pattern and the reference pattern vector, when a recognition reject or a recognition error for the input pattern is occurred during execution of recognition processing or learning processing at said recognition unit; and said characteristic kernel memory section (22,64) is arranged for memorizing the characteristic kernel of a correct category and the characteristic kernel of an erroneous category, for the recognition rejection or the recognition error, to be used in the learning processing. A pattern recognition device as claimed in any preceding claim, in which said learning means comprises a control section (40) for sorting out the information from the keyboard (20) to be used for modification processing to the recognition operation control section (32) or the learning operation control section (34), by judging whether or not it is possible to learn the reference pattern vector at the first arithmetic processing unit (30) based on the signals from the recognition operation control section (32) and the learning operation control section (34). A pattern recognition device as claimed in any preceding claim, in which said learning means comprises a control section (40) for sorting out the information from the keyboard (20) to be used for modification processing to the recognition operation control section (32) or the learning operation control section (34), by judging whether or not the input pattern is inputted continuously to the first arithmetic processing unit (30,68) based on the signals from the recognition operation control section (32), and, when the input pattern is inputted interruptedly, the learning operation control section (34) operates so as to enable the learning process to be carried out at the first arithmetic processing unit (30,68) based on the information from the control section (40) for the modification processing. A pattern recognition device as claimed in any preceding claim, in which the matching processing between the input pattern and the reference pattern vector by the first arithmetic processing unit (30,68) is carried out with the method of compound degree of similarity or the partial space method, and the learning of the reference pattern vector by the arithmetic processing unit (30,68) is carried out by varying the degree of contribution of the input pattern to the correlation matrix (characteristic kernel) or the correlation matrix by the use of K-L expansion method. A pattern recognition device as claimed in any previous claim, arranged so that the first arithmetic processing unit (30,68) that is in an idle state can carry out learning of the reference pattern vector by matching of the input pattern which generated the recognition or error memorized in said error/reject memory section (26,62), and the reference pattern vector. A pattern recognition device according to any preceding claim, further comprising enabling means to enable learning, when it becomes necessary to learn the reference pattern vector, at said learning means in parallel with the recognition processing at said recognition means whereby the recognition processing in said recognition means is not interrupted. A pattern recognition unit as claimed in claim 8, in which said enabling means can detect the occurrence of a recognition error or rejection for an input pattern, and can allow the recognition processing in said recognition means and the learning processing in said learning means proceed in parallel. A pattern recognition device as claimed in any preceding claim, which further comprises a control section (80) for detecting the occurrence of a recognition error or rejection for the input pattern based on the signal from the recognition means control section, and for supplying the information for modification processing from said terminal equipment (18,20) to both of the recognition means (32) and the learning means (34) in order to enable the learning processing to be carried out in parallel with the recognition processing. A pattern recognition device according to any preceding claim, in which said learning means (34) is arranged to carry out recognition processing whereby the recognition processing by said recognition means (32) is not interrupted. A method of operating a pattern recognition device including: an input means through which patterns to be recognized are inputted to said device; an operation means (30) including a first arithmetic unit, for obtaining products and sums for the operation of multiplication-accumulation ie. inner product; reference pattern vector memory (16); a recognition means (32) for recognizing input patterns with the aid of said operation means by matching the input patterns with reference patterns, each reference pattern being prepared in the form of a pattern vector and stored in the reference pattern vector memory (16); and a learning means (34) for carrying out a learning of the reference patterns, reference pattern vectors being formed by KL expansion of characteristic kernels, characterized by: storing said characteristic kernels in a characteristic kernel memory (22); performing operations other than the operation of multiplication-accumulation i.e. inner products in the recognition processing by means of a second arithmetic processing unit (36) ; performing operations other than multiplication-accumulation i.e. inner products in the learning processing by means of a third arithmetic processing unit (38); carrying out learning of reference patterns by modifying elements of each characteristic kernel by said operation means (30) with reference to the input patterns; beginning the operation for gaining products and sums in said operation means (30) that are required for the modification of said characteristic kernels when the recognition process is halted and said operation means need not carry out the operation necessary for the recognition, the halt ensuing from an intermittent absence of pattern input while an operator is inputting a series of patterns to be recognized; halting the modification when the recognition is resumed; and storing the intermediate result of the operation for the modification in a memory (24) in order to enable said operation means (30) to resume the operation for the modification from said intermediate result, when the recognition process is halted again.

Inventors: MAEDA, KENICHI, TAKEBAYASHI, YOICHI, TSUBOI, HIROYUKI
Applicants: KABUSHIKI KAISHA TOSHIBA
Application year: 1985
Main classification: G06K9
IPC codes: G06K9