Abstract:
A method and means for identifying valuable documents is proposed. Wherein the method includes the following steps: collect multi-mode information of the valuable document to be identified, according to the pre-generated fusion strategy and the multi-mode information of the valuable document to be identified, identify the valuable document to be identified and obtain identification results. Through the embodiment of the present invention, identifying valuable documents based on the multi-mode information is implemented, and the identified reliability

Description:
[0001]    This application claims the priority of Chinese Patent Application No. 200910037735.0, entitled “METHOD AND DEVICE FOR RECOGNIZING VALUABLE DOCUMENT”, filed with the Chinese Patent Office on Mar. 10, 2009, which is hereby incorporated by reference in its entirety. 
       FIELD OF THE INVENTION 
       [0002]    The present invention relates to the field of pattern recognition, and in particular to a method and device for recognizing a valuable document. 
       BACKGROUND OF THE INVENTION 
       [0003]    With the development of social economy, there is an increasing demand for anti-counterfeiting detection of a valuable document such as banknotes and valuable securities. 
         [0004]    In the field of pattern recognition, taking a banknote as an example, the recognition of the valuable document is typically performing by recognizing the denomination of the banknote and determining whether the banknote is true or fake and whether the banknote is complete or worn and so on according to one type of modal information of the banknote (such as optical information or physical information). 
         [0005]    In conceiving the present invention, the inventor finds out that the prior art has at least the following problems: 
         [0006]    Single-modal information of the valuable document such as the banknote describes the banknote only on a certain level or in a certain aspect, which may not represent the characteristics of the banknote completely, and therefore it is incomplete. Moreover, the single-modal information of the banknote is easy to be interfered by external factors, for example, the single-modal information can be tampered and counterfeited easily; and thus it is uncertain and unstable. 
       SUMMARY OF THE INVENTION 
       [0007]    Embodiments of the present invention provide a method and device for recognizing a valuable document, so as to recognize the valuable document based on multimodal information and improve reliability and accuracy of the recognition. 
         [0008]    In view of the above objects, an embodiment of the present invention provides a method for recognizing a valuable document, and the method includes the following steps: 
         [0009]    collecting multimodal information of the valuable document to be recognized, wherein the multimodal information includes two or more of optical information, electrical information, magnetic information and physical information of the valuable document to be recognized; and 
         [0010]    recognizing the valuable document to be recognized according to a pre-generated fusion strategy and the multimodal information of the valuable document to be recognized, and obtaining a recognition result. 
         [0011]    In addition, an embodiment of the present invention also provides a device for recognizing a valuable document, and the device includes: 
         [0012]    a collection module for collecting multimodal information of the valuable document to be recognized, wherein the multimodal information includes two or more of optical information, electrical information, magnetic information and physical information of the valuable document to be recognized; and 
         [0013]    a recognition module for recognizing the valuable document to be recognized according to a pre-generated fusion strategy and the collected multimodal information of the valuable document to be recognized, and obtaining a recognition result. 
         [0014]    The advantageous effects of the embodiments of the present invention are as follows: 
         [0015]    According to the embodiments of the present invention, the recognition of a valuable document based on multimodal information is achieved by: collecting multimodal information of the valuable document to be recognized; and then recognizing the valuable document to be recognized according to a pre-generated fusion strategy and the multimodal information of the valuable document to be recognized and obtaining a recognition result. The multimodal information may represent the characteristics of the valuable document in the round, such as its authenticity, denomination and type, thus improving the reliability and accuracy of the recognition through the recognition method employing multimodal information. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0016]    In order to illustrate the technical solutions according to the embodiments of the present invention or in the prior art more clearly, drawings to be used in the description of the prior art or the embodiments will be described briefly hereinafter. Apparently, the drawings described hereinafter are only some embodiments of the present invention, and other drawings may be obtained by those skilled in the art according to those drawings without creative work. 
           [0017]      FIG. 1  is a comparison diagram of spectrograms of a valuable document under irradiation of different wavelengths according to an embodiment of the present invention; 
           [0018]      FIG. 2  is a reference diagram of position relationship between optical information and magnetic information of a valuable document provided by an embodiment of the present invention; 
           [0019]      FIG. 3  is a schematic flow chart of the first embodiment of a method for recognizing a valuable document provided by an embodiment of the present invention; 
           [0020]      FIG. 4  is a schematic flow chart of the second embodiment of a method for recognizing a valuable document provided by an embodiment of the present invention; 
           [0021]      FIG. 5  is a schematic flow chart of the third embodiment of a method for recognizing a valuable document provided by an embodiment of the present invention; 
           [0022]      FIG. 6  is a schematic flow chart of the fourth embodiment of a method for recognizing a valuable document provided by an embodiment of the present invention; 
           [0023]      FIG. 7  is a schematic diagram of composition of the first embodiment of a device for recognizing a valuable document provided by an embodiment of the present invention; 
           [0024]      FIG. 8  is a schematic diagram of composition of the second embodiment of a device for recognizing a valuable document provided by an embodiment of the present invention; 
           [0025]      FIG. 9  is a schematic diagram of composition of the third embodiment of a device for recognizing a valuable document provided by an embodiment of the present invention; and 
           [0026]      FIG. 10  is a schematic composition diagram of a first recognition unit in the third embodiment of the device for recognizing a valuable document provided by the embodiment of the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0027]    The technical solutions according to the embodiments of the present invention will be described clearly and completely hereinafter in conjunction with the drawings in the embodiments of the present invention. Apparently, the described embodiments are only a part of rather than all the embodiments of the present invention. All the other embodiments can be obtained by those skilled in the art based on the embodiments of the present invention without any creative work, which all fall within the scope of protection of the present invention. 
         [0028]    The method and device for recognizing a valuable document provided by the embodiments of present invention includes: collecting multimodal information of a valuable document to be recognized; and recognizing the valuable document to be recognized according to a preset (i.e. pre-generated) fusion strategy and the multimodal information of the valuable document to be recognized, and obtaining a recognition result. By implementing the embodiments of present invention, the recognition of the valuable document based on multimodal information is achieved, and the reliability and accuracy of the recognition are improved. 
         [0029]    In practice, information exists in various modes. Multimodal information of an objective thing refers to description information of the same objective thing obtained in different manners. Particularly, the multimodal information of a valuable document such as a banknote is able to represent the characteristics of the banknote in the round, such as its authenticity, state, type and denomination. 
         [0030]    An embodiment of the present invention provides a technical solution for recognizing characteristics of a valuable document based on multimodal information, which includes: firstly collecting multimodal information of a valuable document to be recognized, wherein the multimodal information includes two or more of optical information, electrical information, magnetic information and physical information of the valuable document to be recognized; then generating a fusion strategy based on the multimodal information of the valuable document, according to the unique and determinate relationship between inherent characteristics of a standard valuable document, such as two or more of the optical information, the electrical information, the magnetic information, the physical information and so on of the standard valuable document; then processing the collected multimodal information of the valuable document to be recognized according to the fusion strategy; and finally obtaining a recognition result of the valuable document, for example accepting or rejecting the valuable document. 
         [0031]    In order to facilitate the understanding of the technical solution of the embodiments according to the present invention, the method for providing the fusion strategy will be described in detail here. 
         [0032]    The multimodal information of the standard valuable document is collected, wherein the multimodal information includes two or more of optical information, electrical information, magnetic information, physical information and so on of the valuable document. The unique and determinate relationship between the multimodal information can be obtained by integratively analyzing the multimodal information. Knowledge rule is formed by using these relationships, and the fusion strategy including one or more of a fusion strategy of collection level, a fusion strategy of quantization level, a fusion strategy of feature level and a fusion strategy of decision level is established under the instruction of the knowledge rule. 
         [0033]    Referring to two examples of the fusion strategy of feature level and the fusion strategy of decision level, the technical solution will be further described hereinafter. 
         [0034]    Reference to  FIG. 1 , a comparison diagram of spectrograms of a valuable document under irradiation of different wavelengths provided by an embodiment of the present invention is illustrated. For valuable documents made of the same physical material and in the same physical manner, there is a stable relationship between their imaging contents under irradiation of different wavelengths. As shown in  FIG. 1 , for a certain area A in a valuable document, the imaging contents under irradiation of three wavelengths λ 1 , λ 2 , λ 3  are respectively) ƒ(λ 1 ) 11 ,  ƒ(λ   2 ) 12 , ƒ(λ 3 ) 13 . It can be seen that there is constant differences in brightness between the three imaging contents, and the features extracted from these optical information will keep the relationships, so that the optical information under different wavelengths may be fused on the feature level. 
         [0035]    Reference to  FIG. 2 , a reference diagram of position relationship between optical information and magnetic information of a valuable document provided by an embodiment of the present invention is illustrated. For a valuable document with magnetic safety line, such as a banknote, the magnetic safety line will be displayed obviously in the visible light information of the banknote. As shown in this figure, the image of the magnetic safety line of the banknote in the optical information (the visible light image) is a dark line, and the position  21   a  of the dark line is the imaging position of the magnetic safety line. When the magnetic information is collected, the imaging position  21   a  of the magnetic safety line may be taken as an auxiliary criterion for judging the validity of the magnetic information. Specifically, the magnetic information detected at the position  21   a  corresponding to the position of the dark line is valid, while the magnetic information detected at the position  22  not corresponding to the position of the dark line may be invalid. Conversely, the magnetic information may be taken as an auxiliary criterion for judging the validity of the imaging of the magnetic safety line, which will not be described in detail here. According to reference relationship between the imaging and the magnetic information of the magnetic safety line, it can be seen that the validity of the recognition of the valuable document utilizing the magnetic information may directly affect the validity of the recognition of the valuable document utilizing the optical information. Therefore, the magnetic information and the optical information may be fused on the decision level. 
         [0036]    After the above fusion strategy is obtained, the valuable document to be recognized may be recognized based on the fusion strategy. It is to be noted that, when the similar or same valuable documents are recognized many times, the fusion strategy may be set only once and used many times. For example, when banknotes of 100-yuan RMB are recognized, the fusion strategy may be set before the first recognition, and then the set fusion strategy may be utilized many times to recognize 100-yuan RMB banknotes, without setting the fusion strategy before every recognition. In the following embodiments, the method for recognizing the valuable document to be recognized will be described in detail. 
         [0037]    Reference to  FIG. 3 , a schematic flow chart of a first embodiment of a method for recognizing a valuable document provided by an embodiment of the present invention is shown. The method includes the following steps: 
         [0038]    Step  301 : collecting multimodal information of a valuable document to be recognized, wherein the multimodal information includes two or more of optical information, electrical information, magnetic information, physical information and so on of the valuable document to be recognized; and the valuable document may include a banknote, valuable securities, a ticket, a bill and so on. In this step, for example, the optical information is a spectral characteristic or the like; the electrical information is conductibility or the like; the physical information is information such as material, format and printed image. Practically, the mentioned information is not limited to those but may include other information, which is not limited by this embodiment; and 
         [0039]    Step  302 : recognizing the valuable document to be recognized according to a preset fusion strategy (i.e. a pre-generated fusion strategy, the same below) and the multimodal information of the valuable document to be recognized, and obtaining a recognition result. 
         [0040]    Optionally, the method may further include: generating in advance a fusion strategy based on the multimodal information of the valuable document, according to the inherent characteristics of a standard valuable document. 
         [0041]    According to this embodiment, the recognition of a valuable document based on multimodal information is achieved by collecting multimodal information of the valuable document to be recognized, and then recognizing the valuable document to be recognized according to a preset fusion strategy and the multimodal information of the valuable document to be recognized and obtaining a recognition result, thus improving the reliability and the accuracy of the recognition. 
         [0042]    During the multimodal recognition of the valuable document to be recognized, the multimodal information may be fused on four levels, such as the collection level, the feature level, the quantization level and/or the decision level. In the following embodiments of the method according to the present invention, the fusion strategies of decision level, feature level and combination thereof are taken as examples to describe the method for recognizing the valuable document. However, the method is not limited to thereto. 
         [0043]    Reference to  FIG. 4 , a schematic flow chart of a second embodiment of a method for recognizing a valuable document provided by an embodiment of the present invention is shown. The method includes the following steps: 
         [0044]    Step  401 : collecting multimodal information of a valuable document to be recognized, wherein the multimodal information includes two or more of optical information, electrical information, magnetic information, physical information and so on of the valuable document; and the valuable document may include a banknote, valuable securities, a ticket, a bill and so on; 
         [0045]    Step  402 : analyzing the multimodal information of the valuable document to be recognized and extracting features of the multimodal information, wherein the features of the multimodal information includes two or more of the feature of the optical information, the feature of the electrical information, the feature of the magnetic information, the feature of the physical information of the valuable document to be recognized. For example, by analyzing the multimodal information of the valuable document such as the banknote, the stable corresponding relationship between optical imaging position and the magnetic information of the magnetic safety line of the banknote may be obtained, and this corresponding relationship may be described by the textural characteristics of the optical image of the banknote, therefore the textural characteristics may be chosen as the features of the optical information; 
         [0046]    Step  403 : recognizing respectively each of the extracted features of the multimodal information and obtaining recognition results corresponding to these features. For example, a classifier is used to recognize the features, in which the feature of the magnetic information of the valuable document may be a first input feature of the classifier, and the feature of the physical information may be a second input feature of the classifier, and then the classification calculation can be performed for the above two input features respectively to obtain the classified recognition results; and 
         [0047]    Step  404 : performing a decision fusion for the recognition results according to a preset fusion strategy, and obtaining a decided recognition result, wherein the fusion strategy is the fusion strategy of the decision level, such as the AND method, i.e. when all the classification results satisfy the conditions for the decision fusion, for example, when the optical information, the magnetic information and the physical information of the banknote are all correct, the banknote can be accepted. 
         [0048]    According to this embodiment, the decision fusion is performed for the recognition results corresponding to the features of the multimodal information, and the recognition result is the conclusion obtained by synthesizing the recognized results of many features. Therefore, the reliability and accuracy of the recognition of the valuable document can be improved by the decision fusion. 
         [0049]    Reference to  FIG. 5 , a schematic flow chart of a third embodiment of a method for recognizing a valuable document provided by an embodiment of the present invention is shown. The method includes the following steps: 
         [0050]    Step  501 : collecting multimodal information of a valuable document to be recognized; 
         [0051]    Step  502 : analyzing the multimodal information of the valuable document to be recognized and extracting features of the multimodal information, wherein the features include features to be fused and features not to be fused, wherein the features to be fused are the features which will be fused and the number of these features is at least two; and the features not to be fused refer to the features which are not to be fused and the number of these features is not limited and of course may be zero; 
         [0052]    Step  503 : fusing the features to be fused according to a preset fusion strategy, and obtaining a new fused feature of the multimodal information. For example, the optical information under irradiation of different wavelengths, such as red light, infrared light and ultraviolet light, are fused to obtain a new feature which contains three types of optical information of the valuable document to be recognized. It is to be noted that the fusion strategy in this step is the fusion strategy of feature level, such as the weighted average method; and 
         [0053]    Step  504 : recognizing the valuable document to be recognized according to the features not to be fused and the new fused feature, and obtaining a recognition result. It is to be noted that, when the number of the features not to be fused is zero, the valuable document to be recognized may be recognized according to the new fused feature only to obtain the recognition result. 
         [0054]    According to this embodiment, the features of the multimodal information of the valuable document are fused to obtain a new fused feature which may represent the characteristics of the valuable document more accurately and completely. 
         [0055]    Reference to  FIG. 6 , a schematic flow chart of the fourth embodiment of a method for recognizing a valuable document provided by an embodiment of the present invention is shown. Steps  601  to  603  in this method are the same as the steps  501  to  503  in the third embodiment of the method for recognizing the valuable document, which will not be described in detail any more. Moreover, the step  504  in the third embodiment specifically is corresponding to step  604  and step  605  in this embodiment: 
         [0056]    Step  604 : recognizing respectively the features not to be fused and the new fused feature and obtaining recognition results corresponding to these features. For example, the new fused feature is a new feature of the optical information formed by fusing the red light, the infrared light and the ultraviolet light; and the features not to be fused include the features of the magnetic and physical information of the valuable document. The new feature of the optical information of the valuable document may be set as a first input feature of a classifier, the feature of the magnetic information of the valuable document may be set as a second input feature of the classifier, and the feature of the physical information may be set as a third input feature of the classifier, and then the classification calculation is performed for the above three input features respectively to obtain the classified results; and 
         [0057]    Step  605 : performing the decision fusion for the recognition results according to a preset fusion strategy, and obtaining a decided recognition result. 
         [0058]    According to this embodiment, the features of the multimodal information of the valuable document are fused and the decision fusion is performed for the recognition results of the features to obtain a decided recognition result. After two levels of fusion, the reliability and accuracy of the recognition of the valuable document are improved. 
         [0059]    In order to facilitate the understanding of the technical solution of the embodiments according to the present invention, the specific implementation of the embodiments according to the present invention will be described in detail hereinafter, by taking the banknote in the valuable document as an example. 
         [0060]    Step  1 : collecting the multimodal information of the banknote by a sensor, and in this example, the following information are chosen as the modal information of the banknote: 
         [0061]    1. red light information of the banknote; 
         [0062]    2. infrared light information of the banknote; 
         [0063]    3. ultraviolet light information of the banknote; 
         [0064]    4. magnetic information of the banknote; and 
         [0065]    5. physical information (thickness, format etc.) of the banknote. 
         [0066]    Step  2 : analyzing the relationship between the multimode information; forming knowledge rules; and storing the knowledge rules into a memory. According to the knowledge rules formed in this step, the fusion strategy may be established and the features of the multimodal information may be extracted. 
         [0067]    For the printed documents made of the same physical material in the same physical manner, there is a stable relationship between the imaging contents under irradiation of different wavelengths. Accordingly, the feature level fusion strategy is established, which is referred to as a first fusion rule herein: the optical information of different wavelengths may be fused on the feature level and the fusion strategy of the weighted average method is employed. 
         [0068]    Because of the uniformity among the optical, magnetic and physical information of the banknote, during the recognition, it may be rejected to recognize the banknote once one of the above information fails to meet the requirement. Accordingly, the fusion strategy of decision level is established, which is referred to as a second fusion rule here: the magnetic and physical information nay be fused on the decision level and the fusion strategy of AND is employed. 
         [0069]    Because there exists a stable corresponding relationship between the optical imaging position and the magnetic information of the magnetic safety line of the banknote, and these corresponding relationships may be represented by the textural characteristics of the optical image of the banknote, and thus the textural characteristics are chosen as the features of the optical information in this embodiment. 
         [0070]    Step  3 : extracting the features of the multimodal information of the banknote, wherein these features are the textural characteristics of the optical image of the banknote. 
         [0071]    1. The feature X 1 ={x 11 , x 12 , • • •, x 1n } is extracted from the red light information of the banknote; 
         [0072]    2. the feature X 2 ={x 21 , x 22 , • • •, x 2n } is extracted from the infrared light information of the banknote; 
         [0073]    3. the feature X 3 ={x 31 , x 32 , • • •, x 3n } is extracted from the ultraviolet light information of the banknote; 
         [0074]    4. the feature X 4 ={x 41 , x 42 , • • •, x 4n } is extracted from the magnetic information of the banknote; and 
         [0075]    5. the feature X 5 ={x 51 , x 52 , • • •, x 5n } is extracted from the physical information of the banknote. 
         [0076]    Where the symbol X k (k=1,2,3,4,5) represents a characteristic vector and the symbol x ki (i=1,2, • • •, n) represents a characteristic component in the characteristic vector. 
         [0077]    Step  4 : fusing the features. 
         [0078]    According to the fusion rule 1, the features X 1 , X 2 , X 3  of the optical information of the banknote are fused according to the weighted average method. The computation formula for the weighted average method is as follows: 
         [0000]        x′   i =ƒ( x   1i   , x   2i   , • • •, x   mi )= W   1   x   1i   +W   2   x   2i   + • • • +W   m   x   mi    (1)
 
         [0000]    where x′ i  is the characteristic component of the new fused feature X′; x ki  is the characteristic component of the feature X k , and x ki  ∈ X k ; W k  is the weight factor, W k &gt;0 and 
         [0000]    
       
         
           
             
               
                 ∑ 
                 
                   k 
                   = 
                   1 
                 
                 m 
               
                
               
                 W 
                 k 
               
             
             = 
             1. 
           
         
       
     
         [0079]    X 1 , X 2  and X 3  are fused according to the formula (1), where m=3, then X′={x′ 1 , x′ 2 , • • •, x′ n }. 
         [0080]    The advantageous effects of this step are as follows: the features of the three light information (red light, infrared light, ultraviolet light) are fused to obtain a new feature X′ which contains all the three types of light information of the banknote and may represent the banknote more accurately and completely. 
         [0081]    Step  5 : classifying the features. 
         [0082]    1. Classifier 
         [0083]    (1) Descriptions of the Classifier 
         [0084]    Provided that D={D 1 ,D 2 ,D 3 } represents a group of classifiers, where D i (i=1,2,3) represents a component classifier. 
         [0085]    In this embodiment, the Bayesian network is chosen as the classifier D 1 , the three-layer BP network, namely the three-layer feed-forward network, is chosen as the classifier D 2  and the decision tree is chosen as the classifier D 3 . 
         [0086]    (2) input 
         [0087]    The characteristic vector X ∈           is input; and different component classifiers correspond to different input characteristic vectors. 
         [0088]    The input of the classifier D 1  is the fused feature X′ of the optical information; 
         [0089]    the input of the classifier D 2  is the feature X 4  of the magnetic information; and 
         [0090]    the input of the classifier D 3  is the feature X 5  of the physical information. 
         [0091]    (3) Descriptions of Classes 
         [0092]    Provided that Θ={ω 1 , ω 2 , • • •, ω L } represents a group of class marks, ω i  represents the i-th class. 
         [0093]    (4) Output 
         [0094]    The output of the component classifier is a vector having a length of L: D i (X)=[d i1 (X),d i2 (X), • • •, d iL (X)] T , 
         [0095]    where d ij (X) represents the support degree of D i  to X belonging to ω i , and 
         [0000]    
       
         
           
             
               
                 ∑ 
                 
                   j 
                   = 
                   1 
                 
                 L 
               
                
               
                 
                   d 
                   ij 
                 
                  
                 
                   ( 
                   X 
                   ) 
                 
               
             
             = 
             1. 
           
         
       
     
         [0096]    The output of the classified D 1  is D 1 (X′)−[d 11 (X′), d 12 (X′), • • •, d 1L (X′)] T ; 
         [0097]    the output of the classifier D 2  is D 2 (X 4 )=[d 21 (X 4 , d 22 (X 4 ), • • •, d 2L (X 4 )] T ; and 
         [0098]    the output of the classified D 3  is D 3 (X 5 )=[d 31 (X 5 ), d 32 (X 5 ), • • •, d 3L (X 5 )] T . 
         [0099]    The classification result of each component classifier is that 
         [0000]        O   i =max{ d   y ( X )| X ∈              ″}   (2)
 
         [0100]    where O i  represents class, i=1,2,3, j=1,2, • • •, L. 
         [0101]    2. Training 
         [0102]    A group of banknotes are chosen as training samples. Provided that a sample set with N samples is Ω={B 1 ,B 2 , • • •, B N }, where B k (k=1,2, • • •, N) represents the k-th sample. 
         [0103]    The class mark is assigned to the sample B k (k1,2, • • •, N) in the training sample set Ω. Provided that the mark of B k  is ω t , then the outputs of the component classifiers meet the following constraint conditions: 
         [0104]    (1) for the classified D 1 : 
         [0000]    
       
         
           
             
               
                 
                   O 
                   1 
                 
                  
                 
                   ( 
                   
                     B 
                     k 
                   
                   ) 
                 
               
               = 
               
                 
                   
                     ω 
                     t 
                   
                   ⇔ 
                   
                     
                       d 
                       
                         1 
                          
                         
                             
                         
                          
                         t 
                       
                     
                      
                     
                       ( 
                       
                         X 
                         ′ 
                       
                       ) 
                     
                   
                 
                 = 
                 
                   
                     max 
                     
                       
                         j 
                         = 
                         1 
                       
                       , 
                       2 
                       , 
                       
                           
                       
                        
                       … 
                        
                       
                           
                       
                       , 
                       L 
                     
                   
                    
                   
                     { 
                     
                       
                         d 
                         
                           1 
                            
                           
                               
                           
                            
                           j 
                         
                       
                        
                       
                         ( 
                         X 
                         ) 
                       
                     
                     } 
                   
                 
               
             
             ; 
           
         
       
     
         [0105]    (2) for the classifier D 2 : 
         [0000]    
       
         
           
             
               
                 
                   O 
                   2 
                 
                  
                 
                   ( 
                   
                     B 
                     k 
                   
                   ) 
                 
               
               = 
               
                 
                   
                     ω 
                     t 
                   
                   ⇔ 
                   
                     
                       d 
                       
                         2 
                          
                         
                             
                         
                          
                         t 
                       
                     
                      
                     
                       ( 
                       
                         X 
                         4 
                       
                       ) 
                     
                   
                 
                 = 
                 
                   
                     max 
                     
                       
                         j 
                         = 
                         1 
                       
                       , 
                       2 
                       , 
                       
                           
                       
                        
                       … 
                        
                       
                           
                       
                       , 
                       L 
                     
                   
                    
                   
                     { 
                     
                       
                         d 
                         
                           2 
                            
                           j 
                         
                       
                        
                       
                         ( 
                         
                           X 
                           4 
                         
                         ) 
                       
                     
                     } 
                   
                 
               
             
             ; 
           
         
       
     
         [0000]    and 
         [0106]    (3) for the classified D 3 : 
         [0000]    
       
         
           
             
               
                 O 
                 3 
               
                
               
                 ( 
                 
                   B 
                   k 
                 
                 ) 
               
             
             = 
             
               
                 
                   ω 
                   t 
                 
                 ⇔ 
                 
                   
                     d 
                     
                       3 
                        
                       
                           
                       
                        
                       t 
                     
                   
                    
                   
                     ( 
                     
                       X 
                       5 
                     
                     ) 
                   
                 
               
               = 
               
                 
                   max 
                   
                     
                       j 
                       = 
                       1 
                     
                     , 
                     2 
                     , 
                     
                         
                     
                      
                     … 
                      
                     
                         
                     
                     , 
                     L 
                   
                 
                  
                 
                   
                     { 
                     
                       
                         d 
                         
                           3 
                            
                           
                               
                           
                            
                           j 
                         
                       
                        
                       
                         ( 
                         
                           X 
                           5 
                         
                         ) 
                       
                     
                     } 
                   
                   . 
                 
               
             
           
         
       
     
         [0107]    Each component classifier D i (i=1,2,3) is trained by the training samples set until the outputs of each component classifier meet the above three constraint conditions for any sample B k . 
         [0108]    3. Classification 
         [0109]    The features of multimodal information of the target, namely the banknote to be recognized, are computed by the trained classifier to obtain a group of classification output results O 1 , O 2 , O 3 . 
         [0110]    The advantageous effects of this step are as follows: one implementation of each component classifier may be obtained by training the classifier; the features of the multimodal information of the target banknote are computed utilizing the component classifiers obtained by training to obtain a group of candidate classification results O 1 , O 2 , O 3  which may be used for the decision fusion. 
         [0111]    Step 6: performing the decision fusion. 
         [0112]    According to the fusion rule 2, the decision fusion is performed by utilizing the AND method, and the computation formula of the decision fusion is as follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     O 
                      
                     
                       ( 
                       B 
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             
                               ω 
                               t 
                             
                              
                             
                               ( 
                               
                                 
                                   t 
                                   = 
                                   1 
                                 
                                 , 
                                 2 
                                 , 
                                 … 
                                  
                                 
                                     
                                 
                                 , 
                                 L 
                               
                               ) 
                             
                           
                         
                         
                           
                             
                               when 
                                
                               
                                   
                               
                                
                               
                                 
                                   O 
                                   1 
                                 
                                  
                                 
                                   ( 
                                   B 
                                   ) 
                                 
                               
                             
                             = 
                             
                               
                                 
                                   O 
                                   2 
                                 
                                  
                                 
                                   ( 
                                   B 
                                   ) 
                                 
                               
                               = 
                               
                                 
                                   
                                     O 
                                     3 
                                   
                                    
                                   
                                     ( 
                                     B 
                                     ) 
                                   
                                 
                                 = 
                                 
                                   ω 
                                   t 
                                 
                               
                             
                           
                         
                       
                       
                         
                           rejected 
                         
                         
                           otherwise 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0113]    where B represents the target to be recognized, such as the banknote, O i (B)(i=1,2,3) represents the classification results of the component classifiers, and ω 1  represents the class. 
         [0114]    The decision fusion is performed for the results obtained by classifying by the classifier according to the formula (3) to get the final recognition result. That is to say, the target banknote will be accepted if the classification result O 1  of the features of the optical information, the classification result O 2  of the features of the magnetic information, and the classification result O 3  of the features of the physical information all meet the conditions in the formula of the decision fusion, and the target banknote will be rejected if one of the conditions is not satisfied. 
         [0115]    According to this step, the decision fusion is performed for a group of the candidate classification results, so as to improve the reliability and accuracy of the final recognition result. 
         [0116]    According to this embodiment, the recognition of the banknote is achieved utilizing the multimodal information of the banknote via two levels of fusion. During the recognition, types of multimodal information of the banknote are synthesized, and the multimodal information may represent the characteristics of the valuable document more accurately and completely, so as to improve the reliability and accuracy of the recognition of the banknote. 
         [0117]    Taking the banknote as an example, the counterfeit banknote recognition using the fusion technique of the multimodal information described above is only a simple example. The fusion of the multimodal information may also be divided into three levels: source data level fusion, feature level fusion, and decision level fusion. 
         [0118]    Among the three levels, the source data level fusion is aimless and is not recommended to fusing the information in principle. 
         [0119]    In the aspect of the feature level fusion according to the present invention, besides the weighted average method in the embodiment, the following fusion rules may be employed as required; and in the aspect of the decision level fusion, besides the AND method in the embodiment, the following fusion rules may also be employed as required. 
         [0120]    The feature level fusion may be divided into two types as follows: 
         [0121]    (1) Target State Information Fusion 
         [0122]    For date parameters correlation and state estimation, the target state information fusion mainly includes information fusion rules such as the sequential estimation method and Kalman filtering method. 
         [0123]    (2) Target Characteristic Fusion 
         [0124]    For the combinations of feature vectors, the target characteristic fusion mainly includes fusion rules such as the clustering, the neural network, the weighted average method, the maximum value method, the minimum value method and the average summation method. 
         [0125]    In the aspect of the decision level fusion: 
         [0126]    For the joint decision problems, the decision level fusion mainly includes fusion rules such as the logic combination of “AND” and “OR”, Bayes theory, D-S evidence theory, the production rules, the fuzzy set theory, the rough set theory and the expert system. 
         [0127]    Reference to  FIG. 7 , a schematic composition diagram of a first embodiment of a device for recognizing a valuable document provided by an embodiment of the present invention is shown. As shown in the figure, a device  70  for recognizing a valuable document includes: 
         [0128]    a collection module  71  for collecting multimodal information of a valuable document to be recognized, wherein the multimodal information includes two or more of optical information, electrical information, magnetic information, physical information and so on of the valuable document to be recognized; and the valuable document may include a banknote, valuable securities, a ticket, a bill and so on; 
         [0129]    a storage module  72  for storing a preset fusion strategy and the multimodal information collected by the collection module  71 ; wherein the preset fusion strategy is a fusion strategy based on the multimodal information of the valuable document which is generated according to inherent characteristics of a standard valuable document; and 
         [0130]    a recognition module  73  for recognizing the valuable document to be recognized according to the fusion strategy stored by the storage module  72  and the multimodal information of the valuable document to be recognized, and obtaining a recognition result. 
         [0131]    According to this embodiment, the recognition of a valuable document based on multimodal information is achieved by collecting multimodal information of the valuable document to be recognized; and recognizing the valuable document to be recognized according to a preset fusion strategy and the multimodal information of the valuable document to be recognized, and obtaining a recognition result, thus improving the reliability and accuracy of the recognition. 
         [0132]    Reference to  FIG. 8 , a schematic composition diagram of a second embodiment of a device for recognizing a valuable document provided by an embodiment of the present invention is shown. As shown in the figure, compared with the first embodiment of the device for recognizing a valuable document, the device in this embodiment has the same collection module  71  and the same storage module  72 , while the recognition module  73  includes: 
         [0133]    a second feature extraction unit  731  for analyzing multimodal information of the valuable document to be recognized which is stored by the storage module  72  and extracting the features of the multimodal information; 
         [0134]    a second recognition unit  732  for recognizing respectively the features of the multimodal information extracted by the second feature extraction unit  731  and obtaining recognition results corresponding to these features; and 
         [0135]    a decision fusion unit  733  for performing a the decision fusion for the recognition results obtained by the second recognition unit  732  according to a fusion strategy of decision level in the fusion strategy stored by the storage module  72 , and obtaining a decided recognition result, in which the fusion strategy is a fusion strategy on the decision level. 
         [0136]    It is to be noted that the functions performed by the above units of the recognition module  73  refer to the corresponding description of the second embodiment of the method for recognizing the valuable document. 
         [0137]    According to this embodiment, the decision fusion is performed for the recognition results corresponding to the features of the multimodal information. The recognition result is the conclusion obtained by synthesizing the recognized results of many features. Therefore, the reliability and the accuracy of the recognition of the valuable document are improved by the decision fusion. 
         [0138]    Reference to  FIG. 9 , a schematic composition diagram of a third embodiment of a device for recognizing a valuable document provided by an embodiment of the present invention is shown. As shown in the figure, compared with the first embodiment of the device for recognizing a valuable document, the device for recognizing in this embodiment has the same collection module and the same storage module, while the recognition module  73  includes: 
         [0139]    a first feature extraction unit  734  for analyzing multimodal information of the valuable document to be recognized which is stored by the storage module  72 , and extracting features of the multimodal information, wherein the features include features to be fused and features not to be fused; 
         [0140]    a feature fusion unit  735  for fusing the features to be fused which are extracted by the first feature extraction unit  734  according to a fusion strategy of feature level in the fusion strategies stored by the storage module  72 , and obtaining a new fused feature of the multimodal information; and 
         [0141]    a first recognition unit  736  for recognizing the valuable document to be recognized according to the features not to be fused which are extracted by the first feature extraction unit  734  and the new fused feature obtained by the feature fusion unit  735 , and obtaining a recognition result. 
         [0142]    It is to be noted that the functions performed by each above units of the recognition module  73  refer to the corresponding description of the third embodiment of the method for recognizing the valuable document. 
         [0143]    According to this embodiment, the new fused feature is obtained by fusing the features of the multimodal information of the valuable document. The new feature contains types of modal information of the valuable document and may represent the characteristics of the valuable document more accurately and completely. 
         [0144]    Reference to  FIG. 10 , a schematic composition diagram of a first recognition unit of the third embodiment of the device for recognizing a valuable document provided by a third embodiment of the present invention is shown; and reference to  FIG. 9  together, in this embodiment, the first recognition unit  736  includes: 
         [0145]    a recognition subunit  7361  for recognizing respectively the features not to be fused which are extracted by the first feature extraction unit  734  and the new fused feature obtained by the feature fusion unit  735 , and obtaining recognition results corresponding to these features; and 
         [0146]    a decision subunit  7362  for performing the decision fusion for the recognition results obtained by the recognition subunit  7361  according to a fusion strategy of decision level in the fusion strategy stored by the storage module  72 , and obtaining a decided recognition result. 
         [0147]    It is to be noted that, the functions performed by each above subunits of the first recognition unit  736  refer to the corresponding description of the fourth embodiment of the method for recognizing the valuable document. 
         [0148]    Moreover, the device for recognizing the valuable document in an embodiment according to the present invention may include only a collection module and a recognition module, in which the collection module is adapted to collect multimodal information of a valuable document to be recognized, and the multimodal information includes two or more of the optical information, the electrical information, the magnetic information and the physical information of the valuable document to be recognized; and the recognition module is adapted to recognize the valuable document to be recognized according to a pre-generated fusion strategy and the collected multimodal information of the valuable document to be recognized, and obtain a recognition result. 
         [0149]    Optionally, the device may further include a pre-generation module for generating in advance, a fusion strategy based on the multimodal information of the valuable document, according to the inherent characteristics of a standard valuable document, wherein the fusion strategy generated by the pre-generation module is a pre-generated fusion strategy. 
         [0150]    Optionally, the device may further include a storage module for storing the pre-generated fusion strategy, and the multimodal information collected by the collection module. 
         [0151]    In this embodiment, the recognition module may include a first feature extraction unit, a feature fusion unit and a first recognition unit; and the first recognition unit may include a recognition subunit and a decision subunit; the recognition module may include a second feature extraction unit, a second recognition unit and a decision fusion unit, wherein the descriptions of the functions of each units or subunits are as above and will not be described in detail any more. 
         [0152]    According to this embodiment, the features of the multimodal information of the valuable document are fused, the recognition results of the features are fused on decision level to obtain a decided recognition result. After two levels of fusion, the reliability and the accuracy of the recognition of the valuable document are improved. 
         [0153]    In other embodiments according to the present invention, a product related to the recognition of a valuable document includes a part of or all of the units in the recognition device in the embodiments according to the present invention. For example, a control sensor can be the collection module  71  in the embodiments of the present invention; a memory can be the storage module  72  in the embodiments of the present invention; a processor can be the recognition module  73  in the embodiments of the present invention. Further, the processor also includes a second feature extraction unit  731 , a second recognition unit  732 , a decision fusion unit  733 , a first feature extraction unit  734 , a feature fusion unit  735 , a first recognition unit  736 , a recognition subunit  7361  and a decision subunit  7362 . 
         [0154]    It is to be noted that, except the single level fusion and the two level fusion based on the feature level and the decision level described above in embodiments, in other embodiments according to the present invention, the multimodal information can be fused on the collection level and/or the multimodal information of a valuable documents can be fused on the quantization level. In a word, the multimodal information of a valuable document may be fused on a group of levels selected from the four levels, i.e. the collection level, the quantization level, the feature level and the decision level. Further, the quantization level fusion includes two steps: normalizing and fusing; the fusion strategy of feature level is not limited to the weighted average method mentioned in the above embodiments and may further include the average summation method, the maximum value method and the minimum value method etc; the fusion strategy of decision level is also not limited to the AND method mentioned in the above embodiments, which is mainly divided into two kinds: one is a method in which parameters are not to be trained, such as the voting method, the AND method and the OR method, and the other is a method in which parameters are to be trained, such as the D-S evidence theory, Bayes estimation method, fuzzy clustering method. 
         [0155]    All those disclosed above are only preferred embodiments according to the present invention and can certainly not be used to define the scope of protection of the claims of the present invention. Accordingly, the equivalent alternation made based on the present invention still falls within the scope of protection of the present invention.