Patent Publication Number: US-11380348-B2

Title: Method and system for correcting infant crying identification

Description:
RELATED APPLICATIONS 
     This application claims priority to Taiwan Application Serial Number 109124847, filed Jul. 22, 2020, which is herein incorporated by reference. 
     BACKGROUND 
     Technical Field 
     The present disclosure relates to a method and a system for correcting sound identification. More particularly, the present disclosure relates to a method and a system for correcting infant crying identification with an incremental learning. 
     Description of Related Art 
     Before infants learn to speak, they express their physical, psychological, and emotional needs in terms of crying or facial expressions. Therefore, for infants, crying and facial expressions can be regarded as an innate special language. In general, the causes of infant crying are mostly uncomfortable environment, needing to be taken care of or feeding and emotional dependence. When the caregivers or parents look after the infants, they mostly guess the reason behind the infant crying with experience. There is no objective and reliable basis for judgment, especially for novice parents who lack parenting experience, and they often feel helpless when the infant is crying. 
     In the conventional system for crying identification, because of the difference in infant growth and environmental factors, the identification result of the infant differs from the real result of the infant. When the conventional system for crying identification is used by the caregiver in the actual environment, the training samples for crying are usually impossible to get all at once, and the information reflected by the training samples can also change over time. When the traditional method with machine learning encounters newly added training samples, the traditional method with machine learning usually needs to retrain the entire data, which can easily cause catastrophic forgetting effects. 
     Therefore, how to improve the application of machine learning to the system for sound identification, make the system avoid catastrophic forgetting effects and reduce the time for re-training and identification to train personalized models become the goal of relevant industry efforts. 
     SUMMARY 
     According to an embodiment of a methodical aspect of the present disclosure, a method for correcting infant crying identification is for identifying a crying of an infant and includes a detecting step, a converting step, an extracting step, an incremental training step, a judging step and a correcting step. The detecting step provides an audio unit to detect a sound around the infant to generate a plurality of audio samples. The converting step provides a processing unit to convert the audio samples to generate a plurality of audio spectrograms. The extracting step provides a common model to extract the audio spectrograms to generate a plurality of infant crying features. The incremental training step provides an incremental model to train the infant crying features to generate an identification result. The judging step provides the processing unit to judge whether the identification result is correct according to a real result of the infant. When the identification result is different from the real result, an incorrect result is generated. The correcting step provides the processing unit to correct the incremental model according to the incorrect result. 
     According to an embodiment of a structural aspect of the present disclosure, a system for correcting infant crying identification is for identifying a crying of an infant. The system for correcting infant crying identification includes an audio unit and a processing module. The audio unit detects a sound around the infant to generate a plurality of audio samples. The processing module is electrically connected to the audio unit and includes a memory and a processing unit. The memory includes a real result, a common model and an incremental model. The real result is corresponding to the crying of the infant. The common model is created from the audio samples. The incremental model is created from a plurality of infant crying features. The processing unit is connected to the audio unit and the memory and receives the audio samples. The processing unit converts the audio samples to generate a plurality of audio spectrograms. The processing unit extracts the audio spectrograms through the common model to generate a plurality of infant crying features. The processing unit trains the infant crying features through the incremental model to generate an identification result and judges whether the identification result is correct according to the real result. When the identification result is different from the real result, the processing unit generates an incorrect result and corrects the incremental model according to the incorrect result. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The present disclosure can be more fully understood by reading the following detailed description of the embodiment, with reference made to the accompanying drawings as follows: 
         FIG. 1  is a block diagram of a system for correcting infant crying identification according to an embodiment of a structural aspect of the present disclosure. 
         FIG. 2  is a flow chart of a method for correcting infant crying identification according to an embodiment of a methodical aspect of the present disclosure. 
     
    
    
     DETAILED DESCRIPTION 
     The embodiment will be described with the drawings. For clarity, some practical details will be described below. However, it should be noted that the present disclosure should not be limited by the practical details, that is, in some embodiment, the practical details is unnecessary. In addition, for simplifying the drawings, some conventional structures and elements will be simply illustrated, and repeated elements may be represented by the same labels. 
     It will be understood that when an element (or device) is referred to as be “connected to” another element, it can be directly connected to the other element, or it can be indirectly connected to the other element, that is, intervening elements may be present. In contrast, when an element is referred to as be “directly connected to” another element, there are no intervening elements present. In addition, the terms first, second, third, etc. are used herein to describe various elements or components, these elements or components should not be limited by these terms. Consequently, a first element or component discussed below could be termed a second element or component. 
       FIG. 1  is a block diagram of a system  100  for correcting infant crying identification according to an embodiment of a structural aspect of the present disclosure. In  FIG. 1 , the system  100  for correcting infant crying identification is for identifying a crying of an infant. The system  100  for correcting infant crying identification includes an audio unit  110  and a processing module  120 . In the embodiment, the audio unit  110  can be a recording device, and the processing module  120  can be a computer, but the present disclosure is not limited thereto. 
     The audio unit  110  detects a sound around the infant to generate a plurality of audio samples  111 . The processing module  120  is electrically connected to the audio unit  110  and includes a processing unit  121  and a memory  122 . The memory  122  includes a real result  1221 , a common model  1222  and an incremental model  1223 . The real result  1221  is corresponding to the crying of the infant. 
     The processing unit  121  is connected to the audio unit  110  and the memory  122  and receives the audio samples  111 . The processing unit  121  converts the audio samples  111  to generate a plurality of audio spectrograms  112 . The processing unit  121  extracts the audio spectrograms  112  through the common model  1222  to generate a plurality of infant crying features  113 . The common model  1222  is created in advance from the first one of the audio samples  111 , and the incremental model  1223  is created in advance from the first one of the infant crying features  113 . The training of the above models is a conventional art and is not the focus of the present disclosure, and will not be detailedly described herein. The processing unit  121  trains the infant crying features  113  through the incremental model  1223  to generate an identification result  114  and judges whether the identification result  114  is correct according to the real result  1221 . Particularly, when the identification result  114  is the same as the real result  1221 , the processing unit  121  generates a correct result (not shown), and the correct result represents the cause of infant crying. More particularly, when the identification result  114  is different from the real result  1221 , the processing unit  121  generates an incorrect result (not shown) and then corrects the incremental model  1223  according to the incorrect result. 
     Therefore, the incremental model  1223  can be gradually updated with adding different infant crying features  113 , so that the incremental model  1223  need not to learn all the audio samples  111  again. 
     Please refer to  FIGS. 1 and 2 .  FIG. 2  is a flow chart of a method  200  for correcting infant crying identification according to an embodiment of a methodical aspect of the present disclosure. In  FIG. 2 , the method  200  for correcting infant crying identification can be applied to the system  100  for correcting infant crying identification in  FIG. 1 . The method  200  for correcting infant crying identification is for identifying the crying of the infant and includes a detecting step S 1 , a converting step S 2 , an extracting step S 3 , an incremental training step S 4 , a judging step S 5  and a correcting step S 6 . 
     The detecting step S 1  provides the audio unit  110  to detect the sound around the infant to generate the audio samples  111 . The converting step S 2  provides the processing unit  121  to convert the audio samples  111  to generate the audio spectrograms  112 . The extracting step S 3  provides the common model  1222  to extract the audio spectrograms  112  to generate the infant crying features  113 . The incremental training step S 4  provides the incremental model  1223  to train the infant crying features  113  to generate the identification result  114 . The judging step S 5  provides the processing unit  121  to judge whether the identification result  114  is correct according to the real result  1221  of the infant. Especially, when the identification result  114  is the same as the real result  1221 , the processing unit  121  generates a correct result (not shown), the correct result represents the cause of infant crying, and the correcting step S 6  will not be executed. More especially, when the identification result  144  is different from the real result  1221 , the incorrect result is generated. The correcting step S 6  is executed and provides the processing unit  121  to correct the incremental model  1223  according to the incorrect result. Furthermore, an execution order of the steps of the method  200  for correcting infant crying identification is the detecting step S 1 , the converting step S 2 , the extracting step S 3 , the incremental training step S 4 , the judging step S 5  and the correcting step S 6 . 
     Therefore, the infant crying feature  113  can be identified through the incremental model  1223 . When the identification result  114  is different from the real result  1221 , the method  200  for correcting infant crying identification can correct the previous incremental model  1223  so as to generate a crying identification model exclusively for each of the infants. 
     In detail, as the infant grows and changes in the environment, the identification result  114  is easily to be different from the real result  1221 . For example, when the identification result  114  indicates that the infant crying is that the infant is frightened, but the actual situation is that the infant is not fed, the processing unit  121  judges that the identification result  114  and the actual situation are different according the real result  1221 . Therefore, during the next identification, the incremental model  1223  is corrected and updated at the same time. Especially, the method  200  for correcting infant crying identification can adjust a cumulative number (not shown) of the incorrect results to correct the incremental model  1223 . In the method  200  for correcting infant crying identification, the judging step S 5  is performed to set the cumulative number of the incorrect results to 0. In response to the identification result  114  is different from the real result  1221 , the cumulative number is accumulated. In response to the cumulative number is between 5 and 10, the correcting step S 6  is executed. 
     Furthermore, in the converting step S 2 , the processing unit  121  can convert audio samples  111  through a Fast Fourier Transform (FFT) to generate the audio spectrograms  112 , and each of the audio spectrograms is a Mel-spectrogram. In the processing of signals, the Mel-spectrogram is a spectrogram that can be used to represent a short-term audio. FFT and the Mel-spectrogram are both conventional arts and not the focus of the present disclosure, and will not be detailedly described herein. 
     Moreover, in the extracting step S 3 , the common model  1222  is a convolutional neural network (CNN). It is worth noting that when each of the audio samples  111  is an infant crying sample, the processing unit  121  trains the infant crying samples to generate the common model  1222 , so that the processing unit  121  extracts the audio spectrograms  112  according to the common model  1222  to generate the infant crying features  113 . In detail, the audio spectrograms  112  can be any kind of sound spectrogram. Therefore, the common model  1222  is pre-trained from the processing unit  121  for extracting the sound spectrograms of the infant crying in the audio spectrograms  112 , so that the incremental model  1223  receives the marked crying samples (that is, the infant crying features  113 ). 
     Moreover, in the incremental training step S 4 , the incremental model  1223  is a bidirectional recurrent neural network (BRNN). The incremental model  1223  is based on an algorithm of an incremental learning to gradually update parameters of a model and can correct and strengthen the parameters, so that the updated parameters can adapt to the added data and do not need to relearn all the data. Therefore, the incremental model  1223  reduces the demand for time and space and can more satisfy the actual requirements so as to avoid the problem of the catastrophic forgetting effects. 
     In summary, the present disclosure has the following advantages: First, it is favorable for greatly reducing the time for retraining and identification. Second, it is favorable for the previous incremental model to be corrected and updated so as to generate the crying identification model exclusively for each of the infants. 
     Although the present disclosure has been described in considerable detail with reference to certain embodiments thereof, other embodiments are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the embodiments contained herein. 
     It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the present disclosure without departing from the scope or spirit of the disclosure. In view of the foregoing, it is intended that the present disclosure cover modifications and variations of this disclosure provided they fall within the scope of the following claims.