Patent Description:
The present invention relates to a method and apparatus for processing subject sounds for automatic detection of cough sounds therein.

Any references to methods, apparatus or documents of the prior art are not to be taken as constituting any evidence or admission that they formed, or form part of the common general knowledge.

It is known to electronically process subject sounds to predict the presence of respiratory maladies. Where a symptom of the malady is coughing in the subject then it is important to be able to identify segments of the subject sounds that contain coughs, as opposed to background noise for example.

A number of approaches to identifying cough segments of patient sounds are known in the prior art. For example, in <CIT>at al. there is described a method for cough detection which involves determining a number of features for each of a plurality of segments of a subject's sound, forming a feature vector from those features and applying them to a pre-trained classifier. The output from the classifier is then processed to deem the segments as either "cough" or "non-cough".

A more recent approach to identifying portions of subject sounds that contain coughs is described in <CIT> (sometimes called the "LW2" method herein) in which feature vectors from the subject sound are applied to two pre-trained neural nets being respectively trained for detecting an initial phase of a cough sound and a subsequent phase of a cough sound. The first neural net is weighted in accordance with positive training to detect the initial, explosive phase, and the second neural net is positively weighted to detect one or more post-explosive phases of the cough sound. In a preferred embodiment of the LW2 method the first neural net is further weighted in accordance with positive training in respect of the explosive phase and negative training in respect of the post-explosive phases. LW2 is particularly good at identifying cough sounds in a series of connected coughs.

The Inventors have noticed that a problem that can occur with prior art cough identification methods is that they may have undesirably low specificity which means that they identify sound segments as being cough sounds when in fact they are not. Such false positive detection may make those methods infeasib le for long term use in high background noise environments where the number of non-cough events in the subject sound recording is much greater than the number of cough events.

It is known from the publication<NPL>, a technique for detecting cough sounds using a deep neural network and for confirming the potential cough sound using a Hidden Markov Model.

It is also known according to the patent <CIT>, a technique for confirming a potential cough sound by using a set of four confirmatory frequencies.

It is also known from the publication<NPL>, a technique for detecting a cough sound using a trained convolutional neural network using an image representation.

It would be desirable if a method and apparatus were provided that can reduce the number of false positives.

A method for identifying cough sounds in an audio recording of a subject comprising:.

In an embodiment the method includes, operating said processor to transform the one or more sounds into the image representations wherein the image representations relate frequency and time.

In an embodiment, the one or more image representations comprise spectrograms.

In an embodiment, the one or more image representations comprise mel-spectrograms.

In an embodiment, the method includes, operating said processor to identify the potential cough sounds as cough audio segments of the audio recording by using first and second cough sound pattern classifiers trained to respectively detect initial and subsequent phases of cough sounds.

In an embodiment, the one or more image representations have a dimension of N x M pixels and are formed by said processor processing N windows of each of the cough audio segments wherein each of the N windows is analyzed in M frequency bins.

In an embodiment, each of the N windows overlaps with at least one other of the N windows.

In an embodiment, length of the windows is proportional to length of its associated cough audio segment.

In an embodiment the method includes operating said processor to calculate a Fast Fourier Transform (FFT) and a power value per frequency bin to arrive at a corresponding pixel value of the corresponding image representation of the or more image representations.

In an embodiment the method includes, operating said processor to calculate a power value per frequency bin in the form of M power values, being power values for each of the M frequency bins.

In an embodiment, the M frequency bins comprise M mel-frequency bins, the method including operating said processor to concatenate and normalize the M power values to thereby produce the corresponding image representation in the form of a mel-spectrogram image.

In an embodiment, the image representations are square and wherein M equals N.

In an embodiment, the representation pattern classifier comprises a neural network.

In an embodiment, the neural network is a convolutional neural network (CNN).

In an embodiment the method includes, operating said processor to compare a probability value comprising, or based upon, an output of the representation pattern classifier with a predetermined threshold value.

In an embodiment the method includes, operating said processor to flag one or more of the potential cough sounds as confirmed cough sounds upon the probability value exceeding the predetermined threshold value.

In an embodiment the method includes, operating said processor to flag the confirmed cough sounds by recording begin and end times of the corresponding cough audio segment as being begin and end times of a confirmed cough sound.

In an embodiment the method includes, operating said processor to generate a screen on a display responsive to said processor, the screen indicating the number of potential cough sounds processed and the number of confirmed cough sounds.

According to a further apparatus there is provided an apparatus for identifying cough sounds in a subject comprising:.

In an embodiment the apparatus includes, or more cough sound classifiers trained to identify portions of the digital audio recording to thereby produce the pre-identified potential cough sounds.

In an embodiment, the one or more cough sound classifiers comprise a first cough sound pattern classifier and a second cough sound pattern classifiers trained to respectively detect initial and subsequent phases of cough sounds.

In an embodiment, the first cough sound pattern classifier and the and second cough sound pattern classifier each comprise neural networks.

In an embodiment, the sound segment-to-image representation assembly is arranged to transform the pre-identified potential cough sounds into corresponding image representations comprising spectrograms.

In an embodiment, the sound segment-to-image representation assembly is arranged to transform the pre-identified potential cough sounds into corresponding image representations by calculating a Fast Fourier Transform and a power per bin for M to the pre-identified potential cough sounds.

In an embodiment, the sound segment-to-image representation assembly is arranged to transform the pre-identified potential cough sounds into spectrograms.

In an embodiment, the spectrograms comprise mel-spectrograms.

In an embodiment, the apparatus includes at least one electronic processor in communication with the electronic memory, wherein the processor is configured by instructions stored in the electronic memory to implement the sound segment-to-image representation assembly.

In an embodiment, the at least one electronic processor is configured by instructions stored in the electronic memory to implement the representation pattern classifier.

In an embodiment, the at least one electronic processor is configured by instructions stored in the electronic memory to implement the at least one cough sound pattern classifier arranged to identify the potential cough sounds.

According to a further aspect of the present invention there is provided a method for training a pattern classifier to confirm a potential cough sound as a confirmed cough sound from a sound recording of the subject, the method comprising:.

According to another aspect there is provided a method for identifying cough sounds in an audio recording of a subject including transforming potential cough sounds in the audio recording into corresponding image representations and then applying the image representations to a pre-trained classifier and based on output from the pre-trained classifier flagging the potential cough sounds as confirmed cough sounds or not.

According to a further aspect there is provided an apparatus for processing potential cough sounds identified in an audio recording of a subject, the apparatus including at least one electronic processor in communication with a digital memory storing instructions to configure said processor to implement the method.

According to another aspect of the present invention there is provided a computer readable media bearing tangible, non-transitory machine readable instructions for one or more processors to implement a method for confirming a potential cough sound to be a confirmed cough sound based on an image representation of the potential cough sound.

Preferred features, embodiments and variations of the invention may be discerned from the following Detailed Description which provides sufficient information for those skilled in the art to perform the invention. The Detailed Description is not to be regarded as limiting the scope of the preceding Summary of the Invention in any way. The Detailed Description will make reference to a number of drawings as follows:.

<FIG> presents a flowchart of a method according to a preferred embodiment of the present invention for automatic cough detection.

A hardware platform that is configured to implement the method comprises a cough identification machine. The machine may be a desktop computer or a portable computational device such as a smartphone that contains at least one processor in communication with an electronic memory that stores instructions that specifically configure the processor in operation to carry out the steps of the method as will be described. It will be appreciated that it is impossible to carry out the method without the specialized hardware, i.e. either a dedicated machine or a machine that is comprised of specially programmed one or more processors. Alternatively, the machine may be implemented as a dedicated assembly that includes specific circuitry to carry out each of the steps that will be discussed. The circuitry may be largely implemented using a Field Programmable Gate Array (FPGA) configured according to a Hardware Descriptor Language (HDL) or Verilog specification.

<FIG> is a block diagram of an apparatus comprising a cough identification machine <NUM> that, in the presently described embodiment, is implemented using the one or more processors and memory of a smartphone. The cough identification machine <NUM> includes at least one processor <NUM> that accesses an electronic memory <NUM>. The electronic memory <NUM> includes an operating system <NUM> such as the Android operating system or the Apple iOS operating system, for example, for execution by the processor <NUM>. The electronic memory <NUM> also includes a cough identification software product or "App" <NUM> according to a preferred embodiment of the present invention. The cough identification App <NUM> includes instructions that are executable by the processor <NUM> in order for the cough identification machine <NUM> to process sounds from a subject <NUM> and present an identification of coughs to a clinician <NUM> by means of LCD touch screen interface <NUM>. The App <NUM> includes instructions for the processor <NUM> to implement a pattern classifier such as a trained predictor or decision machine, which in the presently described preferred embodiment of the invention comprises a specially trained Convolutional Neural Network (CNN) <NUM>.

The processor <NUM> is in data communication with a plurality of peripheral assemblies <NUM> to <NUM>, as indicated in <FIG>, via a data bus <NUM> which is comprised of metal conductors along which digital signals <NUM> are conveyed between the processor and the various peripherals. Consequently, if required the cough identification machine <NUM> is able to establish voice and data communication with a voice and/or data communications network <NUM> via WAN/WLAN assembly <NUM> and radio frequency antenna <NUM>. The machine also includes other peripherals such as Lens & CCD assembly <NUM> which effects a digital camera so that an image of subject <NUM> can be captured if desired. A LCD touch screen interface <NUM> is provided that acts as a human-machine interface and allows the clinician <NUM> to read results and input commands and data into the machine <NUM>. A USB port <NUM> is provided for effecting a serial data connection to an external storage device such as a USB stick or for making a cable connection to a data network or external screen and keyboard etc. A secondary storage card <NUM> is also provided for additional secondary storage if required in addition to internal data storage space facilitated by memory <NUM>. Audio interface <NUM> couples a microphone <NUM> to data bus <NUM> and includes anti-aliasing filtering circuitry and an Analog-to-Digital sampler to convert the analog electrical waveform <NUM> from microphone <NUM> (which corresponds to subject sound wave <NUM>) to a digital audio signal <NUM> (shown stored in memory in <FIG> and shown graphically in <FIG>) that can be stored in memory <NUM> and processed by processor <NUM>. The audio interface <NUM> is also coupled to a speaker <NUM>. The audio interface <NUM> includes a Digital-to-Analog converter for converting digital audio into an analog signal and an audio amplifier that is connected to speaker <NUM> so that audio recorded in memory <NUM> or secondary storage <NUM> can be played back for listening by clinician <NUM>. It will be realized that the microphone <NUM> and audio interface <NUM> along with processor <NUM> programmed with App <NUM> comprise an audio capture arrangement that is configured for storing a digital audio recording <NUM> of subject <NUM> in an electronic memory such as memory <NUM> or secondary storage <NUM>.

The cough identification machine <NUM> is programmed with App <NUM> so that it is configured to operate as a machine for identifying cough segments in the recording of the subject sound.

As previously discussed, although the cough identification machine <NUM> that is illustrated in <FIG> is provided in the form of smartphone hardware that is uniquely configured by App <NUM> it might equally make use of some other type of computational device such as a desktop computer, laptop, or tablet computational device or even be implemented in a cloud computing environment wherein the hardware comprises a virtual machine that is specially programmed with App <NUM>. Furthermore, a dedicated cough identification machine might also be constructed that does not make use of a general purpose processor. For example, such a dedicated machine may have an audio capture arrangement including a microphone and analog-to-digital conversion circuitry configured to store a digital audio recording of the subject in an electronic memory. The machine further includes a potential cough sound identification assembly in communication with the memory and arranged to process the digital audio recording to thereby identify segments of the digital audio potentially containing cough sounds, i.e. potential cough sounds. Preferably the potential cough sound identification assembly is arranged to implement the LW2 method of <CIT>. A sound segment to image representation assembly may be provided that transforms identified cough sound segments into image representations. The dedicated machine further includes a hardware implemented pattern classifier to produce a signal indicating the potential cough sound as being either a confirmed cough sound or a non-cough sound.

An embodiment of the procedure that cough identification machine <NUM> uses to identify cough segments in a recording of subject <NUM>, and which comprises instructions that make up App <NUM> is illustrated in the flowchart of <FIG> and will now be described in detail.

Initially clinician <NUM>, or another carer or even subject <NUM>, selects App <NUM> from an app selection screen generated by OS <NUM> on LCD touchscreen interface <NUM>. In response to that selection the processor <NUM> displays a screen such as screen <NUM> of <FIG> to prompt the clinician <NUM> to operate machine <NUM> to commence recording sound <NUM> from subject <NUM> via microphone <NUM> and audio interface <NUM>. The audio interface <NUM> converts the sound into digital signals <NUM> which are conveyed along bus <NUM> and recorded as one or more digital files <NUM> by processor <NUM> in memory <NUM> and/or secondary storage SD card <NUM>. In the presently described preferred embodiment the recording should proceed for a duration that is sufficient to include a number of cough sounds of the subject <NUM> to be present in the sound recording.

At box <NUM> processor <NUM> identifies potential cough sounds (PCSs) in the audio sound files <NUM>. In a preferred embodiment of the invention the App <NUM> includes instructions that configure processor <NUM> to implement a first cough sound pattern classifier (CSPC <NUM>) 62a and a second cough sound pattern classifier (CSPC <NUM>) 62b, each preferably comprising neural networks trained to respectively detect initial and subsequent phases of cough sounds. Thus, in that preferred embodiment the processor <NUM> identifies the PCSs using the LW2 method that is described in the previously mentioned international patent application publication <CIT>, the disclosure of which is hereby incorporated herein in its entirety by reference. Other methods for identifying potential cough sounds may alternatively be used at box <NUM>, for example the methods described in the previously mentioned international patent publication <CIT>at al might also be used.

<FIG> is a graph showing a portion of the recorded sound wave <NUM> from subject <NUM>. Application of the method described in <CIT> involves applying features of the sound wave to two trained neural networks which are respectively trained to recognize a first phase and a second phase of a cough sound. The output of the first neural network is indicated as line <NUM> in <FIG> and comprises a signal that represents the likelihood of a corresponding portion of the sound wave being a first phase of a cough sound. The output of the second neural network is indicated as line <NUM> in <FIG> and comprises a signal that represents the likelihood of a corresponding portion of the sound wave being a subsequent phase of the cough sound. Based on the outputs <NUM> and <NUM> of the first and second trained neural networks processor <NUM> identifies two Potential Cough Sounds 66a and 66b which are located in segments 68a and 68b.

At box <NUM> the processor <NUM> sets a variable Current PCS to the first PCS that has been previously identified, i.e. "pre-identified" at box <NUM>.

At box <NUM> the processor <NUM> transforms the pre-identified PCS that is stored in the Current PCS variable to produce a corresponding image representation <NUM> which it stores in either memory <NUM> or secondary storage <NUM>.

This image representation may comprise, or be based on, a spectrogram of the Current Cough Sound portion of the digital audio file. Possible image representations include mel-frequency spectrogram (or "mel-spectrogram"), continuous wavelet transform, and derivatives of these representations along the time dimension, also known as delta features. Consequently, the image representations relate frequency, for example on a vertical axis, with time for example on a horizontal axis, over the duration of the PCS.

An example of one particular implementation of box <NUM> is depicted in <FIG>. Initially the processor <NUM> identifies two Potential Cough Sounds (PCS) 66a, 66b in the digital sound file <NUM>.

Processor <NUM> identifies the Potential Cough Sounds 66a and 66b as separate cough audio segments 68a and 68b. Each of the separate cough audio segments 68a and 68b are then divided into N, in the present example N=<NUM>, equal length overlapping windows 72a1,. ,72a5 and 72b1,. For a shorter cough segment, e.g. cough segment 68b which is somewhat shorter than cough segment 68a, the overlapping windows 72b that are used to segment section 68b are proportionally shorter to the overlapping windows 72a that are used to segment section 68a.

Processor <NUM> then calculates a Fast Fourier Transform (FFT) and a power value per mel-bin, for M=<NUM> bins for each of the N=<NUM> windows, to arrive at corresponding pixel values. Machine readable instructions that configure a processor to perform these operations on the sound wave are included in App <NUM>. Such instructions are publicly available, for example at: https://librosa. io/librosa/_modules/librosa/core/spectrum. html (retrieved <NUM> December <NUM>).

In the example illustrated in <FIG>, processor <NUM> extracts Mel-spectrograms 74a, 74b, each with M=<NUM> Mel-frequency bins, from each of the N=<NUM> overlapping windows 72a1,. ,72a5 and 72b1,.

Processor <NUM> concatenates and normalizes the values stored in the spectrograms 74a and 74b to produce corresponding Square Mel-Spectrogram images 76a and 76b representing cough sounds 66a and 66b respectively. Each of images 76a and 76b is an <NUM>-bit greyscale MxN image where M=N.

N may be any positive integer value bearing in mind that at some N, depending on the sampling rate of the audio interface <NUM>, the cough image will contain all information present in the original audio, which is desirable. The number of FFT bins may need to be increased to accommodate higher N.

<FIG> are Square Mel-spectrogram images of non-cough segments of the subject sound recording, obtained using the process described in <FIG> with N = M = <NUM>. In this image, time increases from left to right and frequency increases from bottom to top. Darker areas denote increased amplitude of the mel-frequency bin.

In contrast <FIG> are Square Mel-spectrogram images of cough segments, e.g. one of segments 68a, 68b.

The images in <FIG> have been thresholded to convert them to black and white image for purposes of official publication of this patent specification.

Although it is convenient to use square representations that are N x M pixels derived from N segments, each analyzed for M frequency bins, where N=M, it is also possible to use rectangular representations where N is not equal to M provided that the CNN <NUM> has been trained using similarly dimensioned rectangular images.

From the discussion of box <NUM> it will be understood that processor <NUM>, configured by App <NUM> to perform the procedure of box <NUM>, comprises a sound segment-to-image representation assembly that is arranged to transform sound segments of the recording, previously identified as Potential Cough Sounds, e.g. at box <NUM>, into corresponding image representations.

Returning now to <FIG>, at box <NUM> processor <NUM> applies the image representation, for example image 76a to a representation pattern classifier in the form of the trained convolutional neural network (CNN) <NUM>. The CNN <NUM> is trained to confirm whether or not the image representation of the Potential Cough Sound is indeed a cough sound i.e. a Confirmed Cough Sound (CCS). The CNN <NUM> comprises a representation pattern classifier that generates an output probability signal which ranges between <NUM> and <NUM> wherein <NUM> indicates a certainty that the Potential Cough Sound (PCS) is indeed a cough sound and thus a Confirmed Cough Sound and <NUM>, which indicates that there is no likelihood of the PCS being a cough sound. A probability value p is arrived at box <NUM> from the output of the trained Neural Network (CNN) at box <NUM>. At box <NUM> the p value that has been determined at box <NUM> is compared to a threshold value stored in variable Threshold. The threshold value is preferably <NUM> so that the PCS is deemed to be a CCS provided that the p value indicates that the PCS is more than likely a CCS. Higher or lower threshold values may be used as desired depending on the requirements of the specific situation.

If p is greater than Threshold at box <NUM> then at box <NUM> processor <NUM> flags that the current PCS is a CCS, for example by recording the corresponding sound segment's begin and end times as being the begin and end times of a confirmed cough sound (CCS).

If the p value is not greater than Threshold then the PCS is not flagged as being a CCS. Control then proceeds to decision box <NUM>. At decision box <NUM> processor <NUM> checks if there are any more PCSs to be processed. If there are more PCSs, that were identified at box <NUM>, to be processed then at box <NUM> the Current PCS variable is set to the next identified PCS and control proceeds to box <NUM> where the previously described boxes <NUM> to <NUM> are repeated. If, at box <NUM>, there are no more PCSs to be processed then control proceeds to box <NUM> where processor <NUM> operates a display in the form of LCD Touch Screen Interface <NUM>, which is responsive to processor <NUM>, to display the screen <NUM> shown in <FIG>. Screen <NUM> presents the number of PCSs processed and the number that have been found to be CCSs. It also presents the start and end times for each CCS so that clinician <NUM> can listen to them via speaker <NUM> if desired.

<FIG> is a block diagram of a CNN training machine <NUM> implemented using the one or more processors and memory of a desktop computer configured according to CNN training Software <NUM>. CNN training machine <NUM> includes a main board <NUM> which includes circuitry for powering and interfacing to one or more onboard microprocessors (CPUs) <NUM>.

The main board <NUM> acts as an interface between microprocessors <NUM> and secondary memory <NUM>. The secondary memory <NUM> may comprise one or more optical or magnetic, or solid state, drives. The secondary memory <NUM> stores instructions for an operating system <NUM>. The main board <NUM> also communicates with random access memory (RAM) <NUM> and read only memory (ROM) <NUM>. The ROM <NUM> typically stores instructions for a startup routine, such as a Basic Input Output System (BIOS) or Unified Extensible Firmware Interface (UEFI) which the microprocessor <NUM> accesses upon start up and which preps the microprocessor <NUM> for loading of the operating system <NUM>. For example Microsoft Windows, and Ubuntu Linux Desktop are two examples of such an operating system.

The main board <NUM> also includes an integrated graphics adapter for driving display <NUM>. The main board <NUM> will typically include a communications adapter <NUM>, for example a LAN adaptor or a modem or a serial or parallel port, that places the server <NUM> in data communication with a data network.

An operator <NUM> of CNN training machine <NUM> interfaces with it by means of keyboard <NUM>, mouse <NUM> and display <NUM>.

The operator <NUM> may operate the operating system <NUM> to load software product <NUM>. The software product <NUM> may be provided as tangible, non-transitory, machine readable instructions <NUM> borne upon a computer readable media such as optical disk <NUM> for reading by disk drive <NUM>. Alternatively, it might also be downloaded via port <NUM>.

The secondary storage <NUM> also includes software product <NUM>, being a CNN training software product <NUM> according to an embodiment of the present invention. The CNN training software product <NUM> is comprised of instructions for CPUs <NUM> (or as alternatively and collectively referred to "processor <NUM>") to implement the method that is illustrated in <FIG>.

Initially at box <NUM> of <FIG> processor <NUM> retrieves a training subject audio dataset which in the presently described embodiment is comprised of <NUM>,<NUM> cough segments and non-cough segments. The metadata includes training labels, i.e., whether or not each segment is actually a cough or not.

At box <NUM> the processor <NUM> represents the non-cough events and the cough events as images in the same manner as has previously been discussed at box <NUM> of <FIG> wherein Mel-spectrogram images are created to represent each potential cough sound (PCS).

At box <NUM> processor <NUM> transforms each image produced at box <NUM> to create additional training examples for subsequently training a convolutional neural net (CNN). This data augmentation step at box <NUM> is preferable because a CNN is a very powerful learner and with a limited number of training images it can memorize the training examples and thus over fit the model. The Inventors have discerned that such a model will not generalize well on previously unseen data. The applied image transformations include, but are not limited to, small random zooming, cropping and contrast variations.

At box <NUM> the processor <NUM> trains the CNN <NUM> on the augmented cough and non-cough images that have been produced at box <NUM> and the original training labels. Over fitting of the CNN <NUM> is further reduced by using regularization techniques such as dropout, weight decay and batch normalization.

One example of the process used to produce a CNN <NUM> is to take a pretrained ResNet model, which is a residual network containing shortcut connections, such as ResNet-<NUM>, and use the convolutional layers of the model as a backbone, and replace the final non-convolutional layers with layers that suit the cough identification problem domain. These include fully connected hidden layers, dropout layers and batch normalization layers. Information about ResNet-<NUM> is available at: https://www. com/help/deeplearning/ref/resnet18. html (retrieved <NUM> December <NUM>), the disclosure of which is incorporated herein by reference. ResNet-<NUM> is a convolutional neural network that is trained on more than a million images from the ImageNet database (http://www. The network is <NUM> layers deep and can classify images into <NUM> object categories, such as keyboard, mouse, pencil, and many animals. As a result, the network has learned rich feature representations for a wide range of images. The network has an image input size of <NUM>-by-<NUM> pixels.

The Inventors have found that it is sufficient to fix the ResNet-<NUM> layers and only train the new non-convolutional layers, however it is also possible to retrain both the ResNet-<NUM> layers and the new non-convolutional layers to achieve a working model. A fixed dropout ratio of <NUM> is preferably used. Adaptive Moment Estimation (ADAM) is preferably used as an adaptive optimizer though other optimizer technique may also be used.

At box <NUM> the original (non-augmented) cough and non-cough images from box <NUM> are applied to the CNN <NUM> which is now trained to respond with probabilities for each.

The trained CNN is then distributed as CNN <NUM> as part of cough identification App <NUM> being CNN <NUM>.

To test the performance of the method of <FIG>, the Inventors developed a dataset of <NUM> coughs, and <NUM> non coughs. The non-cough sounds in the dataset were specifically chosen from events that had been incorrectly flagged as a cough by the LW2 algorithm.

<NUM>% of that set was used to train the CNN <NUM> for Deep Cough ID and the remaining <NUM>% (<NUM> coughs and <NUM> non coughs) was used as a test set.

Using LW2, <NUM> coughs (PCS) were identified, while <NUM>,<NUM> non-cough events were false positives (i.e. LW2 said these were coughs whereas further investigation revealed that they were not). When Deep Cough ID was used after LW2, <NUM> coughs were identified (ie. <NUM> coughs were false negatives and incorrectly classified), and <NUM> non-cough events were now correctly classified (rejected) and only <NUM> of these non-cough events were incorrectly classified as coughs.

A summary of the performance of the method of <FIG> on the test set is set out in Table <NUM>:.

It will be observed from the above table that embodiments of the present invention result in an accuracy increase of over <NUM>% over the prior art LW2 method that is the subject of international patent publication <CIT>.

To recap, in one aspect a method is provided for identifying cough sounds, such as cough sounds 66a, 66b in an audio recording, such as digital sound file <NUM>, of a subject <NUM>. The method in this aspect involves operating at least one electronic processor <NUM> to identify potential cough sounds (box <NUM> of <FIG>) in the audio recording <NUM>, for example by, but not limited to, using the LW2 procedure described in relation to <FIG>. The method also involves operating the electronic processor <NUM> to transform (box <NUM> of <FIG>) one or more of the potential cough sounds into corresponding one or more image representations such as image representations 76a, 76b (<FIG>).

The electronic processor <NUM> is operated to apply the one or more image representations 76a, 76b to a representation pattern classifier <NUM> (<FIG>) trained to confirm (box <NUM> of <FIG>) that a potential cough sound is a cough sound or is not a cough sound. The method includes operating the at least one electronic processor <NUM> to flag one or more of the potential cough sounds as confirmed cough sounds (box <NUM> <FIG>) based on an output of the representation pattern classifier <NUM>.

In another aspect an apparatus has been described for identifying cough sounds in a subject. The apparatus includes an audio capture arrangement, for example comprised of microphone <NUM> (<FIG>) and audio interface <NUM> along with processor <NUM> configured by App <NUM> to capture and store a digital audio recording <NUM> of subject <NUM> in an electronic memory such as memory <NUM> or secondary storage <NUM>.

The apparatus has a sound segment-to-image representation assembly arranged to transform pre-identified potential cough sounds into corresponding image representations. For example, the sound segment-to-image representation assembly may comprise processor <NUM>, configured by App <NUM> to perform the procedure of box <NUM> (<FIG> that is arranged to transform sound segments of the recording, previously identified as Potential Cough Sounds, e.g., at box <NUM>, into corresponding image representations.

The apparatus also includes a representation pattern classifier in communication with the sound segment-to-image representation assembly that is configured to process the image representations to thereby produce a signal indicating a probability of the image representations corresponding to the pre-identified potential cough sounds being a confirmed cough sound. The representation pattern classifier may be in the form of a trained convolutional neural network (CNN) <NUM>, which is trained to confirm whether or not the image representation of the Potential Cough Sound is indeed a cough sound i.e. a Confirmed Cough Sound (CCS).

In compliance with the statute, the invention has been described in language more or less specific to structural or methodical features. The term "comprises" and its variations, such as "comprising" and "comprised of" is used throughout in an inclusive sense and not to the exclusion of any additional features.

It is to be understood that the invention is not limited to specific features shown or described since the means herein described comprises preferred forms of putting the invention into effect. The invention is, therefore, claimed in any of its forms or modifications within the proper scope of the appended claims appropriately interpreted by those skilled in the art.

Throughout the specification and claims (if present), unless the context requires otherwise, the term "substantially" or "about" will be understood to not be limited to the value for the range qualified by the terms.

Claim 1:
A method for identifying cough sounds in an audio recording of a subject comprising:
operating at least one electronic processor to identify potential cough sounds in the audio recording; the method being characterised by further comprising:
operating the at least one electronic processor to transform one or more of the potential cough sounds into corresponding one or more image representations;
operating the at least one electronic processor to apply said one or more image representations to a representation pattern classifier trained to confirm that a potential cough sound is a cough sound or is not a cough sound; and
operating the at least one electronic processor to flag one or more of the potential cough sounds as confirmed cough sounds based on an output of the representation pattern classifier.