Patent Description:
Implementations described herein disclose an artificial intelligence (AI) based method for generating an oxygen saturation level output signal using the trained neural network. In one implementation, the method includes receiving a photoplethysmographic (PPG) signal, including a red PPG signal and an infrared PPG signal, generating an input feature matrix or matrices by performing time-frequency transform of the PPG signal, training a neural network using the input feature matrices and an oxygen saturation level input signal, and generating an oxygen saturation level output signal using the trained neural network.

A method, of determining oxygen level saturation includes receiving a photoplethysmographic (PPG) signal, the PPG signal including a red PPG signal and an infrared PPG signal, generating an input feature matrix by performing time-frequency transform of the PPG signal, training a neural network using the input feature matrix and an oxygen saturation level input signal, and generating an oxygen saturation level output signal using the trained neural network. In one implementation, generating an input feature matrix by performing time-frequency transform of the PPG signal further includes generating an input feature matrix by performing a wavelet transform of the PPG signal.

In one implementation, generating the input feature matrix by performing time-frequency transform of the PPG signal further comprising generating a modulus value vector and a phase value vector across a time-frequency plane. According to the invention, generating an input feature matrix by performing time-frequency transform of the PPG signal further comprises generating a real value vector and an imaginary value vector across a time-frequency plane. Yet additionally, generating the input feature matrix by performing a wavelet transform of the PPG signal further comprising generating the input feature matrix by performing a Morlet wavelet transform of the PPG signal.

In one implementation, the method further includes normalizing the PPG signal by a baseline to generate a normalized PPG signal, wherein generating an input feature matrix further comprises generating an input feature matrix by performing time-frequency transform of the normalized PPG signal. Alternatively, the method further includes rescaling the input feature matrix non-linearly before training a neural network using the input feature matrix. Yet alternatively, rescaling the input feature matrix non-linearly further comprising rescaling the input feature matrix using a logarithmic scaling. Alternatively, performing time-frequency transform of the PPG signal further comprising one of performing short time Fourier transform (STFT) of the PPG signal, performing WignerVille transform of the PPG signal, and performing S-transform of the PPG signal. Alternatively, the method further includes combining two or more vectors of the input feature matrix to generate a combined feature vector and wherein training the neural network further comprising training the neural network with the combined feature vector.

In a computing environment, a method performed at least in part on at least one processor, the method including receiving a photoplethysmographic (PPG) signal, the PPG signal including a red PPG signal and an infrared PPG signal, generating an input feature matrix by performing time-frequency transform of the PPG signal, training a neural network using the input feature matrix and an oxygen saturation level input signal, and generating an oxygen saturation level output signal using the trained neural network.

A physical article of manufacture including one or more tangible computer-readable storage media, encoding computer-executable instructions for executing on a computer system a computer process to provide an automated connection to a collaboration event for a computing device, the computer process including receiving a photoplethysmographic (PPG) signal, the PPG signal including a red PPG signal and an infrared PPG signal, generating an input feature matrix by performing time-frequency transform of the PPG signal, and training a neural network using the input feature matrix and an oxygen saturation level input signal.

This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter, which is defined by the appended claims.

A further understanding of the nature and advantages of the present technology may be realized by reference to the figures, which are described in the remaining portion of the specification.

Hypoxemia is a condition that indicates lower than normal concentration of oxygen levels in arterial blood of a patient. Hypoxemia may result in hypoxia or hypoxic condition for the patient, characterized by inadequate oxygen content in patient tissues. Pulse oximeters may be used to measure the oxygen content of arterial blood to indicate existing hypoxia condition as well as to predict impending hypoxia. Pulse oximeters generate various output signals including fluctuating time series indicating pulse oximeter trace or red and infrared photoplethysmographic (PPG) signals. The red an infrared PPG signals may be processed to generate oxygen saturation levels, also referred to as SpO<NUM> values.

Using deep learning for time series analysis such as the determination of an SpO<NUM> value from the red and infrared PPG signals may involve simply inputting the raw PPG into a network, such as a long-short term memory (LSTM) network or a convolutional neural network (CNN). However, improved results can be obtained by preprocessing the data to provide input features that more succinctly capture the information from the PPG signals. An AI based method for determining oxygen saturation levels disclosed herein includes time-frequency processing of the red and infrared PPG signals to generate an input feature matrix that are then input into a machine learning model such as a deep learning AI model based on CNNs.

<FIG> illustrates an example schematic view of an AI based system <NUM> for determining oxygen desaturation levels using time-frequency preprocessing for the determination of input features. An oximeter <NUM> may be used to measure the oxygen saturation (SpO<NUM>) level in the patient. For example, the pulse oximeter <NUM> may be attached to the patient's thumb. The pulse oximeter <NUM> may be communicatively connected to a computing system <NUM>. For example, the pulse oximeter <NUM> may be connected to the computing system <NUM> wirelessly and it may send a sequence of input signals <NUM> measured by the oximeter <NUM> over a period of time. For example, such input signal sequence <NUM> may be communicated at every second. In one implementation, the input signal sequence <NUM> may include PPG signals such as a red signal 110a, an infrared signal 110b, etc. The pulse oximeter <NUM> may also use the values of the red signal 110a and the infrared signal 110b, to generate values of oxygen saturation levels (the SpO<NUM> levels). A sequence of such SpO<NUM> levels generated by the pulse oximeter <NUM> is illustrated by a sequence of oxygen saturation levels <NUM> (or a sequence of SpO<NUM> levels <NUM>).

The computing system <NUM> may be a computing system that includes a microprocessor <NUM> and various other components implemented on a memory <NUM>. An example of such a computing system <NUM> is disclosed in <FIG> below. In a method disclosed herein, the memory <NUM> may be used to store the sequence of input signals <NUM> generated by the pulse oximeter <NUM>, the sequence of SpO<NUM> levels <NUM> generated by the pulse oximeter <NUM>, as well as an input feature matrix or matrices <NUM> generated based on the sequence of input signals <NUM>. The combination of the input matrices <NUM> may also be referred as tensors. For example, a time-frequency transformer <NUM> may be used to generate the input feature matrix <NUM> based on the sequence of input signals <NUM>.

In one implementation, the time-frequency transformer <NUM> may be a wavelet transformer that allows a signal to be decomposed such that both the frequency characteristics and the location of particular features in a time series may be highlighted simultaneously. The nature of the wavelet transform is such that it is well suited to the analysis of signals in which a more precise time resolution is required for higher frequencies than for lower ones, such as PPG signals. Furthermore, by employing a variable width window, it effectively zooms in on the temporal signal when analyzing higher frequencies, thus providing higher temporal resolution where necessary. In one implementation, the wavelet transform of a continuous real-valued time signal, x(t), with respect to the wavelet function, ψ is defined as: <MAT>.

Where t is time, a is the dilation parameter, b is the location parameter, ψ ((t-b)/a) is the analyzing wavelet used in the convolution and , ψ* ((t-b)/a) is its complex conjugate, and x(t) is the signal under investigation which, in this application, may be the PPG signals <NUM> obtained from the pulse oximeter <NUM>. Examples of various wavelets that may be used by the time-frequency transformer <NUM> may include a Morlet wavelet, a Mexican hat wavelet, a Paul Wavelet, etc. Alternatively, discrete wavelet transforms with corresponding discrete wavelets may also be used. In other implementations, variants of the same wavelet, such as a Morlet wavelet having different characteristic (or central) frequencies, may also be used.

In one example implementation using Morlet wavelet at the time-frequency transformer <NUM>, the central frequency ω<NUM>, is set to <NUM>. However, alternative central frequencies, such as values less than <NUM> (e.g. <NUM> or <NUM>), which are better at extracting temporal information may also be used. Alternatively, in other implementations, values greater than <NUM> (e.g. <NUM>, <NUM>), which are better at extracting low frequency measures from the signal may be used as the central frequency. In another implementation, the time-frequency transformer <NUM> may run continuous wavelet transform on the input signal <NUM> using a Morlet wavelet set at two, or more, separate values of the wavelet central frequencies (ω<NUM>) and input the modulus and phase from these additional matrices as additional inputs into the neural network training phase 144a.

The output of the time-frequency transformer <NUM> may be stored as an input feature matrix <NUM>. The input feature matrix <NUM> may be a matrix of complex number values in a time frequency plane. An example of such a matrix may be a matrix of modulus values across the time-frequency plane. Another such example may be a matrix of phase values across the time-frequency plane. Such modulus and phase values across time-frequency plane are illustrated below in <FIG>.

In an alternative implementation, the time-frequency transformer <NUM> may normalize the PPG signal <NUM> by its baseline before computing the wavelet transform. Alternatively, the transform values output from the time-frequency transformer <NUM> may be rescaled in a nonlinear way in order to enhance the subtle features within the signal. For example, the modulus values output from the time-frequency transformer <NUM> may be logarithmically scaled and the scaled values may be used as part of the input feature matrix <NUM>.

Implementations of the time-frequency transformer <NUM> may use alternate transforms such as short time Fourier transform (STFT), Wigner-Ville transform, S-transform, etc. Alternatively, such alternative transforms may be computed and combined with a wavelet transform or with each other and the combination transform may be input to the neural network training phase 144a. Similarly, different time-frequency transforms may be used as additional inputs to the neural network training phase 144a. For example, the phase and modulus matrix from a wavelet transform plus a phase and modulus matrix from a short time Fourier transform may be used as inputs to the neural network training phase 144a. Alternatively, only part of such transform, e.g. the real part, imaginary part, modulus, or phase may be used at input to the neural network training phase 144a.

The input feature matrix <NUM> and the sequence of oxygen saturation levels <NUM> generated by the pulse oximeter <NUM> are used as inputs to train a new neural network at a neural network training phase 144a. For example, the neural network may be a deep learning network such as a CNN.

Once the neural network is trained, as indicated by the trained neural network 144b, the input feature matrix <NUM> may be input to the trained neural network 144b to generate a predicted oxygen saturation level sequence <NUM>. A comparison of the predicted oxygen saturation level sequence <NUM> and the sequence of oxygen saturation levels <NUM> generated by the pulse oximeter <NUM> is depicted at <NUM> (and further illustrated below in <FIG>).

In one implementation, various vectors of the input feature matrix <NUM> may be combined in some way before being input into the deep learning neural network 144a. For example, the modulus values of the red signal wavelet transform may be divided by the modulus values of the infrared signal wavelet transform. Subsequently, this ratio transform may be used as input to the deep learning neural network 144a. Other operations apart from division may also be employed.

<FIG> illustrates example schematic <NUM> of the wavelet transform process to generate the input feature matrix. Specifically, the schematic <NUM> illustrates an input PPG signal <NUM> is transformed at <NUM> to generate a modulus matrix <NUM> in a time-frequency plane and a phase matrix <NUM> in a time-frequency plane.

<FIG> illustrates example depictions <NUM> of the wavelet transform modulus map <NUM> and a phase map <NUM> derived from a PPG signal <NUM>. The matrix representing the modulus map <NUM> and a phase map <NUM> may be input to a deep learning network such as a CNN in order to derive oxygen saturation levels.

<FIG> illustrates an example a schematic of the modulus matrices <NUM> of the red and infrared PPG signals being input to a deep learning model. Specifically, a modulus matrix <NUM> is generated by time-frequency transform of red PPG signal and a modulus matrix <NUM> is generated by time-frequency transform of infrared PPG signal. Both of these matrices are input to a deep learning network.

<FIG> illustrates an example implementation of a deep learning network <NUM>. The illustrated deep learning network <NUM> is a convolutional neural network (CNN). Here a series of blocks <NUM> which may be repeated an arbitrary number of times.

<FIG> illustrates example comparative graph <NUM> of the output of the deep learning model compared to the true oxygen saturation levels using the input from the modulus matrices from the red and infrared signals described in <FIG>. Specifically, <FIG> illustrates sequential desaturation events over a period of time. As illustrated herein, the dashed line <NUM> is the oxygen saturation level truth and the solid line <NUM> is the predicted oxygen saturation level derived from the deep learning model.

<FIG> illustrates an alternative example schematic of another example input to the deep learning model where both the red and infrared signal modulus and phase are input to the network. Specifically, a modulus matrix <NUM> is generated by time-frequency transform of red PPG signal and a modulus matrix <NUM> is generated by time-frequency transform of infrared PPG signal. Furthermore, a phase matrix <NUM> is generated by time-frequency transform of red PPG signal and a phase matrix <NUM> is generated by time-frequency transform of infrared PPG signal. Each of these four matrices are input to a deep learning network. In alternative implementations, the modulus matrices and the phase matrices from a number of wavelet transforms may be used for red and infrared signals. For example, an implementation may use a number of Morlet wavelet transforms of each signal where the characteristic frequency of the Morlet analyzing wavelet is changed for each transform. For example, if three characteristic frequencies are used, three modulus and three phase matrices per signal, thus a total of twelve matrices, may be generated and used as input to the training network.

<FIG> illustrates example depiction <NUM> of real and imaginary parts of a transform as generated by the time-frequency transformer disclosed herein. Specifically, the transform real part <NUM> and transform imaginary part <NUM> make up the real and imaginary parts of the complex numbers making up the transform matrix generated by a time-frequency transform of a short segment PPG signal <NUM>. The transform real part <NUM> and transform imaginary part <NUM> are input into the neural network training phase 144a.

<FIG> illustrates time-frequency plots <NUM> of example transform modulus that is rescaled logarithmically. Specifically, <NUM> contains an original modulus transform matrix and <NUM> contains logarithmically scaled modulus generated by rescaling the transform values of the original transform matrix. Such scaling allows smaller features of the original modulus transform matrix to be less dominated by the higher energy features in the signal. Scaling methods other than logarithmic scaling may also be used to scale the transform values in a nonlinear way in order to enhance he subtle features within the signal.

<FIG> illustrates example operations <NUM> of the AI based method of determining oxygen saturation levels disclosed herein. An operation <NUM> acquires red and infrared PPG signals from an oximeter. An operation <NUM> computes wavelet transforms of the input red an infrared PPG signals to generate transform matrices WTR and WTIR. At operation <NUM>, the transform matrices WTR and WTIR are input into a deep learning neural network for training the deep learning neural network to generate computed oxygen saturation level or SpO<NUM>.

<FIG> illustrates an example system <NUM> that may be useful in implementing the described technology for providing attestable and destructible device identity. The example hardware and operating environment of <FIG> for implementing the described technology includes a computing device, such as a general-purpose computing device in the form of a computer <NUM>, a mobile telephone, a personal data assistant (PDA), a tablet, smart watch, gaming remote, or other type of computing device. In the implementation of <FIG>, for example, the computer <NUM> includes a processing unit <NUM>, a system memory <NUM>, and a system bus <NUM> that operatively couples various system components including the system memory to the processing unit <NUM>. There may be only one or there may be more than one processing unit <NUM>, such that the processor of the computer <NUM> comprises a single central-processing unit (CPU), or a plurality of processing units, commonly referred to as a parallel processing environment. The computer <NUM> may be a conventional computer, a distributed computer, or any other type of computer; the implementations are not so limited.

The system bus <NUM> may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, a switched fabric, point-to-point connections, and a local bus using any of a variety of bus architectures. The system memory may also be referred to as simply the memory, and includes read only memory (ROM) <NUM> and random-access memory (RAM) <NUM>. A basic input/output system (BIOS) <NUM>, containing the basic routines that help to transfer information between elements within the computer <NUM>, such as during start-up, is stored in ROM <NUM>. The computer <NUM> further includes a hard disk drive <NUM> for reading from and writing to a hard disk, not shown, a magnetic disk drive <NUM> for reading from or writing to a removable magnetic disk <NUM>, and an optical disk drive <NUM> for reading from or writing to a removable optical disk <NUM> such as a CD ROM, DVD, or other optical media.

The hard disk drive <NUM>, magnetic disk drive <NUM>, and optical disk drive <NUM> are connected to the system bus <NUM> by a hard disk drive interface <NUM>, a magnetic disk drive interface <NUM>, and an optical disk drive interface <NUM>, respectively. The drives and their associated tangible computer-readable media provide non-volatile storage of computer-readable instructions, data structures, program modules and other data for the computer <NUM>. It should be appreciated by those skilled in the art that any type of tangible computer-readable media may be used in the example operating environment.

A number of program modules may be stored on the hard disk drive <NUM>, magnetic disk <NUM>, optical disk <NUM>, ROM <NUM>, or RAM <NUM>, including an operating system <NUM>, one or more application programs <NUM>, other program modules <NUM>, and program data <NUM>. A user may generate reminders on the personal computer <NUM> through input devices such as a keyboard <NUM> and pointing device <NUM>. Other input devices (not shown) may include a microphone (e.g., for voice input), a camera (e.g., for a natural user interface (NUI)), a joystick, a game pad, a satellite dish, a scanner, or the like. These and other input devices are often connected to the processing unit <NUM> through a serial port interface <NUM> that is coupled to the system bus <NUM>, but may be connected by other interfaces, such as a parallel port, game port, or a universal serial bus (USB) (not shown). A monitor <NUM> or other type of display device is also connected to the system bus <NUM> via an interface, such as a video adapter <NUM>. In addition to the monitor, computers typically include other peripheral output devices (not shown), such as speakers and printers.

The computer <NUM> may operate in a networked environment using logical connections to one or more remote computers, such as remote computer <NUM>. These logical connections are achieved by a communication device coupled to or a part of the computer <NUM>; the implementations are not limited to a particular type of communications device. The remote computer <NUM> may be another computer, a server, a router, a network PC, a client, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer <NUM>. The logical connections depicted in <FIG> include a local-area network (LAN) <NUM> and a wide-area network (WAN) <NUM>. Such networking environments are commonplace in office networks, enterprise-wide computer networks, intranets and the Internet, which are all types of networks.

When used in a LAN-networking environment, the computer <NUM> is connected to the local network <NUM> through a network interface or adapter <NUM>, which is one type of communications device. When used in a WAN-networking environment, the computer <NUM> typically includes a modem <NUM>, a network adapter, a type of communications device, or any other type of communications device for establishing communications over the wide area network <NUM>. The modem <NUM>, which may be internal or external, is connected to the system bus <NUM> via the serial port interface <NUM>. In a networked environment, program engines depicted relative to the personal computer <NUM>, or portions thereof, may be stored in the remote memory storage device. It is appreciated that the network connections shown are examples and other means of communications devices for establishing a communications link between the computers may be used.

In an example implementation, software or firmware instructions for providing attestable and destructible device identity may be stored in memory <NUM> and/or storage devices <NUM> or <NUM> and processed by the processing unit <NUM>. One or more datastores disclosed herein may be stored in memory <NUM> and/or storage devices <NUM> or <NUM> as persistent datastores. For example, an AI based SpO<NUM> determination module <NUM> (illustrated within the personal computer <NUM>) may be implemented on the computer <NUM> (alternatively, the AI based SpO<NUM> determination module <NUM> may be implemented on a server or in a cloud environment). The AI based SpO<NUM> determination module <NUM> may utilize one of more of the processing unit <NUM>, the memory <NUM>, the system bus <NUM>, and other components of the personal computer <NUM>.

In contrast to tangible computer-readable storage media, intangible computer-readable communication signals may embody computer readable instructions, data structures, program modules or other data resident in a modulated data signal, such as a carrier wave or other signal transport mechanism. By way of example, and not limitation, intangible communication signals include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.

The implementations described herein are implemented as logical steps in one or more computer systems. The logical operations may be implemented (<NUM>) as a sequence of processor-implemented steps executing in one or more computer systems and (<NUM>) as interconnected machine or circuit modules within one or more computer systems. The implementation is a matter of choice, dependent on the performance requirements of the computer system being utilized. Accordingly, the logical operations making up the implementations described herein are referred to variously as operations, steps, objects, or modules. Furthermore, it should be understood that logical operations may be performed in any order, unless explicitly claimed otherwise or a specific order is inherently necessitated by the claim language.

Claim 1:
A computer-implemented method, of determining oxygen level saturation, comprising
receiving a photoplethysmographic (PPG) signal, the PPG signal including a red PPG signal (110a) and an infrared PPG signal (110b);
generating an input feature matrix (<NUM>) by performing time-frequency transform of the PPG signal, wherein generating the input feature matrix (<NUM>) further comprises generating a real value vector and an imaginary value vector across a time-frequency plane;
training a neural network using the input feature matrix (<NUM>), an oxygen saturation level input signal and the real value vector and the imaginary value vector across a time-frequency plane; and
generating an oxygen saturation level output signal using the trained neural network.