Learning Blood Oxygen from Respiration Signals
Authors: Hao He
Ying-Cong Chen
Yuan Yuan
Dina Katabi
Venue:  (2021)
Keywords: healthcare
medical application

Abstract: Monitoring blood oxygen is critical in a variety of medical conditions. For almost a century, pulse oximetry has been the only non-invasive method for measuring blood oxygen. While highly useful, pulse oximetry has important limitations. It requires wearable sensors, which can be cumbersome for older patients. It is also known to be biased when used for dark-skinned subjects. In this paper, we demonstrate, for the first time, the feasibility of predicting oxygen saturation from breathing. By eliminating the dependency on oximetry, we eliminate bias against skin color. Further, since breathing can be monitored without body contact by analyzing the radio signal in the environment, we show that oxygen too can be monitored without any wearable devices. We introduce a new approach for leveraging auxiliary variables via a switcher-based multi-headed neural network model. Empirical results show that our model achieves good accuracy on multiple medical datasets.

Under review as a conference paper at ICLR 2021
LEARNING BLOOD OXYGEN FROM RESPIRATION SIG- NALS
Anonymous authors Paper under double-blind review
ABSTRACT
Monitoring blood oxygen is critical in variety of medical conditions. For almost century, pulse oximetry has been the only non-invasive method for measuring blood oxygen. While highly useful, pulse oximetry has important limitations. It requires wearable sensors, which can be cumbersome for older patients. It is also known to be biased when used for dark- skinned subjects. In this paper, we demonstrate, for the first time, the feasibility of predicting oxygen saturation from breathing. By eliminating the dependency on oximetry, we eliminate bias against skin color. Further, since breathing can be monitored without body contact by analyzing the radio signal in the environment, we show that oxygen too can be monitored with- out any wearable devices. We introduce a new approach for leveraging auxiliary variables via switcher-base multi-headed neural network model. Empirical results show that our model achieves good accuracy on multiple medical datasets.
INTRODUCTION
Oxygen saturation refers the amount of oxygen in the blood that is the fraction of oxygen- saturated hemoglobin relative to the total blood hemoglobin. Normal oxygen levels range from 94%
100%. Lower oxygen can be dangerous, and if severe, leads to brain and lung failure (Taylor
ct al. 1951 Diaz-Regañón et al., 22022 Lapinsky et al. 1999). Oxygen Monitoring is important for patients suffering from lung diseases such as COVID- 19, pulmonary embolism (PE), and chronic obstructive pulmonary disease (COPD) who are susceptible to reduced blood oxygen (NIH) 2020 Nordenholz al. 2011). is also recommended in old people since lung functions deteriorate with
age putting the elders at high risk of low oxygen
NLM 2020
Today, the only non-invasive approach for measuring oxygen saturation (SpO2) pulse oximetry. It works by shining light on one's figure (or other body parts). Since has a bright red color, oxygen saturation can be measured by estimating the absorbance of the red- colored waves relative to other colors. While pulse oximetry is a very useful technology, it can be unsuitable for some of the most vulnerable patients. Oximetry requires the patient to wear a finger clip other sensors, which can be cumbersome and hard to remember for older patients with dementia or cognition problems. Further, since it relies on measuring light absorbance, it is affected by skin color and tend to overestimate blood oxygen in dark-skinned subjects (Feiner et al. 2007).
hemoglobin
In this paper, we introduce new approach for monitoring blood oxygen that could be more appro- priate for older patients or those with dark skin color. We propose learn oxygen saturation from respiration signals. Recent advances in wireless sensing have demonstrated the feasibility of obtain- ing accurate measurements of the respiration signal from analyzing the radio waves that bounce off people's 's bodies, without any physical contact (Adib et al. 2015) Yue et al. 2018). Hence, if oxygen saturation is predictable from breathing, we can sense one's oxygen in a contactless way without any wearable sensors. Further, unlike pulse oximetry, such oxygen sensing method would be oblivious to skin color.
We model the problem as pequence-to-squeeee learning and introduce a neural network that maps respiration signals to the corresponding oxygen saturation. A key innovation our model is a new design for leveraging auxiliary variables. The standard approach in the literature for leveraging auxiliary variables would either provide those variables to the model as inputs or try to predict them auxiliary tasks. We demonstrate empirically that for our problem neither of these approaches beneficial. We show that some cases, the gradient of the model can take vastly different and even divergent values given the auxiliary variable. In such scenario, it is better to learn different

---

Under review as a conference paper at ICLR 2021
models for different values of the variable. This motivates to propose alternative design for incorporating auxiliary information. We introduce a multi-headed model where the value of the auxiliary variable is used to gate the learning through the appropriate head.
We evaluate our model on multiple medical datasets, and compare it with various basclines. Our empirical results show that the average absolute error in predicting oxygen saturation the tested medical data is 1.6%, which is comparable to the accuracy of consumer pulse oximeters (whose average error ranges from 0.4% to 3.5% (Lipnick et al., 2016)). Further, by testing the model on data that include the RF modality, we show that the model generalizes to breathing signals extracted from radio waves, opening the door for continuous contactless monitoring of blood oxygen. We also use longitudinal data from one COVID-19 patient as a qualitative case study, showing that the predicted oxygen values are consistent with the patient's 's recovery process.
In summary, this paper makes the following contributions:
The paper is the first to predict blood oxygen from respiration signals. This provides novel non-invane non-invasive way for monitoring oxygen saturation, and improves our understanding of the rela- tionship between these two physiological signals.
The paper introduces new approach for leveraging auxiliary variables. We analyze the gradient of a vanilla model with respect to the variable; if the gradient is divergent or vastly different for different values of the variable, we use a multi-headed neural network gated by the variable.
The paper is the first to demonstrate the feasibility of contactless oxygen prediction from radio signals. Such a design can facilitate in-home oxygen monitoring, particularly for older patients who may have problems remembering to wear and charge oxygen sensors.
2 RELATED WORK
Predicting Oxygen Saturation. Past work on oxygen prediction focuses on inferring future oxygen values from recent measurements. Several papers use auto-regressive models that take past SpO2 readings and predict the SpO2 in the near future (EIMoaqet et al. 2013 2014 2016). Others use an auto-regressive model to model the photoplethysmogrrm (PPG) signal, which is the raw data obtained from pulse oximeter (Lee et al.) 2011). Some past work predicts changes in SpO2 in the five minutes after adjusting ventilator setting (Ghazal et al. 2019). Our work differs from these methods in that it predicts oxygen saturation from breathing, a completely different physiological signal. Further, we show the feasibility of oxygen prediction without body contact by inferring oxygen from radio-based breathing estimates.
Contactless Sensing with Radio Frequency Signals. The past decade has seen a rapid growth in research on passive sensing using radio frequency (RF) signals. Early work has demonstrated the possibility of sensing one's 's respiration and heart rate using radio signals (Adib et al. 2015)). Building on this work, researchers have found that by carefully analyzing the RF signals that bounce off the human body, they can monitor variety of health metrics including sleep, respiration, heart rate, gait, falls, and even human emotions (Nguyen et al.) 2016) Wang et 2017 Yue et al. 2018 Hsu
et al. 2017 Wang et al. 2016, Tian et al. 2018: Zhao et al.) 2016 Jiang et al., 2018). Our work adds a new piece to the global picture of predicting vital signals from radio waves.
Learning with Auxiliary Variables. Improving machine learning model by leveraging auxiliary variables is a well-known idea. Treating auxiliary variables as extra input features is a simple but effective method, and has been shown useful in computer vision (Forsyth & Ponce, 20022) health profiling (Wang et af. 2019), and traffic prediction (Liao et al.) 2018). Auxiliary learning (Liu et al. 2019) provides an alternative approach for leveraging auxilaary variables by taking them as secondary supervisors. It has been shown beneficial in multiple domains including object detec- tion (Mordan al.) 2018), sequence modelling (Collobert & Weston 2008 Dong al.) 2015). reinforcement learning (Wilson et al., 2007), and visual odometry (Valada et al., 20.8). Our work explores a third way of using auxiliary variables. We focus on partitioning the data space by those auxiliary variables based on analyzing how they affect the gradient of the primary task. We show that a wise partition facilitates model learning and leads to better results.
2

---

Under review as a conference paper at ICLR 2021
gate
Input
9
VI
Output
Accessible Variable
F
F
Fu
Inaccessible Variable
Switch Status
E
E
Predictor
Encoder
(a) Backbone
(b) Multi-Task
(c) Multi-Headed
Figure I: Illustrations of different models. (a) is the vanilla backbone model, which takes a sequence of breathing signal input, and predicts the corresponding sequence of oxygen saturation. (b) is the multi-task model that has extra input v, and predicts extra output ú. (c) is the multi-headed model that generates multiple predictions under different status, and the final output is obtained by combining these predictions with the estimated status.
METHOD
3.1 PROBLEM FORMULATION
Our task is to predict oxygen saturation based on respiration signal Let the breathing signal be E Rlx and the corresponding oxygen saturation be y E R 1IXX.TT we aim at learning scquence-to-sequeece model that maps z to y. i.e. p(y", "y"," y2,............... y..... .yo.y [2 x² _.,... foT)). Here, I' denotes the length of the segments (in seconds), fo and fo are the sampling frequencies of the breathing and oxygen data. In practice, we have high sampling rates for both respiration and oxygen saturation, i.e., fo 10H and fo 1Hz. This allows to capture instantaneous changes in these signals.
Moreover, we are particularly interested in monitoring oxygen saturation during sleep time. This because blood oxygen typically drops during sleep, and supplemental oxygen is medically necessary oxygen level drops to 88% or lower for 5 consecutive minutes during sleep (Becattini et al. 2018
3.2 BACKBONE MODEL.
As the relation between respiration x and oxygen y could be highly complicated, we use deep neural network to map to y. We start with the backbone model shown in Figure [a), which consists of an encoder E(:; and a predictor Fy): The encoder composed of a fully con- volution network (FCN) followed by bidirectional transformers (BERT) module (Devlin et al.) 2018). The FCN extracts local features from the raw respiration signal, then the BERT module cap- tures long-term dependencies based on those features. This design has two benefits. First, it can extract both local and global information from the respiration signal. Second, it works for variable length respiration signals. The predictor F composed of several De-convolution layers, which up-samples the extracted features to the same time resolution of oxygen saturation. Formally, we have E R > R and F RaxofT - RIxfTT where is the size of the breathing feature and a is the down-sampling factor (1/240 in our model). The model is trained with a Li loss, i.e., (F, E) ll yil1- where ń F(E(x)) is model prediction. Thanks to elaborately designed model architecture, even with simple L1 loss, this model has achieved 1.66% error (see Table. []. which is a reasonably good result. More details on this model are available in Appendix
'The breathing signal is time series that measures the chest displacement with the inhale-exhale motion. Please refer to Section Band and Figure for detailed introduction and illustrative figures.
3

---

Under review as a conference paper at ICLR 2021
3.3 LEVERAGING AUXILIARY VARIABLES & THE MULTI-HEADED MODEL
In the medical domain, typically, there exist many physiological indices that could potentially help improve the model. From the viewpoint of machine learning, physiological indices can be viewed as auxiliary variables that could be beneficial to train the model. During testing, auxiliary variables can be categorized into two types: accessible and inaccessible. We denote accessible variables as v and inaccessible variables as u. In the learning procedure, common practice to use accessible variable as extra inputs and use inaccessible variables for auxiliary tasks. Figure [bb illustrates such framework where F. denotes predictor of the auxiliary task u. We call the design in Figure Ib)) the Multi- Task model. Despite its intuitive appeal, in practice, this model may not always lead to improvements. As shown in Table the Multi-Task model even causes a performance drop on the SHHS datase
In our work, we investigate an alternative approach that uses auxiliary variables as switches. The idea is based on the hypothesis that some variable h affects the target input-output relationship in non-smooth way. For example, assuming h a binary variable. The mapping from input æ to output y conditioned on variable hi equal to 0 is significantly different from the mapping when the variable is 1. i.e., y( /h = 0) and y(r|h ha = 1) are two distinct functions. In this case, instead of learning function yzz, ft., we propose to use variable h as a switch to partition the data space and learn the functions y( = 0) and y( = 1) separately. We realize this idea in our Multi-Header model illustrated in Figure (c). We first define switch function S(u, u)) V x U [N] where E V and u E lI are accessilleimaccessihhee variables and N is the number of switch status. The model has N heads { to make the prediction it = Fz(()))) for every switch status separately. also has a predictor Fu to infer u. During testing time, based on the accessible auxiliaries v and estimated inaccessible auxiliaries ú, we evaluate the switch status S = S(v, di, and output the final prediction أ Ug. In the case of oxygen prediction, DJ and the switch S are time series, we use of and S denote their value at time step f. As shown in Figure (e), the final prediction is the gated combination of every head's output, jf = -i'g.- One might argue that the Multi-Task model
can also learn such compositional function by implicitly learning each piece of the target function, and learning to assign the input to the target piece based on the value of the auxiliary variable. However, in practice, such incorrect model inductive bias introduces unnecessary difficulty in the learning procedure and typically leads to bad performance. In our experiments, we show, with a more preferable inductive bias, the Multi-Headed model outperforms the Multi-Task model with clear gap.
3.4 IDENTIFYING GOOD SWITCHES VIA GRADIENT DIAGNOSIS
In the above description, we assume a good switch function exists. In this paragraph, we answer the critical question: which auxiliary variables should be used as switches? We introduce, gradient diagnosis, simple but effective way to test the influence of variable v on the task of learning function y(x)). We first train vanilla model with no auxiliary variables, c.g., our backbone model. We then use the validation set to check the predictor F(:; 0 loss gradient Vo For each possible value a E V of variable v, we compute the gradients averaged on data samples when v equals o, i... Vo,Clu=o
b(s.p)))(...//=a)) Vo, ((F(())) y). Such gradient indicates the direction to improve the model when v = a. Intuitively, if gradients Ve, Llv=a are similar for all values a, it means variable does not change the loss function drastically. So is reasonable to use one model to handle all values of v. On the other hand, if the gradients are vastly different from each other, it preferable to separate the predictor for different values of v.
4 EXPERIMENTS
We present the results of empirical evaluation medical datasets and RF data. We evaluate the models using the average absolute error between the predicted oxygen saturation ÿ and the ground- truth y. The error is first averaged for every night and each subject, i.c., $ E - - We then report the mean and standard deviation of per-night averaged errors of the dataset.
-
Mooe dataset details are provided in Section 4.1
4

---

Under review as a conference paper at ICLR 2021
4.1 DATASETS
Medical Datasets. We leverage three publicly- aaviiale medical datasets: Sleep Heart Health Study (SHHS) (Zhang et al.),, Multi-Ethnic Study of Atherosclerosis (MESA) (Chen et al.] 2015), and Osteoporotic Fractures in Men Study (MrOS) (Blackwell et al., 201T). Each dataset contains the polysomnography (PSG) signals of subjects. The first two datasets have males and females, whereas the last one has male subjects. The PSG data includes full-night respiration signals and the corre- sponding oxygen saturation (SpO2) and sleep stage labels. The data has three types of respiration channel measured with: a nasal probe, breathing belt the chest, and breathing belt on the ab- domen. We denote it as BB-respiration signals. The datasets also contain subjects' bio-information such as gender, age, etc. SHHS, MESA, and MrOS contains 2651, 2056, and 1026 subjects, respec- tively. We randomly split the training and testing sets for SHHS, MESA, and MrOS as 1855/796, 1439/617, and 718/308, and keep the same splits in all experiments. We note that the subjects in these studies have an age range between 40 and 95, and suffer from variety of diseases as indicated by the studies names. This allows for a wider range of oxygen variability beyond the typical range of healthy individuals.
RF Dataset. We collect a small PSG dataset that contains the standard PSG channels mentioned above as well as an RF-based inference of the respiration signal, i.e., RF-respiration signals. The dataset collected the sleep lab in the university and contains 31 nights of data from 14 indi- viduals. All subjects in this study are healthy. Each subject is monitored using the standard PSG channels as in the medical datasets above (SpO2, respiration belts, sleep stages, etc.). Additionally, radio device is installed in the room to collect RF signals, which are then processed to extract the subject's 's respiration in accordance with Adib et al. (2015). Therefore, in this dataset, each subject has both the Berreppiration signals and the RF-respiration signals.
4.2 MODELS
As we describe in Section our models focus on predicting oxygen from breathing while leveraging auxiliary gender and sleep stage information. We evaluate the following models: (a) Single-Headed the vanilla backbone model. (b) Multi-Task: the model that takes gender as an extra input and is trained with sleep stages as an auxiliary task. (c) -Hldii-Headed: the model that uses multiple heads to predict oxygen level for different switch status. (d) We also compare with an adaptation of the Projecting Conflicting Gradient PCGrad -ddptedAdapted. The original PCGrad was proposed for improved multi-task learning to deal with scenarios where the various tasks have divergent gradients (Yu et al.) 2020 In that case, PCGrad projects the divergent gradients along consistent directions. In our context, the gradients of the tasks themselves - i.e., oxygen prediction and sleep stage prediction do not diverge. Hence, simply applying PCGrad to our problem should be no different from the Multi- Task model. However, we can adapt PCGrad to be more suitable for our problem. In our case, the gradients of subgroups (i.e.. subjects with different status) in the input data with respect to the main task can be divergent significantly different. We can think of learning mapping from breathing to oxygen for different subject status as different sub-tasks. Then, we can augment our backbone model with PCGrad applied to these sub-tasks. Specifically, train the single predictor F", We compute 6 different losses based on the six subject status corresponding to our 6-headed model. For each subject status i, we have an oxygen prediction loss C for that status. Mathematically, C, EE = (Ers'=s 1) 'E E [F((())))' " y²). We compute the predictor's 's gradients Vo, C. for each loss and update the predictor by the gradient calibrated by PCGrad.
4.3 GRADIENT DIAGNOSIS RESULTS
Before evaluating the various models, we show that the gradient diagnosis method proposed in section 3.3 finds good "switches" for the oxygen prediction task. Among the huge set of physio- logical indices in the medical datasets, we pick the following variables as candidates: (a) Sleep stages: awake, light sleep (NI,N2), deep sleep (N3), rapid eye movement (REM); (b) Gender: male vs. female; (c) Asthma: having asthma or not; (d): Smoking: non-smoker, current smoker, for- mer smoker; (e) Education: less than 15 years vs. greater than 15 years; (f): Height: lower than 160em or higher than 160em; (g) Aspirin: taking aspirin or not. Figure 2 visualizes the cosine similarity of the gradients computed on the medical dataset SHHS. We clearly see that the gradient directions are altered by the sleep stage, gender, asthma and smoking while less affected by the
5

---

Under review as a conference paper at ICLR 2021
1.0
0.5
0.37
0.32
0.24
Pale
Never
1.0
0.05
9
4.2
1.0
0.85
111
0.5
1.0
0.88
0.74
0.43
remale
Kamm
0.05
Remale
Conrent
1.0
0.97
Gender
(d) Smoking
Aspirin
0.32
0.79
0.97
1.0
0.28
<-160
1.0
-15
1.0
0.95
REM
0.24
0.32
0.20
0.73
1.0
1-15
0.95
1.0
Aadio
R2M
(a) Sleep Stage
(c) Asthma
(e)
Height
(g) Edocation (years)
Figure Gradient cosine similarities of different variables.
height, aspirin and education level. These result is quite interesting. For example, although it is hard see why gender affects the exygen-breathing function from the first glance, there are medi- cal evidence that show gender matters. Due to differences between women's 's and men's 's respiratory system, women consume more oxygen when breathing compared men (Ramonatxo) 2003), espe- cially during exercise (Reybrouck & Fagard 1999). and men's 's oxygen saturation is naturally higher than women (Ricart et al. 2008). As for sleep stages, is a known fact that it influences the oxygen saturation. For example, pointed out by Choi et al. (2016), that oxygen saturation differs signifi- cantly during REM and Non-REM sleep. It is also understandable that asthma affects oxygenation since asthma causes difficulty in breathing. It is also supported by medical findings that smoking reduces oxygen saturation (Ozdal et al. (017)). Since it is a long-term effect, no matter if smoker quits not, his/her oxygen pattern is already altered. This may explains why the gradient similarity is high between current smoker and former smoker.
Based on our gradient diagnosis, pick sleep stage and gender as auxiliary variables for our task. Other variables are possible. However, we pick these two variables because gender is almost always accessible during testing: and while sleep stages are usually not available during testing. luckily past work shows that sleep stages can be inferred from respiration signals (Zhao et al.) 2017). Hence, our model can estimate the sleep stages and use the estimates to do the switching. Further, from Figure[js a), we can see the gradients for awake, REM and sleeping (N1,32,N3) are very dissimilar from each other while the gradients are much more similar within the group of N1,N2,,3. So we design the switch function to consider N N1,N2,N3 one sleep stage. In total, we have six switch status due to the combinations of gender and sleep stages. Our model has six heads for each switch status.
4.4 EVALUATION ON MEDICAL DATASETS
Quantitative Results. Table reports the absolute error of all models for the three medical datasets. The table reveals two main results. First, all baselines perform reasonably well and have average prediction eπrors between 1.58% and 1.7%. These eπгогs are relatively low and comparable to the errors of consumer pulse oximeters whose average error is in the range 0.4-3.5% (Lipnick et al., 2016). It is important to keep in mind that all baselines are based on our backbone model, which already embodies our design for learning this new task. Thus, these results show for the first time that respiration signals have a predictive power for learning oxygen saturation.
Second, Table D also shows that the Multi-Headee model outperforms the other baselines on all datasets. This demonstrates that the Multi- design is more effective at leveraging relevant auxiliary variables than the other methods.
Qualitative Results. Figure visualizes the predicted oxygen saturation of the Multi-Headed model and the Multi-Task model on a male subject in the SHHS dataset. As the ground- oxygen satu- ration are integers, we rounded the predicted oxygen saturation into integers. The background color indicates different sleep stages. The 'light grey' and "white" corresponds to 'REM' and 'Awake', respectively. For clarity, we one color 'dark grey' for all three non- REM 'Sleep' stages, N1, N2, and N3. We observe that the headed model consistently performs better than the Multi-Task model over the whole night. In Ron-REM 'Sleep', the ground-truth oxygen saturation is more stable.
visualizes

---

Under review as a conference paper at ICLR 2021
Table 1: Prediction error (mean + std) of different models for three medical datasets.
Model
SHHS
MESA
MrOS
Overall
Single-Headed
1.68 1.56
1.39 1.58
1.76 + 1.56
1.66 + 1.53
Multi-Task
1.69 + 1.57
1.51 1.56
1.71 + 1.57
1.62 + 1.53
PCGrad-Adapted
1.74 + 1.58
1.62 1.57
1.74 1.53
1,71 = 1.54
Multi-Headed
1.62 + 1.54
1.48 1.51
1.66 = 1.51
1.58 + 1.49
Oxygen (%)
Ground Truth
Multi-readeed
Multi-Takk
16600
12600
Time (in seconds)
96
100
90
Sleep
REM
Awake
Figure 3: Visualization of the oxygen saturation predicted by the Multi-Task model and the Multi- Headed model on male subject in SHHS dataset. The background color indicates sleep stages. The 'dark grey', 'light grey" and "white" corresponds to 'Sleep' (N1, N2, N3), 'REM' and 'Awake'.
The Multi-Headed model predicts more accurate oxygen than the Multi-Task model. In the 'REM' stage, people usually have lower oxygen saturation with larger variance. The Multi-Headed model predicts the ups and downs and captures the changes blood oxygen. In the 'Awake' stage, the Multi-Headee model predicts the more dramatic changes, while the Multi-Task model has larger bias. This experiment demonstrates that the way we incorporate the sleep stages into the model improves the performance across different sleep stages.
Error Analysis and Relation to Skin Color. One important issue to keep in mind is that our model learns from ground-truth data measured using pulse oximeters (today oximetry the only way for continuous noninvasive monitoring of blood oxygen). Hence, to understand the prediction errors and their potential causes, we plot Figure 4 the distributions of the ground-truth SpO2 and the predicted SpO2 by the Multi-Headed model for different races, for all datasets. Interest- ingly, the ground-trutt -tuuh SpO2 from oximetry shows a clear discrepancy between black and white subjects. In particular, the ground-truth oxygen distributions show that black subjects have higher blood oxygen. This is compatible with past findings that pulse oximeter overestimates blood oxygen in dark-skinned subjects (Feiner al. 2007). In contrast, the model prediction shows much more similar oxygen distributions for the two races. is interesting that while the model learned from what seems to be biased ground-truth, the results show that the model is able to correct, to some extent, for this bias. Admittedly, these observations are inductive since there is no way to measure the exact errors in the ground-truth labels.
The distributions in Figure also show that the model tends to miss the very high and very low oxygen values. This is expected given that only very small percentage of the ground-truth data falls in the tails. One would expect that the performance on very high and very low oxygen values would improve if there is sufficient training data those ranges.
4.5 EVALUATION ON RF DATASET
In this section, evaluate whether model, which is trained on respiration measured using a breathing belt, can generalize to predicting oxygen from respiration signals measured via radio waves. We directly apply Multi-Headed model trained on the BB-respiration signals in medical
7

---

Under review as a conference paper at ICLR 2021
Deralty
Denohyy
Dendity
5p02
Spo2
5po2
5p02
(a) SHHS GT (b) SHHS Pred (c) MESA GT (d) MESA Pred
(c) MrOS GT
(f) MrOS Pred
Figure 4: Distributions of the ground-truth SpO2 readings ((a), (c), (e)), and the predicted SpO2 ((b),
(d), (f)) by meati- model for the two races.
Prodicted Par-Night Oxygon
00.5
05.0
94.
94.0
96.5
06.0
96.
dates
Figure 5: The predicted per-night averaged oxygen saturation for a COVID-19 patient from Apr 28 July 31, 2020. The patient turned negative for COVID-19 on May 2020 (marked by the star).
datasets, to the RF dataset. For comparison, we test the model on both RF-respiration signals and BB-respiration signals from the RF dataset. The results are shown in Table 2 We observe that the model works well RF dataset with small prediction error, no matter which breathing signal is used. The prediction error of using the RF-respiration signals (1.28%) is even slightly lower than using the B--repiiration signals (1.30%). This is because the noise in the respiration measured using RF signal is different from the noise in the breathing belt measurements. The noise in the RF-based measurements tend to increase with motion and hence we see that the model prediction error is higher for the aware stage. In contrast, the error in the breathing belt depends on the sleep posture and the positioning of the belt on the body. Also, comparing with the error from previous section, we find the model is performing better on the RF dataset than on the medical datasets, i.e., 1.30% vs. 1.62%. We believe the reason is that the RF dataset is collected with healthy individuals and hence has less complexity and is easier to predict. These results show the first demonstration of predicting oxygen saturation from radio signals without any body contact.
Table 2: Prediction error for the RF dataset using BB-respiration signals or RF-respiration signals.
BB-respiration signals
RF-respiration signals
Awake
Sleep
REM
All
Awake
Sleep
REM
All
1.37
1.30
1.27
1.30
1.65
1.24
1.35
1.28
4.6 APPLICATION TO MONITORING COVID-19 PATIENTS
Finally, we show case study of a COVID-19 patient. An 88-year old male was monitored from April 28 till July 31, in home, with a contactless radio device. The subject tested COVID-19 positive prior to the monitoring, and was declared COVID-19 19 negative May 5. The objective of the case study to compare the results of a completely passive contactless monitoring approach to information collected from the medical doctor and the professional caregiver attending to the subject.
Figure shows the average oxygen saturation predicted by the Multi-Headed model for each of the monitored nights. May 5 is highlighted with star as the day on which the patient was declared COVID-19 negative. The results the figure show a clear trend of increased oxygen level as the patient transitioned to becoming COVID-19 negative. They also show that improvement in blood oxygen did not stop on the day the patient became COVID-free it rather continued to ramp up for almost two weeks after that date, and eventually reached steady level. While there is ground-
8

---

Under review as a conference paper at ICLR 2021
truth measurements of the oxygen for this completely passive case study, the results are consistent with the doctor notes about patient recovery and general expectations for a symptomatic COVID-19 patient who recovered without escalation of his COVID-19 symptoms.
5 DISCUSSION AND CONCLUDING REMARKS
This paper introduces model for predicting oxygen saturation from breathing signals, and shows that it performs well on three medical datasets and an RF-based dataset. In this section, we discuss some of the clinical implications, properties, and limitations of such model.
Data Diversity. We note that the medical datasets used in our evaluation include many patients with diverse diseases and health conditions. In particular, the Multi-Ethnic Study of Atherosclerosis (MESA) dataset focuses on patients with Atherosclerosis, a disease in which cholesterol plaques build up in the walls of arteries, and the minimum oxygen saturation predicts disease progno- sis (Gunnarsson et al. 2015). As shown in the results, our model has low error and good per- formance on this dataset. In fact, all three evaluation datasets (SHHS, MrOS and MESA) contain people who have various diseases including chronic bronchitis (323 subjects), cardiovascular dis- eases (1196 subjects), coronary heart disease (869 subjects), myocardial infarctions (508 subjects), diabetes (307 subjects), Alzheimer's (40 subjects), Parkinson's 's (69 subjects), and others. This means that our reported accuracy already takes into account unhealthy individuals. We note that many of these diseases, including diabetes and pulmonary and cardiovascular discases, interact with oxygen. Our model performs well on this population, as shown in our evaluation results. Furthermore, all three datasets are focused on old people. In particular, the median age in MrOS is 80 years. Hence, 50% of the subjects are in their 80's and 90's. Such old subjects are natural beneficiaries from monitoring oxygen saturation since they are at high risk of low oxygen NLM 2020
Relationship between breathing signals and Oxygen. Oxygen affects breathing through the pe- ripheral chemoreceptors, located in the carotid bodies (Plataki al. 2013). There feedback loop that relates blood oxygen and ventilation, allowing changes in oxygen to cause changes in breath- ing. However, the details of the chemoreflex physiology are complex and not yet fully understood. Our approach is data-driven and demonstrates that custom neural networks can extract significant and useful information about oxygen level from the breathing signal. It is important to note that the model learns more than the relationship between breathing rate and oxygen level. In Section B we show several examples where two subjects breath at the same rate, yet the model detects that their oxygen levels are quite different. Such empirical results show that the model not only leverages the breathing rate, but also other more sophisticated patterns the detailed breathing signals.
Limitations and Clinical Usefulness. Oxygen saturation and breathing are physiologically related, but this does not mean that the model can discover all situations that lead to low oxygen. Even pulse oximetry does not always work. The literature reports that in the critically ill, oximetry does not reliably predict changes in oxygen saturation because the calibration of red-light absorbance is based caleulations made from healthy volunteers (Ralston al.) [191]. Oximetry also fails in subjects with COPD (Amalakanti & Pentakota 2016). anemia (Severinghaus & Kohl 1990). sickle cell disease (Bliisdell et al.] (200)), skin pigmentation (Bickler et al. 2005). Similarly, the model herein has its own limitations and cannot detect low oxygen if the breathing signal is not impacted.
Our objective is to introduce a method that is non-invasive, easy to use, non-contact, and can be useful in many cases, particularly for old people, who are naturally at high risk of reduced oxygen (NMM 2020 This population is well represented in our evaluation datasets, on which the model is demonstrated to have good performance. Furthermore, old patients can have onset of dementia or Alzheimer's 's and cannot remember to regularly measure themselves using a pulse oximeter. Thus, they would benefit from system that passively and continuously measures oxygen using radio signals, without asking the patient to remember to actively measure themselves.
The model can also provide complementary oxygen monitoring system at home or in the hospital. As explained above pulse oximetry fails in various scenarios. One can add a radio device in the patient room to collect radio signals and use our system to augment oximetry. This does not add any overhead to the patient since they will not wear any additional sensor but rather be monitored in a passive contactless manner using radio signals. Further, since oximetry and our model use intrinsically different methods, they have independent errors, and hence can complement each other.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, and Robert C Miller. Smart homes that monitor breathing and heart rate. In Proceedings of the 33rd annual ACM conference on human factors in computing systems, 837-846, 2015.
Sridhar Amalakanti and Mohan Rao Pentakota. Pulse oximetry overestimates oxygen saturation in copd. Respiratory Care, 61(4):423-427, 2016.
C Becattini, MC Vedovati, P Pruszczyk. Vanni, M Cotugno, LA Cimini, V Stefanone, MG de Na- tale, M Kozlowska, F Mannucci, et al. Oxygen saturation or respiratory rate to improve risk stratification in hemodynamically stable patients with acute pulmonary embolism. Journal of Thrombosis and Haemostasis, 16(12):2397-2402, 2018.
Philip E Bickler, John R Feiner, and John W Severinghaus. Effects of skin pigmentation on pulse oximeter accuracy at low saturation. Anesthesiology: The Journal of the American Society of Anesthesiologisis, 102(4):715-719, 2005.
Terri Blackwell, Kristine Yaffe, Sonia Ancoli-Israel, Susan Redline, Kristine E Ensrud, Marcia L. Stefanick, Alison Laffan, Katie L Stone, and Osteoporotic Fractures in Men Study Group. Associations between sleep architecture and sleep-disordered breathing and cognition in older community-dwelling men: the osteoporotic fractures in men sleep study. Journal of the American Geriatrics Society, (9111):2217-2223 2011.
Carol J Blaisdell, Steve Goodman, Karen Clark, James F Casella, and Gerald M Loughlin. Pulse oximetry is a poor predictor of hypoxemia in stable children with sickle cell disease. Archives of pediatrics & adolescent medicine, 154(9)9900-99)) 903, 2000.
Xiaoli Chen, Rui Wang, Phyllis Zee, Pamela I. Lutsey, Sogol Javaheri, Carmela Alcántara, Chan- dra L Jackson, Michelle A Williams, and Susan Redline. Racial/ethnic differences in sleep dis- turbances: the multi-ethnic study of atherosclerosis (mesa). Sleep, 38(6):877-885 888, 2015.
Funkyung Choi, Doo-Heum Park, Jae-hak Yu, Seung-Ho Ryu, and Ji-Hyeon Ha. The severity of sleep disordered breathing induces different decrease in the oxygen saturation during rapid eye movement and non-rapid eye movement sleep. Psychiatry investigation, 13(6):652, 2016.
Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pp. 160-167, 2008.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
Genaro Díaz-Regatón, Eduardo Miñambres, Marisol Holanda, Segundo González-Herrera, Fran- cisco Sôpez-Espadas, and Carlos Garrido-Díaz. Usefulness of venous oxygen saturation in the jugular bulb for the diagnosis of brain death: report of 118 patients. Intensive care medicine, 28 (12):1724-117) 1728, 2002.
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. Multi-task learning for multiple language translation. In Proceedings of the 53rd Annual Meeting of the Association for Compu- tational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1723-1732, 2015.
Hisham ElMoaget, Dawn M Tilbury, and Satya-Krishna Krishna Ramachandran. Predicting oxygen satu- ration levels in blood using autoregressive models: A threshold metric for evaluating predictive models. In 2013 American Control Conference, pp. 734-739. IEEE, 2013.
Hisham ElMoaqet, Dawn M Tilbury, and Satya- Krissaa Ramachandran. A novel dynamic model to predict abnormal oxygen desaturations in blood. In 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA), pp. 1-6. IEEE, 2014.
Hisham ElMoaqet, Dawn M Tilbury, and Satya Krishna Ramachandran. Multi-step ahead predic- tions for critical levels in physiological time series. IEEE transactions on cybernetics, 46(7); 1704-1714, 2016.
10

---

Under review as a conference paper at ICLR 2021
John R Feiner, John W Severinghaus, and Philip E Bickler. Dark skin decreases the accuracy of pulse oximeters at low oxygen saturation: the effects of oximeter probe type and gender. Anesthesia & Analgesia, 105(6):S18 S23, 2007.
David A Forsyth and Jean Ponce. Computer vision: modern approach. Prentice Hall Professional Technical Reference, 2002.
Sam Ghazal, Michael Sauthier, David Brossier, Wassim Bouachir, Philippe A Jouvet, and Rita Noumeir. Using machine learning models to predict oxygen saturation following ventilator sup- port adjustment in critically ill children: A single center pilot study, PloS one, 14(2):c0189221 2019.
Sverrir I Gunnarsson, Paul E Peppard, Claudia Korcarz, Jodi H Barnet, Erika W Hagen, Mae Hla, Mari Palta, Terry Young, and James H Stein. Minimal nocturnal oxygen saturation predicts future subclinical carotid atherosclerosis: the wisconsin sleep cohort. Journal of sleep research, 24(6))680-668 2015.
Chen-Yo Yu Hsu, Yuchen Liu, Zachary Kabelac, Rumen Hristov, Dina Katabi, and Christine Liu. Ex- tracting gait velocity and stride length from surrounding radio signals. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 21 16-2126, 2017.
Hongbo Jiang, Chao Cai, Xiaoqiang Ma, Yang Yang, and Jiangehuan Liu. Smart home based on wifi sensing: A survey, IEEE Access, 6:13317-13325, 2018.
SE Lapinsky, Aubin, S Mehta, P Boiteau, and AS Slutsky. Safety and efficacy of sustained inflation for alveolar recruitment in adults with respiratory failure. Intensive care medicine, 25 (11):1297-1301, 1999.
Jinscok Lee, John Florian, and Ki Chon. Respiratory rate extraction from pulse oximeter and electrocardiographic recordings. Physiological measurement, 32(11):1766 2011.
Binbing Liao, Jingqing Zhang, Chao Wu, Douglas Mcllwraith, Tong Chen, Shengwen Yang, Yike Guo, and Fei Wu. Deep sequence learning with auxiliary information for traffic prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 537-546, 2018.
Michael S Lipnick, John R Feiner, Paul Au, Michael Bernstein, and Philip E Bickler. The accuracy of 6 inexpensive pulse oximeters not cleared by the food and drug administration: the possible global public health implications. Anesthesia & Anallesia, 123(2):338-32) 2016.
Yifan Liu, Bohan Zhuang, Chunhua Shen, Hao Chen, and Wei Yin. Auxiliary learning for deep multi-task learning. arXiv preprint 4rXhv:1909.02214 2019.
Taylor Mordan, Nicolas Thome, Gilles Henaff, and Matthieu Cord. Revisiting multi-task learning with rock: deep residual auxiliary block for visual detection. In Advances Neural Information Processing Systems, pp. 1310-1322, 2018.
Phuc Nguyen, Xinyu Zhang, Ann Halbower, and Tam Vu. Continuous and fine-grained breathing volume monitoring from afar using wireless signals. In IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, pp. 1-9. IEEE, 2016.
NIH. Covid-19 treatment guidelines.
nih
2020. Accessed:
2020-10-01.
NLM. Aging changes in the lungs. http:://eeelnnelll...cc//eerrrrr 004011. htm 2020. Accessed: 9020-10-01.
Kristen Nordenholz, Jordan Ryan, Benjamin Atwood, and Kennon Heard. Pulmonary embolism risk stratification: pulse oximetry and pulmonary embolism severity index. The Journal of emergency medicine, 40(1):95-102, 2011.
11

---

Under review as a conference paper at ICLR 2021
Mustafa Özdal, Zarife Pancar, Vedat Çinar, and Murat Bilgiç. Effect of smoking on oxygen satu- ration in healthy sedentary men and women. EC Pulmonology and Respiratory Medicine, 4(6): 178-182, 2017.
Rosalind W. Picard, Elias Vyzas, and Jennifer Healey. Toward machine emotional intelligence: Analysis of affective physiological state. IEEE transactions pallern analysis and machine intelligence, 23(10): 2001.
Maria Plataki, Scott A Sands, and Atul Malhotra. Clinical consequences of altered chemoreflex control. Respiratory physiology & neurobiology. 189(2)3354-366 2013.
AC Ralston, RK Webb, and WB Runciman. Potential errors in pulse oximetry: I. pulse oximeter evaluation. Anaesthesia, 46(3):202-206, 1991.
M Ramonatxo. Gender influence on the oxygen consumption of the respiratory muscles in young and older healthy individuals. Int J Sports Med, 24:559-564, 2003.
Tony Reybrouck and Robert Fagard. Gender differences in the oxygen transport system during maximal exercise in hypertensive subjects. Chest, 11(())788-779) 1999.
A Ricart, T Pages, G Viscor, C Leal, and JL Ventura. Sex-linked differences in pulse oxymetry. British journal of sports medicine, 42(())620-62) 2008.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241. Springer, 2015.
John W Severinghaus and Shin O Koh. Effect of anemia on pulse oximeter accuracy at low satura- tion. Journal of clinical monitoring, 6(2):85-88, 1990.
E. Stewart Taylor, William C Scott, and Clifton D Govan. Studies of blood oxygen saturation and causes of death in premature infants. American journal of obstetrics and gynecology, 62(4))764- 777. 1951.
Yonglong Tian, Guang-He Lec, Hao He, Chen Yu Hsu, and Dina Katabi. Rf-based fall monitoring using convolutional neural networks. Proceedings of the ACM on Interactive, Mobile, Wearable and Uhiquitous Technologies, 2(3):1-24, 2018.
Abhinav Valada, Noha Radwan, and Wolfram Burgard. Deep auxiliary learning for visual localiza- tion and odometry. In 2018 IEEE international conference on robotics and automation (ICRA), pp. 6939-6946. IEEE, 2018.
Hao Wang, Chengzhi Mao, Hao He, Mingmin Zhao, Tommi S Jaakkola, and Dina Katabi. Bidirec- tional inference networks: A class of deep bayesian networks for health profiling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 766-773, 2019.
Xuyu Wang, Chao Yang, and Shiwen Mao. Tensorbeat: Tensor decomposition for monitoring mul- tiperson breathing beats with commodity wifi. ACM Transactions on Intelligent Systems and Technology (TIST), 9(1):1-27, 2017.
Yuxi Wang, Kaishun Wu, and Lionel MNi. Wifall: Device-free fall detection by wireless networks. IEEE Transactions on Mobile Computing, 16(2):581-594, 2016.
Aaron Wilson, Alan Fern, Soumya Ray, and Prasad Tadepalli. Multi-task reinforcement learning: hierarchical bayesian approach. In Proceedings of the 24th international conference on Machine learning, pp. 1015-1022, 2007.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. arXiv preprint arXiv:2001.0678) 2020.
Shichao Yue, Hao He, Hao Wang, Hariharan Rahul, and Dina Katabi. Extracting multi- multi-person person respiration from entangled rf signals. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(2):1-22, 2018
12

---

Under review as a conference paper at ICLR 2021
Guo-Qian Guo-Qiang Zhang, Licong Cui, Remo Mueller, Shiqiang Tao, Matthew Kim, Michael Rueschman, Sara Mariani, Daniel Mobley, and Susan Redline. The national sleep research resource: towards sleep data commons. Journal of the American Medical Informatics Association.
Mingmin Zhao, Fadel Adib, and Dina Katabi. Emotion recognition using wireless signals. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, pp. 108, 2016.
Mingmin Zhao, Shichao Yue, Dina Katabi, Tommi Jaakkola, and Matt Bianchi. Learning sleep stages from radio signals: A conditional adversarial architecture. In International Conference on Machine Learning, pp. 4100-4109, 2017.
13

---

Under review as a conference paper at ICLR 2021
A MODEL ARCHITECTURE AND IMPLEMENTATION DETAILS
As we described section S our vanilla model consists of an encoder and a predictor. The en- coder has nine 1-D convolutional layers Monv-Norm-RReLL that shrink the temporal dimension of the features by 240 times. It then followed by several bi-directional multi-head self-attention layers (Devlin al. 2018) aggregate the temporal information at the bottleneck. We use layers, 8 heads with hidden-sian size of 256, intermediate-ss size of 512 for self-attention, and the max position embeddings is 2400. The decoder contains 7 layers 1-D de- de-convolutional layers (DeConv-Norm- RReLU). We also use skip connection (Ronneberger et al. 2015) by concatenating the convolu- tional layers in the encoder the de-convolutional layers the predictor. Figure ( illustrate the overall network architecture. We keep the encoder and predictor architectures the same for all mod- els. All models are implemented using Py TyToch. The number of parameters for the multi-headed model is 26,821,113 and the model size is (07.28MB. In the training process, optimizer with a learning rate of 2 x 10 10-4, and train the model for 500 epochs. Due to the varying input length, set the batch size for all the models to I (i.e., night of breathing signals and the corresponding oxygen time series).
the Adam
Because oxygen saturation is affected by sleep stages and sleep cycles, it exhibits non-local depen- dencies. To allows the model to capture these dependencies, we use the breathing signals from night of sleep as one input sample. To process such a long signal the input, we use a UNET structure with a bottleneck that shrinks by a factor of 240, and has max position embedding of 2400. The breathing signal is measured 10 fps (frames per second). Thus, the model can process 240 2400/ 10/100033600 = 16 hours of breathing data.
Output
De Conv
De Conv
De Conv
De Conv
De Conv
De Conv
Predictor
Temporal-Aware
Features
BERT
Encoder
Position Embeddings
Breathing Features
Conv
Conv
Conv
Conv
Conv
Conv
Input
Figure 6: Network architecture for the vanilla model.
14

---

Under review as a conference paper at ICLR 2021
B BREATHING SIGNALS
The input respiration signal used in the paper is one-dimensional time series with frame rate of 10 fps (frames per second). contains full information about the inhale and exhale motions. Figure iillustrates typical patterns of breathing signals with the corresponding oxygen saturation. Figure a shows a normal breathing pattern, which leads to constant oxygen saturation. Figure b- d) are abnormal breathing signals.
M
MmwW M
Grownd Trcc
10.00
05.3
4130
4930
61008
100K0
10000
10000
143440
(a)
M00
005
500
03.5
Gre
187250 327100
307/02
1M00E
181000
30R022
380712
Figure 7: Visualization breathing signals and the corresponding oxygen level. (a) is an example of normal breathing signals. (b-d) are examples of different abnormal breathing signals. In every figure, the first row shows the breathing signals while the second row shows the ground truth oxygen level (in blue) and the oxygen level predicted by model (in orange).
B.1 FULL BREATHING SIGNALS VS. BREATHING RATE
is important note that we use the full breathing signals instead of the breathing rate. Full breath- ing signals contain richer information about oxygen saturation than the breathing rate. Figure g shows several cases where two subjects have the same breathing rate but different oxygen saturation values. The figure also shows that our model is able to predict the correct oxygen levels and is not confused by the fact that the subjects have the same breathing rates.
15

---

Under review as a conference paper at ICLR 2021
breathing
Full breathing signals
Silaed i dround
Sudject moddodd
Osygen
Okygen
subje Lneadingg
4.002
rale
Brealling (b)
(a)
S-becc Brachin
w bf h www.oowwwwwwwooooooooooooooo
Saeec2 Dracting
Full breathing signats
Full breathing signals
Selest Grond Trat
augeed & indicod
Osygen saluration
Osygen eaturation
1 Exeriiing kiathing
15
233003
227080
Breathing rale
222033
220099
13008
Breathing rate
6302
10:00
(c)
(d)
Figure 8: Visualization of full breathing signals having more information about oxygen level than breathing rates. Here are four examples of subjects having very similar breathing rate but different oxygen saturation. However, if we look at the full breathing signals, we can still differentiate them. Our model can capture the shape difference of full breathing signals and make the correct prediction.
C MORE VISUALIZATIOONS
We include more visualizations of the breathing signals and the corresponding oxygen saturation predicted by the MrOS (Figure I1 and Figure 12). In the plots, The background color indicates different sleep stages. The 'dark grey 'light grey and 'white" corresponds to sleep stages (N1, N2, N3), 'REM' and 'Awake", respectively.
model for the medical datasets: MESA (Figure 9 and Figure I0),
C.I MESA
The examples in Figure S show our model's ability to capture the fluctuations oxygen saturation. At the same time, the examples in Figure 10 shows our model accurately detects the region of low oxygen saturation, which highlights its usefulness monitoring patients.
C.2 MROS
Looking at the zoomed-in range (a) and (b) from Figure | we see that the model exhibits a larger when the ground truth SpO2 reading is very low. It is mainly caused by the imbalanced labels in the training set since subjects usually experience much less time of having low oxygen level (c.g., below 90%) than having normal oxygen level between 94% to 100%. Figure 12 shows another
16

---

Under review as a conference paper at ICLR 2021
example of the dynamics. As shown in the zoomed range, oxygen fluctuations correlate with one's sleep stage: in 'REM' (colored by light gray), the oxygen fluctuates drastically (colored by dark gray), the oxygen is much more stable and fluctuates in small range. Our model makes accurate predictions since leverages the sleep stage information.
while in 'slee''
Oxygen Saturation (%6
50-
Ground-Trum
80
Multi Headed
15
Oxygen Saturation (%)
Ground-Truth
Multi-Headed
0003
7000
E000
$000
10000
Figure MESA Example
Oxygen Saturation
Ground-Truth
Multi-Headed
3580
10000
35400
2000
25000
Oxygen Saturation (%)
Ground-Truth Multi-Headed
37500
30003
18992
19920
13550
20000
20500
23000
Figure 10: MESA Example 2.
17

---

Under review as a conference paper at ICLR 2021
Oxygen Saturation (96)
(b)
Ground-Truth Headed
-0-
5000
20200
15000
20000
20000
Oxygen Saturation (9)
(a)
Ground-Tran Multi-Heade
27888
Oxygen Saturation (%)
51.3
MM V
85.0
62.5
(b)
10.0-
Ground Truth
Head-
Figure 11: MrOS Example 1.
Oxygen (96)
Ground-Truth Multi-Headed
5000
20000
X5000
Oxygen Saturation (%))
60
Ground-Truth Multi-Headed
12500
Figure 12: MrOS Example 2.
18

---

Under review as a conference paper at ICLR 2021
D COMPARISON AGAINST NAIVE BASELINE MODELS
We have compared approaches with two naive baseline models, i.c., Linear Regression and Random Forest Regression. To choose features, we use the six statistical time series feature sug- gested by (Picard et al 200T). Table compares These baselines produce average prediction errors that are significantly higher than the multi-headed model. Figure T shows illustrative example of the prediction. It can be noticed that both the linear regression and random forest models tend to make predictions concentrated at the averaged oxygen level without capturing the dynamics of oxygen fluctuations across time. While our model's prediction follows the ground truth oxygen changes well. It is because our model is designed to capture the non-local temporal dependencies in the signals.
model with the naive baselines. As shown,
Table 3: Prediction error (mean + std) of naive baseline models compared with our multi-headed model.
Model
SHHS
MESA
MrOS
Overall
Linear Regression
1,86 1.72
1,73 = 1.73
1.87 + 1.69
1.81 = 1.72
Random Forest
1.85 + 1.73
1.72 1.73
1.89 + 1.71
1.81 1.73
Multi-Headed
1.62 + 1.54
1.48 + 1.51
1.66 + 1.51
1.58 = 1.49
Oxygen Saturation (%)
100
90
GT Ours
Lincar
Random Forest
80
5000
10000
15000
20000
25000
Oxygen Saturation (%)
98
GT
Ours
Linear
Random Forest
90
9500
10000
10500
11000
11500
12000
12500
Figure 13:
Comparison between our model and linear regression and random forest.
19

---

Under review as a conference paper at ICLR 2021
E SWITCHER-BASED AUXILIARY LEARNING PIECE-WISH LINEAR FUNCTION
To highlight benefits of switcher-based auxiliary learning and gradient diagnosis, we conduct experiments the toy dataset shown in Figure [4] The target function is 1-dimensional piece- wise linear function. The observed data has small white noise added to the label. Visually the function looks like string composed of 'W", 'N', and 'M", Assume have auxiliary variable, class, that indicates which letter a data point belongs to. Our goal is to learn the function with access the auxiliary variable. By using gradient diagnosis, it suggests that this auxiliary variable is a good "switcher" candidate. Denote 92, 93 as the gradients computed by the data points from the three letters. The cosine similarity between them are cos(g) 92) 0.48, cos(g1. 0.002, (os( 9a) = 0.39, meaning the gradients are not aligned.
We compare our Multi- Helli- model with three baselines: Single-Header Multi Multi-Task, PCGrad- Adapted described in section 4 Figure [5] displays the learned function of the four methods. Only the Multi-Headed model successfully learns the target function and makes a prediction that visually looks like "WNM". Quantitatively, the mean squared errors of the predictions are 0.06, 0.06, 0.05, 0.01 for Single-Headed, Multi- Task, PCGrad- Adaated,, and Multi-Headed respectively. We attribute the Multi-Hcaded model's success to its usage of the auxiliary variable "switcher".
1.2
1.0
Clsss1
0.8
-22
0.0
0.5
3.0
2.0
Figure 14: Visualization of the toy dataset. a ID piecewise linear function. The color indicates the auxiliary variable for every data point.
18-
W
14-
W
08-
68
06-
6.6
04-
67 coss-0
64
castee Ganed
02
CT caams
63
0.-
Tree daset
64
Pred cesue?
-02
26
(a) Single- Headed
(b) Multi Task
1.2
14
W
WNA
1.6-
14-
08-
68-
68-
G7 casueo
64
of Usome danee
casuml
daset
Pred caund
Fiee dayse
Prod cossee
cound
(c) PPPPCGrddAAie -daated
(d)
(Ours)
Figure 15: The benefits of switcher based auxiliary learning. The figure shows the results of different models for the toy dataset. Only the this non soothho function effectively.
model which uses a switcher based method learns
20


Reviews:
[{"review_id": "m50rCm3yvoG", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n##########################################################################\n\nSummary:\n\n \nThe paper proposes neural network models to predict oxygen saturation from breathing signals. The architecture implements \"feature switches\" that partition the data in a multi-head manner with the ultimate goal of predicting auxiliary variables which will help the prediction. The results are very encouraging especially given the immediate application to COVID-19 non-invasive monitoring. \n\n##########################################################################\n\nReasons for score: \n\n \nOverall, I vote for marginal accept. I like the idea of predicting one modality from another. My major concern is about the experimental section of the paper and some additional ablations (see cons below). Hopefully, the authors can address my concern in the rebuttal. \n\n \n##########################################################################Pros: \n\n \n1. The paper leverages one of the most important biomarkers for COVID-19 monitoring. If it's possible to predict SpO2 robustly from other signals, it's a big advance.\n \n2. The proposed architecture seems to be superior to vanilla models, as reported in the experiments section. \n\n \n3. This paper provides comprehensive experiments, including both qualitative analysis and quantitative results, to show the effectiveness of the proposed framework.\n\n \n##########################################################################\n\nCons: \n\n \n1. My major concern is the use of PSG and RF signals. Since both of them require prohibitively-expensive equipment (a minimum of 22 wire attachments, and a big antenna I suppose, respectively), I am curious why the paper is not using easier to obtain sensors. For example, some of the medical datasets already used here include heart rate measured through PPG signals or ECG (now common in smartwatches). Could the same framework be applied to heart rate->SpO2?\n\n \n2. The motivation behind the feature switches seems a bit inflated. The paper ends up using the gradient cosine similarities to inform the final outputs. Why do we need to inspect the gradients in contrast to -say- the validation loss? Couldn't this be achieved with traditional feature importance methods? For example, permutation feature importance could inform which feature set is insignificant or you could even apply SHAP to the vanilla model with all features.\n \n3. In terms of baselines, the paper starts from a single-headed neural network. However, it would be good to explore the \"lower bound\" of the dataset with a naive model. What would be the error of linear regression or a random forest regressor trained on features extracted by the signals? I am aware that here the output is multi-step, but you could apply the models iteratively on the predicted values. Also, in general, I would like to see more information about the parameter size of the models, batch size etc. In the appendix, we read that \"the breathing signals we used in all the experiments last several hours per night\", which means that the output is also a signal that spans hours? How do you average the error for that kind of multi-step forecasting? Does the error increase as we forecast further in the future?\n\n \n##########################################################################\n\nQuestions during the rebuttal period: \n\n \nPlease acknowledge the potential use of cheaper and more accessible sensors.\n\nThe structure of the sections could be improved as well. Section 3 is not the right place to mention (some) results.\n\nFigure 4 --> is this an observation made for the Multi-head model only? How is the distribution of the single-headed model?\n\nlearning to throw the data to the right function --> informal, please rephrase\n\n\n \n#########################################################################\n\nMany typos (please proofread the entire paper again): \n\n-These result is quite interesting \n\n-there are medical evidence \n\n-..\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "hnGCnaGOHXW", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: A model that appears to very effectively predict SpO2 from respiration signals. I find the model and experiments well-designed, but have serious concerns about the motivation and prediction task itself.\n\n**Update**: After reading the rebuttal and revised paper, I am keeping my score the same. There is a lot in this paper I really like -- an important prediction target and race analysis among them -- but still have concerns about clinical utility. In standard photoplethysmography, the causal graph is PaO2-->light absorbance, which leads to a clear story of what a fingertip SpO2 sensor is doing (modeling an imperfect but mostly unidirectional relationship). The causal graph in this paper is less clear but probably has some kind of cycle like ventilation-->SpO2-->ventilation, which is  a much more challenging story. I think there are two good routes this paper can take: (1) focus on how to clinically justify a model with such a potentially complicated causal graph; is the model learning the vent->O2 relationship, O2->vent, or some combination of the two? How can we be confident we know that? To which scenarios will and won't a model that has learned such relationships generalize? (The rebuttal has analysis that is a good start, but I think not enough to fully answer these questions) Or (2) acknowledge that the scope will necessary be limited when causal structure is so complicated and poorly understood, and actually limit the tasks the model is capable performing to those on which we can be confident it will do well. I think either (1) or (2) would involve major changes to the structure and goals of the paper, and probably require new experiments. \n\nObjective: Predict blood oxygen sequence information from respiration sequence information, leverage auxiliary variables and demonstrate feasibility of RF-based contactless oxygen prediction. I view the first as the primary goal, bolstered by the auxiliary variables, which enables the RF-based prediction.\n\nStrengths:\nOverall, I think the model design and evaluation is very good! I like the paper a lot overall and think the authors did a good job addressing most methodological and experimental concerns.\n* Impressive, accurate predictions. \n* Experiments are well-designed. In particular, analysis w.r.t. race is well-motivated and well conducted. \n* RF experiment demonstrates both RF feasibility and ability of the model to generalize across respiration measurement devices/methodologies.\n\nWeaknesses:\nMajor weaknesses (only 1):\nMy most important concern is the choice of problem and prediction task. There is a well-discussed medical distinction between ventilation and respiration -- simply moving air vs actually adding oxygen to the blood stream. There is a good reason why respiratory rate and pulse oximetry are measured separately: resp. rate measures ventilation while oximetry measures respiration. Many conditions, i.e. pulmonary edema or embolism, affect the lungs' ability to exchange oxygen without affecting the ability to move air, and many conditions, i.e. traumatic or neurological, affect the ability to move air without affecting the ability to exchange oxygen in the lungs. Because of this, I see SpO2 as a variable that contains *independent* information not in the ventilation signal, and believe it is dangerous to display a patient SpO2 signal that is entirely *dependent* on ventilation - containing no respiration signal. Such a signal could look plausible enough but fail in the most important clinical cases.\n\nI understand there is low error on the tasks shown, but how does the model work in cases where we'd expect respiratory problems but no ventilation problems -- or vice versa? Beyond sleep studies or ambulatory settings, is it likely to work in cases of pulmonary embolism or ARDS in an acutely ill COVID-19 patient? Some of the problems discussed in the paper (e.g., sleep-time drops in oxygen saturation) seem like they could plausibly be predicted from ventilation signal, but for these cases, why not predict a binary target instead of the full SpO2 signal? The only reason for predicting the SpO2 signal would seem to be if you think it's a true inference of the physiology that will hold true even in cases you didn't examine in the paper -- and I think there are a lot of prior physiological reasons to believe that's not the case. My prior is that there are at least 2 causal paths where you can predict SpO2 from ventilation signal: (1) hypoventilation-->hypoxemia and (2) hypoxemia-->hyperventilation. I'm concerned that the model may be able to pick up SpO2 signals that follow these patterns (and are accurate in the datasets used) without learning a general, \"true\" relationship between breathing and blood oxygen. This is important because the paper is primarily clinical: the value is the ability to predict a new clinical outcome. Thus, I think the paper should only be accepted if it gives a good idea of in what cases the model would be clinically useful at predicting SpO2, and I don't see much of this analysis in the paper. If these issues were discussed at all (and ideally in detail), I'd be more open to seeing clinical value in the work. \n\nMinor points:\n* Methodologically, it seems the auxiliary variable strategy works well but I'm not convinced it's the only way or the best way to solve the problem -- both multitask and multi-headed seem good at capturing the shape of the signal, with multitask often off by a constant. This is what we'd expect from an MSE model trained on a large dataset -- the overall signal shape will be pulled towards the mean. Gating into separate models for groups with different baselines would reduce this problem (because there will be less variance within each group). This is not a huge issue (and I know L1 loss is used here rather than MSE), but there are other ways to handle such problems -- for example using shape and time distortion losses like DILATE.\n* I did not find the COVID analysis particularly compelling because there's only a single patient and mostly qualitative analysis -- it's hard to draw any clear conclusions form the example about the method's value overall in such cases. I think I'd prefer a more rigorous evaluation on a non-COVID task to something that's a bit speculative but COVID related.\n* Some writing could be smoother and more terse in the Method section, i.e. \" In this paragraph, we answer\nthe critical question...\"\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "JiZKTmc5QMT", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Overview: The authors present an approach to learn blood oxygen levels using respiratory signal data. This might be relevant for various contexts such as ARDS or COVID-19. Unlike using pulse oximetry, it doesnt rely on wearing a sensor and less biased on darker skinned individuals.\n\nClarity and Quality: The problem description and aim of the paper are very clear. Unfortunately, I think the abstract of the paper and the introduction are quite misleading however as the authors entirely focus on COVID-19, while only a tiny fraction of the results are actually demonstrated on COVID. The paper thus makes bigger claims for COVID than what is demonstrated in the experiments which is misleading.\n\nOriginality and Significance: Certainly the problem of learning blood oxygen levels is a significant one in high-risk patients in ICU for instance. However, the originality of the modelling approach is not that unique. Specifically the authors use auxiliary learning to improve on standard ML approaches. The model proposed is a multiheaded model that incorporates auxiliary oxygen-related variables into the prediction process.\n\nPros: 1. The paper addresses a relevant biomedical problem\n2.  The authors demonstrate the performance of their approach on three separate datasets\n\nCons:\n1. Unfortunately, the authors never explicitly clarify what is meant by a \"breathing signal\". Is this the respiratory rate or a pulse or something else? I find this term neither technically sound from the medical perspective nor informative to a non-expert. This is also crucial to understanding how good the model is. For instance, if the task is to predict blood oxygen from \"breathing signals\", how noisy are these measurements? How are these recorded? Are these recorded via contactless sensing? \n\n2. In the experiment for COVID, the model is only applied to a case of a single COVID-19 patient. The reader needs more information of what is the prior condition of the patient, is he in ICU? If so, how do you account for treatment policies or protocol changing for the duration of stay; this could affect overall outcomes (and oxygen saturation)? Moreover, the paper oversells this demonstration on a single COVID-19 patient almost as its key contribution. Yet this is only a single patient.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Two referees indicate reject, one supports (weak) accept. My impression is that major points of criticism raised by the reviewers -- mostly about limited novelty and somewhat inconclusive experimental results -- could not be addressed in a clearly convincing way during the rebuttal phase. I will therefore recommend rejection.

Decision: Reject