Original: L EARNING B LOOD O XYGEN FROM R ESPIRATION S IGNALS
Revised: L EARNING B LOOD O XYGEN FROM R ESPIRATION S IGNALS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Monitoring blood oxygen is critical in a variety of medical conditions including COVID-19.
Revised: Monitoring blood oxygen is critical in a variety of medical conditions.

Original: For almost a century, pulse oximetry has been the only noninvasive method for measuring blood oxygen.
Revised: For almost a century, pulse oximetry has been the only non-invasive method for measuring blood oxygen.

Original: While highly useful, pulse oximetry has important limitations.
Revised: While highly useful, pulse oximetry has important limitations.

Original: It requires wearable sensors, which can be cumbersome for older patients.
Revised: It requires wearable sensors, which can be cumbersome for older patients.

Original: It is also known to be biased when used for dark-skinned subjects.
Revised: It is also known to be biased when used for darkskinned subjects.

Original: In this paper, we demonstrate, for the ﬁrst time, the feasibility of predicting oxygen saturation from breathing.
Revised: In this paper, we demonstrate, for the ﬁrst time, the feasibility of predicting oxygen saturation from breathing.

Original: By eliminating  dependency on oximetry, we eliminate bias against skin color.
Revised: By eliminating the dependency on oximetry, we eliminate bias against skin color.

Original: Further, since breathing can be monitored without body contact by analyzing the radio signal in the environment, we show that oxygen too can be monitored without any wearable devices.
Revised: Further, since breathing can be monitored without body contact by analyzing the radio signal in the environment, we show that oxygen too can be monitored without any wearable devices.

Original: We introduce a new approach for leveraging auxiliary variablesand tasks via a  multi-headed neural network model.
Revised: We introduce a new approach for leveraging auxiliary variables via a switcher-based multi-headed neural network model.

Original: Empirical results show that our model achieves good accuracy on existing medical datasets and can potentially be used to monitor the recovery of COVID-19 patients.
Revised: Empirical results show that our model achieves good accuracy on multiple medical datasets.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Oxygen saturation refers to the amount of oxygen in the blood – that is the fraction of oxygensaturated hemoglobin relative to the total blood hemoglobin.
Revised: Oxygen saturation refers to the amount of oxygen in the blood – that is the fraction of oxygensaturated hemoglobin relative to the total blood hemoglobin.

Original: Normal oxygen levels range from 94% to 100%.
Revised: Normal oxygen levels range from 94% to 100%.

Original: Lower oxygen can be dangerous, and if severe, leads to brain and lung failure (Taylor et al., 1951; D´ıaz-Rega˜n´on et al., 2002; Lapinsky et al., 1999).
Revised: Lower oxygen can be dangerous, and if severe, leads to brain and lung failure (Taylor et al., 1951; D ´ ıaz-Rega ˜ n ´ on et al., 2002; Lapinsky et al., 1999).

Original: COVID-19 patients  areparticularly susceptible to reduced bloodoxygen. The National Institutes of Health (NIH) recommends monitoring blood oxygen in COVID-19 patients and aiming to keep their oxygen about 92–96% (NIH).
Revised: Oxygen Monitoring is important for patients suffering from lung diseases such as COVID-19, pulmonary embolism (PE), and chronic obstructive pulmonary disease (COPD) who are susceptible to reduced blood oxygen (NIH, 2020; Nordenholz et al., 2011).

Original: Oxygen monitoring is particular important for older COVID-19 patients, who tend to arrive in the hospital with alarmingly low oxygen saturation, reducing their chances of survival (Levitan).
Revised: It is also recommended in old people since lung functions deteriorate with age putting the elders at a high risk of low oxygen (NLM, 2020).

Original: Today, the only noninvasive approach for measuring oxygen saturation (SpO2) is pulse oximetry.
Revised: Today, the only non-invasive approach for measuring oxygen saturation (SpO2) is pulse oximetry.

Original: It works by shining light on one’s ﬁgure (or other body parts).
Revised: It works by shining light on one’s ﬁgure (or other body parts).

Original: Since oxygen-saturated hemoglobin has a bright red color, oxygen saturation can be measured by estimating the absorbance of the redcolored waves relative to other colors.
Revised: Since oxygen-saturated hemoglobin has a bright red color, oxygen saturation can be measured by estimating the absorbance of the redcolored waves relative to other colors.

Original: While pulse oximetry is a very useful technology, it can be unsuitable for some of the most vulnerableCOVID patients.
Revised: While pulse oximetry is a very useful technology, it can be unsuitable for some of the most vulnerable patients.

Original: Oximetry requires the patient to wear a ﬁnger clip or other sensors, which can be cumbersome and hard to remember for older patients with dementia or cognition problems.
Revised: Oximetry requires the patient to wear a ﬁnger clip or other sensors, which can be cumbersome and hard to remember for older patients with dementia or cognition problems.

Original: Further, since it relies on measuring light absorbance, it is affected by skin color and tend to overestimate blood oxygen in dark-skinned subjects (Feiner et al., 2007).
Revised: Further, since it relies on measuring light absorbance, it is affected by skin color and tend to overestimate blood oxygen in dark-skinned subjects (Feiner et al., 2007).

Original: In this paper, we introduce a new approach for monitoring blood oxygen that could be more appropriate for older patients or those with dark skin color.
Revised: In this paper, we introduce a new approach for monitoring blood oxygen that could be more appropriate for older patients or those with dark skin color.

Original: We propose to learn oxygen saturation from respiration signals.
Revised: We propose to learn oxygen saturation from respiration signals.

Original: Recent advances in wireless sensing have demonstrated the feasibility of obtaining accurate measurements of the respiration signal from analyzing the radio waves that bounce off people’s bodies, without any physical contact (Adib et al., 2015;
Revised: Recent advances in wireless sensing have demonstrated the feasibility of obtaining accurate measurements of the respiration signal from analyzing the radio waves that bounce off people’s bodies, without any physical contact (Adib et al., 2015;

Original: Yue et al., 2018).
Revised: Yue et al., 2018).

Original: Hence, if oxygen saturation is predictable from breathing, we can sense one’s oxygen in a contactless way without any wearable sensors.
Revised: Hence, if oxygen saturation is predictable from breathing, we can sense one’s oxygen in a contactless way without any wearable sensors.

Original: Further, unlike pulse oximetry, such oxygen sensing method would be oblivious to skin color.
Revised: Further, unlike pulse oximetry, such oxygen sensing method would be oblivious to skin color.

Original: We model the problem asa sequence-to-sequence learning and introduce a neural network that maps respiration signals to the corresponding oxygen saturation.
Revised: We model the problem as sequence-to-sequence learning and introduce a neural network that maps respiration signals to the corresponding oxygen saturation.

Original: A key innovation in our model is a new design for leveraging auxiliary variables.
Revised: A key innovation in our model is a new design for leveraging auxiliary variables.

Original: The standard approach in the literature for leveraging auxiliary variables would either provide those variables to the model as inputs or try to predict them as auxiliary tasks.
Revised: The standard approach in the literature for leveraging auxiliary variables would either provide those variables to the model as inputs or try to predict them as auxiliary tasks.

Original: We demonstrate empirically that for our problem neither of these approaches is beneﬁcial.
Revised: We demonstrate empirically that for our problem neither of these approaches is beneﬁcial.

Original: We show that in some cases, the gradient of the model can take vastly different and even divergent values given the auxiliary variable.
Revised: We show that in some cases, the gradient of the model can take vastly different and even divergent values given the auxiliary variable.

Original: In such a scenario, it is better to learn different models for different values of the variable.
Revised: In such a scenario, it is better to learn different models for different values of the variable.

Original: This motivates us to propose an alternative design for incorporating auxiliary information.
Revised: This motivates us to propose an alternative design for incorporating auxiliary information.

Original: We introduce a multi-headed model where the value of the auxiliary variable is used to gate the learning through the appropriate head.
Revised: We introduce a multi-headed model where the value of the auxiliary variable is used to gate the learning through the appropriate head.

Original: We evaluate our model on multiple medical datasets, and compare it with various baselines.
Revised: We evaluate our model on multiple medical datasets, and compare it with various baselines.

Original: Our empirical results show that the average absolute error in predicting oxygen saturation on the tested medical data is 1.6%, which is comparable to the accuracy of consumer pulse oximeters (whose average error ranges from 0.4% to 3.5% (Lipnick et al., 2016)).
Revised: Our empirical results show that the average absolute error in predicting oxygen saturation on the tested medical data is 1.6%, which is comparable to the accuracy of consumer pulse oximeters (whose average error ranges from 0.4% to 3.5% (Lipnick et al., 2016)).

Original: Further, by testing the model on data that include the RF modality, we show that the model generalizes to breathing signals extracted from radio waves, opening the door for continuous contactless monitoring of blood oxygen.
Revised: Further, by testing the model on data that include the RF modality, we show that the model generalizes to breathing signals extracted from radio waves, opening the door for continuous contactless monitoring of blood oxygen.

Original: Finally, using longitudinal data from one COVID-19 patient, we validate that the predicted oxygen values are consistent with the patient’s recovery process.
Revised: We also use longitudinal data from one COVID-19 patient as a qualitative case study, showing that the predicted oxygen values are consistent with the patient’s recovery process.

Original: In summary, this paper makes the following contributions:
Revised: In summary, this paper makes the following contributions:

Original: • The paper is the ﬁrst to predict blood oxygen from respiration signals.
Revised: • The paper is the ﬁrst to predict blood oxygen from respiration signals.

Original: This provides a novel noninvasive way for monitoring oxygen saturation, and improves our understanding of the relationship between these two physiological signals.
Revised: This provides a novel non-invasive way for monitoring oxygen saturation, and improves our understanding of the relationship between these two physiological signals.

Original: • The paper introduces a new approach for leveraging auxiliary variables.
Revised: • The paper introduces a new approach for leveraging auxiliary variables.

Original: We analyze the gradient of a vanilla model with respect to the variable; if the gradient is divergent or vastly different for different values of the variable, we use a multi-headed neural network gated by the variable.
Revised: We analyze the gradient of a vanilla model with respect to the variable; if the gradient is divergent or vastly different for different values of the variable, we use a multi-headed neural network gated by the variable.

Original: • The paper is the ﬁrst to demonstrate the feasibility of contactless oxygen prediction from radio signals.
Revised: • The paper is the ﬁrst to demonstrate the feasibility of contactless oxygen prediction from radio signals.

Original: Such a design can facilitate in-home monitoring of COVID-19 patients, particularly  older patients who may have problems remembering to wear and charge oxygen sensors.
Revised: Such a design can facilitate in-home oxygen monitoring, particularly for older patients who may have problems remembering to wear and charge oxygen sensors.

Original: This is validated by predicting oxygen saturation from RF-based breathing signals, and providing a case study of a COVID-19 patient as he transitions from being COVID-19 positive to negative.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Predicting Oxygen Saturation.
Revised: Predicting Oxygen Saturation.

Original: Past work on oxygen prediction focuses on inferring future oxygen values from recent measurements.
Revised: Past work on oxygen prediction focuses on inferring future oxygen values from recent measurements.

Original: Several papers use auto-regressive models that take past SpO readings and predict the SpO2 in the near future (ElMoaqet et al., 2013; 2014; 2016).
Revised: Several papers use auto-regressive models that take past SpO readings and predict the SpO2 in the near future (ElMoaqet et al., 2013; 2014; 2016).

Original: Others use an auto-regressive model to model the photoplethysmogram (PPG) signal, which is the raw data obtained from a pulse oximeter (Lee et al., 2011).
Revised: Others use an auto-regressive model to model the photoplethysmogram (PPG) signal, which is the raw data obtained from a pulse oximeter (Lee et al., 2011).

Original: Some past work predicts changes in SpO2 in the ﬁve minutes after adjusting ventilator setting (Ghazal et al., 2019).
Revised: Some past work predicts changes in SpO2 in the ﬁve minutes after adjusting ventilator setting (Ghazal et al., 2019).

Original: Our work differs from these methods in that it predicts oxygen saturation from breathing, a completely different physiological signal.
Revised: Our work differs from these methods in that it predicts oxygen saturation from breathing, a completely different physiological signal.

Original: Further, we show the feasibility of oxygen prediction without body contact by inferring oxygen from radio-based breathing estimates.
Revised: Further, we show the feasibility of oxygen prediction without body contact by inferring oxygen from radio-based breathing estimates.

Original: Contactless Sensing with Radio Frequency Signals.
Revised: Contactless Sensing with Radio Frequency Signals.

Original: The past decade has seen a rapid growth in research on passive sensing using radio frequency (RF) signals.
Revised: The past decade has seen a rapid growth in research on passive sensing using radio frequency (RF) signals.

Original: Early work has demonstrated the possibility of sensing one’s respiration and heart rate using radio signals (Adib et al., 2015).
Revised: Early work has demonstrated the possibility of sensing one’s respiration and heart rate using radio signals (Adib et al., 2015).

Original: Building on this work, researchers have found that by carefully analyzing the RF signals that bounce off the human body, they can monitor a variety of health metrics including sleep, respiration, heart-rate, gait, falls, and even human emotions (Nguyen et al., 2016;Wang et al., 2017;
Revised: Building on this work, researchers have found that by carefully analyzing the RF signals that bounce off the human body, they can monitor a variety of health metrics including sleep, respiration, heart rate, gait, falls, and even human emotions (Nguyen et al., 2016;

Original:  Yue et al., 2018; Hsu et al., 2017; Wang et al., 2016; Tian et al., 2018; Zhao et al., 2016; Jiang et al., 2018).
Revised: Wang et al., 2017; Yue et al., 2018; Hsu et al., 2017; Wang et al., 2016; Tian et al., 2018; Zhao et al., 2016; Jiang et al., 2018).

Original: Our work adds a new piece to the global picture of predicting vital signals from radio waves.
Revised: Our work adds a new piece to the global picture of predicting vital signals from radio waves.

Original: Learning with Auxiliary Variables.
Revised: Learning with Auxiliary Variables.

Original: Improving machine learning model by leveraging auxiliary variables is a well-known idea.
Revised: Improving machine learning model by leveraging auxiliary variables is a well-known idea.

Original: Treating auxiliary variables as extra input features is a simple but effective method, and has been shown useful in computer vision (Forsyth & Ponce, 2002), health proﬁling (Wang et al., 2019), and trafﬁc prediction (Liao et al., 2018).
Revised: Treating auxiliary variables as extra input features is a simple but effective method, and has been shown useful in computer vision (Forsyth & Ponce, 2002), health proﬁling (Wang et al., 2019), and trafﬁc prediction (Liao et al., 2018).

Original: Auxiliary learning (Liu et al., 2019) provides an alternative approach for leveraging auxiliary variables by taking them as secondary supervisors.
Revised: Auxiliary learning (Liu et al., 2019) provides an alternative approach for leveraging auxiliary variables by taking them as secondary supervisors.

Original: It has been shown beneﬁcial in multiple domains including object detection (Mordan et al., 2018), sequence modelling (Collobert & Weston, 2008; Dong et al., 2015), reinforcement learning (Wilson et al., 2007), and visual odometry (Valada et al., 2018).
Revised: It has been shown beneﬁcial in multiple domains including object detection (Mordan et al., 2018), sequence modelling (Collobert & Weston, 2008; Dong et al., 2015), reinforcement learning (Wilson et al., 2007), and visual odometry (Valada et al., 2018).

Original: Our work explores a third way of using auxiliary variables.
Revised: Our work explores a third way of using auxiliary variables.

Original: We focus on partitioning the data space by those auxiliary variables based on analyzing how they affect the gradient of the primary task.
Revised: We focus on partitioning the data space by those auxiliary variables based on analyzing how they affect the gradient of the primary task

Original: We show that a wise partition facilitates model learning and leads to better results.
Revised: We show that a wise partition facilitates model learning and leads to better results

Original: 𝐸 !
Revised: 𝐸 !

Original: 𝑢 𝐹 !
Revised: 𝑢 𝐹 !

Original: 𝐹 " "𝑆 (c) Multi-Headed 𝐹 # … … 𝑥 𝑣 … $𝑦 $𝑦 !
Revised: 𝐹 " "𝑆 (c) Multi-Headed 𝐹 # … … 𝑥 𝑣 … $𝑦 $𝑦 !

Original: $𝑦 " (a) Backbone 𝐸 𝐹 𝑥 $𝑦 (b) Multi-Task 𝐸 𝐹 !
Revised: $𝑦 " (a) Backbone 𝐸 𝐹 𝑥 $𝑦 (b) Multi-Task 𝐸 𝐹 !

Original: 𝑢 𝑥 𝐹 # $𝑦 𝑣 gate "𝑆
Revised: 𝑢 𝑥 𝐹 # $𝑦 𝑣 gate "𝑆

Original: Switch Status
Revised: Switch Status

Original: Input
Revised: Input

Original: Output 𝐸 𝑣 𝐹 !
Revised: Output 𝐸 𝑣 𝐹 !

Original: 𝑢
Revised: 𝑢

Original: Accessible Variable
Revised: Accessible Variable

Original: Inaccessible Variable
Revised: Inaccessible Variable

Original: Predictor
Revised: Predictor

Original: Encoder 𝑥 $𝑦
Revised: Encoder 𝑥 $𝑦

Original:  M ETHOD 3. P ROBLEM F ORMULATION
Revised:  M ETHOD 3. P ROBLEM F ORMULATION

Original: Our task is to predict oxygen saturation based on respiration signals.
Revised: Our task is to predict oxygen saturation based on respiration signals 1 .

Original: Let the breathing signal be x ∈ R 1 × f b T and the corresponding oxygen saturation be y ∈ R 1 × f o T , we aim at learning a sequence-tosequence model that maps x to y  .
Revised: Let the breathing signal be x ∈ R 1 × f b T and the corresponding oxygen saturation be y ∈ R 1 × f o T , we aim at learning a sequence-to-sequence model that maps x to y , i.e. p ( y 1 , y 2 , · · · , y f o T | x 1 , x 2 , · · · , x f b T ) .

Original: Here, T denotes the length of the segments (in seconds), f b and f o are the sampling frequencies of the breathing and oxygen data.
Revised: Here, T denotes the length of the segments (in seconds), f b and f o are the sampling frequencies of the breathing and oxygen data.

Original: In practice, we have high sampling rates for both respiration and oxygen saturation, i.e., f b = 10 Hz and f o = 1 Hz .
Revised: In practice, we have high sampling rates for both respiration and oxygen saturation, i.e., f b = 10 Hz and f o = 1 Hz .

Original: This allows us to capture instantaneous changes in these signals.
Revised: This allows us to capture instantaneous changes in these signals.

Original: Moreover, we are particularly interested in monitoring oxygen saturation during sleep time.
Revised: Moreover, we are particularly interested in monitoring oxygen saturation during sleep time.

Original: This is because blood oxygen typically drops during sleep, and supplemental oxygen is medically necessary if oxygen level drops to 88% or lower for 5 consecutive minutes during sleep (Becattini et al., 2018).
Revised: This is because blood oxygen typically drops during sleep, and supplemental oxygen is medically necessary if oxygen level drops to 88% or lower for 5 consecutive minutes during sleep (Becattini et al., 2018).

Original: 3. B ACKBONE M ODEL
Revised: 3. B ACKBONE M ODEL

Original: As the relation between respiration x and oxygen y could be highly complicated, we use a deep neural network to map x to y .
Revised: As the relation between respiration x and oxygen y could be highly complicated, we use a deep neural network to map x to y .

Original: We start with the backbone model shown in Figure 1(a), which consists of an encoder E ( · ; θ e ) and a predictor F ( · ; θ f ) .
Revised: We start with the backbone model shown in Figure 1(a), which consists of an encoder E ( · ; θ e ) and a predictor F ( · ; θ f ) .

Original: The encoder is composed of a fully convolution network (FCN) followed by a bidirectional transformers (BERT) module (Devlin et al., 2018).
Revised: The encoder is composed of a fully convolution network (FCN) followed by a bidirectional transformers (BERT) module (Devlin et al., 2018).

Original: The FCN extracts local features from the raw respiration signal, then the BERT module captures long-term dependencies based on those features.
Revised: The FCN extracts local features from the raw respiration signal, then the BERT module captures long-term dependencies based on those features.

Original: This design has two beneﬁts.
Revised: This design has two beneﬁts.

Original: First, it can extract both local and global information from the respiration signal.
Revised: First, it can extract both local and global information from the respiration signal.

Original: Second, it works for variable length respiration signals.
Revised: Second, it works for variable length respiration signals.

Original: The predictor F is composed of several De-convolution layers, which up-samples the extracted features to the same time resolution of oxygen saturation.
Revised: The predictor F is composed of several De-convolution layers, which up-samples the extracted features to the same time resolution of oxygen saturation.

Original: Formally, we have E : R 1 × f b T → R n × αf b T and F : R n × αf b T → R 1 × f o T where n is the size of the breathing feature and α is the down-sampling factor ( 1 / 240 in our model).
Revised: Formally, we have E : R 1 × f b T → R n × αf b T and F : R n × αf b T → R 1 × f o T where n is the size of the breathing feature and α is the down-sampling factor ( 1 / 240 in our model).

Original: The model is trained with a Lloss, i.e., L o ( F, E ) = 1 T (cid:107) ˆ y − y (cid:107) 1 , where ˆ y = F ( E ( x )) is the model prediction.
Revised: The model is trained with a Lloss, i.e., L o ( F, E ) = 1 T (cid:107) ˆ y − y (cid:107) 1 , where ˆ y = F ( E ( x )) is the model prediction.

Original: Thanks to the elaborately designed model architecture, even with a simple L 1 loss, this model has achieved 1 . 66% error (see Table. 1), which is a reasonably good result.
Revised: Thanks to the elaborately designed model architecture, even with a simple L 1 loss, this model has achieved 1 . 66% error (see Table. 1), which is a reasonably good result.

Original: More details on this model are available in appendix A.
Revised: More details on this model are available in Appendix A.

Original: 3. L EVERAGING A UXILIARY V ARIABLES & THE M ULTI - HEADED M ODEL
Revised: 3. L EVERAGING A UXILIARY V ARIABLES & THE M ULTI - HEADED M ODEL

Original: In the medical domain, typically, there exist many physiological indices that could potentially help improve the model.
Revised: In the medical domain, typically, there exist many physiological indices that could potentially help improve the model.

Original: From the viewpoint of machine learning, physiological indices can be viewed as auxiliary variables that could be beneﬁcial to train the model.
Revised: From the viewpoint of machine learning, physiological indices can be viewed as auxiliary variables that could be beneﬁcial to train the model.

Original: During testing, auxiliary variables can be categorized into two types: accessible and inaccessible.
Revised: During testing, auxiliary variables can be categorized into two types: accessible and inaccessible.

Original: We denote accessible variables as v and inaccessible variables as u .
Revised: We denote accessible variables as v and inaccessible variables as u .

Original: In the learning procedure, a common practice is to use accessible variable as extra inputs and use inaccessible variables for auxiliary tasks.
Revised: In the learning procedure, a common practice is to use accessible variable as extra inputs and use inaccessible variables for auxiliary tasks.

Original: Figure 1(b) illustrates such framework where F u denotes a predictor of the auxiliary task u .
Revised: Figure 1(b) illustrates such framework where F u denotes a predictor of the auxiliary task u .

Original: We call the design in Figure 1(b) the Multi-Task model.
Revised: We call the design in Figure 1(b) the Multi-Task model.

Original: Despite its intuitive appeal, in practice, this model may not always lead to improvements.
Revised: Despite its intuitive appeal, in practice, this model may not always lead to improvements.

Original: As shown in Table 1, the Multi-Task model even causes a performance drop on the
Revised: As shown in Table 1, the Multi-Task model even causes a performance drop on the

Original: SHHS dataset 1 .
Revised: SHHS dataset 2 .

Original: In our work, we investigate an alternative approach that uses auxiliary variables as switches.
Revised: In our work, we investigate an alternative approach that uses auxiliary variables as switches.

Original: The idea is based on the hypothesis that some variable h affects the targeted input-output relationship in avery non-smooth way.
Revised: The idea is based on the hypothesis that some variable h affects the target input-output relationship in a non-smooth way.

Original: For example, assuming h is a binary variable.
Revised: For example, assuming h is a binary variable.

Original: The mapping from input x to output y conditioned on variable h equal to 0 is signiﬁcantly different from the mapping when the variablev is 1 , i.e., y ( x | h
Revised: The mapping from input x to output y conditioned on variable h equal to 0 is signiﬁcantly different from the mapping when the variable is 1 , i.e., y ( x | h

Original: = 0) and y ( x | h = 1) are two distinct functions.
Revised: = 0) and y ( x | h = 1) are two distinct functions.

Original: In this case, instead of learning function y ( x, h ) , we propose to use variable h as a switch to partition the data space and learn the functions y
Revised: In this case, instead of learning function y ( x, h ) , we propose to use variable h as a switch to partition the data space and learn the functions y

Original: ( x | h = 0) and y ( x | h
Revised: ( x | h = 0) and y ( x | h

Original: = 1) separately.
Revised: = 1) separately.

Original: We realize this idea in our Multi-Headed model illustrated in Figure 1(c).
Revised: We realize this idea in our Multi-Headed model illustrated in Figure 1(c).

Original: We ﬁrst deﬁne a switch function S ( v, u ) :
Revised: We ﬁrst deﬁne a switch function S ( v, u ) :

Original: V × U →
Revised: V × U →

Original: [ N ] where v ∈ V and u ∈ U are accessible/inaccessible variables and N is the number of switch status.
Revised: [ N ] where v ∈ V and u ∈ U are accessible/inaccessible variables and N is the number of switch status.

Original: The model has N heads { F i } Ni =1 to make the prediction ˆ y i
Revised: The model has N heads { F i } Ni =1 to make the prediction ˆ y i

Original: = F i ( E ( x )) for every switch status separately.
Revised: = F i ( E ( x )) for every switch status separately.

Original: It also has a predictor F u to infer u .
Revised: It also has a predictor F u to infer u .

Original: During testing time, based on the accessible auxiliaries v and estimated inaccessible auxiliaries ˆ u , we get theestimated switch status ˆ S = S ( v, ˆ u ) . Then we output the ﬁnal prediction ˆ y
Revised: During testing time, based on the accessible auxiliaries v and estimated inaccessible auxiliaries ˆ u , we evaluate the switch status ˆ S = S ( v, ˆ u ) , and output the ﬁnal prediction ˆ y

Original: = ˆ y ˆ S .
Revised: = ˆ y ˆ S .

Original: In the case of oxygen prediction, ˆ y i and the switch ˆ S are time series, we use ˆ y ti and ˆ S t to denote their value at time step t .
Revised: In the case of oxygen prediction, ˆ y i and the switch ˆ S are time series, we use ˆ y ti and ˆ S t to denote their value at time step t .

Original: As shown in Figure 1(c), the ﬁnal prediction is the gated combination of every head’s output, ˆ y t
Revised: As shown in Figure 1(c), the ﬁnal prediction is the gated combination of every head’s output, ˆ y t

Original: = ˆ y t ˆ S t .
Revised: = ˆ y t ˆ S t .

Original: One might argue that the Multi-Task model can also learn sucha compositional function via ﬁrst learning  the function in different cases separately, then learning to throw the data to the right function based on the  auxiliary variables.
Revised: One might argue that the Multi-Task model can also learn such compositional function by implicitly learning each piece of the target function, and learning to assign the input to the target piece based on the value of the auxiliary variable.

Original: However, in practice, such incorrect model inductive bias introduces unnecessary difﬁculty in the learning procedure and typically leads to bad performance.
Revised: However, in practice, such incorrect model inductive bias introduces unnecessary difﬁculty in the learning procedure and typically leads to bad performance.

Original: In our experiments, we show, with a more preferable inductive bias, the Multi-Headed model outperforms the Multi-Task model with a clear gap. 
Revised: In our experiments, we show, with a more preferable inductive bias, the Multi-Headed model outperforms the Multi-Task model with a clear gap. 3. I DENTIFYING G OOD S WITCHES VIA G RADIENT D IAGNOSIS

Original: In the above description, we assume a good switch function exists.
Revised: In the above description, we assume a good switch function exists.

Original: In this paragraph, we answer the critical question: which auxiliary variables should be used as switches?
Revised: In this paragraph, we answer the critical question: which auxiliary variables should be used as switches?

Original: We introduce, gradient diagnosis, a simple but effective way to test the inﬂuence of variable v on the task of learning function y ( x ) .
Revised: We introduce, gradient diagnosis, a simple but effective way to test the inﬂuence of variable v on the task of learning function y ( x ) .

Original: We ﬁrst train a vanilla model with no auxiliary variables, e.g., our backbone model.
Revised: We ﬁrst train a vanilla model with no auxiliary variables, e.g., our backbone model.

Original: We then use the validation set to check the predictor F ( · ; θ f ) ’s loss gradient ∇ θ f L .
Revised: We then use the validation set to check the predictor F ( · ; θ f ) ’s loss gradient ∇ θ f L .

Original: For each possible value a ∈ V of variable v , we compute the gradients averaged on data samples when v equals a , i.e., ∇ θ f L| v = a = E ( x,y ) ∼ p ( x,y | v = a ) ∇ θ f L ( F ( E ( x )) , y ) .
Revised: For each possible value a ∈ V of variable v , we compute the gradients averaged on data samples when v equals a , i.e., ∇ θ f L| v = a = E ( x,y ) ∼ p ( x,y | v = a ) ∇ θ f L ( F ( E ( x )) , y ) .

Original: Such gradient indicates the direction to improve the model when v = a .
Revised: Such gradient indicates the direction to improve the model when v = a .

Original: Intuitively, if gradients ∇ θ f L| v = a are similar for all values a , it means variable v does not change the loss function drastically.
Revised: Intuitively, if gradients ∇ θ f L| v = a are similar for all values a , it means variable v does not change the loss function drastically.

Original: So it is considerable to use one model to handle all values of v .
Revised: So it is reasonable to use one model to handle all values of v .

Original: However, if those gradients are vastly different from each other, forcing one model to work for all values of v may cause the learning to fail.

Original: Taking our oxygen prediction as an example, we conduct such gradient diagnosis on multiple variables including (a) Sleep stages: awake, light sleep (N1,N2), deep sleep (N3), rapid eye movement (REM); (b) Gender: male vs. female; (c) Asthma: having asthma or not; (d): Smoking: nonsmoker, current smoker, former smoker e

Original: Education: less

Original: 15 years vs. greater than 15 years

Original: : Height: lower than 160cm or higher than 160cm

Original: (g) Aspirin: taking aspirin or not.

Original: Figurevisualizes the cosine similarity of the gradients computed on the medical dataset SHHS.

Original: We clearly see that the gradient directions are altered by the sleep stage, gender, asthma and smoking while less affected by the height, aspirin and education level.

Original: These result is quite interesting.

Original: For example, although it is hard to see why gender affects the oxygen-breathing function from the ﬁrst glance, there are medical evidence that show gender matters.

Original: Due to differences between women’s and men’s respiratory system, women consume more oxygen when breathing compared to men (Ramonatxo, 2003), especially during exercise (Reybrouck & Fagard, 1999), and men’s oxygen saturation is naturally higher than women (Ricart et al., 2008).

Original: As for sleep stages, it is a known fact that it inﬂuences the oxygen saturation.

Original: For example, pointed out by Choi et al.

Original: (2016), that oxygen saturation differs signiﬁcantly during REM and Non-REM sleep.

Original: It is also understandable that asthma affects oxygenation since asthma causes difﬁculty in breathing.

Original: It is also supported by medical ﬁndings that smoking reduces oxygen saturation ( ¨Ozdal et al., 2017).

Original: Since it is a long-term effect, no matter if a

Original: [Figure]

Original: smoker quits or not, his/her oxygen pattern is already altered. This may explains why the gradient similarity is high between current smoker and former smoker. Based on our gradient diagnosis, we pick sleep stage and gender as auxiliary variables for our task. Other variables are possible.
Revised: On the other hand, if the gradients are vastly different from each other, it is preferable to separate the predictor for different values of v .

Original: However, we pick these two variables because gender is almost always accessible during testing; and while sleep stages are usually not available during testing, luckily past work shows that sleep stages can be inferred from respiration signals (Zhao et al., 2017).

Original: Hence, our model can estimate the sleep stages and use the estimates to do the switching.

Original: Further, from

Original: Figure 2(a), we can see the gradients for awake, REM and sleeping (N1,N2,N3) are very dissimilar from each other while the gradients are much more similar within the group of N1,N2,N3.

Original: So we design the switch function to consider N1,N2,N3 as one sleep stage.

Original: In total, we have six switch status due to the combinations of gender and sleep stages.

Original: Our model has six heads for each switch status.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We present the results of empirical evaluation on medical datasets and RF data.
Revised: We present the results of empirical evaluation on medical datasets and RF data.

Original: We evaluate the models using the average absolute error between the predicted oxygen saturation ˆ y and the groundtruth y .
Revised: We evaluate the models using the average absolute error between the predicted oxygen saturation ˆ y and the groundtruth y .

Original: The error is ﬁrst averaged for every night for each subject, i.e., 1 T (cid:80)
Revised: The error is ﬁrst averaged for every night and each subject, i.e., 1 T (cid:80)

Original: Tt =1 (cid:107) y t − ˆ y t (cid:107) .
Revised: Tt =1 (cid:107) y t − ˆ y t (cid:107) .

Original: We then report the mean and standard deviation of per-night averaged errors of the dataset.
Revised: We then report the mean and standard deviation of per-night averaged errors of the dataset.

Original: 4. D ATASETS
Revised: 4. D ATASETS

Original: Medical Datasets.
Revised: Medical Datasets.

Original: We leverage three publicly-available medical datasets: Sleep Heart Health Study ( SHHS ) (Zhang et al.), Multi-Ethnic Study of Atherosclerosis ( MESA ) (Chen et al., 2015), and
Revised: We leverage three publicly-available medical datasets: Sleep Heart Health Study ( SHHS ) (Zhang et al.), Multi-Ethnic Study of Atherosclerosis ( MESA ) (Chen et al., 2015), and

Original: Osteoporotic Fractures in Men Study ( MrOS ) (Blackwell et al., 2011).
Revised: Osteoporotic Fractures in Men Study ( MrOS ) (Blackwell et al., 2011).

Original: Each dataset contains the polysomnography (PSG) signals of subjects.
Revised: Each dataset contains the polysomnography (PSG) signals of subjects.

Original: The ﬁrst two datasets have males and females, whereas the last one has male subjects.
Revised: The ﬁrst two datasets have males and females, whereas the last one has male subjects.

Original: The PSG data includes full-night respiration signals and the corresponding oxygen saturation (SpO2) and sleep stage labels.
Revised: The PSG data includes full-night respiration signals and the corresponding oxygen saturation (SpO2) and sleep stage labels.

Original: The data has three types of respiration channel measured with: a nasal probe, a breathing belt on the chest, and a breathing belt on the abdomen.
Revised: The data has three types of respiration channel measured with: a nasal probe, a breathing belt on the chest, and a breathing belt on the abdomen.

Original: We denote it as BB-respiration signals .
Revised: We denote it as BB-respiration signals .

Original: The datasets also contain subjects’ bio-information such as gender, age, etc.
Revised: The datasets also contain subjects’ bio-information such as gender, age, etc.

Original: SHHS, MESA, and MrOS contains 2651, 2056, and 1026 subjects, respectively.
Revised: SHHS, MESA, and MrOS contains 2651, 2056, and 1026 subjects, respectively.

Original: We randomly split the training and testing sets for SHHS, MESA, and MrOS as 1855/796, 1439/617, and 718/308, and keep the same splits in all experiments.
Revised: We randomly split the training and testing sets for SHHS, MESA, and MrOS as 1855/796, 1439/617, and 718/308, and keep the same splits in all experiments.

Original: We note that the subjects in these studies have an age range between 40 and 95, and suffer from a variety of diseases as indicated by the studies names.
Revised: We note that the subjects in these studies have an age range between 40 and 95, and suffer from a variety of diseases as indicated by the studies names.

Original: This allows for a wider range of oxygen variability beyond the typical range of healthy individuals.
Revised: This allows for a wider range of oxygen variability beyond the typical range of healthy individuals.

Original: RF Dataset.
Revised: RF Dataset.

Original: We collect a small PSG dataset that contains the standard PSG channels mentioned above as well as an RF-based inference of the respiration signal, i.e., RF-respiration signals .
Revised: We collect a small PSG dataset that contains the standard PSG channels mentioned above as well as an RF-based inference of the respiration signal, i.e., RF-respiration signals .

Original: The dataset is collected in the sleep lab in the university and contains 31 nights of data from 14 individuals.
Revised: The dataset is collected in the sleep lab in the university and contains 31 nights of data from 14 individuals.

Original: All subjects in this study are healthy.
Revised: All subjects in this study are healthy.

Original: Each subject is monitored using the standard PSG[Table] channels as in the medical datasets above (SpO2, respiration belts, sleep stages, etc.).
Revised: Each subject is monitored using the standard PSG channels as in the medical datasets above (SpO2, respiration belts, sleep stages, etc.).

Original: Additionally, a radio device is installed in the room to collect RF signals, which are then processed to extract the subject’s respiration in accordance with Adib et al.
Revised: Additionally, a radio device is installed in the room to collect RF signals, which are then processed to extract the subject’s respiration in accordance with Adib et al.

Original: Therefore, in this dataset, each subject has both the BB-respiration signals and the RF-respiration signals.
Revised: Therefore, in this dataset, each subject has both the BB-respiration signals and the RF-respiration signals.

Original: 4. M ODELS
Revised: 4. M ODELS

Original: As we describe in Section 3, our models focus on predicting oxygen from breathing while leveraging auxiliary gender and sleep stage information.
Revised: As we describe in Section 3, our models focus on predicting oxygen from breathing while leveraging auxiliary gender and sleep stage information.

Original: We evaluate the following models: (a) Single-Headed : the vanilla backbone model.
Revised: We evaluate the following models: (a) Single-Headed : the vanilla backbone model.

Original: (b) Multi-Task : the model that takes gender as an extra input and is trained with sleep stages as an auxiliary task.
Revised: (b) Multi-Task : the model that takes gender as an extra input and is trained with sleep stages as an auxiliary task.

Original: (c) Multi-Headed : the model that uses multiple heads to predict oxygen level for different switch status.
Revised: (c) Multi-Headed : the model that uses multiple heads to predict oxygen level for different switch status.

Original: (d) We also compare with an adaptation of the
Revised: (d) We also compare with an adaptation of the

Original: Projecting Conﬂicting Gradient PCGrad-Adapted .
Revised: Projecting Conﬂicting Gradient PCGrad-Adapted .

Original: The original PCGrad was proposed for improved multi-task learning to deal with scenarios where the various tasks have divergent gradients (Yu et al., 2020).
Revised: The original PCGrad was proposed for improved multi-task learning to deal with scenarios where the various tasks have divergent gradients (Yu et al., 2020).

Original: In that case, PCGrad projects the divergent gradients along consistent directions.
Revised: In that case, PCGrad projects the divergent gradients along consistent directions.

Original: In our context, the gradients of the tasks themselves – i.e., oxygen prediction and sleep stage prediction – do not diverge.
Revised: In our context, the gradients of the tasks themselves – i.e., oxygen prediction and sleep stage prediction – do not diverge.

Original: Hence, simply applying PCGrad to our problem should be no different from the
Revised: Hence, simply applying PCGrad to our problem should be no different from the

Original: Multi-Task model.
Revised: Multi-Task model.

Original: However, we can adapt PCGrad to be more suitable for our problem.
Revised: However, we can adapt PCGrad to be more suitable for our problem.

Original: In our case, the gradients of subgroups (i.e., subjects with different status) in the input data with respect to the main task can be divergent or signiﬁcantly different.
Revised: In our case, the gradients of subgroups (i.e., subjects with different status) in the input data with respect to the main task can be divergent or signiﬁcantly different.

Original: We can think of learning a mapping from breathing to oxygen for different subject status as different sub-tasks.
Revised: We can think of learning a mapping from breathing to oxygen for different subject status as different sub-tasks.

Original: Then, we can augment our backbone model with PCGrad applied to these sub-tasks.
Revised: Then, we can augment our backbone model with PCGrad applied to these sub-tasks.

Original: Speciﬁcally, to train the single predictor F ,
Revised: Speciﬁcally, to train the single predictor F ,

Original: We compute 6 different losses based on the six subject status corresponding to our 6-headed model.
Revised: We compute 6 different losses based on the six subject status corresponding to our 6-headed model.

Original: For each subject status i , we have an oxygen prediction loss L io for that status.
Revised: For each subject status i , we have an oxygen prediction loss L io for that status.

Original: Mathematically,
Revised: Mathematically,

Original: L io ( F, E ) =
Revised: L io ( F, E ) =

Original: ( (cid:80) t : S t =
Revised: ( (cid:80) t : S t =

Original: i 1) − 1 (cid:80) t : S t = i | F ( E ( x b ))
Revised: i 1) − 1 (cid:80) t : S t = i | F ( E ( x b ))

Original: t − y t | .
Revised: t − y t | .

Original: We compute the predictor’s gradients ∇ θ f L io for each loss and update the predictor by the gradient calibrated by PCGrad. 
Revised: We compute the predictor’s gradients ∇ θ f L io for each loss and update the predictor by the gradient calibrated by PCGrad. 4. G RADIENT D IAGNOSIS R ESULTS

Revised: Before evaluating the various models, we show that the gradient diagnosis method proposed in section 3.3 ﬁnds good “switches” for the oxygen prediction task.

Revised: Among the huge set of physiological indices in the medical datasets, we pick the following variables as candidates: (a) Sleep stages: awake, light sleep (N1,N2), deep sleep (N3), rapid eye movement (REM); (b) Gender: male vs. female; (c) Asthma: having asthma or not; (d): Smoking: non-smoker, current smoker, former smoker; (e) Education: less than 15 years vs. greater than 15 years; (f): Height: lower than 160cm or higher than 160cm; (g) Aspirin: taking aspirin or not.

Revised: Figure 2 visualizes the cosine similarity of the gradients computed on the medical dataset SHHS.

Revised: We clearly see that the gradient directions are altered by the sleep stage, gender, asthma and smoking while less affected by the

Original: 4. E VALUATION ON M EDICAL D ATASETS
Revised: [Figure] height, aspirin and education level. These result is quite interesting. For example, although it is hard to see why gender affects the oxygen-breathing function from the ﬁrst glance, there are medical evidence that show gender matters.

Revised: Due to differences between women’s and men’s respiratory system, women consume more oxygen when breathing compared to men (Ramonatxo, 2003), especially during exercise (Reybrouck & Fagard, 1999), and men’s oxygen saturation is naturally higher than women (Ricart et al., 2008).

Revised: As for sleep stages, it is a known fact that it inﬂuences the oxygen saturation.

Revised: For example, pointed out by Choi et al.

Revised: (2016), that oxygen saturation differs signiﬁcantly during REM and Non-REM sleep.

Revised: It is also understandable that asthma affects oxygenation since asthma causes difﬁculty in breathing.

Revised: It is also supported by medical ﬁndings that smoking reduces oxygen saturation ( ¨ Ozdal et al., 2017).

Revised: Since it is a long-term effect, no matter if a smoker quits or not, his/her oxygen pattern is already altered.

Revised: This may explains why the gradient similarity is high between current smoker and former smoker.

Revised: Based on our gradient diagnosis, we pick sleep stage and gender as auxiliary variables for our task.

Revised: Other variables are possible.

Revised: However, we pick these two variables because gender is almost always accessible during testing; and while sleep stages are usually not available during testing, luckily past work shows that sleep stages can be inferred from respiration signals (Zhao et al., 2017).

Revised: Hence, our model can estimate the sleep stages and use the estimates to do the switching.

Revised: Further, from

Revised: Figure 2(a), we can see the gradients for awake, REM and sleeping (N1,N2,N3) are very dissimilar from each other while the gradients are much more similar within the group of N1,N2,N3.

Revised: So we design the switch function to consider N1,N2,N3 as one sleep stage.

Revised: In total, we have six switch status due to the combinations of gender and sleep stages.

Revised: Our model has six heads for each switch status.

Revised: 4. E VALUATION ON M EDICAL D ATASETS

Original: Quantitative Results.
Revised: Quantitative Results.

Original: Table 1 reports the absolute error of all models for the three medical datasets.
Revised: Table 1 reports the absolute error of all models for the three medical datasets.

Original: The table reveals two main results.
Revised: The table reveals two main results.

Original: First, all baselines perform reasonably well and have average prediction errors between 1.58% and 1.7%.
Revised: First, all baselines perform reasonably well and have average prediction errors between 1.58% and 1.7%.

Original: These errors are relatively low and comparable to the errors of consumer pulse oximeters whose average error is in the range 0.4-3.5% (Lipnick et al., 2016).
Revised: These errors are relatively low and comparable to the errors of consumer pulse oximeters whose average error is in the range 0.4-3.5% (Lipnick et al., 2016).

Original: It is important to keep in mind that all baselines are based on our backbone model, which already embodies our design for learning this new task.
Revised: It is important to keep in mind that all baselines are based on our backbone model, which already embodies our design for learning this new task.

Original: Thus, these results show for the ﬁrst time that respiration signals have a predictive power for learning oxygen saturation.
Revised: Thus, these results show for the ﬁrst time that respiration signals have a predictive power for learning oxygen saturation.

Original: Second, Table 1 also shows that the Multi-Headed model outperforms the other baselines on all datasets.
Revised: Second, Table 1 also shows that the Multi-Headed model outperforms the other baselines on all datasets.

Original: This demonstrates that the Multi-Headed design is more effective at leveraging relevant auxiliary variables than the other methods.
Revised: This demonstrates that the Multi-Headed design is more effective at leveraging relevant auxiliary variables than the other methods.

Original: Qualitative Results.
Revised: Qualitative Results.

Original: Figure 3 visualizes the predicted oxygen saturation of the Multi-Headed model and the Multi-Task model on a male subject in the SHHS dataset.
Revised: Figure 3 visualizes the predicted oxygen saturation of the Multi-Headed model and the Multi-Task model on a male subject in the SHHS dataset.

Original: As the ground-truth oxygen saturation are integers, we rounded the predicted oxygen saturation into integers.
Revised: As the ground-truth oxygen saturation are integers, we rounded the predicted oxygen saturation into integers.

Original: The background color indicates different sleep stages.
Revised: The background color indicates different sleep stages.

Original: The ‘light grey’ and ‘white’ corresponds to ‘REM’ and ‘Awake’, respectively.
Revised: The ‘light grey’ and ‘white’ corresponds to ‘REM’ and ‘Awake’, respectively.

Original: For clarity, we use one color ‘dark grey’ for all three non-REM ‘Sleep’ stages, N1, N2, and N3.
Revised: For clarity, we use one color ‘dark grey’ for all three non-REM ‘Sleep’ stages, N1, N2, and N3.

Original: We observe that the Multi-Headed model consistently performs better than the Multi-Task model over the whole night.
Revised: We observe that the Multi-Headed model consistently performs better than the Multi-Task model over the whole night.

Original: In non-REM ‘Sleep’, the ground-truth oxygen saturation is more stable.
Revised: In non-REM ‘Sleep’, the ground-truth oxygen saturation is more stable.

Original: The Multi-Headed model predicts more accurate oxygen than the Multi-Task model.

Original: In the ‘REM’ stage, people usually have lower oxygen saturation with larger variance.

Original: The Multi-Headed model

Original: [Figure]

Original: [Figure]

Original: predicts the ups and downs and captures the changes in blood oxygen.

Original: In the ‘Awake’ stage, the

Original: Multi-Headed model predicts the more dramatic changes, while the Multi-Task model has a larger bias.

Original: This experiment demonstrates that the way we incorporate the sleep stages into the model improves the performance across different sleep stages

Original: [Figure]
Revised: [Table]

Original: Error Analysis and Relation to Skin Color.
Revised: Error Analysis and Relation to Skin Color.

Original: One important issue to keep in mind is that our model learns from ground-truth data measured using pulse oximeters (today oximetry is the only way for continuous noninvasive monitoring of blood oxygen).
Revised: One important issue to keep in mind is that our model learns from ground-truth data measured using pulse oximeters (today oximetry is the only way for continuous noninvasive monitoring of blood oxygen).

Original: Hence, to understand the prediction errors and their potential causes, we plot in Figure 4 the distributions of the ground-truth SpO and the predicted SpO2 by the Multi-Headed model for different races, for all datasets.
Revised: Hence, to understand the prediction errors and their potential causes, we plot in Figure 4 the distributions of the ground-truth SpO and the predicted SpO2 by the Multi-Headed model for different races, for all datasets.

Original: Interestingly, the ground-truth SpO2 from oximetry shows a clear discrepancy between black and white subjects.
Revised: Interestingly, the ground-truth SpO2 from oximetry shows a clear discrepancy between black and white subjects.

Original: In particular, the ground-truth oxygen distributions show that black subjects have higher blood oxygen.
Revised: In particular, the ground-truth oxygen distributions show that black subjects have higher blood oxygen.

Original: This is compatible with past ﬁndings that pulse oximeter overestimates blood oxygen in dark-skinned subjects (Feiner et al., 2007).
Revised: This is compatible with past ﬁndings that pulse oximeter overestimates blood oxygen in dark-skinned subjects (Feiner et al., 2007).

Original: In contrast, the model prediction shows much more similar oxygen distributions for the two races.
Revised: In contrast, the model prediction shows much more similar oxygen distributions for the two races.

Original: It is interesting that while the model learned from what seems to be biased ground-truth, the results show that the model was able to correct, to some extent, for this bias.
Revised: It is interesting that while the model learned from what seems to be biased ground-truth, the results show that the model is able to correct, to some extent, for this bias.

Original: Admittedly, these observations are inductive since there is no way to measure the exact errors in the ground-truth labels.
Revised: Admittedly, these observations are inductive since there is no way to measure the exact errors in the ground-truth labels.

Original: The distributions in Figure 4 also show that the model tends to miss the very high and very low oxygen values.
Revised: The distributions in Figure 4 also show that the model tends to miss the very high and very low oxygen values.

Original: This is expected given that only a very small percentage of the ground-truth data falls in the tails.
Revised: This is expected given that only a very small percentage of the ground-truth data falls in the tails.

Original: One would expect that the performance on very high and very low oxygen values would improve if there is sufﬁcient training data in those ranges.
Revised: One would expect that the performance on very high and very low oxygen values would improve if there is sufﬁcient training data in those ranges.

Original: 4. E VALUATION ON RF D ATASET
Revised: 4. E VALUATION ON RF D ATASET

Original: In this section, we evaluate whether our model, which is trained on respiration measured using a breathing belt, can generalize to predicting oxygen from respiration signals measured via radio waves.
Revised: In this section, we evaluate whether our model, which is trained on respiration measured using a breathing belt, can generalize to predicting oxygen from respiration signals measured via radio waves.

Original: We directly apply a Multi-Headed model trained on the BB-respiration signals in medical 
Revised: We directly apply a Multi-Headed model trained on the BB-respiration signals in medical 86 88 90 92 94 96 98 100 SpO D e n s i t y BlackWhite 86 88 90 92 94 96 98 100 SpO

Original: P r e d ic t  e d P e rN ig h t 
Revised: D e n s i t y BlackWhite 86 88 90 92 94 96 98 100 SpO D e n s i t y

Revised: BlackWhite 86 88 90 92 94 96 98 100 SpO

Revised: D e n s i t y

Revised: BlackWhite 86 88 90 92 94 96 98 100 SpO

Revised: D e n s i t y

Revised: BlackWhite 86 88 90 92 94 96 98 100 SpO

Revised: D e n s i t y

Revised: BlackWhite

Revised: [Figure]

Revised: P r e d i c t e d P e rN i g h t

Original: A v e r a g e O x y g e n ( % )
Revised: A v e r a g e O x y g e n ( % )

Original: [Table]
Revised: [Figure]

Original: datasets, to the RF dataset.
Revised: datasets, to the RF dataset.

Original: For comparison, we test the model on both RF-respiration signals and
Revised: For comparison, we test the model on both RF-respiration signals and

Original: BB-respiration signals from the RF dataset.
Revised: BB-respiration signals from the RF dataset.

Original: The results are shown in Table 2.
Revised: The results are shown in Table 2.

Original: We observe that the model works well on RF dataset with a small prediction error, no matter which breathing signal is used.
Revised: We observe that the model works well on RF dataset with a small prediction error, no matter which breathing signal is used.

Original: The prediction error of using the RF-respiration signals ( 1 . 28% ) is even slightly lower than using the BB-respiration signals ( 1 . 30% ).
Revised: The prediction error of using the RF-respiration signals ( 1 . 28% ) is even slightly lower than using the BB-respiration signals ( 1 . 30% ).

Original: This is because the noise in the respiration measured using RF signal is different from the noise in the breathing belt measurements.
Revised: This is because the noise in the respiration measured using RF signal is different from the noise in the breathing belt measurements.

Original: The noise in the
Revised: The noise in the

Original: RF-based measurements tend to increase with motion and hence we see that the model prediction error is higher for the aware stage.
Revised: RF-based measurements tend to increase with motion and hence we see that the model prediction error is higher for the aware stage.

Original: In contrast, the error in the breathing belt depends on the sleep posture and the positioning of the belt on the body.
Revised: In contrast, the error in the breathing belt depends on the sleep posture and the positioning of the belt on the body.

Original: Also, comparing with the error from previous section, we ﬁnd the model is performing better on the RF dataset than on the medical datasets, i.e., 1 . 30% vs. 1 . 62% .
Revised: Also, comparing with the error from previous section, we ﬁnd the model is performing better on the RF dataset than on the medical datasets, i.e., 1 . 30% vs. 1 . 62% .

Original: We believe the reason is that the RF dataset is collected with healthy individuals and hence has less complexity and is easier to predict.
Revised: We believe the reason is that the RF dataset is collected with healthy individuals and hence has less complexity and is easier to predict.

Original: These results show the ﬁrst demonstration of predicting oxygen saturation from radio signals without any body contact. 
Revised: These results show the ﬁrst demonstration of predicting oxygen saturation from radio signals without any body contact. [Table]

Original: 4. A PPLICATION TO M ONITORING COVID-19 P ATIENTS
Revised: 4. A PPLICATION TO M ONITORING COVID-19 P ATIENTS

Original: Finally, we show a case study of a COVID-19 patient.
Revised: Finally, we show a case study of a COVID-19 patient.

Original: An 88-year old male was monitored from
Revised: An 88-year old male was monitored from

Original: April 28 till July 31,  with a contactless radio device.
Revised: April 28 till July 31, in home, with a contactless radio device.

Original: The subject tested COVID-19 positive prior to the monitoring, and was declared COVID-19 negative on May 5.
Revised: The subject tested COVID-19 positive prior to the monitoring, and was declared COVID-19 negative on May 5.

Original: The objective of the case study is to compare the results of a completely passive contactless monitoring approach to information collected from the medical doctor and the professional caregiver attending to the subject.
Revised: The objective of the case study is to compare the results of a completely passive contactless monitoring approach to information collected from the medical doctor and the professional caregiver attending to the subject.

Original: Figure 5 shows the average oxygen saturation predicted by the Multi-Headed model for each of the monitored nights.
Revised: Figure 5 shows the average oxygen saturation predicted by the Multi-Headed model for each of the monitored nights.

Original: May 5 is highlighted with a star as the day on which the patient was declared
Revised: May 5 is highlighted with a star as the day on which the patient was declared

Original: COVID-19 negative.
Revised: COVID-19 negative.

Original: The results in the ﬁgure show a clear trend of increased oxygen level as the patient transitioned to becoming COVID-19 negative.
Revised: The results in the ﬁgure show a clear trend of increased oxygen level as the patient transitioned to becoming COVID-19 negative.

Original: They also show that improvement in blood oxygen did not stop on the day the patient became COVID-free; it rather continued to ramp up for almost two weeks after that date, and eventually reached a steady level.
Revised: They also show that improvement in blood oxygen did not stop on the day the patient became COVID-free; it rather continued to ramp up for almost two weeks after that date, and eventually reached a steady level.

Original: While there is no groundtruth measurements of the oxygen for this completely passive case study, the results are consistent with the doctor notes about patient recovery and general expectations for a symptomatic COVIDpatient who recovered without escalation of his COVID-19 symptoms.  
Revised: While there is no ground- truth measurements of the oxygen for this completely passive case study, the results are consistent with the doctor notes about patient recovery and general expectations for a symptomatic COVIDpatient who recovered without escalation of his COVID-19 symptoms.  D ISCUSSION AND C ONCLUDING R EMARKS This paper introduces a model for predicting oxygen saturation from breathing signals, and shows that it performs well on three medical datasets and an RF-based dataset.

Original: C ONCLUSION
Revised: In this section, we discuss some of the clinical implications, properties, and limitations of such model. Data Diversity. While a comprehensive clinical evaluation is beyond the scope of this paper, we note that medical datasets used in our evaluation include many patients with diverse diseases and health conditions. In particular, the Multi-Ethnic Study of Atherosclerosis (MESA) dataset focuses on patients with Atherosclerosis, a disease in which cholesterol plaques build up in the walls of arteries, and the minimum oxygen saturation predicts future plaque score (Gunnarsson et al., 2015).

Original: We demonstrate the feasibility of predicting blood oxygen saturation from respiration signals.
Revised: As shown in the paper, our model has low error and good performance on this dataset. In fact, all three evaluation datasets (SHHS, MrOS and MESA) contain people who have various diseases including chronic bronchitis (323 subjects), cardiovascular diseases (1196 subjects), coronary heart disease (869 subjects), myocardial infarctions (508 subjects), diabetes (307 subjects), Alzheimer’s (40 subjects), Parkinson’s (69 subjects), and others. This means that our reported accuracy already takes into account unhealthy individuals. We note that many of these diseases, including diabetes and pulmonary and cardiovascular diseases, interact with oxygen.

Revised: Our model performs well on this population, as shown in our evaluation results.

Revised: Furthermore, all three datasets are focused on old people.

Revised: In particular, the median age in MrOS is 80 years.

Revised: Hence, 50% of the subjects are in their 80’s and 90’s.

Revised: Such old subjects are natural beneﬁciaries from monitoring oxygen saturation since they are at a high risk of low oxygen (NLM, 2020).

Revised: Relationship between breathing signals and Oxygen.

Revised: Intuitively, there could be two casual paths between breathing and oxygen, i.e., hypoventilation may cause hypoxemia, while hypoxemia could stimulate hyperventilation.

Revised: Our model can leverage such relationships.

Revised: However, the model also learns more complex relationships between oxygen level and the patterns in the breathing signals and their temporal features.

Revised: In Section ??

Original: We start by proposing a vanilla model, and then explore the oxygen-related physiological variables and propose a better preformed multi-headed model which effectively incorporate these variables.
Revised: , we show several examples where two subjects breath at the same rate, yet the model detects that their oxygen levels are quite different. Such empirical results show that the model not only leverages the breathing rate, but also other more sophisticated patterns in the detailed breathing signals. Limitations and Clinical Usefulness. Of course the above does not mean that the model presented in this paper can discover all situations that lead to low oxygen.

Revised: Even pulse oximetry does not always work.

Revised: The literature reports that in the critically ill, oximetry does not reliably predict changes in oxygen saturation because the calibration of red-light absorbance is based on calculations made from healthy volunteers (Ralston et al., 1991).

Revised: Indeed, oximetry fails in many important scenarios including subjects having chronic obstructive pulmonary disease (e.g., chronic bronchitis and emphysema)

Revised: (Amalakanti & Pentakota, 2016), anemia (Severinghaus & Koh, 1990), sickle cell disease (Blaisdell et al., 2000), skin pigmentation (Bickler et al., 2005).

Original: Our model  isdirectly applicable to respiration measured by wireless signals for contactless oxygen prediction.
Revised: Similarly, the model herein has its own limitations and cannot detect low oxygen if the breathing signal is not impacted. Our objective is to introduce a method that is non-invasive, easy to use, non-contact, and can be useful in many cases, particularly for old people, who are naturally at high risk of reduced oxygen (NLM, 2020). This population is well represented in our evaluation datasets, on which the model is demonstrated to have good performance. Furthermore, old patients can have onset of dementia or

Revised: Alzheimer’s and cannot remember to regularly measure themselves using a pulse oximeter.

Original: We also show a potential use case of monitoring  the recovery of COVID-19 patients.
Revised: Thus, they would beneﬁt from a system that passively and continuously measures oxygen using radio signals, without asking the patient to remember to actively measure themselves. The model can also provide a complementary oxygen monitoring system at home or in the hospital. As explained above pulse oximetry fails in various scenarios. One can add a radio device in the patient room to collect radio signals and use our system to augment oximetry.

Revised: This does not add any overhead to the patient since they will not wear any additional sensor but rather be monitored in a passive contactless manner using radio signals.

Revised: Further, since oximetry and our model use intrinsically different methods, they have independent errors, and hence can complement each other.

Original: A M ODEL A RCHITECTURE AND I MPLEMENTATION D ETAILS
Revised: A M ODEL A RCHITECTURE AND I MPLEMENTATION D ETAILS

Original: As we described in section 3, our vanilla modelmainly consists of an encoder and a predictor.
Revised: As we described in section 3, our vanilla model consists of an encoder and a predictor.

Original: The encoder has 9 layers of 1-D convolutional layers (Conv-Norm-RReLU) and shrink the temporal dimension of the features by 240 times, and then followed by several bi-directional multi-head self-attention layers (Devlin et al., 2018) to aggregate the temporal information at the bottleneck.
Revised: The encoder has nine 1-D convolutional layers (Conv-Norm-RReLU) that shrink the temporal dimension of the features by 240 times. It then followed by several bi-directional multi-head self-attention layers (Devlin et al., 2018) to aggregate the temporal information at the bottleneck.

Original: Speciﬁcally, we use 8 layers, 8 heads with hidden-size  256, intermediate-size512 of self-attention and the max position embeddings is 2400.
Revised: We use 8 layers, 8 heads with hidden-size of 256, intermediate-size of 512 for self-attention, and the max position embeddings is 2400.

Original: The decoder contains 7 layers of 1-D de-convolutional layers (DeConv-Norm-RReLU).
Revised: The decoder contains 7 layers of 1-D de-convolutional layers (DeConv-NormRReLU).

Original: We also use a skip connection (Ronneberger et al., 2015) by concatenating the convolutional layers in the encoder to the de-convolutional layers in the predictor.
Revised: We also use a skip connection (Ronneberger et al., 2015) by concatenating the convolutional layers in the encoder to the de-convolutional layers in the predictor.

Original: Figure 6 illustrate the overall network architecture.
Revised: Figure 6 illustrate the overall network architecture.

Original: We keep  encoder and predictor architecturesbe the same for all models.
Revised: We keep the encoder and predictor architectures the same for all models.

Original: All models are implemented using PyTorch.
Revised: All models are implemented using PyTorch.

Revised: The number of parameters for the multi-headed model is 26,821,113 and the model size is 107.28MB.

Revised: In the training process, we use the Adam optimizer with a learning rate of 2 × 10 − 4 , and train the model for 500 epochs.

Revised: Due to the varying input length, we set the batch size for all the models to 1 (i.e., one night of breathing signals and the corresponding oxygen time series).

Revised: Because oxygen saturation is affected by sleep stages and sleep cycles, it exhibits non-local dependencies.

Revised: To allows the model to capture these dependencies, we use the breathing signals from a night of sleep as one input sample.

Original:  [Figure]
Revised: To process such a long signal at the input, we use a UNET structure with a bottleneck that shrinks by a factor of 240, and has a max position embedding of 2400. The breathing signal is measured at 10 fps (frames per second). Thus, the model can process 240 ∗ 2400 / 10 / 3600 ≈ 16 hours of breathing data. [Figure]

Original: Encoder
Revised: Encoder

Original: Predictor
Revised: Predictor

Original: Input
Revised: Input

Original: Output
Revised: Output

Revised: B B REATHING S IGNALS

Original: Because the blood oxygen saturation is related to the sleep stage and sleep cycle, which indicates the non-local dependencies.
Revised: The input respiration signal we used in the paper is a one-dimensional time series with a frame rate of 10 fps (frames per second). It contains full information about the inhale and exhale motions. Figure 7 illustrates typical patterns of breathing signals with the corresponding oxygen saturation. Figure 7(a) shows a normal breathing pattern, which leads to constant oxygen saturation.

Original: Instead to segment the breathing signal into short-clips, the  breathing signals we used in all the experiments lasts several hours per night.
Revised: Figure 7(bd) are abnormal breathing signals. [Figure] B. F ULL B REATHING S IGNALS VS . B REATHING R ATE It is important to note that we use the full breathing signals instead of the breathing rate.

Original: Typically, as the bottleneck shrinks 240 times and has a max position embedding 2400, we can process as long as 240 ∗ 2400 / 3600 ≈hours breathing data.
Revised: Full breathing signals contain richer information about oxygen saturation than the breathing rate. Figureshows several cases where two subjects have the same breathing rate but different oxygen saturation values. The ﬁgure also shows that our model is able to predict the correct oxygen levels and is not confused by the fact that the subjects have the same breathing rates. [Figure]

Original: B M ORE VISUALIZATIONS ON MEDICAL DATASETS
Revised: C M ORE V ISUALIZATIONS ON M EDICAL D ATASETS

Original: We include more visualizations of the oxygen saturation predicted by the Multi-Headed model for the medical datasets: MESA (Figure 7 and Figure 8), MrOS (Figure 9 and Figure 10).
Revised: We include more visualizations of the oxygen saturation predicted by the Multi-Headed model for the medical datasets: MESA (Figure 9 and Figure 10), MrOS (Figure 11 and Figure 12).

Original: In the plots,
Revised: In the plots,

Original: The background color indicates different sleep stages.
Revised: The background color indicates different sleep stages.

Original: The ‘dark grey’, ‘light grey’ and ‘white’ corresponds to ‘Sleep’ (N1, N2, N3), ‘REM’ and ‘Awake’, respectively.
Revised: The ‘dark grey’, ‘light grey’ and ‘white’ corresponds to ‘Sleep’ (N1, N2, N3), ‘REM’ and ‘Awake’, respectively.

Original: B. M ESA
Revised: C. M ESA

Original: The examples in Figure 7 show our model’s ability to capture the regular ﬂuctuations of the oxygen saturation dynamics.
Revised: The examples in Figure 9 show our model’s ability to capture the regular ﬂuctuations of the oxygen saturation dynamics.

Original: While the examples in Figure 8 show our model accurately detecting the region of the low oxygen saturation which highlights its usefulness of monitoring patients.
Revised: At the same time, the examples in Figure 10 show our model accurately detecting the region of the low oxygen saturation, which highlights its usefulness in monitoring patients.

Original: [Figure]
Revised: C. M R OS

Original: Looking at the zoomed in range (a) and (b) from Figure 9, we ﬁnd the model exhibits  larger error when the ground truth SpO2 reading goes lower.
Revised: Looking at the zoomed-in range (a) and (b) from Figure 11, we ﬁnd the model exhibits a larger error when the ground truth SpO2 reading goes lower.

Original: It is mainly caused by the imbalanced label from the training set, since subjects usually experience much less time of having low oxygen level (like below 90%) then having a normal oxygen level between 94% to 100%. 
Revised: It is mainly caused by the imbalanced label from the training set since subjects usually experience much less time of having low oxygen level (like below 90%) than having a normal oxygen level between 94% to 100%. Figure 12 shows another example of the oxygenation dynamics.

Revised: As shown in the zoomed range, the oxygenation ﬂuctuation correlated with one’s sleep stage: in ’REM’ (colored by light gray), the oxygen ﬂuctuating drastically while in ’sleep’ (colored by dark gray), the oxygen is much more stable and ﬂuctuates in a small range.

Revised: Our model makes accurate prediction since it leverages the sleep stage information.

Original: [Figure]
Revised: [Figure]

Revised: D C OMPARISON AGAINST N AIVE B ASELINE M ODELS

Revised: We have compared our approaches with two naive baseline models, i.e., Linear Regression and

Revised: Random Forest Regression .

Revised: We use the six statistical time series feature suggested by (Picard et al., 2001).

Revised: Table 3 compares our model with the naive baselines.

Revised: As shown, These baselines produce average prediction errors that are 14% to 17% higher than our multi-headed model.

Revised: The gap between the number may do not look signiﬁcant.

Revised: However, if looking into the predicted oxygen of the naive baselines, we ﬁnd both linear regression and random forest tend to predict a value that concentrated at the averaged oxygen level without capturing the real dynamics of the oxygen ﬂuctuation.

Revised: Figure 13 visualizes one illustrative example.

Revised: Our model’s prediction follows the ground truth oxygen changes instead of being constant all the time.

Revised: It is because we design our model to capture the non-local temporal dependencies in the signals.

Revised: [Table]

Revised: E T OY E XAMPLE : L EARNING A P IECE - WISE L INEAR F UNCTION

Revised: We conduct experiments on a toy dataset shown in Figure 14 to demonstrate the effectiveness of the gradient diagnosis and our multi-headed model.

Revised: The target function is an 1-dimensional piecewise linear function.

Revised: The observed data has a small white noise added to the label.

Revised: Visually the function looks like a string composed of ‘W’,‘N’, and ‘M’.

Revised: Assuming we have an auxiliary variable, class , indicating which letter the data point belongs.

Revised: Our goal is to learn the function with access to the auxiliary variable.

Revised: By using gradient diagnosis, it suggests that this auxiliary variable is a good “switcher” candidate.

Revised: Denote g 1 , g 2 , g 3 as the gradients computed by the data points from the three letters.

Revised: The cosine similarity between them are cos ( g 1 , g 2 ) = 0 .

Revised: 48 , cos ( g 1 , g 3 ) = − 0 .

Revised: 02 , cos ( g 2 , g 3 ) = 0 .

Revised: 39 , meaning the gradients are not aligned.

Revised: We compare our Multi-Headed model with three baselines: Single-Headed, Multi-Task, PCGradAdapted described in section 4.

Revised: Figure 15 displays the learned function of the four methods.

Revised: Only the Multi-Headed model successfully learns the target function and makes a prediction that visually looks like “WNM”.

Revised: Quantitatively, the mean squared errors of the predictions are 0.06, 0.06, 0.05, 0.01 for Single-Headed, Multi-Task, PCGrad-Adapted, and Multi-Headed, respectively.

Revised: We attribute the Multi-Headed model’s success to its usage of the auxiliary variable as the “switcher”.

Revised: [Figure]

Revised: [Figure]