Original: Are you asleep when your phone is asleep? Semi-supervised methods to infer sleep from smart devices
Revised: Are you asleep when your phone is asleep? Semi-supervised methods to infer sleep from smart devices

Original: Abstract
Revised: Abstract

Original: Sleep is a vital aspect of our life.
Revised: Sleep is a vital aspect of our life.

Original: Having good quality sleep is necessary for our well-being and health.
Revised: Having good quality sleep is necessary for our well-being and health.

Original: Therefore, sleep measurements can aid us in improving our sleep quality.
Revised: Therefore, sleep measurements can aid us in improving our sleep quality.

Original: While many users are reluctant to use intrusive sleep sensing techniques such as wearables, passive sensing such as network activity of smartphone devices can be utilized to measure the sleep duration of a user.
Revised: While many users are reluctant to use intrusive sleep sensing techniques such as wearables, passive sensing such as network activity of smartphone devices can be utilized to measure the sleep duration of a user.

Original: However, we need large amounts of labeled data to develop accurate sleep prediction models.
Revised: However, we need large amounts of labeled data to develop accurate sleep prediction models.

Original: In addition, due to heterogeneity in user behaviors, hardware and software of the devices used, a single model may not generalize to every user in a given population.
Revised: In addition, due to heterogeneity in user behaviors, hardware, and software of the devices used, a single model may not generalize to every user in a given population.

Original: Although ground truth data collection from a large population is costly and challenging, unlabelled network activity data is easy to gather using mobile applications or network logs.
Revised: Although ground truth data collection from a large population is costly and challenging, unlabeled network activity data is easy to gather using mobile applications or network logs.

Original: This motivates us to look for semi-supervised learning approaches to leverage unlabelled data from the users to develop accurate sleep prediction models.
Revised: This motivates us to look for semi-supervised learning approaches to leverage unlabeled data from the users to develop accurate sleep prediction models.

Original: Our results show that  semi-supervised learning techniquescan be used to improve the accuracy of sleep duration estimation from smart devices.
Revised: Our results show that we can use semi-supervised learning techniques to improve the accuracy of sleep duration estimation from smart devices.

Original:  Introduction
Revised:  Introduction

Original: Despite the importance of sleep for human wellbeing, over a third of the human population sleeps less than seven hours per day and sleep disorders are commonplace amongthe adults (CDC, et al. 
Revised: Despite the importance of sleep for human well-being, over a third of the human population sleeps less than seven hours per day, and sleep disorders are commonplace among adults (CDC et al. 2009).

Original: Sleep monitoring via wearables has emerged as an approach for improving sleep health via daily tracking of sleep patterns.
Revised: Sleep monitoring via wearables has emerged as an approach for improving sleep health via daily tracking of sleep patterns.

Original: However, many users tend to be reluctant to wear trackers when sleeping, which has led to the emergence of contactless sensing methods to monitor sleep (Tauhidur Rahman, et al, 2015).
Revised: However, many users tend to be reluctant to wear trackers when sleeping, which has led to the emergence of contactless sensing methods to monitor sleep (Tauhidur Rahman et al., 2015).

Original: One promising approach is to use smartphone activity as a proxy for users sleep and awake state, where long periods of phone inactivity, particularly in the night, are used to infer the user’s nocturnal sleep durations.
Revised: One promising approach is to use smartphone activity as a proxy for the user’s sleep and awake state, where long periods of phone inactivity, particularly in the night, are used to infer the user’s nocturnal sleep durations.

Original: Recent research has shown the feasibility of using time series data of phone network WiFi activity (e g. event rates) to infer bed time and wake-up times.
Revised: Recent research has shown the feasibility of using time series data of phone network WiFi activity (e.g., event rates) to infer bedtime and wake-up times.

Original: When good quality labelled ground truth data is available, such techniques can even approach the accuracy of wearable sleep trackers such as oura ring (Ghorbani S et al, 2022).
Revised: When good quality labeled ground truth data is available, such techniques can even approach the accuracy of wearable sleep trackers such as oura ring (Ghorbani S et al, 2022).

Original: However such approaches suffer from many challenges due to differences in data.
Revised: However, such approaches suffer from many challenges due to differences in data.

Original: Such differences arises due to user heterogeneity, device heterogeneity and study heterogeneity (Blunck et al, 2016).
Revised: Such differences arise due to user heterogeneity, device heterogeneity and study heterogeneity (Blunck et al, 2016).

Original: User heterogeneity refers to the differences in user’s device usage practices and differences in sleeping patterns.
Revised: User heterogeneity refers to the differences in user’s device usage practices and differences in sleeping patterns.

Original: Device heterogeneity refers to the differences in the hardware and software version of the device.
Revised: Device heterogeneity refers to the differences in the hardware and software version of the device.

Original: Study heterogeneity arises due to the fact that users can change their devices over the time or new users can be added to the later stages of the study.
Revised: Study heterogeneity arises due to the fact that users can change their devices over the time or new users can be added to the later stages of the study.

Original: Consequently, current approaches have several limitations.
Revised: Consequently, current approaches have several limitations.

Original: First of all, a supervised model trained on the labelled data from user studies will not be able to generalize to new users, which limits the scalability of the technique.
Revised: First of all, a supervised model trained on the labeled data from user studies will not be able to generalize to new users, which limits the scalability of the technique.

Original: Second, obtaining high quality ground truth data from large number of users imposes high overheads and high cost for many user studies.
Revised: Second, obtaining high-quality ground truth data from large number of users imposes high overheads and high cost for many user studies.

Original: At the same time, unlabelled data is easy to gather since mobile app or standard network logs can be used to gather data of a phone’s WiFi activity.
Revised: At the same time, unlabeled data is easy to gather since mobile apps or standard network logs can be used to gather data on a phone’s WiFi activity.

Original: This motivates the need for semi-supervised learning (SSL) approaches that can combine a small amount of labelled ground truth data with large amounts of unlabelled data to improve the efficacy of ML models to predict sleep using phone activity and thereby personalize the models using a user’s phone’s unlabelled data.
Revised: This motivates the need for semi-supervised learning (SSL) approaches that can combine a small amount of labeled ground truth data with large amounts of unlabeled data to improve the efficacy of ML models to predict sleep using phone activity and thereby personalize the models using a user’s phone’s unlabeled data.

Original: In this paper, we mitigate the limitation of current supervised methods by providing interesting ideas on developing semi-supervised methods for sleep duration estimation.
Revised: In this paper, we mitigate the limitation of current supervised methods by providing interesting ideas on developing semi-supervised methods for sleep duration estimation.

Original:  Motivation 2.1 Problem Statement
Revised:  Motivation 2.1 Problem Statement

Original: We state the problem of sleep duration estimation as follows:
Revised: We state the problem of sleep duration estimation as follows:

Original: For each user, we are given a time series of network activity (event rates) from a smartphone (refer Fig. 1).
Revised: For each user, we are given a time series of network activity (event rates) from a smartphone (refer Fig. 1).

Original: We split a 24 hour period starting from 6 p.m of day d to 6 p.m of day d + 1 into 15 minute windows and classify each window into one of the two activity lables - "sleep" or "awake".
Revised: We split a 24 hour period starting from 6 p.m of day d to 6 p.m of day d + 1 into 15 minute windows and classify each window into one of the two activity labels - "sleep" or "awake".

Original: We then find the longest continuous streak of sleep to calculate the bed time and wake-up time for that user.
Revised: We then find the longest continuous streak of sleep to calculate the bed time and wake-up time for that user.

Original: Although a user may have multiple sleep periods in a day, we are considering only the sleeping period with the longest duration to calculate the bed time and wake up time.
Revised: Although a user may have multiple sleep periods in a day, we are considering only the sleeping period with the longest duration to calculate the bed time and wake up time.

Original: We are also considering the problem of generalizing our model for users with only unlabelled data.
Revised: We are also considering the problem of generalizing our model for users with only unlabeled data.

Original: For a population of size N , we have labelled data D for n users and unlabelled data D ′ for N − n users where N − n ≫ n .
Revised: For a population of size N , we have labeled data D for n users and unlabeled data D ′ for N − n users where N − n ≫ n .

Original: Our goal is to train models in a generalized and personalized way that can leverage both D and D ′ .
Revised: Our goal is to train models in a generalized and personalized way that can leverage both D and D ′ .

Original: 2.2 Dataset
Revised: 2.2 Dataset

Original: We collected ground truth data from 20 undergraduate students who are on-campus residents under an IRB approved study.
Revised: We collected ground truth data from 20 undergraduate students who are on-campus residents under an IRB approved study.

Original: The data-collection for each participant lasted for a duration of 4 weeks.
Revised: The data-collection for each participant lasted for a duration of 4 weeks.

Original: The participants were given Fitbit devices and manuals for logging the sleep data.
Revised: The participants were given Fitbit devices and manuals for logging the sleep data.

Original: We gathered logs of network activity from their smartphones.
Revised: We gathered logs of network activity from their smartphones.

Original: More details about the study is added in the Appendix
Revised: More details about the study is added in the Appendix

Original: Ethical Considerations: We conducted the data collection and analysis for this work under safeguards and restrictions approved by our Institutional Review Board (IRB) and Data Usage Agreement (DUA) with our campus network IT group.
Revised: Ethical Considerations: We conducted the data collection and analysis for this work under safeguards and restrictions approved by our Institutional Review Board (IRB) and Data Usage Agreement (DUA) with our campus network IT group.

Original: We anonymized all device MAC addresses and authentication information using a strong hashing algorithm.
Revised: We anonymized all device MAC addresses and authentication information using a strong hashing algorithm.

Original: The identities of the users were blinded by assigning numeric identifiers.
Revised: The identities of the users were blinded by assigning numeric identifiers.

Original: Ground truth was collected within the IRB-approved protocol.
Revised: Ground truth was collected within the IRB-approved protocol.

Original: Individual analyses were performed on users who had consented to this study.
Revised: Individual analyses were performed on users who had consented to this study.

Original: 2.3 Supervised Learning
Revised: 2.3 Supervised Learning

Original: Below we describe the baseline architecture and the features that we used to perform supervised learning on labelled data.
Revised: Below we describe the baseline architecture and the features that we used to perform supervised learning on labeled data.

Original: Baseline Architecture:
Revised: Baseline Architecture:

Original: For the baseline model, we use a simple Convolutional Neural Network.
Revised: For the baseline model, we use a simple Convolutional Neural Network.

Original: The model has three temporal 1D convolutional layers with 32, 64 and 96 filters.
Revised: The model has three temporal 1D convolutional layers with 32, 64 and 96 filters.

Original: Each kernel size in each layer are 24, 16, and 8 respectively with uniform a stride 1.
Revised: Each kernel size in each layer are 24, 16, and 8 respectively with uniform a stride 1.

Original: We then use a Sigmoid activation function with L2 regularization with a factor of 0.0001 for the weights.
Revised: We then use a Sigmoid activation function with L2 regularization with a factor of 0.0001 for the weights.

Original: In the end, a global 1D maximum pooling layer is connected to the last convolutional layer.
Revised: In the end, a global 1D maximum pooling layer is connected to the last convolutional layer.

Original: Features Used: We derived features from the WiFi syslog data of the devices.
Revised: Features Used: We derive features from the WiFi syslog data of the devices.

Original: The features include
Revised: The features include

Original: Unique Access Points (AP) visited, Access Point Transitions, Location Type, Unique Buildings visited, Building Transitions and Time.
Revised: Unique Access Points (AP) visited, Access Point Transitions, Dorm or not, and Time.

Original: The rationale for choosing these features is, when a user is active or awake, the value of these features is expected to be different compared to when a user is inactive or asleep.
Revised: The rationale for choosing these features is, when a user is active or awake, the value of these features is expected to be different compared to when a user is inactive or asleep.

Original: Efficacy of the Model: We picked 10 users and did 80:20 splits of the labelled data for training and testing.
Revised: Efficacy of the Model: We picked 10 users and did 80:20 splits of the labelled data for training and testing.

Original: The results are shown in Table 1 and we can see that the model is able to predict the bed time and wake up time with an average error of less than 30 mins.
Revised: The results are shown in Table 1 and we can see that the model is able to predict the bed time and wake up time with an average error of less than 30 mins.

Original: [Table]
Revised: [Table]

Original: Inability to generalize: Next, we study the ability of the SL models to generalize.
Revised: Inability to generalize: Next, we study the ability of the SL models to generalize.

Original: Data collection from various sources is an important step in mobile sensing.
Revised: Data collection from various sources is an important step in mobile sensing.

Original: However, data collection and analysis is often challenged by the the differences in training data.
Revised: However, data collection and analysis is often challenged by the the differences in training data.

Original: In our experiment, we try to gradually increase the size of the test data by adding more users.
Revised: In our experiment, we try to gradually increase the size of the test data by adding more users.

Original: We can see from Table.
Revised: We can see from Table.

Original: 2 that the models accuracy decreases as we bring in more variability in the data by adding new users.
Revised: 2 that the models accuracy decreases as we bring in more variability in the data by adding new users.

Original: This shows that the model is not scalable.
Revised: This shows that the model is not scalable.

Original: [Table]
Revised: [Table]

Original: This motivates us to look for scalable and generalizable techniques that does not involve additional labelled data collection, leading us to the semi-supervised techniques.
Revised: This motivates us to look for scalable and generalizable techniques that does not involve additional labelled data collection, leading us to the semi-supervised techniques.

Original:  Approach
Revised:  Approach

Original: In this section, we experiment with a state-of-the-art SSL technique to compare against the supervised learning model (baseline model) on sleep prediction.
Revised: In this section, we experiment with a state-of-the-art SSL technique to compare against the supervised learning model (baseline model) on sleep prediction.

Original: 3.1 Multi-head Single-view Co-training
Revised: 3.1 Multi-head Single-view Co-training

Original: In single-view co-training, we train multiple classifiers without splitting the features unlike multi-view co-training.
Revised: In single-view co-training, we train multiple classifiers without splitting the features, unlike multiview co-training.

Original: In this paper we try to overcome the overhead of training multiple classifiers.
Revised: In this paper, we try to overcome the overhead of training multiple classifiers.

Original: We do this by using multiple heads in a shared module instead of training multiple classifiers separately.
Revised: We do this by using multiple heads in a shared module instead of training multiple classifiers separately.

Original: Thus in multi-head co-training, there will be a shared parametric module and multiple classification heads with identical structure.
Revised: Thus in multi-head co-training, there will be a shared parametric module and multiple classification heads with identical structures.

Original: We use a similar architecture mentioned in (Chen, et al, 2022), however the training pipeline is different.
Revised: We use a similar architecture mentioned in (Chen et al., 2022); however, the training pipeline is different.

Original: Instead of augmenting the dataset using data perturbations, we use the unlabelled data from the new users or unseen users in a population.
Revised: Instead of augmenting the dataset using data perturbations, we use the unlabeled data from the new users or unseen users in a population.

Original: At first, we train the model with only one classification head with labelled data.
Revised: At first, we train the model with only one classification head with labeled data.

Original: We then use this model to generate the pseudo labels for the unlabelled data using a majority voting from all the classification heads.
Revised: We then use this model to generate the pseudo labels for the unlabeled data using a majority voting from all the classification heads.

Original: Each 15 min window is classified as "sleep" or "awake" if the majority is above a certain threshold.
Revised: Each 15 min window is classified as "sleep" or "awake" if the majority is above a certain threshold.

Original: However, since we are interested in getting bed time and wake-up time of an entire day, knowing  individual window’s classification is not sufficient.
Revised: However, since we are interested in getting the bedtime and wake-up time of an entire day, knowing the individual window’s classification is not sufficient.

Original: We evaluate if the classification of the bins overa 24 hour period is good enough to mix with labelled data.
Revised: We evaluate if the classification of the bins over 24 hours is good enough to mix with labeled data.

Original: We take the average majority scores of all the bins forhour period and check if this average is above a certain threshold, we consider that days data good enough to be mixed with labelled data.
Revised: We take the average majority scores of all the bins for 24 hour period and check if this average is above a certain threshold; we consider that days data good enough to be mixed with labeled data.

Original: After mixing the datasets, we again train the model using only one classification head using the new augmented dataset.
Revised: After combining the datasets, we again train the model using only one classification head using the new augmented dataset.

Original: 3.2 Performance Evaluation
Revised: 3.2 Performance Evaluation

Original: Experiments were run on Google colaboratory notebooks.
Revised: Experiments were run on Google colaboratory notebooks.

Original: We split the labelled dataset into 50 :splits, with training data set and test data set consisting of data from 10 users each.
Revised: We split the labeled dataset into 50 :splits, with training data set and test data set consisting of data from 10 users each.

Original: We also took one week of unlabelled data from users in the test dataset.
Revised: We also took one week of unlabeled data from users in the test dataset.

Original: The performance metrics we use here are sleep time and wake up time estimation errors which are defined as the difference between the estimated value and the true value.
Revised: The performance metrics we use here are sleep time and wake-up time estimation errors which are defined as the difference between the estimated value and the true value.

Original: From Table 3, we can see that the Multihead single-view co-training is able to perform much better in comparison to the supervised learning without labelled data from test set users.Technique
Revised: From Table 3, we can see that the Multihead single-view co-training is able to perform much better in comparison to the supervised learning without labeled data from test set users.

Original: [Table]
Revised: [Table]

Original:  Conclusions and Future Directions
Revised:  Conclusions and Future Directions

Original: Sleep sensing can potentially help us improve our sleep quality.
Revised: Sleep sensing can potentially help us improve our sleep quality.

Original: As most of the users are reluctant to use intrusive sleep sensing techniques,  network activity ofthe smartphones can be utilized to predict our sleep.
Revised: As most of the users are reluctant to use intrusive sleep-sensing techniques, the network activity of smartphones can be utilized to predict our sleep.

Original: However, due to limited labelled data and ground truth data collection overheads in a larger population, we have to rely on semi supervised learning techniques.
Revised: However, due to limited labeled data and ground truth data collection overheads in a larger population, we have to rely on semi-supervised learning techniques.

Original: Our experiments show that SSL techniques are viable options to improve the accuracy of sleep prediction models when there is limited labelled data.
Revised: Our experiments show that SSL techniques are viable options to improve the accuracy of sleep prediction models when there is limited labeled data.

Original: The results we presented are preliminary and as a future direction, we plan to extend the work by experimenting with different model configurations and other SSL techniques to achieve closer accuracy with supervised learning with labelled data.
Revised: The results we presented are preliminary and as a future direction, we plan to extend the work by experimenting with different model configurations and other SSL techniques to achieve closer accuracy with supervised learning with labeled data.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: We would like to thank the anonymous reviewers for their constructive feedback.
Revised: We would like to thank the anonymous reviewers for their constructive feedback.

Original: This research was supported in part by NSF grants 1836752, 1722792, 1763834, 1802523 and Army grant W911NF-172-
Revised: This research was supported in part by NSF grants 1836752, 1722792, 1763834, 1802523 and Army grant W911NF-172-

Original: A Appendix A.1 User Study Procedure
Revised: A Appendix A.1 User Study Procedure

Original: We advertised about the study through flyers and using undergraduate mailing groups.
Revised: We advertised about the study through flyers and using undergraduate mailing groups.

Original: We recruitedundergraduate students who are on-campus residents.
Revised: We recruitedundergraduate students who are on-campus residents.

Original: The data-collection for each participant lasted for a duration of 4 weeks, while the overall recruiting process ran from October to mid-December.
Revised: The data-collection for each participant lasted for a duration of 4 weeks, while the overall recruiting process ran from October to mid-December.

Original: The participants were given Fitbit devices and manuals for logging the sleep data.
Revised: The participants were given Fitbit devices and manuals for logging the sleep data.

Original: We precisely identified the participants’ hashed MAC addresses by monitoring their WiFi events from a dedicated
Revised: We precisely identified the participants’ hashed MAC addresses by monitoring their WiFi events from a dedicated

Original: AP on campus.
Revised: AP on campus.

Original: We made sure the following guidelines were followed through out the study by sending periodic reminders to the participants :
Revised: We made sure the following guidelines were followed through out the study by sending periodic reminders to the participants :

Original: • The students were asked to wear a fitness tracker (like a Fitbit or similar) for the duration of the study.
Revised: • The students were asked to wear a fitness tracker (like a Fitbit or similar) for the duration of the study.

Original: They were advised to wear the device as much as possible, but it is critical that they wore it at night.
Revised: They were advised to wear the device as much as possible, but it is critical that they wore it at night.

Original: We used the sleep-tracking function of the fitness tracker to help us understand the relationship between sleep and network activity.
Revised: We used the sleep-tracking function of the fitness tracker to help us understand the relationship between sleep and network activity.

Original: It is critical to keep the device charged and paired to the phone at night.
Revised: It is critical to keep the device charged and paired to the phone at night.

Original: Putting the device on and taking it off and charging as needed should take less than 10 minutes per day.
Revised: Putting the device on and taking it off and charging as needed should take less than 10 minutes per day.

Original: • They were asked to keep a sleep journal.
Revised: • They were asked to keep a sleep journal.

Original: This is used to verify the sleep data we get from the wearable, and vice versa.
Revised: This is used to verify the sleep data we get from the wearable, and vice versa.

Original: This should take 30 seconds or less per day.
Revised: This should take 30 seconds or less per day.

Original: In addition to the above data collection activities, the students made two visits to the lab:
Revised: In addition to the above data collection activities, the students made two visits to the lab:

Original: • Participants attended a 30-minute orientation session at the beginning of the study.
Revised: • Participants attended a 30-minute orientation session at the beginning of the study.

Original: During the orientation session they are given a wearable fitness tracker along with instructions on how to use and charge it.
Revised: During the orientation session they are given a wearable fitness tracker along with instructions on how to use and charge it.

Original: The study’s smartphone app was installed on your Android phone during the session, and they were instructed on how to use it.
Revised: The study’s smartphone app was installed on your Android phone during the session, and they were instructed on how to use it.

Original: • They came for a 30-minute debriefing session at the end of the study during which they turn over any study equipment they still own and deleted the study app from their phone.
Revised: • They came for a 30-minute debriefing session at the end of the study during which they turn over any study equipment they still own and deleted the study app from their phone.

Original: During this session they also received compensation for their participation in the study.
Revised: During this session they also received compensation for their participation in the study.

Original: The students received compensation in the form of Fitbits or Amazon coupons worth 100 USDs if they wore Fitbit every day for at least 4 weeks while on campus.
Revised: The students received compensation in the form of Fitbits or Amazon coupons worth 100 USDs if they wore Fitbit every day for at least 4 weeks while on campus.