Original: Empirical Evaluation of Data Augmentations for Biobehavioral Time Series Data with Deep Learning
Revised: Empirical Evaluation of Data Augmentations for Biobehavioral Time Series Data with Deep Learning

Original: Abstract
Revised: Abstract

Original: Deep learning has performed remarkably well on many tasks recently.
Revised: Deep learning has performed remarkably well on many tasks recently.

Original: However, the superior performance of deep models relies heavily on the availability of a large number of training data, which limits the wide adaptation of deep models on various clinical and affective computing tasks, as the labeled data are usually very limited.
Revised: However, the superior performance of deep models relies heavily on the availability of a large number of training data, which limits the wide adaptation of deep models on various clinical and affective computing tasks, as the labeled data are usually very limited.

Original: As an effective technique to increase the data variability and thus train deep models with better generalization, data augmentation (DA) is a critical step for the success of deep learning models on biobehavioral time series data.
Revised: As an effective technique to increase the data variability and thus train deep models with better generalization, data augmentation (DA) is a critical step for the success of deep learning models on biobehavioral time series data.

Original: However, the effectiveness of various DAs for different datasets with different tasks and deep models is understudied for biobehavioral time series data.
Revised: However, the effectiveness of various DAs for different datasets with different tasks and deep models is understudied for biobehavioral time series data.

Original: In this paper, we first systematically review eight basic DA methods for biobehavioral time series data, and evaluate the effects on seven datasets with three backbones.
Revised: In this paper, we first systematically review eight basic DA methods for biobehavioral time series data, and evaluate the effects on seven datasets with three backbones.

Original: Next, we explore adapting more recent DA techniques ( i.e., automatic augmentation, random augmentation ) to biobehavioral time series data by designing a new policy architecture applicable to time series data.
Revised: Next, we explore adapting more recent DA techniques ( i.e., automatic augmentation, random augmentation ) to biobehavioral time series data by designing a new policy architecture applicable to time series data.

Original: Last, we try to answer the question of why a DA is effective ( or not) by first summarizing two desired attributes for augmentations ( challenging and faithful ),and then utilizing two metrics to quantitatively measure the corresponding attributes, which can  guide us in the search for more effective DA for biobehavioral time series data by designing more challenging but still faithful transformations.
Revised: Lastly, we try to answer the question of why a DA is effective ( or not by first summarizing two desired attributes for augmentations ( challenging and faithful ), then utilizing two metrics to quantitatively measure them. The measurement can therefore guide us in the search for more effective DA for biobehavioral time series data by designing more challenging but still faithful transformations.

Original: Our code and results are available at this Link.
Revised: Our code and results are available at this Link.

Original:  Introduction
Revised:  Introduction

Original: Deep learning performs remarkably well in many fields, including computer vision (CV), natural language processing (NLP), and recently time series-related tasks [11, 32, 7].
Revised: Deep learning performs remarkably well in many fields, including computer vision (CV), natural language processing (NLP) and recently time series-related tasks [11, 32, 7].

Original: Those successful applications increasingly inspire researchers to embrace deep learning for solving issues in humancentered applications that use physiological and behavioral time series data.
Revised: Those successful applications increasingly inspire researchers to embrace deep learning for solving issues in humancentered applications that use physiological and behavioral time series data.

Original: However, the superior performance of deep models relies heavily on the availability of a large number of training data, but unfortunately, many human centered applications ( i e., healthcare tasks ) usually do not have enough labeled samples, which may limit the wide adaptation of deep models to various computing tasks.
Revised: However, the superior performance of deep models relies heavily on the availability of a large number of training data, but unfortunately, many human centered applications ( i e., healthcare tasks ) usually do not have enough labeled samples, which may limit the wide adaptation of deep models to various computing tasks.

Original: As an effective technique to increase the data variability and thus train deep models with better generalization, data augmentation (DA) is a critical step for the successful applications of deep learning models.
Revised: As an effective technique to increase the data variability and thus train deep models with better generalization, data augmentation (DA) is a critical step for the successful applications of deep learning models.

Original: While DA can yield considerable performance improvements, they do require domain knowledge and are task- and domain-dependent.
Revised: While DA can yield considerable performance improvements, they do require domain knowledge and are task- and domain-dependent.

Original: For example, image rotation, a likely class-preserving behavior, is designed to rotate the input by some number of degrees.
Revised: For example, image rotation, a likely class-preserving behavior, is designed to rotate the input by some number of degrees.

Original: The image’s class can still be recognized by humans, thus allowing the model to generalize in a way humans expect it to generalize.
Revised: The image’s class can still be recognized by humans, thus allowing the model to generalize in a way humans expect it to generalize.

Original: However, such an effective random angle-based rotation operation may not be applicable to other domains, i.e., wearable data .
Revised: However, such an effective random angle-based rotation operation may not be applicable to other domains, i.e., wearable data .

Original: In addition, searching for the most effective
Revised: In addition, searching for the most effective

Original: DA methods for a new dataset is very time-consuming, and this motivated the proposal of several automatic DA search algorithms [5, 16, 6, 15, 17].
Revised: DA methods for a new dataset is very time-consuming, and this motivated the proposal of several automatic DA search algorithms [5, 16, 6, 15, 17].

Original: The existing DA literature mainly focuses on computer vision, but its application to other domains, i.e, biobehavioral time series data , is understudied.
Revised: The existing DA literature mainly focuses on computer vision, but its application to other domains, i.e, biobehavioral time series data , is understudied.

Original: A few works investigated the effectiveness of basic data augmentation methods for time series and wearable data [30, 12, 32, 1].
Revised: A few works investigated the effectiveness of basic data augmentation methods for time series and wearable data [30, 12, 32, 1].

Original: However, those works only investigated the very basic DAs, leaving the more recent DA techniques ( i.e., automatic DA ) unexplored.
Revised: However, those works only investigated the very basic DAs, leaving the more recent DA techniques ( i.e., automatic DA ) unexplored.

Original: More importantly, it is still an open question why a data augmentation method works, and how to quantify its effectiveness.
Revised: More importantly, it is still an open question why a data augmentation method works, and how to quantify its effectiveness.

Original: Therefore, in this paper, we first systematically review various basic
Revised: Therefore, in this paper, we first systematically review various basic

Original: DA methods for biobehavioral time series data, evaluating the effects on different datasets with varied backbones and tasks.
Revised: DA methods for biobehavioral time series data, evaluating the effects on different datasets with varied backbones and tasks.

Original: Next, we validate the effectiveness of adapting more recent DA techniques ( i.e., automatic DA ) to biobehavioral time series data.
Revised: Next, we validate the effectiveness of adapting more recent DA techniques ( i.e., automatic DA ) to biobehavioral time series data.

Original: Following the DADA[15], we designed a different policy architecture where the operations are differentiable with respect to different time series DA methods.
Revised: Following the DADA[15], we designed a different policy architecture where the operations are differentiable with respect to different time series DA methods.

Original: Therefore, the model can be applied to biobehavioral time series data and the
Revised: Therefore, the model can be applied to biobehavioral time series data and the

Original: DA parameters and deep model weights can be jointly optimized.
Revised: DA parameters and deep model weights can be jointly optimized.

Original: Lastly, we try to answer the open question of why a DA works( or not ), by first summarizing two desired attributes ( challenging and faithful ) for an effective DA, and then utilizing two metrics to quantitatively measure the two attributes.
Revised: Lastly, we try to answer the open question of why a DA works( or not ), by first summarizing two desired attributes ( challenging and faithful ) for an effective DA, and then utilizing two metrics to quantitatively measure the two attributes.

Original: We find that an effective DA needs to generate challenging but still faithful transformations, which can guide us for the search of more effective DA for biobehavioral time series data.
Revised: We find that an effective DA needs to generate challenging but still faithful transformations, which can guide us for the search of more effective DA for biobehavioral time series data.

Original:  Related Works
Revised:  Related Works

Original: Augmentations for Biobehavioral Time Series Data Most of the basic DA methods are borrowed from image or time series data augmentation, such as flipping, cropping and noise addition.
Revised: Augmentations for Biobehavioral Time Series Data Most of the basic DA methods are borrowed from image or time series data augmentation, such as flipping, cropping and noise addition.

Original: These augmentation methods rely on adding random transformations to the training data.
Revised: These augmentation methods rely on adding random transformations to the training data.

Original: Um et al.
Revised: Um et al.

Original: [30] systematically evaluated six DA methods for wearable sensor data based Parkinson’s disease monitoring, and found that the combination of rotational and permutational data augmentation methods improve the baseline performance the most.
Revised: [30] systematically evaluated six DA methods for wearable sensor data based Parkinson’s disease monitoring, and found that the combination of rotational and permutational data augmentation methods improve the baseline performance the most.

Original: Ohashi et al.
Revised: Ohashi et al.

Original: [21] proposed a rotation based DA method for wearable data.
Revised: [21] proposed a rotation based DA method for wearable data.

Original: Alawneh et al.
Revised: Alawneh et al.

Original: [1] investigated the benefits of adopting time series DA methods to biomedical time series data.
Revised: [1] investigated the benefits of adopting time series DA methods to biomedical time series data.

Original: Besides, DA methods have been also used to balance the dataset.
Revised: Besides, DA methods have been also used to balance the dataset.

Original: For example, Cao et al.
Revised: For example, Cao et al.

Original: [4] used DA methods to balance the number of samples among different categories for automated heart disease detection.
Revised: [4] used DA methods to balance the number of samples among different categories for automated heart disease detection.

Original: However, those related works only investigated the very basic DAs, and the effectiveness of adapting more advanced DAs is not explored yet for biobehavioral time series data.
Revised: However, those related works only investigated the very basic DAs, and the effectiveness of adapting more advanced DAs is not explored yet for biobehavioral time series data.

Original: More importantly, the previous works did not explore the question of why a DA is effective, and vice versa.
Revised: More importantly, the previous works did not explore the question of why a DA is effective, and vice versa.

Original: Automatic Data Augmentation DA can be very useful for the training of deep models, but the success relies heavily on domain knowledge and also the extensive experiments to select the effective
Revised: Automatic Data Augmentation DA can be very useful for the training of deep models, but the success relies heavily on domain knowledge and also the extensive experiments to select the effective

Original: DA policies for a target dataset.
Revised: DA policies for a target dataset.

Original: Otherwise, a model may be negatively impacted by some DA policies [12, 32].
Revised: Otherwise, a model may be negatively impacted by some DA policies [12, 32].

Original: Therefore, it is nontrivial and desired to select the effective DA policy for a new dataset automatically.
Revised: Therefore, it is nontrivial and desired to select the effective DA policy for a new dataset automatically.

Original: The pioneering work, AutoAugment [5], formats the process of searching
Revised: The pioneering work, AutoAugment [5], formats the process of searching

Original: DA as an optimization problem to search for the parameters of augmentation, and follwing work [10, 16, 15, 17, 6] were later proposed to improve the efficiency.
Revised: DA as an optimization problem to search for the parameters of augmentation, and follwing work [10, 16, 15, 17, 6] were later proposed to improve the efficiency.

Original: Despite the success of automatic
Revised: Despite the success of automatic

Original: DA for computer vision tasks, the adaption to biobehavioral time series data is understudied.
Revised: DA for computer vision tasks, the adaption to biobehavioral time series data is understudied.

Original: As long as we know, [25] in the only work which investigated the automatic differentiable data augmentation for EEG signals.
Revised: As long as we know, [25] in the only work which investigated the automatic differentiable data augmentation for EEG signals.

Original: However, our work is different with [25], as we target more diverse types of data and DA methods, and more importantly, we explore to explain why a DA works or not.
Revised: However, our work is different with [25], as we target more diverse types of data and DA methods, and more importantly, we explore to explain why a DA works or not.

Original: Quantitative Effectiveness Measurement for DA
Revised: Quantitative Effectiveness Measurement for DA

Original: Although the effectiveness of DA is well acknowledged, a quantitative evaluation of the effectiveness of DA is still an open question.
Revised: Although the effectiveness of DA is well acknowledged, a quantitative evaluation of the effectiveness of DA is still an open question.

Original: Currently, the most well-known hypothesis is that effective DA can produce samples from an "overlapping but different" distribution [2, 18], therefore improving generalization by training with the diverse samples.
Revised: Currently, the most well-known hypothesis is that effective DA can produce samples from an "overlapping but different" distribution [2, 18], therefore improving generalization by training with the diverse samples.

Original: However, the role of distribution shift in training remains unclear.
Revised: However, the role of distribution shift in training remains unclear.

Original: A more recent work [9] studied to quantify how DA improves model generalization, and introduced two measures: Affinity and Diversity , to predict the performance of an augmentation method.
Revised: A more recent work [9] studied to quantify how DA improves model generalization, and introduced two measures: Affinity and Diversity , to predict the performance of an augmentation method.

Original: During our experiments, we adopt those two metrics to jointly evaluate the performance of different augmentations.
Revised: During our experiments, we adopt those two metrics to jointly evaluate the performance of different augmentations.

Original:  Methods and Experiments
Revised:  Methods and Experiments

Original: Through the experiments, we aim to answer the following research questions:
Revised: Through the experiments, we aim to answer the following research questions:

Original: R1:
Revised: R1:

Original: What is the most effective DA method for a given backbone and dataset? R2:
Revised: What is the most effective DA method for a given backbone and dataset? R2:

Original: what are the factors that impact the selection of DA? R3:
Revised: What are the factors that impact the selection of DA? R3:

Original: What are the general conclusion for DA methods in biobehavioral time series data?
Revised: What are the general conclusion for DA methods in biobehavioral time series data?

Original: R4: Why are some DA methods more effective than the others?
Revised: R4: Why are some DA methods more effective than the others?

Original: 3.1 Methods
Revised: 3.1 Methods

Original: To answer those questions, we first performed extensive experiments with various DA methods on different datasets with different tasks and backbones.
Revised: To answer those questions, we first performed extensive experiments with various DA methods on different datasets with different tasks and backbones.

Original: Let T α denote an augmentation operations parameterized by α , given input data x , this procedure outputs augmented data ˆ x = T α ( x ) , where α controls the magnitude of the operation T .
Revised: Let T α denote an augmentation operations parameterized by α , given input data x , this procedure outputs augmented data ˆ x = T α ( x ) , where α controls the magnitude of the operation T .

Original: Our augmentation set consists of eight operations:
Revised: Our augmentation set consists of eight operations:

Original: Jittering, Scaling, Rotation, Permutation, Magnitude Warping, Time Warping, Window Slicing, and
Revised: Jittering, Scaling, Rotation, Permutation, Magnitude Warping, Time Warping, Window Slicing, and

Original: Window Warping , which are widely used DAs drawn from the traditional time series processing literature.
Revised: Window Warping , which are widely used DAs drawn from the traditional time series processing literature.

Original: Further, we explored recent DA techniques to automatically search for effective DAs.
Revised: Further, we explored recent DA techniques to automatically search for effective DAs.

Original: DADA[15] was originally designed to improve the optimization efficiency for visual data through relaxing the optimization process as differentiable.
Revised: DADA[15] was originally designed to improve the optimization efficiency for visual data through relaxing the optimization process as differentiable.

Original: To make it applicable to biobehavioral time series data and the DA parameters and deep model weights can be jointly optimized, we designed a different policy architecture where the operations are differentiable with respect to different time series DA methods.
Revised: To make it applicable to biobehavioral time series data and the DA parameters and deep model weights can be jointly optimized, we designed a different policy architecture where the operations are differentiable with respect to different time series DA methods.

Original: To avoid the complicated searching procedure of automatic DA a random DA [6] procedure was also adapted in our experiments.
Revised: To avoid the complicated searching procedure of automatic DA a random DA [6] procedure was also adapted in our experiments.

Original: Lastly, we also explored to answer the open questions of why some DA methods are more effective, and vice versa.
Revised: Lastly, we also explored to answer the open questions of why some DA methods are more effective, and vice versa.

Original: Following previous work[29], we first summarized the desired attributes ( challenging and faithful ) for effective DA, and then we adopted a recent work [9] to quantitatively measure the two attributes respectively.
Revised: Following previous work[29], we first summarized the desired attributes ( challenging and faithful ) for effective DA, and then we adopted a recent work [9] to quantitatively measure the two attributes respectively.

Original: Augmentations are designed to prevent models from over-fitting by increasing the number of samples in the training set.
Revised: Augmentations are designed to prevent models from over-fitting by increasing the number of samples in the training set.

Original: Based on the intuition that more diverse data should be more difficult for a model to fit, the augmented data should be complex and strong enough that a model must learn useful representation to perform the task.
Revised: Based on the intuition that more diverse data should be more difficult for a model to fit, the augmented data should be complex and strong enough that a model must learn useful representation to perform the task.

Original: We use challenging to measure the intuition, which is defined as the ratio of final training loss of a model trained with a given augmentation, and the loss of the model trained on original data.
Revised: We use challenging to measure the intuition, which is defined as the ratio of final training loss of a model trained with a given augmentation, and the loss of the model trained on original data.

Original: Formally, let τ be an augmentation and D train and D ′ train be the training data and augmented training data respectively.
Revised: Formally, let τ be an augmentation and D train and D ′ train be the training data and augmented training data respectively.

Original: Further, let
Revised: Further, let

Original: L ( θ |D ) be the training loss of a model with parameter θ on the training data D .
Revised: L ( θ |D ) be the training loss of a model with parameter θ on the training data D .

Original: Then, we can define
Revised: Then, we can define

Original: Challenging as:
Revised: Challenging as:

Original: [Equation]
Revised: [Equation]

Original: The augmentation should generate challenging augmented data, but must not too challenging that make tasks impossible.
Revised: The augmentation should generate challenging augmented data, but must not too challenging that make tasks impossible.

Original: For example, the random noise added to the signal are so strong that all the useful information might been destroyed.
Revised: For example, the random noise added to the signal are so strong that all the useful information might been destroyed.

Original: We use faithful to measure the requirement, which is defined as the ratio between the validation accuracy of a model trained on clean data and tested on an augmented validation set, and the accuracy of the same model tested on clean data.
Revised: We use faithful to measure the requirement, which is defined as the ratio between the validation accuracy of a model trained on clean data and tested on an augmented validation set, and the accuracy of the same model tested on clean data.

Original: More formally, let
Revised: More formally, let

Original: D train and D val be the training and validation datasets, and let D ′ val be an augmented dataset derived from D val .
Revised: D train and D val be the training and validation datasets, and let D ′ val be an augmented dataset derived from D val .

Original: Further, let M ( · ) be a model trained on D train , and A cc ( M , D ) denote the accuracy of the model when evaluated on dataset D .
Revised: Further, let M ( · ) be a model trained on D train , and A cc ( M , D ) denote the accuracy of the model when evaluated on dataset D .

Original: Then, for an augmentation τ , Faithful is defined as:
Revised: Then, for an augmentation τ , Faithful is defined as:

Original: [Equation]
Revised: [Equation]

Original: 3.2 Datasets
Revised: 3.2 Datasets

Original: We conducted extensive experiments on seven biomedical time series datasets, including electrocardiogram (ECG) data: PTB-XL [31] and Apnea-ECG [22]; electroencephalogram (EEG) data:
Revised: We conducted extensive experiments on seven biomedical time series datasets, including electrocardiogram (ECG) data: PTB-XL [31] and Apnea-ECG [22]; electroencephalogram (EEG) data:

Original: Sleep-EDF (expanded) [13] and MMIDB-EEG [27]; electrodermal activity (EDA) data: CLAS [19]; inertial measurement unit (IMU) data: PAMAP2 [23] and UCI-HAR [24].
Revised: Sleep-EDF (expanded) [13] and MMIDB-EEG [27]; electrodermal activity (EDA) data: CLAS [19]; inertial measurement unit (IMU) data: PAMAP2 [23] and UCI-HAR [24].

Original: Note that due to page[Table] limits, we listed more details in the supplementary materials, including the details of the individual dataset and tasks, magnitude range and default value for augmentations, model structures, etc. 
Revised: Note that due to page limits, we listed more details in the supplementary materials, including the details of the individual dataset and tasks, magnitude range and default value for augmentations, model structures, etc. [Table]

Original: 3.3 Experimental Results
Revised: 3.3 Experimental Results

Original: Performance evaluation of eight basic DA methods.
Revised: Performance evaluation of eight basic DA methods.

Original: Experiments were first conducted on seven datasets with three different backbones and eight basic DA methods, and the results are shown in
Revised: Experiments were first conducted on seven datasets with three different backbones and eight basic DA methods, and the results are shown in

Original: Table.1.
Revised: Table.1.

Original: We may find that: I).
Revised: We may find that: I).

Original: All the Tasks can benefit from DAs.
Revised: All the Tasks can benefit from DAs.

Original: ( R1,R3 )
Revised: ( R1,R3 )

Original: Comparing with the models trained w/o augmentation, training with proper DA usually can achieve higher performance, and the improvement ranges from 0.5% to 15% over different datasets.
Revised: Comparing with the models trained w/o augmentation, training with proper DA usually can achieve higher performance, and the improvement ranges from 0.5% to 15% over different datasets.

Original: II).
Revised: II).

Original: The effectiveness of DA depends on many factors.
Revised: The effectiveness of DA depends on many factors.

Original: ( R2 )
Revised: ( R2 )

Original: From the grey colored areas, our first impression is that there is no such a single augmentation method works equally well for all the different datasets.
Revised: From the grey colored areas, our first impression is that there is no such a single augmentation method works equally well for all the different datasets.

Original: The dataset varies in terms of data type ( e.g., ECG, EEG, EDA and IMU ) and tasks ( sleep quality, heart disease, human activities, etc ).
Revised: The dataset varies in terms of data type ( e.g., ECG, EEG, EDA and IMU ) and tasks ( sleep quality, heart disease, human activities, etc ).

Original: We can find that the effectiveness of an augmentation not only depends on the dataset itself ( data type, task ), but also the selection of backbones.
Revised: We can find that the effectiveness of an augmentation not only depends on the dataset itself ( data type, task ), but also the selection of backbones.

Original: For example, with the same backbone ( i.e., ResNet-1D ) and data type ( ECG ), the top three augmentations for PTB-XL and Apnea-ECG are totally different.
Revised: For example, with the same backbone ( i.e., ResNet-1D ) and data type ( ECG ), the top three augmentations for PTB-XL and Apnea-ECG are totally different.

Original: scaling and jittering work the best for PTB-XL dataset, while rotation and permutation show the highest improvement for the Apnea-ECG dataset.
Revised: scaling and jittering work the best for PTB-XL dataset, while rotation and permutation show the highest improvement for the Apnea-ECG dataset.

Original: Besides, we also observe that permutation appears 13 times in the top three augmentations, which counts for around half of the rows.
Revised: Besides, we also observe that permutation appears 13 times in the top three augmentations, which counts for around half of the rows.

Original: Therefore, it is suggested to try permutation first for the related applications.
Revised: Therefore, it is suggested to try permutation first for the related applications.

Original: Performance evaluation of automatic and practical DA methods.
Revised: Performance evaluation of automatic and practical DA methods.

Original: The performance evaluation of automatic and random DA is displayed in the last two columns of Table.1, and we can conclude that I).
Revised: The performance evaluation of automatic and random DA is displayed in the last two columns of Table.1, and we can conclude that I).

Original: Random DA is both effective and efficient (R3) .
Revised: Random DA is both effective and efficient (R3) .

Original: As we can see from Table.1, the performance trained with random DA generally outperforms the baselines ( No Aug ), and are comparable with the best performance across different datasets and backbones.
Revised: As we can see from Table.1, the performance trained with random DA generally outperforms the baselines ( No Aug ), and are comparable with the best performance across different datasets and backbones.

Original: Therefore, random DA is an efficient, effective, and practical automated data augmentation method for biobehavioral time series data.
Revised: Therefore, random DA is an efficient, effective, and practical automated data augmentation method for biobehavioral time series data.

Original: II).
Revised: II).

Original: Automatic
Revised: Automatic

Original: DA outperforms most comparable baselines ( R1 ).
Revised: DA outperforms most comparable baselines ( R1 ).

Original: From the various possible DA combinations and their corresponding magnitudes, the automatic DA method helps us search for the more effective DA policies (operations and magnitudes).
Revised: From the various possible DA combinations and their corresponding magnitudes, the automatic DA method helps us search for the more effective DA policies (operations and magnitudes).

Original: As we may find that the automatic DA method ( Auto ) generally outperforms the baselines across different datasets and backbones, and achieves comparable or higher performance than the best performance, which is achieved by manually selected DAs.
Revised: As we may find that the automatic DA method ( Auto ) generally outperforms the baselines across different datasets and backbones, and achieves comparable or higher performance than the best performance, which is achieved by manually selected DAs.

Original: Quantitative measurement of the effectiveness for different DA methods. ( R4 )
Revised: Quantitative measurement of the effectiveness for different DA methods. ( R4 )

Original: We tried to answer the question of why some DA methods are more effective than others, and how can we quantitatively measure the effectiveness of different DA methods?
Revised: We tried to answer the question of why some DA methods are more effective than others, and how can we quantitatively measure the effectiveness of different DA methods?

Original: First, we summarized the desired attributes for effective DAs:
Revised: First, we summarized the desired attributes for effective DAs:

Revised: 1) challenging : the augmented data should be complex and strong enough that a deep model must learn useful representations to perform the task.

Original: 1.0 Faithful 0.
Revised: 1.0 Faithful 0.

Original: C h a ll e n g
Revised: C h a ll e n g

Original: i n g <- - - - -   2 R e l a t i v e T e s t
Revised: i n g <- - - - -   2 R e l a t i v e T e s t

Original: A cc u r a c
Revised: A cc u r a c

Original: y
Revised: y

Original: (a) PTB
Revised: (a) PTB

Original: [Figure]
Revised: [Figure]

Original: XL 0.{+0.
+}
Revised: XL {+0.
+}

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original:  0.9 1.0 Faithful
Revised: 0.8 0.9 1.0 Faithful

Original: C h a ll e n g
Revised: C h a ll e n g

Original: i n g <- - - - -
Revised: i n g <- - - - -

Original: R e l a t i v e
Revised: R e l a t i v e

Original: T e s t
Revised: T e s t

Original: A cc u r a c y
Revised: A cc u r a c y

Original: [Figure]
Revised: [Figure]

Original: |
Revised: |

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original:  C h a ll e n g i n g ----------
Revised: -- C h a ll e n g i n g ----------

Original: > H i g h
Revised: > H i g h

Original: [Figure]
Revised: [Figure]

Original: 1) challenging : the augmented data should be complex and strong enough that a deep model must learn useful representations to perform the task.

Original: 2) faithful : the DA must not make the task impossible, being so strong that they destroy all features of the input.
Revised: 2) faithful : the DA must not make the task impossible, being so strong that they destroy all features of the input.

Original: For example, the random noise added to jittering should not be so strong that makes learning impossible from the augmented data.
Revised: For example, the random noise added to jittering should not be so strong that makes learning impossible from the augmented data.

Original: Next, we use the metrics: C ( τ ) and F ( τ ) , to quantify how DA improves model’s performance by quantitatively measuring challenging and faithful for different DAs.
Revised: Next, we use the metrics: C ( τ ) and F ( τ ) , to quantify how DA improves model’s performance by quantitatively measuring challenging and faithful for different DAs.

Original: As shown in Fig.1 (c), the horizontal axis is used to measure how faithful of a DA, while vertical axis measures the level of challenge resulting from applying an augmentation.
Revised: As shown in Fig.1 (c), the horizontal axis is used to measure how faithful of a DA, while vertical axis measures the level of challenge resulting from applying an augmentation.

Original: Fig.1 (a) and (b) measure both faithful and challenging across 65 different augmentations for PTB-XL and PAMAP2 dataset.
Revised: Fig.1 (a) and (b) measure both faithful and challenging across 65 different augmentations for PTB-XL and PAMAP2 dataset.

Original: We find that many augmentations that dramatically decrease the performance have low value in faithful measurement and high value in challenging measurement (upper left).
Revised: We find that many augmentations that dramatically decrease the performance have low value in faithful measurement and high value in challenging measurement (upper left).

Original: On the other hand, many successful augmentations ( blue points ) lay in the area with high value in faithful , and for fixed (high) value of faithful , test accuracy generally increases with the increase of challenging .
Revised: On the other hand, many successful augmentations ( blue points ) lay in the area with high value in faithful , and for fixed (high) value of faithful , test accuracy generally increases with the increase of challenging .

Original: In summary, challenging and faithful together provide an explanation of an augmentation policy’s benefit to a model’s performance.
Revised: In summary, challenging and faithful together provide an explanation of an augmentation policy’s benefit to a model’s performance.

Original: In other words, effective DA is a trade-off between generating more diverse data that should be more difficult for a model to fit, while remaining faithful that the learning task is still possible from the augmented data.
Revised: In other words, effective DA is a trade-off between generating more diverse data that should be more difficult for a model to fit, while remaining faithful that the learning task is still possible from the augmented data.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we first conducted a comprehensive and systematic evaluation of various basic DAs on different biobehavioral datasets, finding that all the datasets can benefit from DA if using them carefully.
Revised: In this work, we first conducted a comprehensive and systematic evaluation of various basic DAs on different biobehavioral datasets, finding that all the datasets can benefit from DA if using them carefully.

Original: The effectiveness of an augmentation depended on both the dataset itself ( data type, task ) and the used backbone.
Revised: The effectiveness of an augmentation depended on both the dataset itself ( data type, task ) and the used backbone.

Original: Further, we explored to adopt more recent DA techniques for biobehavioral signals by designing a different policy architecture.
Revised: Further, we explored to adopt more recent DA techniques for biobehavioral signals by designing a different policy architecture.

Original: The results demonstrated that automatic DA can help learn effective policies but at a cost of high computational complexity, while random DA was demonstrated to be both effective and efficient during our experiments.
Revised: The results demonstrated that automatic DA can help learn effective policies but at a cost of high computational complexity, while random DA was demonstrated to be both effective and efficient during our experiments.

Original: At last, we attempted to answer the question of why data augmentation is effective ( or not ), by first summarizing two desired attributes ( challenging and faithful ), and then two we used two quantitative metrics to measure the corresponding attributes of DA.
Revised: At last, we attempted to answer the question of why data augmentation is effective ( or not ), by first summarizing two desired attributes ( challenging and faithful ), and then two we used two quantitative metrics to measure the corresponding attributes of DA.

Original: This can help understand why a DA works ( or not ), and guide us for the search of more effective DA methods in the future.
Revised: This can help understand why a DA works ( or not ), and guide us for the search of more effective DA methods in the future.

Original: We hope our experimental results could shed some light on DA for biobehavioral time series data, as a carefully selected DA could outperform many claimed improvements in the literature.
Revised: We hope our experimental results could shed some light on DA for biobehavioral time series data, as a carefully selected DA could outperform many claimed improvements in the literature.

Original: Our next plan is to investigate domain knowledge inspired DA, and also develop more efficient way to quantitatively measure the effectiveness of a DA method.
Revised: Our next plan is to investigate domain knowledge inspired DA, and also develop more efficient way to quantitatively measure the effectiveness of a DA method.

Revised: Acknowledgments

Revised: This work is supported by NSF #2047296 and #1840167.

Original: Appendix A Basic Data Augmentation Methods
Revised: Appendix A Basic Data Augmentation Methods

Original: Data augmentation methods rely on adding random transformations to the training data, and the transformations can be generally classified into three categories: magnitude-based, time-based, and frequency-based transformations.
Revised: Data augmentation methods rely on adding random transformations to the training data, and the transformations can be generally classified into three categories: magnitude-based, time-based, and frequency-based transformations.

Original: Magnitude-based transformations are applied to the wearable data along the variate or value axes.
Revised: Magnitude-based transformations are applied to the wearable data along the variate or value axes.

Original: Time-based transformations change the time steps, and frequencybased transformations warp the frequencies, respectively.
Revised: Time-based transformations change the time steps, and frequencybased transformations warp the frequencies, respectively.

Original: During our experiments, we will mainly focus on the magnitude and time-based transformations.
Revised: During our experiments, we will mainly focus on the magnitude and time-based transformations.

Original: Let T α denote an augmentation operations parameterized by α , given input data x , this procedure outputs augmented data ˆ x = T α ( x ) , where α controls the magnitude of the operation T .
Revised: Let T α denote an augmentation operations parameterized by α , given input data x , this procedure outputs augmented data ˆ x = T α ( x ) , where α controls the magnitude of the operation T .

Original: We consider a set of data augmentation methods drawn from the traditional time series processing literature.
Revised: We consider a set of data augmentation methods drawn from the traditional time series processing literature.

Original: Specifically, our augmentation set consists of eight operations: Jittering, Scaling, Rotation,
Revised: Specifically, our augmentation set consists of eight operations: Jittering, Scaling, Rotation,

Original: Permutation, Magnitude Warping, Time Warping, Window Slicing, and Window Warping .
Revised: Permutation, Magnitude Warping, Time Warping, Window Slicing, and Window Warping .

Original: Given an input data x = [ x 1 , x 2 , . . .
Revised: Given an input data x = [ x 1 , x 2 , . . .

Original: , x T ] with T , the number of time steps, and each element x t can be univariate or multivariate.
Revised: , x T ] with T , the number of time steps, and each element x t can be univariate or multivariate.

Original: Jittering.
Revised: Jittering.

Original: A random noise is added to the input data:
Revised: A random noise is added to the input data:

Original: [Equation]
Revised: [Equation]

Original: where ˆx is the augmented data, ϵ is a random noise ( i.e., Gaussian noise) added to each time step t .
Revised: where ˆx is the augmented data, ϵ is a random noise ( i.e., Gaussian noise) added to each time step t .

Original: Assume ϵ ∼ N (0 , σ 2 ) , and the standard deviation α = { σ } of the added noise is a hyper-parameter that needs to be pre-determined.
Revised: Assume ϵ ∼ N (0 , σ 2 ) , and the standard deviation α = { σ } of the added noise is a hyper-parameter that needs to be pre-determined.

Original: Scaling.
Revised: Scaling.

Original: The magnitude of the data in a window is changed by multiplying a random scalar.
Revised: The magnitude of the data in a window is changed by multiplying a random scalar.

Original: [Equation]
Revised: [Equation]

Original: (6) where ϵ can be determined by a Gaussian distribution ϵ ∼ N (1 , σ 2 ) with α = { σ } as a hyperparameter.
Revised: (6) where ϵ can be determined by a Gaussian distribution ϵ ∼ N (1 , σ 2 ) with α = { σ } as a hyperparameter.

Original: Rotation.
Revised: Rotation.

Original: Rotate each element by a random rotation matrix.
Revised: Rotate each element by a random rotation matrix.

Original: Although rotating data by a random angle can create plausible patterns for images, it might not be suitable for time series data.
Revised: Although rotating data by a random angle can create plausible patterns for images, it might not be suitable for time series data.

Original: A widely used alternative is flipping, which is defined as:
Revised: A widely used alternative is flipping, which is defined as:

Original: [Equation]
Revised: [Equation]

Original: Permutation.
Revised: Permutation.

Original: Perturb the location of the data in a single window.
Revised: Perturb the location of the data in a single window.

Original: It should be noted that permutation operation does not preserve time dependencies.
Revised: It should be noted that permutation operation does not preserve time dependencies.

Original: Permutation can be performed in two ways: equal sized segments and variable sized segments.
Revised: Permutation can be performed in two ways: equal sized segments and variable sized segments.

Original: First, the data x is split into N segments, each of the segment has a length of TN ( assuming equal sized segments ); then the location of those segments are randomly permuted.
Revised: First, the data x is split into N segments, each of the segment has a length of TN ( assuming equal sized segments ); then the location of those segments are randomly permuted.

Original: [Equation]
Revised: [Equation]

Original: where segment i ∗ is the i * -th segment of the input data x , and i ∗ ∈ [1 , N ] .
Revised: where segment i ∗ is the i * -th segment of the input data x , and i ∗ ∈ [1 , N ] .

Original: The number of segments α = { N } is a hyperparameter to be pre-determined.
Revised: The number of segments α = { N } is a hyperparameter to be pre-determined.

Original: Magnitude Warping.
Revised: Magnitude Warping.

Original: Change the magnitude of each sample by convolving the data window with a smooth curve.
Revised: Change the magnitude of each sample by convolving the data window with a smooth curve.

Original: [Equation]
Revised: [Equation]

Original: where γ 1 , γ 2 , . . .
Revised: where γ 1 , γ 2 , . . .

Original: γ T is a sequence created by interpolating a cubic spline function S ( u ) with knots u
Revised: γ T is a sequence created by interpolating a cubic spline function S ( u ) with knots u

Original: = u 1 , u 2 , . . .
Revised: = u 1 , u 2 , . . .

Original: u I .
Revised: u I .

Original: I is the number of knots and each knot u i is sampled from a Gaussian distribution
Revised: I is the number of knots and each knot u i is sampled from a Gaussian distribution

Original: N (1 , σ 2 ) , therefore, the operation parameters α = { σ, I } .
Revised: N (1 , σ 2 ) , therefore, the operation parameters α = { σ, I } .

Original: Time Warping.
Revised: Time Warping.

Original: Perturb the temporal location by smoothly distorting the time intervals between samples, which is similar to the magnitude warping operation and defined as:
Revised: Perturb the temporal location by smoothly distorting the time intervals between samples, which is similar to the magnitude warping operation and defined as:

Original: [Equation]
Revised: [Equation]

Original: (10) where γ ( · ) : i → j, and i, j ∈ [1 , T ] is a warping function that warps the time steps based on a smooth curve.
Revised: (10) where γ ( · ) : i → j, and i, j ∈ [1 , T ] is a warping function that warps the time steps based on a smooth curve.

Original: The smooth curve is defined by a cubic spline S ( u ) , which is exactly the same as used in the magnitude warping operation, and the operation parameters α = { σ, I } .
Revised: The smooth curve is defined by a cubic spline S ( u ) , which is exactly the same as used in the magnitude warping operation, and the operation parameters α = { σ, I } .

Original: Window Slicing.
Revised: Window Slicing.

Original: Slice time steps off the ends of the pattern, which is equivalent to cropping for image data augmentation.
Revised: Slice time steps off the ends of the pattern, which is equivalent to cropping for image data augmentation.

Original: The operation is defined as:
Revised: The operation is defined as:

Original: [Equation]
Revised: [Equation]

Original: where 0 ≤ W ≤ T is the size of a window that needs to be pre-determined, and δ is a random integer such that 0 ≤ δ ≤ T − W , and the operation parameters α = { W } .
Revised: where 0 ≤ W ≤ T is the size of a window that needs to be pre-determined, and δ is a random integer such that 0 ≤ δ ≤ T − W , and the operation parameters α = { W } .

Original: Window Warping.
Revised: Window Warping.

Original: Randomly select a window with the length of W from the time series and stretch it by K or contract it by 1 K .
Revised: Randomly select a window with the length of W from the time series and stretch it by K or contract it by 1 K .

Original: Linear interpolation is used for other part of the time series, so that the output will have equal length to the input.
Revised: Linear interpolation is used for other part of the time series, so that the output will have equal length to the input.

Original: Note that, in this paper we only consider K = 2 , but other ratios could be used as well.
Revised: Note that, in this paper we only consider K = 2 , but other ratios could be used as well.

Original: The length of window is a hyper-parameter to be pre-determined and α = { W }
Revised: The length of window is a hyper-parameter to be pre-determined and α = { W }

Original: With different operations and their corresponding magnitude parameters α , the generated augmented data are also different.
Revised: With different operations and their corresponding magnitude parameters α , the generated augmented data are also different.

Original: As shown in Fig.2, the two rows represent the examples of the augmented data where the eight basic data augmentation methods are applied to the same input with two different magnitude parameters α for each operation.
Revised: As shown in Fig.2, the two rows represent the examples of the augmented data where the eight basic data augmentation methods are applied to the same input with two different magnitude parameters α for each operation.

Original: B Datasets
Revised: B Datasets

Original: PTB-XL: The PTB-XL [31] dataset is a large dataset containing 21,837 clinical 12-lead electrocardiogram (ECG) records from 18,885 patients of 10 second length, where 52% are male and 48% are female with ages range from 0 to 95 years (median 62 and interquantile range of 22).
Revised: PTB-XL: The PTB-XL [31] dataset is a large dataset containing 21,837 clinical 12-lead electrocardiogram (ECG) records from 18,885 patients of 10 second length, where 52% are male and 48% are female with ages range from 0 to 95 years (median 62 and interquantile range of 22).

Original: There are two sampling rates: 100 Hz and 500 Hz, available in the dataset, but in our experiments, only data sampled at 100 Hz are used.
Revised: There are two sampling rates: 100 Hz and 500 Hz, available in the dataset, but in our experiments, only data sampled at 100 Hz are used.

Original: The raw ECG data are annotated by two cardiologists into five major categories, including normal ECG (NORM), myocardial infarction (MI), ST/T Change (STTC), Conduction
Revised: The raw ECG data are annotated by two cardiologists into five major categories, including normal ECG (NORM), myocardial infarction (MI), ST/T Change (STTC), Conduction

Original: Disturbance (CD) and Hypertrophy (HYP).
Revised: Disturbance (CD) and Hypertrophy (HYP).

Original: The dataset contains a comprehensive collection of various co-occurring pathologies and a large proportion of healthy control samples.
Revised: The dataset contains a comprehensive collection of various co-occurring pathologies and a large proportion of healthy control samples.

Original: We experimented classifying all 5 cardiac conditions as learning tasks.
Revised: We experimented classifying all 5 cardiac conditions as learning tasks.

Original: Further, to ensure a fair comparison of machine learning algorithms trained on the dataset, we follow the recommended splits of training and test sets, which results in a training/testing ratio of 8/1.
Revised: Further, to ensure a fair comparison of machine learning algorithms trained on the dataset, we follow the recommended splits of training and test sets, which results in a training/testing ratio of 8/1.

Original: Apnea-ECG: The Apnea-ECG [22] dataset studies the relationship between human sleep apnea symptoms and heart activities (monitored by ECG).
Revised: Apnea-ECG: The Apnea-ECG [22] dataset studies the relationship between human sleep apnea symptoms and heart activities (monitored by ECG).

Original: This database can be accessed through Physionet[8].
Revised: This database can be accessed through Physionet[8].

Original: This dataset contains 70 records with a sampling rate of 100 Hz, from where 35 records were divided into training, and the other 35 were divided into the test set.
Revised: This dataset contains 70 records with a sampling rate of 100 Hz, from where 35 records were divided into training, and the other 35 were divided into the test set.

Original: The duration of the records varies from slightly less than 7 hours to nearly 10 hours.
Revised: The duration of the records varies from slightly less than 7 hours to nearly 10 hours.

Original: The labels were the annotation of each minute of each recording indicating the presence or absence of sleep apnea.
Revised: The labels were the annotation of each minute of each recording indicating the presence or absence of sleep apnea.

Original: Thus, we split the ECG recording into each minute, which was a total of 6000 data points for each separation.
Revised: Thus, we split the ECG recording into each minute, which was a total of 6000 data points for each separation.

Original: We extracted 17233 samples for the training set and 17010 samples for the test set.
Revised: We extracted 17233 samples for the training set and 17010 samples for the test set.

Original: And the ratio of non-apnea and apnea samples in the training set was 61.49% to 38.51%.
Revised: And the ratio of non-apnea and apnea samples in the training set was 61.49% to 38.51%.

Original: Sleep-EDFE: The Sleep-EDF (expanded) [13] dataset contains whole-night sleep recordings from 822 subjects with physiological signals and sleep stages that were annotated manually by well-trained technicians.
Revised: Sleep-EDFE: The Sleep-EDF (expanded) [13] dataset contains whole-night sleep recordings from 822 subjects with physiological signals and sleep stages that were annotated manually by well-trained technicians.

Original: In this dataset, the physiological signals, including Fpz-Cz/Pz-Oz electroencephalogram (EEG), electrooculogram (EOG), and chin electromyogram (EMG), were sampled at 100 Hz.
Revised: In this dataset, the physiological signals, including Fpz-Cz/Pz-Oz electroencephalogram (EEG), electrooculogram (EOG), and chin electromyogram (EMG), were sampled at 100 Hz.

Original: We targeted to detect 5 sleep stages.
Revised: We targeted to detect 5 sleep stages.

Original: including wakefulness, stage N1, N2, N3, and REM [3].
Revised: including wakefulness, stage N1, N2, N3, and REM [3].

Original: To model the relationship between the sleep patterns and physiological data, we split the whole-night recordings into 30-second Fpz-Cz ECG segments as in [28], which resulted in a total of 42308 ECG and sleep pattern pairs.
Revised: To model the relationship between the sleep patterns and physiological data, we split the whole-night recordings into 30-second Fpz-Cz ECG segments as in [28], which resulted in a total of 42308 ECG and sleep pattern pairs.

Original: We divided 25% of the samples into a testing set according to the order of the subject
Revised: We divided 25% of the samples into a testing set according to the order of the subject

Original: IDs.
Revised: IDs.

Original: MMIDB-EEG: The MMIDB-EEG dataset [27] studies the relationship between physiological EEG and human body physical/imaginary movement.
Revised: MMIDB-EEG: The MMIDB-EEG dataset [27] studies the relationship between physiological EEG and human body physical/imaginary movement.

Original: This dataset contains over 1,500 EEG recordings in 1-2 minutes with a sampling rate of 160Hz from 109 subjects.
Revised: This dataset contains over 1,500 EEG recordings in 1-2 minutes with a sampling rate of 160Hz from 109 subjects.

Original: Each subject performed baseline (eyes open and close) and four tasks, including open and close left or right fist, imagine opening and closing left or right fist, open and close both fists or feet, and imagine opening and closing both fists or both feet.
Revised: Each subject performed baseline (eyes open and close) and four tasks, including open and close left or right fist, imagine opening and closing left or right fist, open and close both fists or feet, and imagine opening and closing both fists or both feet.

Original: Following [26], we omit the data from 6 subjects due to the incorrect annotations and split the remaining data into 4s segments, which results in 4635 segments in total.
Revised: Following [26], we omit the data from 6 subjects due to the incorrect annotations and split the remaining data into 4s segments, which results in 4635 segments in total.

Original: Our task focuses on the classification between baseline and physical hand movement.
Revised: Our task focuses on the classification between baseline and physical hand movement.

Original: Further, data from 22 subjects, based on the order of subject ids, is split into the test set, and the rest samples are employed as the training set.
Revised: Further, data from 22 subjects, based on the order of subject ids, is split into the test set, and the rest samples are employed as the training set.

Original: CLAS:
Revised: CLAS:

Original: The CLAS dataset [19] aims to support research on the automated assessment of certain states of mind and emotional conditions using physiological data.
Revised: The CLAS dataset [19] aims to support research on the automated assessment of certain states of mind and emotional conditions using physiological data.

Original: The dataset consists of synchronized
Revised: The dataset consists of synchronized

Original: [Table]
Revised: [Table]

Original: recordings of ECG, photopletysmogram (PPG), electrodermal activity (EDA), and acceleration (ACC) signals.
Revised: recordings of ECG, photopletysmogram (PPG), electrodermal activity (EDA), and acceleration (ACC) signals.

Original: There are 62 healthy subjects who participated and were involved in three interactive tasks and two perceptive tasks.
Revised: There are 62 healthy subjects who participated and were involved in three interactive tasks and two perceptive tasks.

Original: The perceptive tasks, which leveraged the images and audio-video stimuli, were purposely selected to evoke emotions in the four quadrants of arousal-valence space.
Revised: The perceptive tasks, which leveraged the images and audio-video stimuli, were purposely selected to evoke emotions in the four quadrants of arousal-valence space.

Original: In this study, our goal was to use the EDA signal to detect binary high/low stress states that are annotated in arousal-valence space.
Revised: In this study, our goal was to use the EDA signal to detect binary high/low stress states that are annotated in arousal-valence space.

Original: We processed the raw EDA data with a lowpass Butterworth filter with a cutoff frequency of 0.2 Hz, then split the sequences into 10-second segments.
Revised: We processed the raw EDA data with a lowpass Butterworth filter with a cutoff frequency of 0.2 Hz, then split the sequences into 10-second segments.

Original: We divided the train/test set in a subject-independent manner and utilized the data from 17 subjects as the test set according to subject ids ( > 45)
Revised: We divided the train/test set in a subject-independent manner and utilized the data from 17 subjects as the test set according to subject ids ( > 45)

Original: [Table]
Revised: [Table]

Original: PAMAP2:
Revised: PAMAP2:

Original: The PAMAP2 [23] physical activity monitoring dataset consists data of 18 different physical activities, including household activities (sitting, walking, standing, vacuum cleaning, ironing, etc) and a variety of exercise activities (Nordic walking, playing soccer, rope jumping, etc), performed by 9 participants wearing three inertial measurement units (IMU) and a heart rate monitor.
Revised: The PAMAP2 [23] physical activity monitoring dataset consists data of 18 different physical activities, including household activities (sitting, walking, standing, vacuum cleaning, ironing, etc) and a variety of exercise activities (Nordic walking, playing soccer, rope jumping, etc), performed by 9 participants wearing three inertial measurement units (IMU) and a heart rate monitor.

Original: Accelerometer, gyroscope, magnetometer and temperature data are recorded from the 3 IMUs placed on three different locations (1 IMU over the wrist on the dominat arm, 1 IMU on the chest and
Revised: Accelerometer, gyroscope, magnetometer and temperature data are recorded from the 3 IMUs placed on three different locations (1 IMU over the wrist on the dominat arm, 1 IMU on the chest and

Original: IMU on the dominant side’s ankle) with sampling frequency 100Hz.
Revised: IMU on the dominant side’s ankle) with sampling frequency 100Hz.

Original: Heart rate data are recorded from the heart rate monitor with sampling frequency 9Hz.
Revised: Heart rate data are recorded from the heart rate monitor with sampling frequency 9Hz.

Original: The resulting dataset has 52 dimensions (3 x 17 (IMU) + 1 (heart rate) = 52).
Revised: The resulting dataset has 52 dimensions (3 x 17 (IMU) + 1 (heart rate) = 52).

Original: Following the same setting as used in [20, 29], we linearly interpolated the missing data (upsampling the sampling frequency from 9Hz to 100Hz for heart rate), then took random 10s windows from subject recordings with an overlap of 7s, using the same train/validation/test splits.
Revised: Following the same setting as used in [20, 29], we linearly interpolated the missing data (upsampling the sampling frequency from 9Hz to 100Hz for heart rate), then took random 10s windows from subject recordings with an overlap of 7s, using the same train/validation/test splits.

Original: As mentioned in [20, 29], 12 of the total 18 different physical activities are used in the experiments.
Revised: As mentioned in [20, 29], 12 of the total 18 different physical activities are used in the experiments.

Original: UCI-HAR: The UCI-HAR [24] dataset was collected from a group of 30 volunteers with an age range from 19 to 48 years.
Revised: UCI-HAR: The UCI-HAR [24] dataset was collected from a group of 30 volunteers with an age range from 19 to 48 years.

Original: During the data collection, all the subjects wore a smartphone (Samsung
Revised: During the data collection, all the subjects wore a smartphone (Samsung

Original: Galaxy S II) with embedded inertial sensors around their waist and were instructed to follow an activity protocol performing six basic activities, including three static postures (standing, sitting, lying) and three dynamic activities (walking, walking downstairs and walking upstairs).
Revised: Galaxy S II) with embedded inertial sensors around their waist and were instructed to follow an activity protocol performing six basic activities, including three static postures (standing, sitting, lying) and three dynamic activities (walking, walking downstairs and walking upstairs).

Original: The dataset also included postural transitions that occurred between the static postures, but they were discarded in our experiments (also in related works), due to a much smaller size for those transitions.
Revised: The dataset also included postural transitions that occurred between the static postures, but they were discarded in our experiments (also in related works), due to a much smaller size for those transitions.

Original: 3-axial linear acceleration and 3-axial angular velocity were captured using the embedded accelerometer and gyroscope of the smartphone at a constant rate of 50Hz.
Revised: 3-axial linear acceleration and 3-axial angular velocity were captured using the embedded accelerometer and gyroscope of the smartphone at a constant rate of 50Hz.

Original: Following the authors’ suggestion, the sensor signals were sampled in a fixed-width sliding windows of 2.56 second and 50% overlap, resulting in 128 readings
Revised: Following the authors’ suggestion, the sensor signals were sampled in a fixed-width sliding windows of 2.56 second and 50% overlap, resulting in 128 readings

Original: [Table]
Revised: [Table]

Original: C Details ofUsed Data Augmentation Methods  C.1 Random Augmentation
Revised: C Details of Data Augmentation Methods Used in This Paper C.1 Random Augmentation

Original: The final algorithm is defined as: randaugment ( J, M ) , where J is the number of augmentation transformations to be selected from the K operation pool, M is the magnitude for all the transformations, then the J operations with magnitude M will be sequentially applied to the input data.
Revised: The final algorithm is defined as: randaugment ( J, M ) , where J is the number of augmentation transformations to be selected from the K operation pool, M is the magnitude for all the transformations, then the J operations with magnitude M will be sequentially applied to the input data.

Original: C.2 Differentiable Data Augmentation
Revised: C.2 Differentiable Data Augmentation

Original: A collection of DA policies contains K sub-policies P k , k ∈ [1 , K ] , and each sub-policy P k includes
Revised: A collection of DA policies contains K sub-policies P k , k ∈ [1 , K ] , and each sub-policy P k includes

Original: N basic data augmentation operations ( i.e., jittering, scaling, rotation ) that are applied to input signal sequentially.
Revised: N basic data augmentation operations ( i.e., jittering, scaling, rotation ) that are applied to input signal sequentially.

Original: Each operation can be represented as O ik (cid:0) x ; p ik , m ik (cid:1) , k ∈ [1 , K ] , i ∈ [0 , J ] , where
Revised: Each operation can be represented as O ik (cid:0) x ; p ik , m ik (cid:1) , k ∈ [1 , K ] , i ∈ [0 , J ] , where

Original: K is the total number of sub-policies, J is the operations included in each sub-policy, p nk is the probability of applying the operation and m nk represents the magnitude of the operation.
Revised: K is the total number of sub-policies, J is the operations included in each sub-policy, p nk is the probability of applying the operation and m nk represents the magnitude of the operation.

Original: The objective is to jointly optimize a model’s parameter θ and also the α = { p, m } , thus the trained deep models can generalize well on the testing dataset.
Revised: The objective is to jointly optimize a model’s parameter θ and also the α = { p, m } , thus the trained deep models can generalize well on the testing dataset.

Original: Due to the page limits, readers are encouraged to check more details in the original paper of DADA [15].
Revised: Due to the page limits, readers are encouraged to check more details in the original paper of DADA [15].

Original: The details of the training steps for the automatic differentiable data augmentation method is illustrated in Algorithm.1, where L ( θ |D ) represents the cross-entropy loss on the data D .
Revised: The details of the training steps for the automatic differentiable data augmentation method is illustrated in Algorithm.1, where L ( θ |D ) represents the cross-entropy loss on the data D .

Original: D Implementation Details
Revised: D Implementation Details

Original: We use three deep learning architectures, including MLP, Conv-1d, and ResNet-1d.
Revised: We use three deep learning architectures, including MLP, Conv-1d, and ResNet-1d.

Original: These networks were chosen due to being effective in time series data and also being used in a wide range of biomedical applications.
Revised: These networks were chosen due to being effective in time series data and also being used in a wide range of biomedical applications.

Original: We use an Adam optimizer with initial learning rate of 1 × 10 − 3 , and the learning rate is decayed by 0 .
Revised: We use an Adam optimizer with initial learning rate of 1 × 10 − 3 , and the learning rate is decayed by 0 .

Original: 9 after every 5 epoches.
Revised: 9 after every 5 epoches.

Original: The batch size is 100, and we train the model for 50 epochs.
Revised: The batch size is 100, and we train the model for 50 epochs.

Original: Our model is implemented in the PyTorch deep learning framework, and is trained and tested on the NVIDIA
Revised: Our model is implemented in the PyTorch deep learning framework, and is trained and tested on the NVIDIA

Original: GeForce 3090Ti GPU.
Revised: GeForce 3090Ti GPU.

Original: To evaluate the performance, average accuracy of three runs is reported.
Revised: To evaluate the performance, average accuracy of three runs is reported.

Original: For fair comparison, we use the default value as magnitude for different basic augmentations on different datasets, therefore we can guarantee that the varied performance is caused by augmentation rather than fine-tuning.
Revised: For fair comparison, we use the default value as magnitude for different basic augmentations on different datasets, therefore we can guarantee that the varied performance is caused by augmentation rather than fine-tuning.

Original: For random data augmentation, we set J = 2 , K = 8 and magnitude M = 12 during our experiments.
Revised: For random data augmentation, we set J = 2 , K = 8 and magnitude M = 12 during our experiments.

Original: For automatic data augmentation, K = 14 sub-policies are randomly generated from the 8 basic data augmentation methods, and each sub-policy contains J = 2 operations.
Revised: For automatic data augmentation, K = 14 sub-policies are randomly generated from the 8 basic data augmentation methods, and each sub-policy contains J = 2 operations.

Original: E Model Architecture
Revised: E Model Architecture

Original: The detailed information of the Conv-1D and ResNet-1D are illustrated in Table.
Revised: The detailed information of the Conv-1D and ResNet-1D are illustrated in Table.

Original: 5 and Table.
Revised: 4 and Table.

Original: Input: Training set D
Revised: Input: Training set D

Original: = { x i , y i } Ni =1 , D valid = { x i , y i } Mi =1 , training epochs E, batch size B,
Revised: = { x i , y i } Ni =1 , D valid = { x i , y i } Mi =1 , training epochs E, batch size B,

Original: Model parameters θ , DA  parameters T α , learning rate ξ 1 , ξ 2 and ϵ
Revised: Model parameters θ , DA policy parameters T α , learning rate ξ 1 , ξ 2 and ϵ

Original: Output: Model Parameters θ and DA parameters α 1 for epoch ← 1 to E do for batch ← 1 to max_iter do //Randomly sample batches
Revised: Output: Model Parameters θ and DA parameters α 1 for epoch ← 1 to E do for batch ← 1 to max_iter do //Randomly sample batches

Original: D
Revised: D

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: F Supplementary Results F.1 The effect of different number of operations and different value of magnitude in random DA
Revised: F Supplementary Results F.1 The effect of different number of operations and different value of magnitude in random DA

Original: The random DA method achieves improved or comparable performances across different tasks and datasets using the fixed number of transformations and fixed value of magnitude.
Revised: The random DA method achieves improved or comparable performances across different tasks and datasets using the fixed number of transformations and fixed value of magnitude.

Original: To further study the sensitivity of random DA to the selection of transformations and magnitude, we run experiments of random DA with different number of operations and magnitude on the PTB-XL dataset.
Revised: To further study the sensitivity of random DA to the selection of transformations and magnitude, we run experiments of random DA with different number of operations and magnitude on the PTB-XL dataset.

Original: The results in Fig.3 (a) ( fixed M ) suggest that the random DA improves performance as the number of operations is increased, even with only 1 operation ( J = 1 ).
Revised: The results in Fig.3 (a) ( fixed M ) suggest that the random DA improves performance as the number of operations is increased, even with only 1 operation ( J = 1 ).

Original: However, the performance is dropped once J > 4 , which is potentially caused by the fact that the augmented data is too challenging for the model to learn anything meaningful after J consecutive augmentations, therefore, leading to dropped performance.
Revised: However, the performance is dropped once J > 4 , which is potentially caused by the fact that the augmented data is too challenging for the model to learn anything meaningful after J consecutive augmentations, therefore, leading to dropped performance.

Original: A constant magnitude M sets the distortion magnitude to a constant number during training.
Revised: A constant magnitude M sets the distortion magnitude to a constant number during training.

Original: To validate the impact of different magnitudes, we run experiments with fixed J = 2 and varied numbers in terms of magnitude.
Revised: To validate the impact of different magnitudes, we run experiments with fixed J = 2 and varied numbers in terms of magnitude.

Original: The results in Fig.3 (b) suggest that the model is not very sensitive to the change of magnitude ( except MLP, as it is a relatively weak backbone ).
Revised: The results in Fig.3 (b) suggest that the model is not very sensitive to the change of magnitude ( except MLP, as it is a relatively weak backbone ).

Original: That is because first, the magnitude is limited in a small reasonable range for all the operations ( details in supplementary materials ); and second, the composition of J random consecutive operations could potentially alleviate the sensitivity to the different magnitudes.
Revised: That is because first, the magnitude is limited in a small reasonable range for all the operations ( details in supplementary materials ); and second, the composition of J random consecutive operations could potentially alleviate the sensitivity to the different magnitudes.

Original: [Figure]
Revised: [Figure]

Original: F.2 Visualization of policies in automatic DA
Revised: F.2 Visualization of policies in automatic DA

Original: An illustration of the development of probability ( normalized ) for individual DA policy over training epochs is shown in Fig.4, where all the 14 sub-policies start from the same probability, and change with the training epochs.
Revised: An illustration of the development of probability ( normalized ) for individual DA policy over training epochs is shown in Fig.4, where all the 14 sub-policies start from the same probability, and change with the training epochs.

Original: For example, the probability is gradually increased for sub-policy_1, sub-policy_5 and sub-policy_14 , indicting those policies are more effective than others.
Revised: For example, the probability is gradually increased for sub-policy_1, sub-policy_5 and sub-policy_14 , indicting those policies are more effective than others.

Original: Although automatic differentiable DA shows improved performance, it comes at a cost in terms of computational complexity, which is roughly four times than the corresponding baseline models.
Revised: Although automatic differentiable DA shows improved performance, it comes at a cost in terms of computational complexity, which is roughly four times than the corresponding baseline models.