# Individualized Stress Mobile Sensing Using Self-Supervised Pre-Training
## 3 Methodology
### Self-Supervised Pre-Training  
The use of self-supervised pre-training, an unsupervised approach in which data are self-labeled and, subsequently, fine-tuned using supervised learning approaches to extract important feature representations, has proven to be more effective than supervised training alone in an increasing number of applications [19]. The self-supervised pre-training approach focuses on learning a high-level representation of data without using ground truth labels, which is also known as the pretext task. A pre-trained model enables efficient transfer to a target task since the representation it has learned is already suited (or readily transferable) for a given job. SSL in this manner is the same technology that powers large-scale language models like ChatGPT and natural language training tasks powered by BERT [30]. We pre-trained our model through forecasting, a similar technique used to pre-train OpenAI's GPT language model [31]. A separate model was pre-trained for each human subject using only the data from that subject. To pre-train our model, we employed a 1D CNN, a standard neural network architecture for signal prediction [32]. Prior research has demonstrated the robustness of the 1D CNN in 1D signal processing tasks, particularly in the domain of biosignals or 1D signals [6]. One of the key advantages of 1D CNNs is their ability to automatically learn relevant features from the input signal, eliminating the need for manual feature engineering. This can save time and effort while also improving the model's performance, as it can adaptively learn the most-discriminative features for a given task. Moreover, 1D CNNs can capture information at different time scales due to the use of multiple neural layers and multiple independent learned kernels. This makes this family of architectures particularly helpful when dealing with signals with complicated hierarchical patterns or events that occur at different time scales. Additionally, 1D CNNs can handle input data with noise better than other machine learning algorithms, as the convolutional layers can learn to focus on the most-important features and ignore noise or irrelevant signals. Figure 2 illustrates the architecture that was used in this study for pre-training the model as a knowledge representation of the raw EDA signal. To maintain consistency, we used the same model architecture for both self-supervised pre-training and supervised training. We used the root-mean-squared error (RMSE) as our evaluation metric. Figure 2: The 1D CNN architecture for the pretext task. The pretext 1D CNN model was implemented with an input shape of (7000, 1), consisting of four 1D convolution layers, each utilizing a leaky ReLU activation function. The first convolution layer has 40 filters, followed by a layer with 30 filters, and then, a layer with 18 filters. The fourth convolution layer has 30 filters and uses a leaky ReLU activation function. Moreover, the model includes two dense layers, one with 70 nodes and the other with 30 nodes. The final layer is an output layer with a linear activation function. The convolution layers are responsible for extracting important features from the EDA signal, while the dense layers utilize these features to make a stress prediction. The model was pre-trained on the entirety of the EDA signal for each participant. We divided the whole signal into fixed windows (each containing a 7000-dimensional vector) (Figure 3). Figure 3 illustrates an example of the data points in the training set with overlapping information. We represent the signal as \(X_{1}\cdot\cdot\cdot\cdot\cdot\cdot X_{n}\). Our method segments this signal into multiple fixed windows, each of length 7000 points. So, the first window captures the sequence \(X_{1},X_{2}\cdot\cdot\cdot\cdot\cdot X_{7000}\). The purpose of this window is to predict the next 40 points, i.e., \(X_{7001},X_{7002}\cdot\cdot\cdot\cdot\cdot X_{7040}\). However, instead of progressing to a completely new window after this, we introduced overlap.