## 4 Our Solution: DeepChaos+
### End-to-End Learning Objective  
Mathematically, we further decompose the Variational Autoencoder (VAE) model \(\mathcal{F}_{\theta}\) into two models: the encoder denoted as \(\mathcal{G}_{\psi}\) and the decoder denoted as \(\mathcal{M}_{\phi}\). Formally, we have:  
\[\mathcal{F}_{\theta}=\mathcal{G}_{\psi}\circ\mathcal{M}_{\phi}, \tag{7}\]  
\[\mathcal{X}^{train}=\mathcal{F}_{\theta}(\mathcal{X}^{train})=\mathcal{M}_{ \phi}\Big{(}\mathcal{G}_{\psi}(\mathcal{X}^{train})\Big{)}=\mathcal{M}_{ \phi}(\mathcal{Z}). \tag{8}\]  
Figure 2: Overview of the DeepChaos+ framework. The framework introduces two key models: the Variational Autoencoder (VAE) and the lightweight Informer Network. The VAE is trained to generate interpolated data from the set \(\mathcal{X}\). The generated data are then combined with the dataset \(\mathcal{D}\) and used to iteratively retrain the VAE. The lightweight Informer Network, with fewer parameters but functionality equivalent to the VAE’s decoder, is trained to predict a set \(\hat{\mathcal{X}}\) that minimizes the bit error rate \(\mathcal{B}(\hat{\mathcal{X}},\mathcal{X})\). Knowledge Distillation is employed to ensure the Informer achieves similar performance to the decoder while enabling faster inference time.  
The encoder \(\mathcal{G}\psi\) maps \(\mathcal{X}^{train}\) to a latent space \(\mathcal{Z}\), and the decoder \(\mathcal{M}\phi\) reconstructs \(\mathcal{X}^{train}\) from \(\mathcal{Z}\). The VAE assumes a latent variable \(\mathcal{Z}\in\mathbb{R}^{1\times v}\), with \(v\) as the latent space dimension. This latent variable captures the features of the original signal and follows a latent distribution \(p_{\phi}(\mathcal{Z})\). The complete generative process can be described by:  
\[p_{\phi}(\mathcal{Z}\mid\mathcal{X})=\frac{p_{\phi}(\mathcal{X}\mid\mathcal{ Z})p_{\phi}(\mathcal{Z})}{p_{\phi}(\mathcal{X})}. \tag{9}\]  
To approximate the intractable posterior distribution \(p_{\phi}(\mathcal{Z}\mid\mathcal{X})\), the model \(\mathcal{G}_{p}si\) learns a simpler distribution \(q_{\psi}(\mathcal{Z}\mid\mathcal{X})\). The objective is to have \(p_{\phi}(\mathcal{Z}\mid\mathcal{X})\approx q_{\psi}(\mathcal{Z}\mid\mathcal{ X})\), which is achieved by minimizing the KL divergence \(D_{KL}(q_{\psi}\|p_{\phi})\). This is equivalent to maximizing the evidence lower bound objective (ELBO):  
\[\mathcal{L}^{\text{ELBO}}=\mathbb{E}_{q_{\psi}}\big{[}\log p_{\phi}(\mathcal{ X}\mid\mathcal{Z})\big{]}-\mathbb{E}_{q_{\psi}}\bigg{[}\log\frac{q_{\psi}( \mathcal{Z}\mid\mathcal{X})}{p_{\phi}(\mathcal{Z})}\bigg{]}. \tag{10}\]  
The ELBO includes the expected reconstruction error \(\log p_{\phi}(\mathcal{X}\mid\mathcal{Z})\) learned by the decoder model. The DeepChaos+ framework introduces a student model \(f_{\omega}^{\text{student}}\) trained on augmented data \(\mathcal{D}\) generated by the VAE (teacher model). The student model is trained using a mean squared error (MSE) loss, defined as:  
\[\mathcal{L}^{\text{MSE}}=\frac{1}{N}\sum_{i=1}^{N}\left|\mathcal{\hat{X}}^{(i )}-\mathcal{\hat{X}}_{\xi}^{(i)}\right|_{2}^{2}, \tag{11}\]  
where \(N\) is the number of samples in the dataset \(\mathcal{D}\). The overall objective function for training the student model combines the ELBO loss and the MSE loss:  
\[\mathcal{L}^{\text{total}}=\lambda\mathcal{L}^{\text{ELBO}}+(1-\lambda). \mathcal{L}^{\text{MSE}} \tag{12}\]  
Here, \(\lambda\) is a hyperparameter that balances the trade-off between the ELBO and MSE losses. By optimizing this combined loss function, DeepChaos+ effectively learns from the synthetic data, enhancing the student model's ability to predict the original signals. This approach addresses the challenge of limited original signal data by generating synthetic data that capture the essential features of the original signals, leading to more accurate predictions by the student model.