# On the Quality of Deep Representations for Kepler Light Curves Using Variational Auto-Encoders
## 2 Materials and Methods
### Extending VRAE to Include Temporal Information
#### 2.2.3 VRAE with Embedded Re-Scaling
Considering a random variable \(a\), the forward pass to obtain a new value \(b\) is given by  
\[b=Norm_{L}(a)=\frac{\log a-\text{mean}(\log a)}{\text{std}(\log a)}. \tag{2}\]  
The mean and standard deviation (std) are previously computed over the whole dataset. Furthermore, another layer is introduced to reverse this transformation, given by  
\[a=RevNorm_{L}(b)=\exp(b\cdot\text{std}(\log a)+\text{mean}(\log a))=Norm_{L} ^{-1}(b). \tag{3}\]  
Compared against the VRAE\({}_{t}\) on Algorithm 1, a pseudo-code of the S-VRAE\({}_{t}\) forward pass is presented on Algorithm 2. Here, it can be seen that the main differences are the re-scaling process inside the model and the additional input pattern to be used. The first thing to formalize is that \(g_{w}(\cdot)\) and \(f_{w}(\cdot)\) are non-linear functions (i.e., deep learning models) parameterized by the weights \(w\). The \(E(\cdot)\), which stands for the _embedding_ function, corresponds to the first layers of a deep learning model. Inside \(f_{\phi}(\cdot)\), the \(E^{1}(\cdot)\) is an RNN model and \(E^{2}(\cdot)\) is a multi-layer perceptron (MLP) model with \(Norm_{L}(\cdot)\) as the first layer. Here, \(E^{1}(\cdot)\) codifies the information from the noisy standardized time series, while \(E^{2}\) codifies the _F-std_. On the decoder phase, the \(g^{1}_{\theta}(\cdot)\) is similar to a mirror model of \(E^{1}(\cdot)\), while reversing what \(E^{2}(\cdot)\) does, the last layer of \(g^{2}_{\theta}(\cdot)\) is \(RevNorm_{L}(\cdot)\). Then, \(g^{1}_{\theta}(\cdot)\) reconstructs the standardized version of the time series and \(g^{2}_{\theta}(\cdot)\) reconstruct the _F-std_. The architecture of the proposed S-VRAE\({}_{t}\) is illustrated in Figure 3. Please note that the Encode and Reconstruct blocks are the same as the RAE\({}_{t}\) in Figure 2 but without the _F-std_ transformations. In summary, S-VRAE\({}_{t}\) learns a coded deep representation of the noisy un-scaled time series in an unsupervised way. The advantage is that those features are optimized for the specific input behavior (rather than classification) and are more disentangled (more independent between each other) than human-crafted counterparts. The evidence for this claim can be found in Section 3.3. ```
0:\(\vec{x}^{(i)}\)-- measurements of the time series \(\vec{\beta}^{(i)}\)-- delta times of the time series Output:\(\vec{\hat{x}}^{(i)}_{s}\)-- reconstructed scaled time series
1:// Encode to distribution:
2:\(\vec{\mu}^{(i)}\gets f^{1}_{\phi}\Big{(}E^{1}(\vec{x}^{(i)}_{s},\vec{\delta} ^{(i)})\Big{)}\)
3:\(\vec{\sigma}^{(i)}\gets f^{2}_{\phi}\Big{(}E^{1}(\vec{x}^{(i)}_{s},\vec{ \delta}^{(i)})\Big{)}\)
4:\(\vec{\varepsilon}\sim\mathcal{N}(\vec{0},\vec{I})\) // Auxiliary noise
5:\(\vec{\hat{z}}^{(i)}\leftarrow\vec{\mu}^{(i)}+\vec{\sigma}^{(i)}\odot\vec{\varepsilon}\)
6:// Decode or Reconstruct:
7:\(\vec{\hat{x}}^{(i)}_{s}\gets g^{1}_{\theta}\Big{(}\vec{\hat{z}}^{(i)}, \vec{\delta}^{(i)}\Big{)}\)
```  
**Algorithm 1** Forward pass \(\textbf{VRAE}_{t}\).