## 2 Proposed Methodology for Generating Synthetic Fermentation Data of Shindari
### Generative Adversarial Networks for Generating Synthetic Fermentation Data for Shindari
#### 2.4.3 TGAN-Skip-WGAN-GP  
TGAN is a general purpose GAN for generating synthetic data for tabular datasets. In general, the TGAN architecture does not possess skip connections. We use the improved version of TGAN that has skip connections to eliminate the vanishing gradient problem and decrease the convergence time with improvements in the overall performance. The skip connections in the TGAN architecture preserve the high magnitude of activation by allowing prompt activation to skip in-between layers. This results in retaining old information, achieves higher gradients, and helps in training deeper models [28]. Figure 5(a) depicts the layout of the TGAN-skip architecture. For the generator of the GAN architecture, we used the long short-term memory (LSTM) cell, as shown in Figure 5(b), and the discriminator is built with multilayer perceptron (MLP), as shown in Figure 5(c). The generator is modeled to generate each continuous variable in two steps and each categorical variable in one step. For continuous variables, we first generate the value scalar \(S_{n}\) and then the cluster vector \(V_{n}\); for categorical variables, we compute the probability distribution \(PD_{n}\) over each label, as shown in Figure 5(b). \(RV\) is the random variable that is forwarded as input to the LSTM cell in each step \(st\) along with the previous hidden vector \(HV_{n}\) or embedding vector \(HV_{n}^{{}^{\prime}}\) and the attention-based context vector \(CV_{n}\)[29]. We can determine the context from the attention weight vector \(\beta_{st}\) by Equation (5):  
\[\beta_{st}=\sum_{p=1}^{st}\frac{\exp\beta_{st,p}}{\sum_{q}\exp\beta_{st,q}}Out _{p}, \tag{5}\]  
where \(Out\) is the output from the LSTM cell. In TGAN, the output from the LSTM passes through two dense layers to generate the final output, then the output of the two dense layers goes through another dense layer to form the attention vector for the next iteration of the LSTM network. We added the skip connection in between the two dense layers. WGAN [30] was built to solve the flaws in Vanilla GAN. WGAN-GP, developed by [PERSON] et al. [31], proposed an improved version of WGAN by introducing a gradient penalty that eliminates the need for clipping and complies with the constraint of 1-Lipschitz. The new objective or loss by WGAN-GP is defined by the combination of the original critic loss and gradient penalty loss, as mentioned in Equations (6) and (7):  
\[OriginalCriticLoss=\mathbb{E}_{\tilde{x}}\sim G_{p}[logD(\tilde{x})]-\mathbb{E }_{\tilde{x}\sim R_{p}}[D(x)], \tag{6}\]  
Figure 6: Layout of TGAN-skip–WGAN-GP: (**a**) Layout of TGAN-skip architecture. (**b**) Architecture of the generator. (**c**) Architecture of the discriminator. TGAN—Temporal generative adversarial network; WGAN—Wasserstein GAN; GP—gradient penalty. \[GradientPenaltyLoss=\lambda\mathbb{E}_{\hat{x}\sim P_{\hat{x}}}[(\|\
abla_{\hat{x}}D( \hat{x})\|_{2}-1)^{2}], \tag{7}\]  
where \(R_{p}\) and \(G_{p}\) are the data and generator distribution with \(\hat{x}\sim P_{\hat{x}}\) denoting random samples. The value of \(\lambda\), according to [31], is selected as 10 since when experimented, it worked well with various datasets and architectures. Using the WGAN-GP architecture for TGAN-skip decreases the discriminator loss and improves the performance of the generator. Normally, TGAN uses the architecture of vanilla GAN, in which any imbalance between the generator and discriminator can cause stagnation of training. By using WGAN-GP architecture in the TGAN-skip model, we can avoid a stagnation scenario and mode collapse. TGAN-skip-WGAN-GP also converges smoothly with higher speed than the TGAN with vanilla GAN architecture. To implement the WGAN-GP architecture in TGAN-skip, we adapted the loss function to the Wasserstein distance and evaluated the gradient penalty. We changed the batch normalization layer of the discriminator in TGAN-skip to layer normalization and eliminated the sigmoid activation from the last layer.