# Serial Decoders-Based Auto-Encoders for Image Reconstruction
## 3 Theory
### Proposed Cascade of Decoders-Based Auto-Encoders
#### 3.3.3 Residual-Adversarial Cascade Decoders-Based Auto-Encoders  
The framework of residual-adversarial cascade decoders-based auto-encoders (RAC-DAE) is shown in Figure 6. The blue signal line denotes the training phase, and the green signal line denotes both phases of training and testing. Each decoder is a residual-adversarial module.  
The recovery data can be resolved by the following minimization-maximization problem:  
\[\begin{split}(\theta;\mathbf{z};\mathbf{r}_{1},\cdots,\mathbf{r }_{\text{N}};\mathbf{y}_{1},\cdots,\mathbf{y}_{\text{N}};\mathbf{y})& =\operatorname*{argmin}_{\text{E,D}\text{DC}_{1},\cdots,\text{DC}_{ \text{N}}=1}\sum_{\text{C}_{\text{N}}=1}^{\text{N}}\big{(}\alpha_{\text{n}} \text{M}\big{(}\text{ln}\big{(}\text{DC}_{\text{n}}\big{(}\mathbf{x}-\mathbf{y }_{\text{n}-1}\big{)}\big{)}\big{)}+\beta_{\text{n}}\text{M}(\text{ln}(1-\text{ DC}_{\text{n}}(\mathbf{r}_{\text{n}})))\big{)}\\ \text{s.t.}&\mathbf{z}=\text{E}(\mathbf{x});\mathbf{r }_{1}=\text{D}_{1}(\mathbf{z});\mathbf{r}_{\text{n}}=\text{D}_{\text{n}} \big{(}\mathbf{y}_{\text{n}-1}\big{)},\text{n}=2,\ldots,\text{N};\\ &\mathbf{C}_{\text{z}};\sum_{n=1}^{\text{N}}\left\|\mathbf{x}- \mathbf{y}_{\text{n}-1}-\mathbf{r}_{\text{n}}\right\|_{2}^{2}<\varepsilon, \text{n}=1,\ldots,\text{N};\\ &\mathbf{y}_{0}=0;\mathbf{y}_{\text{n}}=\mathbf{y}_{\text{n}-1}+ \mathbf{r}_{\text{n}},\text{n}=1,\ldots,\text{N};\mathbf{y}=\mathbf{y}_{\text {N}}\end{split} \tag{11}\]  
For the purpose of gradually and serially training residual adversarial cascade decoders-based auto-encoders, the optimization problem in Equation (11) can be divided into the following suboptimization problems:  
\[\begin{split}(\theta_{1};\mathbf{z};\mathbf{r}_{1};\mathbf{y}_{1 })&=\operatorname*{argmin}_{\text{E,D}\text{DC}_{1}}(\alpha_{1} \text{M}(\text{ln}(\text{DC}_{1}(\mathbf{x}-\mathbf{y}_{0})))+\beta_{1}\text{ M}(\text{ln}(1-\text{DC}_{1}(\mathbf{r}_{1}))))\\ (\theta_{2};\mathbf{r}_{2};\mathbf{y}_{2})&= \operatorname*{argmin}_{\text{D}\text{D}\text{D}\text{D}_{2}}(\alpha_{2} \text{M}(\text{ln}(\text{DC}_{2}(\mathbf{x}-\mathbf{y}_{1})))+\beta_{2}\text{ M}(\text{ln}(1-\text{DC}_{2}(\mathbf{r}_{2}))))\\ &\hskip 28.452756 pt......\\ (\theta_{\text{N}};\mathbf{r}_{\text{N}};\mathbf{y}_{\text{N}}; \mathbf{y})&=\operatorname*{argmin}_{\text{D}\text{D}\text{D}\text{D} \text{D}\text{D}\text{D}\text{D}\text{D}\text{D}\text{D}\text{D}\text{D}\text{D} \text{D}\text{D}\textas generator and added an independent discriminator, employed adversarial learning in latent space and let the hidden variable satisfy a given distribution, and finally achieved better performance in data reconstruction [7]. Compared with the classical auto-encoders, AAE infrastructure holds an extra discriminator, which makes the output of the encoder maximally approach a given distribution. The infrastructure can be expressed by the following equations:  
\[\begin{split}\mathbf{z}&=\text{E}(\mathbf{x})\\ \mathbf{y}&=\text{D}(\mathbf{z})\\ \text{DC}(\mathbf{z}_{\text{h}})&=1\text{,DC}(\mathbf{z })=0\end{split} \tag{13}\]  
where the terms are defined as follows:  
\(\mathbf{z}_{\text{h}}\) is the variable related to \(\mathbf{z}\) which satisfies a given distribution;  
DC is the discriminator.  
The reestablishment data can be resolved by the following minimization-maximization problem:  
\[\begin{split}(\theta,\mathbf{z},\mathbf{y})&= \underset{\text{EC},\text{D}}{\text{argminmax}}(\alpha\text{M}(\ln(\text{DC}( \mathbf{z}_{\text{h}})))+\beta\text{M}(\ln(1-\text{DC}(\mathbf{z}))))\\ &\text{s.t.}\ \mathbf{z}=\text{E}(\mathbf{x}),\mathbf{y}=\text{D}( \mathbf{z}),\|\mathbf{y}-\mathbf{x}\|_{2}^{2}<\varepsilon,\text{C}_{\text{y} }\end{split} \tag{14}\]