# Serial Decoders-Based Auto-Encoders for Image Reconstruction
## 3 Theory
### Proposed Cascade of Decoders-Based Auto-Encoders
#### 3.4.2 Proposed Cascade Decoders-Based Adversarial Auto-Encoders  
The architecture of the proposed cascade decoders-based adversarial auto-encoders (CDAAE) is illustrated in Figure 8. The blue flow line represents the training phase, and the green flow line represents both phases of training and testing. Compared with the cascade decoders-based auto-encoders, the proposed architecture has an extra discriminator, which makes the output of the encoder maximally approximate to a known distribution. The architecture can be described by the following formulas:  
\[\begin{split}\mathbf{z}&=\text{E}(\mathbf{x})\\ \mathbf{y}_{\text{n}}&=\left\{\begin{array}{c}\text{D}_ {1}(\mathbf{z}),\mathbf{n}=1\\ \text{D}_{\text{n}}(\mathbf{y}_{\text{n-1}}),\mathbf{n}=2,\ldots,\text{N}\\ \text{DC}(\mathbf{z}_{\text{h}})=1\text{,DC}(\mathbf{z})=0\end{array}\right. \end{split} \tag{15}\]  
Figure 7: The architecture of classical adversarial auto-encoders.  
The restoration data can be resolved by the following optimization problem:  
\[\begin{split}(\mathbf{\theta};\mathbf{z};\mathbf{y}_{1},\cdots, \mathbf{y}_{\text{N}})&=\arg\min\limits_{\text{E},\text{D}_{1}, \cdots,\text{D}_{\text{N}}}\max\limits_{\text{DC}}(\alpha\text{M}(\ln(\text{ DC}(\mathbf{z}_{\text{h}})))+\beta\text{M}(\ln(1-\text{DC}(\mathbf{z}))))\\ \text{s.t.}&\mathbf{z}=\text{E}(\mathbf{x});\mathbf{y} _{1}=\text{D}_{1}(\mathbf{z});\mathbf{y}_{\text{n}}=\text{D}_{\text{n}}( \mathbf{y}_{\text{n-1}}),\text{n}=2,\ldots,\text{N};\\ &\sum\limits_{\text{n}=1}^{\text{N}}\|\mathbf{y}_{\text{n}}- \mathbf{x}\|_{2}^{2}<\varepsilon;\text{C}_{\mathbf{y}_{\text{n}}},\text{n}=1, \ldots,\text{N}\end{split} \tag{16}\]  
For the purpose of gradually and serially training cascade decoders-based adversarial auto-encoders, the optimization problem in Equation (16) can be partitioned into the following suboptimization problems:  
\[\begin{split}(\mathbf{\theta}_{1};\mathbf{z};\mathbf{y}_{1})= \underset{\text{E},\text{D}_{1}\text{ DC}}{\text{argminmax}}(\alpha\text{M}(\ln(\text{DC}(\mathbf{z}_{\text{h}})))+ \beta\text{M}\ln(1-\text{DC}(\mathbf{z})))\\ (\mathbf{\theta}_{2};\mathbf{y}_{2})&=\underset{ \text{D}_{2}}{\text{argminmin}}\|\mathbf{y}_{2}-\mathbf{x}\|_{2}^{2}\\ &\ldots\ldots\ldots\\ (\mathbf{\theta}_{\text{N}};\mathbf{y}_{\text{N}})=\underset{ \text{D}_{1}}{\text{argminmin}}\|\mathbf{y}_{\text{N}}-\mathbf{x}\|_{2}^{2}\\ \text{s.t.}&\mathbf{z}=\text{E}(\mathbf{x});\mathbf{y} _{1}=\text{D}_{1}(\mathbf{z});\mathbf{y}_{\text{n}}=\text{D}_{n}\big{(} \mathbf{y}_{\text{n-1}}\big{)},\text{n}=2,\ldots,\text{N};\\ &\|\mathbf{y}_{1}-\mathbf{x}\|_{2}^{2}<\varepsilon;\text{C}_{ \mathbf{y}_{\text{n}}},\text{n}=1,\ldots,\text{N}\end{split} \tag{17}\]  
The architecture in Figure 8 represents general cascade decoders-based adversarial auto-encoders (GCDAAE), and it can be easily be expanded to residual cascade decoders-based adversarial auto-encoders (RCDAAE), adversarial cascade decoders-based adversarial auto-encoders (ACDAAE), and residual-adversarial cascade decoders-based adversarial auto-encoders (RACDAAE).