# Two-View Mammogram Synthesis from Single-View Data Using Generative Adversarial Networks
## 2 Materials and Methods
### Training Networks
#### 2.1.1 CR-Gan
In bi-directional training, CC image syntheses from MLO views and MLO image syntheses from CC views are learned with equal probability (MLO\(\Leftrightarrow\)CC). On the other hand, in unidirectional training, only the production of CC views from MLO views (MLO\(\Rightarrow\)CC) is learned. The encoder (\(E\)) and \(D\) are trained keeping \(G\) fixed. \(E\) receives \(\mathbf{x}_{1}\) and outputs a latent representation \(\mathbf{\tilde{z}}\) with the estimated view \(\vartheta\), where \((\vartheta,\,\mathbf{\tilde{z}})=(E_{v}(\mathbf{x_{1}}),\,E_{z}(\mathbf{x_{1}}))\). Hopefully, the outputs of \(E\) preserve the identity of the object, such that a complete representation is accomplished. For this purpose, CR-GAN utilizes inverse mapping that inputs the output from \(G\) back into the latent space, as proposed in BiGAN [38]; in other words, \(E\) is trained to be an inverse of \(G\). For a \(50/50\) chance, \(\mathbf{x}_{1}\) is fed to \(E\) for realistic \(\mathbf{x}_{2}\) reconstruction (we refer to this as forward learning), or \(\mathbf{\tilde{x}}\) is also fed for inverse mapping. In the case of forward learning, \(G\) takes \(\mathbf{\tilde{z}}=E_{z}(\mathbf{x_{1}})\) and \(\mathbf{v}_{2}\) as the input to produce \(\mathbf{\tilde{x}}=G(\mathbf{v}_{2},\mathbf{\tilde{z}})\)--a fake image of \(\mathbf{x_{2}}\). \(D\) attempts to distinguish \(\mathbf{\tilde{x}}\) from the real \(\mathbf{x_{2}}\) by minimizing  
\[\mathop{\mathbb{E}}_{\mathbf{x}_{1}\sim\mathbb{P}_{\mathbf{x}}}[D_{s}(\mathbf{\tilde{x}})] -\mathop{\mathbb{E}}_{\mathbf{x}_{2}\sim\mathbb{P}_{\mathbf{x}}}[D_{s}(\mathbf{x_{2}})] +\lambda_{1}\mathop{\mathbb{E}}_{\mathbf{\hat{x}}\sim\mathbb{P}_{\mathbf{\tilde{x}}}}[ (||\
abla_{\mathbf{\hat{x}}}D(\mathbf{\hat{x}})||_{2}-1)^{2}]-\lambda_{2}\mathop{ \mathbb{E}}_{\mathbf{x}_{2}\sim\mathbb{P}_{\mathbf{x}}}[P(D_{v}(\mathbf{x_{2}})=\mathbf{v}_{2 })], \tag{4}\]  
where \(\mathbf{\hat{x}}\sim\mathbb{P}_{\mathbf{\tilde{x}}}\) is interpolated data from \(\mathbf{x_{2}}\) and \(\mathbf{\tilde{x}}\). \(E\) lets \(G\) generate \(\mathbf{\tilde{x}}\) that resembles \(\mathbf{x_{2}}\) by maximizing  
\[\mathop{\mathbb{E}}_{\mathbf{x}\sim\mathbb{P}_{\mathbf{\tilde{x}}}}[D_{s}(\mathbf{\tilde{x }})+\lambda_{3}P(D_{v}(\mathbf{\tilde{x}})=\mathbf{v}_{2})-\lambda_{4}L_{1}(\mathbf{\tilde {x}},\mathbf{x_{2}})-\lambda_{5}L_{v}(E_{v}(\mathbf{x_{1}}),\mathbf{v}_{1})], \tag{5}\]  
Figure 1: Our training networks based on the CR-GAN framework. In the reconstruction path, the inputs and outputs of each network vary, depending on whether there is a forward learning or inverse mapping case. where \(L_{1}\) is L1 regularization, used to bring \(\mathbf{\tilde{x}}\) as close as possible to \(\mathbf{x_{2}}\). \(L_{v}\), which is the cross-entropy loss of the view estimated by \(E\) and the true view, forces \(E\) to be a good view estimator. Equations (1), (2), (4), and (5) update the weights of the networks during forward learning. For inverse mapping, \(G\) takes \(\mathbf{\tilde{z}}\)=\(E_{z}(\mathbf{\tilde{x}})\) and \(v_{1}\) as the input, where \(\mathbf{\tilde{x}}\) is derived from \(v_{1}\) and random noise in the generator path. \(D\) is expected to distinguish \(\mathbf{\tilde{x}}=G(v_{1},E_{z}(\mathbf{\tilde{x}}))\) from the real image pair. Hence, \(\mathbf{\hat{x}}\) in Equation (4) is modified to interpolated data from \(\mathbf{x_{1}}\) and \(\mathbf{\tilde{x}}\). Equation (5) is also modified to  
\[\underset{\mathbf{\tilde{z}}\sim\mathbb{P}_{\mathbf{\tilde{x}}}}{\mathbb{E}}[D_{s}( \mathbf{\tilde{x}})+\lambda_{3}P(D_{v}(\mathbf{\tilde{x}})=v_{1})-\lambda_{4}L_{1}(\bm {\tilde{x}},\mathbf{\tilde{x}})]-\lambda_{5}\underset{\mathbf{\tilde{x}_{1}}\sim \mathbb{P}_{\mathbf{\tilde{x}}}}{\mathbb{E}}[L_{v}(E_{v}(\mathbf{\tilde{x}}),v_{1})], \tag{6}\]  
such that \(\mathbf{\tilde{x}}\) looks similar to \(\mathbf{\tilde{x}}\). Equations (1), (2), (4), and (6) update the weights of the networks during inverse mapping.