# DC2 Anet: Generating Lumbar Spine MR Images from CT Scan Data Based on Semi-Supervised Learning
## 3 Method
### Objective Function
Voxel-wise loss helps to overcome this problem if aligned data are available. The goal of the discriminator networks remains unchanged, but the synthesis networks are tasked with not only cheating the discriminator networks, but also being similar to the reference image at an L1 distance. The voxel-wise loss of the forward and backward cycle-consistent network is defined as follows:  
\[L_{voxel}\left(Syn_{MR},Syn_{CT}\right) =\mathbb{E}_{I_{CT},h_{\text{MR}}\sim p_{data}\left(I_{CT},h_{ \text{MR}}\right)}\left[\left\|I_{MR}-Syn_{MR}\left(I_{CT}\right)\right\|_{1}\right] \tag{5}\] \[+\mathbb{E}_{h_{\text{MR}},I_{CT}\sim p_{data}\left(h_{\text{MR}},I_{CT}\right)}\left[\left\|I_{CT}-Syn_{CT}\left(h_{\text{MR}}\right)\right\|_{1 }\right]\]  
Direct optimization of voxel-wise loss produces a suboptimal (i.e., blurry) result by minimizing the average loss for all plausible outputs. To deal with the inherently blurry results obtained from voxel-wise loss, gradient difference loss is constrained for the synthesis networks. The gradient difference loss between a synthetic and reference image is given as follows:  
\[L_{grad}\left(Syn_{MR},Syn_{CT}\right) \tag{6}\] \[+\mathbb{E}_{h_{\text{MR}},I_{CT}\sim p_{data}\left(h_{\text{MR} },I_{CT}\right)}\left[\left\|\
abla\left(Syn_{CT}\left(h_{\text{MR}}\right) _{x}\right)\right|-\left|\
abla\left(I_{CT}\right)_{x}\right\|_{1}+\left\| \left|\
abla\left(Syn_{CT}\left(h_{\text{MR}}\right)_{y}\right)\right|-\left| \
abla\left(I_{CT}\right)_{y}\right\|_{1}\right]\]  
where \(I_{MR}\) in the first term and \(I_{CT}\) in the second term are the reference images in the forward and backward cycle-consistent networks, respectively, and the \(x\)- and \(y\)-direction gradients are calculated to emphasize the boundaries of the structural shape. A pretrained VGG16 network is incorporated into the optimization of the synthesis networks to ensure perceptual similarity. We aim for the synthetic and reference images to have similar feature representations when computed by the pretrained VGG16 network \(\phi\). Let \(\phi_{j}(I_{CT})\) and \(\phi_{j}(I_{MR})\) be the activations of the \(j^{\text{th}}\) convolutional layer of the network \(\phi\) when processing CT image \(I_{CT}\) and MR image \(I_{MR}\), respectively. The perceptual loss is defined as follows:  
\[L_{perc}\left(Syn_{MR},Syn_{CT}\right) =\mathbb{E}_{I_{CT},h_{\text{MR}}\sim p_{data}\left(I_{CT},h_{ \text{MR}}\right)}\left[\frac{1}{K}\sum_{j=1}^{K}\frac{1}{H_{j}W_{j}C_{j}} \middle\|\phi_{j}\big{(}Syn_{MR}\left(I_{CT}\right)\big{)}-\phi_{j}\big{(}I_{ MR}\big{)}\right\|_{1}\right] \tag{7}\] \[+\mathbb{E}_{h_{\text{MR}},I_{CT}\sim p_{data}\left(I_{MR},I_{CT} \right)}\left[\frac{1}{K}\sum_{j=1}^{K}\frac{1}{H_{j}W_{j}C_{j}}\middle\|\phi_{ j}\big{(}Syn_{CT}\left(I_{MR}\right)\big{)}-\phi_{j}\left(I_{CT}\right)\right\|_{1}\right]\]  
where \(\phi_{j}(Syn_{MR}(I_{CT}))\) and \(\phi_{j}(Syn_{CT}(I_{MR}))\) are the activations of the synthetic images in the forward and backward cycles, respectively, \(H_{j}\times W_{j}\times C_{j}\) is the shape of the activations from the \(j^{\text{th}}\) convolution layer, and \(K\) is the number of layers in the VGG16 network. By utilizing the activations of the higher layer in the VGG16 network, the synthetic images can preserve the overall spatial structure of the reference images, but not the texture and exact shape. Perceptual loss causes the synthetic images to become more perceptually similar to the reference images, but does not lead to an exact match. The vertebra, spinal nerves, and ligaments in spinal images contain strong interdependencies. Structural similarity (SSIM) [51] is a perceptually-motivated metric that considers the human visual system and performs better in terms of visual pattern recognition than do quantitative metrics, e.g., mean-based metrics.