# Shadow Estimation for Ultrasound Images Using Auto-Encoding Structures and Synthetic Shadows
## 3 Materials and Methods
### Training Using Unlabeled Data with Synthetic Shadows  
Since the proposed model is based on auto-encoders, it is basically trained by minimizing a reconstruction loss given as  
\[L_{\text{recon}}(x,\hat{x}):=\frac{1}{HW}\sum_{h=1}^{H}\sum_{w=1}^{W}(\hat{x}_ {hw}-x_{hw})^{2}, \tag{3}\]  
which is also known as the mean squared error (MSE). Of course, we cannot make the model split the input into the estimated shadow and the estimated clean image only by the reconstruction loss. To address this, we introduce synthetic shadows and a loss function that uses them as pseudo labels.  
Annotating shadows is costly because the pixel-level label is expensive in the first place, and additionally, shadows are ambiguous. It is difficult to make a standard for labeling shadows that come in various intensities and are often blurred. However, the possible shapes of shadows are known when the domain is fixed; the target to be diagnosed and equipment such as probes are set. Once the shapes are determined, we can generate random plausible synthetic shadows in a rule-based manner. Then the synthetic shadows can be injected into the input image and can be used as pseudo labels. In this study, we focus on convex probes [1] that generate shadows shaped annular sectors. Details of the algorithm for generating shadows are described in Appendix A.  
Assuming that a synthetic shadow image \(s\in[0,1]^{H\times W}\) is given, we inject it to an input image \(x\) as follows;  
\[x^{\prime}=x\odot s, \tag{4}\]  
and we use \(x^{\prime}\) as a new input to the model. A loss function for training the model to predict shadows is defined as  
\[L_{\text{synth}}(s,\hat{s}):=\frac{1}{HW}\sum_{h=1}^{H}\sum_{w=1}^{W}\mathbf{1 }[s_{hw}\
eq 1](\hat{s}_{hw}-s_{hw})^{2}, \tag{5}\]  
Figure 1: Overview of our shadow estimation method. (**a**) shows the proposed auto-encoding structure. (**b**,**c**) illustrate the learning process for unlabeled data and pixel-level labeled data, respectively. For unlabeled data, the estimated shadow \(\hat{s}\) is compared to the synthetic shadow with respect to the region that the synthetic shadow exists. For labeled data, the label is made semi-transparent based on the estimated intensity of labeled shadows, and \(\hat{s}\) is compared to it.  
where \(\mathbf{1}[\cdot]\) is a function that returns 1 when the condition is met and returns 0 otherwise. Note that the loss \(L_{\text{synth}}\) evaluates the area that the synthetic shadow exists. This is because we do not know whether the original input already contains shadows. If the masking term \(\mathbf{1}[s_{hw}\
eq 1]\) is omitted, the model learns the region without the synthetic shadow as a shadow-free region regardless of the presence of real shadows. Thanks to the mask, we can train the model to estimate the intensity of the synthetic shadow, but the whole estimated shadow tends to be dark. To prevent the estimated shadow \(\hat{s}\) from being too dark and to make the default output white, we also introduce an auxiliary regularization loss defined as  
\[L_{\text{synthreg}}(\hat{s}):=\frac{1}{HW}\sum_{h=1}^{H}\sum_{w=1}^{W}|\hat{s}_ {hw}-1|. \tag{6}\]  
A linear combination of the three losses introduced above is a loss for training with unlabeled US images. That is, the loss function is given as  
\[L_{\text{unlabeled}}(x,s,\hat{x},\hat{s}):=\lambda_{\text{recon}}L_{\text{ recon}}(x,\hat{x})+\lambda_{\text{synth}}L_{\text{synth}}(s,\hat{s})+ \lambda_{\text{synthreg}}L_{\text{synthreg}}(\hat{s}), \tag{7}\]  
where \(\lambda_{\text{recon}},\lambda_{\text{synth}},\lambda_{\text{synthreg}}>0\) are hyperparameters that decide the weight of each loss. This training procedure using unlabeled data and the synthetic shadows is illustrated in Figure 1b.