SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\log p _ {Z _ {i}} \left(\boldsymbol {z} _ {i}\right) = m _ {i} \cdot \log p _ {Z _ {i} ^ {n}} \left(\boldsymbol {z} _ {i}\right) + (1 - m _ {i}) \cdot \log p _ {Z _ {i} ^ {a}} \left(\boldsymbol {z} _ {i}\right), \tag {5}
$$
where the binary indicator  $m_i$  is set to be 1 when the corresponding location in the binary mask  $M$  is 1, and 0, otherwise. Note that,  $Z_i^n$  represents the corresponding base distribution when  $v_i$  is a normal feature, while  $Z_i^a$  represents the base distribution when  $v_i$  is an abnormal feature. In practice, to compute the likelihood at the  $k$ -th scale, the binary mask  $M$  is resized to match the size of the feature map  $A^k$ , which has height  $H_k$  and width  $W_k$ . The resizing is done using nearest-neighbor interpolation to preserve the binary nature of the mask. In turn, each log-likelihood term can be formulated as follows (detailed derivations can be found in the supplementary material):
$$
\log p _ {Z _ {i} ^ {n}} \left(\boldsymbol {z} _ {i}\right) = - \frac {1}{2} \log 2 \pi - \frac {1}{2} \left(\log \beta_ {i} - \psi \left(\alpha_ {i}\right)\right) - \frac {\alpha_ {i}}{2 \beta_ {i}} | | \boldsymbol {z} _ {i} | | _ {2} ^ {2}, \tag {6}
$$
$$
\log p _ {Z _ {i} ^ {a}} \left(\boldsymbol {z} _ {i}\right) = - \frac {1}{2} \log 2 \pi - \frac {1}{2} \left(\log \beta_ {i} - \psi \left(\alpha_ {i}\right)\right) - \frac {\alpha_ {i}}{2 \beta_ {i}} | | \boldsymbol {z} _ {i} - 1 | | _ {2} ^ {2}. \tag {7}
$$
Moreover, to further aid NF in distinguishing normal and abnormal features, we can also train NF to perform binary classification using the loss function as follows:
$$
\mathcal {L} _ {\mathrm {B C E}} = \mathbb {B} \mathbb {C} \mathbb {E} (s \left(\boldsymbol {z} _ {i}\right), m _ {i}), \tag {8}
$$
where  $\mathbb{B}\mathbb{C}\mathbb{E}$  denotes the conventional binary cross-entropy loss, while  $s(\pmb{z}_i)$  measures the probability of  $\pmb{z}_i$  being classified as normal (i.e.,  $s(\pmb{z}_i) = \frac{p_{Z_i^n}(\pmb{z}_i)}{p_{Z_i^n}(\pmb{z}_i) + p_{Z_i^a}(\pmb{z}_i)}$ ). Note that the log-likelihood of  $\pmb{z}_i$  being abnormal (Equation 7) can be used only during training such that our framework learns to map abnormal features to base distribution  $Z^a$  that has small overlap with the base distributions of normal features  $Z^n$ . This allows NF to transform normal and abnormal features to two distinct base distributions, thereby enhancing the discriminability.
Together with  $\mathcal{L}_{\mathrm{KL}}$  from Equation 4, the overall objective function  $\mathcal{L}_{\mathrm{overall}}$  to minimize is given by,
$$
\mathcal {L} _ {\text {o v e r a l l}} = \mathcal {L} _ {\mathrm {N L L}} + \lambda_ {1} \cdot \mathcal {L} _ {\mathrm {B C E}} + \lambda_ {2} \cdot \mathcal {L} _ {\mathrm {K L}}, \tag {9}
$$
where  $\lambda_{1}$  and  $\lambda_{2}$  are hyperparameters to adjust the associated weights of  $\mathcal{L}_{\mathrm{BCE}}$  and  $\mathcal{L}_{\mathrm{KL}}$ .
3.6 Anomaly score functions
During inference, given a test image  $\pmb{x} \in \mathbb{R}^{H \times W \times 3}$ , we first compute the log-likelihood of each feature  $\pmb{v}_i$  at each  $k$ -th scale via Equation 3, where the log-likelihood of  $\pmb{z}_i$  is computed with Equation 6. Then, the likelihood map  $P^k \in \mathbb{R}^{H_k \times W_k}$  for the  $k$ -th scale is obtained by taking exponential of the log-likelihood 6. We obtain the final likelihood map  $P \in \mathbb{R}^{H \times W}$  by summing the likelihood map  $P^k$  from all scales. These likelihood maps are first upsampled to the image resolution  $\mathbb{R}^{H \times W}$  using bilinear interpolation, following the approach described in [20]. Finally, the anomaly score map is computed as the negative of  $P$ , resulting in  $-P$ .
In this section, we evaluate our framework to validate its capability in both pixel-level anomaly localization and image-level anomaly detection. We will release our code and data upon acceptance, and more details and results can be found in the supplementary material.
Dataset. The experiments are conducted on two commonly used datasets for unsupervised anomaly detection: STC (ShanghaiTech Campus) dataset [28] and MVTec dataset [4]. STC is a video surveillance dataset, which provides static videos of 13 different scenes of  $856 \times 480$  resolution. It contains 274, 515 frames for training and 42, 883 frames for evaluation. The training set consists of only normal sequences, while the evaluation set consists of 300, 308 regular frames and 42, 883 irregular frames. MVTec is a dataset that consists of images of industrial products categorized into 5 texture categories and 10 object categories. To evaluate the unsupervised anomaly detection performance, the training set includes only defect-free (e.g., normal) images: 3, 629 normal images are available for training while 1, 725 normal and abnormal images are used as test set. Among the test images, 1, 258 images contain defects (e.g., abnormal images). For data augmentation, abnormal patches undergo random vertical, horizontal flip and rotation during training, as described in 3.2. We evaluate and compare algorithms in terms of area under the receiver operating characteristic curve (AUROC) and area under the per-region-overlap curve (AUPRO), as used in [4, 20]. AUPRO scores can be found in the supplementary.
Table 1: Anomaly detection and localization results w.r.t. AUROC metric by various methods with WRN-50 backbone for feature extraction on STC [28].
FramePred [28]	MemAE [18]	SPADE [8]	PaDiM [11]	CFLOW-AD [20]	PatchCore-10 [36]	SANFlow (Ours)
Image-wise	72.8	71.2	71.9	-	72.63	-	76.1
Pixel-wise	-	-	89.9	91.2	94.48	91.8	94.8
Table 2: Quantitative comparisons on anomaly localization performance in MVTec dataset [4] with respect to AUROC metric. For each category, the best and second best performance is bolded and underlined, respectively.
Category	AESSIM
WRN-50	γ-VAE+grad
WRN-50	PatchSVDD
WRN-50	PaDiM
WRN-50	CutPaste
WRN-50	CFLOW-AD
WRN-50	PatchCore-10
WRN-50	PatchCore-1
WRN-101	SANFlow
WRN-50	SANFlow
WRN-101
bottle	93.0	93.1	98.1	98.3	97.6	98.76	98.6	98.6	98.6	99.1
cable	82.0	88.0	96.8	96.7	90.0	97.64	98.5	98.4	98.5	98.8
capsule	94.0	91.7	95.8	98.5	97.4	98.98	98.9	99.1	99.1	98.9
carpet	87.0	72.7	92.6	99.1	98.3	99.23	99.1	98.7	99.3	99.4
grid	94.0	97.9	96.2	97.3	97.5	96.89	98.7	98.7	98.5	99.3
hazelnut	97.0	98.8	97.5	98.2	97.3	98.82	98.7	98.8	99.2	99.0
leather	78.0	89.7	97.4	99.2	99.5	99.61	99.3	99.3	99.6	99.8
metal nut	89.0	91.4	98.0	97.2	93.1	98.56	98.4	98.8	98.5	98.7
pill	91.0	93.5	95.1	95.7	95.7	98.95	97.6	97.8	99.2	99.1
screw	96.0	97.2	95.7	98.5	96.7	98.10	99.4	99.3	99.0	99.2
tile	59.0	58.1	91.4	94.1	90.5	97.71	95.9	96.1	98.9	99.1
toothbrush	92.0	98.3	98.1	98.8	98.1	98.56	98.7	98.8	98.9	99.2
transistor	90.0	93.1	97.0	97.5	93.0	93.28	96.4	96.4	94.4	95.1
wood	73.0	80.9	90.8	94.9	95.5	94.49	95.1	95.1	96.4	97.9
zipper	88.0	87.1	95.1	98.5	99.3	98.41	98.9	98.9	98.9	99.6
average	87.0	88.8	95.7	97.5	96.0	97.9	98.1	98.2	98.5	98.8
Table 3: Quantitative comparisons on anomaly detection in MVtec dataset [4] with respect to AUROC metric. For each category, the best and second best performance is bolded and underlined, respectively.
Category	GANomaly WRN-50	OCSVM WRN-50	PatchSVDD WRN-50	DifferNet WRN-50	PaDiM WRN-50	CFLOW-AD WRN-50	CutPaste WRN-50	PatchCore-10 WRN-50	PatchCore-1 WRN-101	SANFlow WRN-50	SANFlow WRN-101
bottle	89.2	99	98.6	99.0	-	100	98.2	100	100	100	100
cable	75.7	80.3	90.3	95.9	-	97.59	81.2	99.4	99.6	99.4	99.7
capsule	73.2	54.4	76.7	86.9	-	97.68	98.2	97.8	98.2	97.7	98.9
carpet	69.9	62.7	92.9	92.9	-	98.73	93.9	98.7	98.4	99.8	99.9
grid	70.8	41	94.6	84.0	-	99.60	100	97.9	99.8	99.3	100
hazelnut	78.5	91.1	92.0	99.3	-	99.98	98.3	100	100	100	100
leather	84.2	88	90.9	97.1	-	100	100	100	100	100	100
metal nut	70.0	61.1	94.0	96.1	-	99.26	99.9	100	100	99.8	100
pill	74.3	72.9	86.1	88.8	-	96.82	94.9	96.0	97.2	96.8	98.2
screw	74.6	74.7	81.3	96.3	-	91.89	88.7	97.0	98.9	94.0	96.2
tile	79.4	87.6	97.8	99.4	-	99.88	94.6	98.9	98.9	100	100
toothbrush	65.3	61.9	100	98.6	-	99.65	99.4	99.7	100	96.7	100
transistor	79.2	56.7	91.5	91.1	-	95.21	96.1	100	100	99.3	99.4
wood	83.4	95.3	96.5	99.8	-	99.12	99.1	99.0	99.5	99.1	99.3
zipper	74.5	51.7	97.9	95.1	-	98.48	99.9	99.5	99.9	99.9	100
average	76.2	71.9	92.1	94.9	97.9	98.26	96.1	99.0	99.4	98.7	99.4
Implementation details. Following other works [4, 20, 36], our models are trained and evaluated separately for each category. Similar to CFLOW-AD [20], we use  $K = 3$  scales for feature pyramid; normalizing flow consists of  $L = 8$  transformation blocks; the dimension of position embedding vector  $D$  is 512; Adam optimizer with a learning rate of 5e-4 and 80 train epochs for training. The loss weight hyperparameters  $\lambda_{1}$  and  $\lambda_{2}$  are set to be 1.0 and 0.2, respectively. The statistics prediction network  $h$  consists of five convolutional layers with a skip connection as illustrated in Figure 3. At last, we use NVIDIA RTX8000 for training and the model shows near real-time performance by running at 13fps.
We provide the anomaly localization and detection performance on the STC dataset in Table 1 while the results on MVTec dataset are reported in Table 2 and Table 3. All results presented in this study are obtained using the same backbone architecture, WRN-50 [53], while additional results on MVTec are reported for PatchCore [36] and our method with a larger backbone, WRN-101 [53]. The experimental results demonstrate that the performance of our proposed method is comparable with other state-of-the-art methods. In Table 1, compare to other anomaly detection models that are not video targeted, our method shows outstanding performances in both detection and localization.
(b)
Figure 4: Visualization of anomaly score map based on (a)  $Z^n$  and (b)  $Z^a$  on MVtec dataset.
Table 4: Ablation study on loss functions in MVTec and STC datasets.
L_NLL	L_BCE	L_KL	STC	MVTec
Model (1a)	Eq.(2)	X	X	72.6	98.3
Model (1b)	Eq.(3)&(5)	X	X	73.5	98.2
Model (1c)	Eq.(3)&(5)	✓	X	73.1	98.3
Model (1d)	Eq.(3)&(5)	X	✓	74.2	98.5
SANFlow	Eq.(3)&(5)	✓	✓	76.1	98.7
Table 5: Ablation study on anomaly augmentation in MVTec dataset.
Augmentation	L_NLL	L_BCE	L_KL	STC	MVTec
Model (1d)	✓	✓	✗	✓	74.2	98.5
Model (2a)	✗	✓	✗	✓	73.6	98.4
SANFlow	✓	✓	✓	✓	76.1	98.7
Plus, Table 2 and Table 3 reports that our proposed method performs better than existing NF-based anomaly detection algorithms (CFLOW-AD [20] and DifferNet [37]) and other anomaly detection algorithms such as  $AE_{SSIM}$  [4],  $\gamma$ -VAE+grad [12], PatchSVDD [49], PaDiM [11] and PatchCore-1, 10 [36]. In particular, the results demonstrate the effectiveness of our proposed semantic-aware base distribution mapping approach in enhancing the density estimation performance of the proposed NF model.
We conduct ablation experiments to evaluate the efficacy of each component of our framework. Specifically, we perform ablation studies on the loss function (Table 4), synthetic anomaly generation (Table 5), and statistics estimated by our statistics prediction network  $h$  (Table 6). All ablation experiments are reported with respect to AUROC, using the WRN-50 backbone, and all average AUROC values over all categories are reported for image-wise anomaly detection.
In Table 4, we analyze the efficacy of our proposed loss functions. To do so, we first compare against a baseline NF model trained using the standard negative log-likelihood loss of NF in Equation 2 (Model (1a)). Compared to Model (1a), SANFlow brings substantial improvement, thereby validating the effectiveness of all three proposed loss functions  $(\mathcal{L}_{\mathrm{NLL}}$  in Equation 3 using Equation 5,  $\mathcal{L}_{\mathrm{KL}}$ , and  $\mathcal{L}_{\mathrm{BCE}})$ . We validate the efficacy of each loss function separately by applying only  $\mathcal{L}_{\mathrm{NLL}}$  using Equation 3 and 5 (Model (1b)) or disabling either  $\mathcal{L}_{\mathrm{KL}}$  (Model (1c)) or  $\mathcal{L}_{\mathrm{BCE}}$  (Model (1d)). While each variant leads to performance improvement, using all three proposed loss functions leads to the largest improvement, suggesting that all three proposed loss functions coordinate and play critical roles in our overall framework.
In Table 5, we evaluate the impact of synthetic anomaly augmentation (generation) on the final performance of our framework. Training without anomaly augmentation and thus without any anomaly data results in performance degradation (Model (2a)). Also, Model (1d) (Table 4) which has only difference at doing anomaly augmentation in training, performs better than Model (2a). Thus, the ablation result suggests that not only embedding semantic features to different base distributions but training NF with synthetic anomalies is also an important process. For better understanding, Fig-
ure 4 visualizes anomaly score maps based on estimated normal distribution  $Z^n$  (a) and abnormal distribution  $Z^a$  (b) for test images. As described in Section 3.6, Figure 4 (a) can be found with Equation 3 and 6 while Figure 4 (b) can be found with Equation 3 and 7. We note that both maps agree on the abnormal region (highlighted with bright colors), validating that training NF to map normal and abnormal features to distant base distributions helps the model discriminate abnormal regions from normal regions.
Table 6: Ablation study on statistics estimation in MVtec and STC datasets. fixed/estimated refers to whether statistics are manually fixed or estimated by our statistics prediction network.
μi	σi2	STC	MVTec
Model (3a)	fixed	fixed	75.9	97.6
Model (3b)	estimated	fixed	75.1	97.0
Model (3c)	estimated	estimated	74.5	98.1
SANFlow (Ours)	fixed	estimated	76.1	98.7
Table 6 reports ablation results after fixing either mean  $(\mu_i)$ , variance  $(\sigma_i^2)$ , or both statistics of base distributions. If  $\mu_i$  is fixed, it is set to be 0 for normal features and 1 for abnormal features. If  $\sigma_i^2$  is fixed, it is empirically set to be 0.1 for both normal and abnormal features. The performance is observed to decrease noticeably when  $\mu_i$  is estimated alone (Model (3b)) or together with  $\sigma_i^2$  (Model (3c)). On the other hand, estimating only  $\sigma_i^2$  while fixing  $\mu_i$  brings significant performance improvement. The performance degradation from estimating  $\mu_i$  may be due to learning complexity. Furthermore, to enhance the persuasiveness of the mean margin, we conducted additional experiments on the MVTec dataset, comparing performance at margin values of 0.5, 1.0, and 1.5 in WRN-50 backbone. For margin values 0.5, 1.0, and 1.5, the proposing method shows 98.6, 98.7, and 98.1 AUROC for image-wise and 98.4, 98.5, and 98.3 AUROC for pixel-wise. As the margin of 1.0 yields the highest results in both image- and pixel- wise, we could confirm that the mean margin of 1.0 is a persuasive choice.
4.4 Experiments on other datasets
Although the MVtec dataset is a benchmark that is widely used for anomaly detection and localization, there is difficulty in identifying the advantage of the proposing method with it due to performance saturation. Therefore, we provide a performance comparison in the VisA dataset [58]. Our proposed method with a WRN-50 backbone achieves an image-wise AUROC of 93.4, 98.6, and a PRO score of 89.4, while CFLOW-AD [20] and PaDiM [11] achieve 91.5, 59.8, and 89.1, 85.9 of an image-wise AUROC and a PRO score, respectively. The results show that our approach demonstrates competitive performance against other methods, including other flow-based approaches, demonstrating the effectiveness of our framework in greatly improving the density estimation capability of normalizing flow for anomaly detection and localization. Plus, while our approach excels at detecting anomalies within images, its effectiveness in tasks involving image-wise semantic outlier detection, like CutPaste [26] and CFLOW-AD, is limited. Nonetheless, we conducted an additional experiment with CIFAR-10 [24], where our proposed method achieved an AUROC of 80.8, outperforming performances of CutPaste (69.4) and CFLOW-AD (79.32).
In this work, we propose to improve the density estimation of normalizing flow (NF) for anomaly detection, by training NF to dynamically embed given feature distribution to different base distributions. In particular, base distributions are Gaussian distribution with statistics estimated by our statistics prediction network. As a result, our NF framework learns to map not only diverse normal features but also abnormal features to corresponding yet different base distributions, enhancing the density estimation capability.
Limitation While SANFlow has the capability of mapping different semantic features to different base distributions, it requires anomaly augmentation to map anomaly features to a base distribution distant from base distributions of normal features for better anomaly detection. However, augmentations require domain knowledge and do not cover new anomalies that may occur. But, we would like to note that our ablation study reveals that our framework brings improvements even without anomaly augmentation. Nonetheless, reducing the dependence on anomaly augmentations in the proposed framework is an interesting future research direction.
Broader Impacts Finding anomalies in product during fabrication is important to ensure the quality of products. Anomaly detection algorithms can automate the process, which can offload the burden from human workers. This will allow human workers to focus on other important aspects of fabrication, improving the overall process of fabrication and thus the quality of products. Furthermore, anomaly detection can be used to frequently check anomalies of objects that may be difficult for humans but crucial to human safety (e.g., tall buildings, bridges, nuclear power plants, etc.).
Acknowledgements This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No. RS2023-00220628, Artificial intelligence for prediction of structure-based protein interaction reflecting physicochemical principles,  $50\%$  ) and Samsung Electronics Co., Ltd, and Samsung Research Funding Center of Samsung Electronics (No. SRFCIT1901- 06,  $50\%$  ).
[49] Jihun Yi and Sungroh Yoon. Patch svdd: Patch-level svdd for anomaly detection and segmentation. In Proceedings of the Asian Conference on Computer Vision (ACCV), 2020.
[50] Hongwei Yong, Deyu Meng, Wangmeng Zuo, and Lei Zhang. Robust online matrix factorization for dynamic background subtraction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(7):1726-1740, 2018.
[51] Jiawei Yu, Ye Zheng, Xiang Wang, Wei Li, Yushuang Wu, Rui Zhao, and Liwei Wu. Fastflow: Unsupervised anomaly detection and localization via 2d normalizing flows. arXiv preprint arXiv:2111.07677, 2021.
[52] Zongsheng Yue, Hongwei Yong, Qian Zhao, Lei Zhang, and Deyu Meng. Variational denoising network: Toward blind noise modeling and removal. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[53] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proceedings of the British Machine Vision Conference (BMVC), 2016.
[54] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. Draem - a discriminatively trained reconstruction embedding for surface anomaly detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021.
[55] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. Reconstruction by inpainting for visual anomaly detection. Pattern Recognition, 2021.
[56] Ye Zheng, Xiang Wang, Rui Deng, Tianpeng Bao, Rui Zhao, and Liwei Wu. Focus your distribution: Coarse-to-fine non-contrastive learning for anomaly detection and localization. In International Conference on Multimedia and Expo (ICME), 2022.
[57] Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In Proceedings of the International Conference on Learning Representations (ICLR), 2018.
[58] Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. Spot-the-difference self-supervised pre-training for anomaly detection and segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 392-408. Springer, 2022.
A Loss Function Derivation
In this section, we show the detailed derivations for Equation 4 in Section 3.4.2 and Equation 6 and 7 in Section 3.5.
A.1 KL-Divergence
We present the detailed derivations of our KL divergence loss  $\mathcal{L}_{\mathrm{KL}}$ , where  $IG(\alpha_{i},\beta_{i})$  and  $IG(\alpha ,\beta)$  represent the probability density function of the inverse Gamma distribution with their corresponding parameters  $\alpha_{i},\beta_{i}$  and  $\alpha ,\beta$ . Furthermore,  $\pmb {s}_i$  and  $\pmb{s}$  denote random variables that follow the corresponding distributions  $IG(\alpha_{i},\beta_{i})$  and  $IG(\alpha ,\beta)$ , respectively. The detailed formulations are as follows:
$$
\begin{array}{l} \mathcal {L} _ {K L} = D _ {K L} \left(I G \left(\alpha_ {i}, \beta_ {i}\right) \mid I G (\alpha , \beta)\right) \\ = \sum_ {i = 1} ^ {H _ {k} \times W _ {k}} E [ \log I G (\alpha_ {i}, \beta_ {i}) - \log I G (\alpha , \beta) ] \\ = \sum_ {i = 1} ^ {H _ {k} \times W _ {k}} \left\{E [ - \log I G (\alpha , \beta) ] - E [ - \log I G (\alpha_ {i}, \beta_ {i}) ] \right\} \\ = \sum_ {i = 1} ^ {H _ {k} \times W _ {k}} \left\{E \left[ - \alpha \log \beta + \log \Gamma (\alpha) + (\alpha + 1) \log s + \frac {\beta}{s} \right] \right. \\ \left. - E \left[ - \alpha_ {i} \log \beta_ {i} + \log \Gamma \left(\alpha_ {i}\right) + \left(\alpha_ {i} + 1\right) \log s _ {i} + \frac {\beta_ {i}}{s _ {i}} \right] \right\} \\ = \sum_ {i = 1} ^ {H _ {k} \times W _ {k}} \left\{\left[ \alpha + \log \beta \Gamma (\alpha) - (\alpha + 1) \psi (\alpha) \right] - \left[ \alpha_ {i} + \log \beta_ {i} \Gamma \left(\alpha_ {i}\right) - \left(\alpha_ {i} + 1\right) \psi \left(\alpha_ {i}\right) \right] \right\} \\ = \sum_ {i = 1} ^ {H _ {k} \times W _ {k}} \left\{\left(\alpha_ {i} - \alpha\right) \psi \left(\alpha_ {i}\right) + \left(\log \Gamma (\alpha) - \log \Gamma \left(\alpha_ {i}\right)\right) + \alpha \left(\log \beta_ {i} - \log \beta\right) + \alpha_ {i} \left(\frac {\beta}{\beta_ {i}} - 1\right) \right\}. \tag {S1} \\ \end{array}
$$
A.2 Log-likelihood Derivation
To calculate the semantic-aware variance  $\sigma_{i}$  for log-likelihood computation, we approximate it using the mode value  $(\sigma_{i})$  from the inverse Gamma distribution  $IG(\alpha_{i},\beta_{i})$ . This approximation is employed for computational efficiency, following the approach used in VDNet [52]. Therefore, in Equation S2 and Equation S3, we use  $h(\sigma_i^2 |x)$  that predicts statistics of semantic-aware variances (i.e.,  $\sigma_{i}$ ) from input image  $\pmb{x}$  as:
$$
\begin{array}{l} \log p _ {Z _ {i} ^ {\eta}} \left(\boldsymbol {z} _ {i}\right) = E _ {h \left(\sigma_ {i} ^ {2} \mid \boldsymbol {x}\right)} \left[ \log p \left(\boldsymbol {z} _ {i} \mid 0, \sigma_ {i} ^ {2}\right) \right] \\ = \int h \left(\sigma_ {i} ^ {2} | \boldsymbol {x}\right) \log p \left(\boldsymbol {z} _ {i} | 0, \sigma_ {i} ^ {2}\right) d \sigma^ {2} \\ = \int h \left(\sigma_ {i} ^ {2} | \boldsymbol {x}\right) \left\{- \frac {1}{2} \log 2 \pi - \frac {1}{2} \log \sigma_ {i} ^ {2} - \frac {\left| \left| \boldsymbol {z} _ {i} \right| \right| _ {2} ^ {2}}{2 \sigma_ {i} ^ {2}} \right\} d \sigma^ {2} \\ = - \frac {1}{2} \log 2 \pi - \frac {1}{2} \int h \left(\sigma_ {i} ^ {2} | \boldsymbol {x}\right) \log \sigma_ {i} ^ {2} d \sigma_ {i} ^ {2} - \frac {1}{2} \left\| \boldsymbol {z} _ {i} \right\| _ {2} ^ {2} \int h \left(\sigma_ {i} ^ {2} | \boldsymbol {x}\right) \frac {1}{\sigma_ {i} ^ {2}} d \sigma_ {i} ^ {2} \tag {S2} \\ = - \frac {1}{2} \log 2 \pi - \frac {1}{2} E [ \log \sigma_ {i} ^ {2} ] - \frac {1}{2} | | \boldsymbol {z} _ {i} | | _ {2} ^ {2} E [ \frac {1}{\sigma_ {i} ^ {2}} ] \\ = - \frac {1}{2} \log 2 \pi - \frac {1}{2} \left(\log \beta_ {i} - \psi (\alpha_ {i})\right) - \frac {\alpha_ {i}}{2 \beta_ {i}} | | \boldsymbol {z} _ {i} | | _ {2} ^ {2}, \\ \end{array}
$$