DisCoNet: Rethinking Adversarial Networks for Discriminator-Driven Distribution Modeling

Authors: Francisco Caetano, Christiaan Viviers, Peter H.N. de With, Fons van der Sommen
Primary area: generative models
Keywords: Covariate Shift, Deep Learning, Generative Adversarial Networks, Generative Modelling, Out-of-Distribution Detection

Abstract:
Out-of-distribution (OOD) detection holds significant importance across various applications. While semantic and domain-shift OOD problems are well-documented, this work focuses on the nuances of covariate shifts, which entail subtle perturbations or variations in the data distribution. These disturbances have proven to negatively impact machine learning performance. We have found that existing OOD detection methods often struggle to effectively distinguish covariate shifts from in-distribution instances, emphasizing the need for specialized solutions. Therefore, we propose DisCoNet, an Adversarial Variational Autoencoder (VAE) that rethinks the Generative Adversarial Networks paradigm. Instead of prioritizing the generator as the network's core, we focus on the discriminator, using the generator as a supporting training tool. DisCoNet uses the VAE's suboptimal outputs as negative samples to train the discriminator, thereby improving its ability to delineate the boundary between in-distribution samples and covariate shifts. By tightening this in-distribution boundary, DisCoNet achieves state-of-the-art results in public OOD detection benchmarks. The proposed model not only excels in detecting covariate shifts, achieving 98.9% AUROC on ImageNet-1K(-C), but also outperforms all prior methods on public semantic OOD benchmarks. With a model size of $\leq$ 25MB, it is highly effective on Far-OOD (OpenImage-O (99.4%) and iNaturalist (100.0%)) and Near-OOD (SSB-hard (99.9%) and NINCO (99.7%)) detection. The code will be made publicly available.

Decision: reject

Reviews and discussion:

Metareview: