Title: A Conformalized Inference on Unobservable Variables
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords: confidence interval, neural network, conformal prediction, latent variable

Abstract: Quantifying uncertainty in predicted unobservable variables is a critical area of research in statistics, artificial intelligence, and empirical science. Most scientific studies assume a specific structure involving unobservable variables for the data-generating process and draw inferences from a parameter of interest within that framework. Conformal prediction is a popular model-agnostic method for constructing prediction intervals for new observations. However, it typically requires observed true labels to build the prediction interval, making it unsuitable for unobserved latent variables. We propose a method to construct a prediction interval by leveraging sample-splitting of the training data and analyzing the discrepancy between two independently trained models. To ensure the identifiability of the distribution of this conformity score, we introduce a few assumptions regarding the distribution of the residuals of the predictions. Furthermore, we propose a residual orthogonalization to satisfy these assumptions with a coordinating regularization term. The performance of the proposed method was evaluated using both simulation and large language model experiments.