Mind the Generation Process: Fine-Grained Confidence Estimation During
  LLM Generation
Authors: name: Jinyi Han, name: Tingyun Li, name: Shisong Chen, name: Jie Shi, name: Xinyi Wang, name: Guanglei Yue, name: Jiaqing Liang, name: Xin Lin, name: Liqian Wen, name: Zulong Chen, name: Yanghua Xiao

While large language models (LLMs) have demonstrated remarkable performance
across diverse tasks, they fundamentally lack self-awareness and frequently
exhibit overconfidence, assigning high confidence scores to incorrect
predictions. Accurate confidence estimation is therefore critical for enhancing
the trustworthiness and reliability of LLM-generated outputs. However, existing
approaches suffer from coarse-grained scoring mechanisms that fail to provide
fine-grained, continuous confidence estimates throughout the generation
process. To address these limitations, we introduce FineCE, a novel confidence
estimation method that delivers accurate, fine-grained confidence scores during
text generation. Specifically, we first develop a comprehensive pipeline for
constructing training data that effectively captures the underlying
probabilistic distribution of LLM responses, and then train a model to predict
confidence scores for arbitrary text sequences in a supervised manner.
Furthermore, we propose a Backward Confidence Integration (BCI) strategy that
leverages information from the subsequent text to enhance confidence estimation
for the current sequence during inference. We also introduce three strategies
for identifying optimal positions to perform confidence estimation within the
generation process. Extensive experiments on multiple benchmark datasets
demonstrate that FineCE consistently outperforms existing classical confidence
estimation methods. Our code and all baselines used in the paper are available
on GitHub.