Efficient Latent Semantic Clustering for Scaling Test-Time Computation of LLM s (2025)

Abstract: Scaling test-time computation, generating and analyzing multiple or sequential outputs for a single input, has become a promising strategy for improving the reliability and quality of large language models (LLMs), as evidenced by advances in uncertainty quantification and multi-step reasoning. A key shared component is semantic clustering, which groups outputs that differ in form but convey the same meaning. Semantic clustering enables estimation of the distribution over the semantics of outputs and helps avoid redundant exploration of reasoning paths. However, existing approaches typically rely on external models, which introduce substantial computational overhead and often fail to capture context-aware semantics. We propose Latent Semantic Clustering (LSC), a lightweight and context-sensitive method that leverages the generator LLM’s internal hidden states for clustering, eliminating the need for external models. Our extensive experiment across various LLMs and datasets shows that LSC significantly improves the computational efficiency of test-time scaling while maintaining or exceeding the performance of existing methods.

## introduction scaling test-time computation has emerged as a promising strategy to improve the reliability and quality of responses from large language models (llms) by generating multiple responses for a single input @xcite @xcite @xcite . for reliability, uncertainty quantification methods assess the semantic diversity among the multiple outputs to estimate model confidence @xcite @xcite . for quality, reasoning methods explore and aggregate multiple reasoning paths @xcite . f1 score is plotted against log-scaled latency; circle size denotes squareroot memory usage. circles with the same hatch pattern denote the same model type (i.e., llm, sentenceembedding, or nli) and numbers inside circles indicate memory usage. lsc achieves the best performance with minimal latency and memory usage. detailed experimental results are described in a key component in such test-time scaling methods is semantic clustering, which groups responses with the same meaning (despite their diverse forms). indeed, effective clustering can improve uncertainty quantification @xcite by computing the distribution over distinct meanings of responses, and accelerate multi-step reasoning by avoiding unnecessary explorations of (semantically) duplicated reasoning paths @xcite . however, semantic clustering is challenging in open-ended tasks, where the output space is large, ambiguous, and diverse. reasoning tasks, in particular, are inherently open-ended. for semantic clustering, existing test-time scaling methods @xcite typically employ external models such as natural language inference (nli) @xcite or sentence embedding a concrete example with generated sequences by llms is shown in figure 5 . models @xcite . such approaches with external models incur significant computational overhead due to additional inference. in addition, they often fail to capture context-dependent semantics, as they operate separately from the generator of responses. as a result, they are both inefficient and less applicable to test-time scaling methods. to address these limitations, we propose a lightweight yet effective method for semantic clustering, called latent semantic clustering @xcite . the key idea is to directly leverage the internal latent representations of the generator, eliminating the need for external models (figure 2 ). lsc offers two main advantages for test-time scaling as shown in figure 1 . first, it is highly efficient-requiring only minimal memory to store hidden vectors and incurring virtually no additional computation. second, it naturally captures context-aware semantics embedded in the generator's representations, leading to more accurate clustering and effective improvement of test-time scaling methods. our experiment across various llms and datasets validates the efficiency and effectiveness of lsc in uncertainty quantification and multi-step reasoning. ## background: semantic clustering this section begins with a formal description of semantic clustering (section 2.1). we then describe how semantic clustering can be applied to en-hance uncertainty quantification (section 2.2) and to facilitate efficient exploration in llm reasoning (section 2.3), both of which commonly leverage test-time computation scaling. the related work is discussed in detail in appendix a. ## method for efficient semantic clustering, we propose latent semantic clustering (lsc) that captures context-aware semantics by leveraging the hidden states produced by the generator llms. our approach involves three key steps: (1) we extract the hidden states from the generator llm during inference; @xcite we construct an adjacency matrix that maps pairwise semantic similarity between the generated sequences; ## experiments in this section, we empirically evaluate whether the proposed lsc effectively leverages the outputs generated with scaled inference-time computation to improve performance and efficiency across various downstream llm tasks. to assess the robustness of lsc, we conduct experiments with multiple llms, including llama3-8b-instruct @xcite and mistral-7b-instruct-v0.1 @xcite . additional results are presented in appendix c. in what follows, we present results on uncertainty quantification tasks (section 4.1) and multi-step reasoning tasks (section 4.2). in both tasks, we use the hidden state from an intermediate llm layer, extracted at the last token of each generated sequence in lsc. ## conclusion in this work, we propose latent semantic clustering (lsc), a lightweight and context-aware approach for identifying semantically equivalent outputs generated by llms for test-time computation scaling. rather than relying on external models as in prior works, lsc leverages the generator llm's internal hidden states to perform contextaware semantic clustering with minimal overhead. our experiments show that lsc achieves strong performance on key tasks like uncertainty quantification and multi-step reasoning, while substantially reducing computational cost. lsc consistently outperforms prior semantic clustering methods in both clustering quality and computational efficiency. we believe lsc offers a practical and scalable solution, applicable to any white-box llms, for enhancing the reliability and quality of llm responses, thereby improving test-time computation scaling strategies.