Breaking the Ceiling of the LLM Community by Treating Token Generation as a Classification for Ensembling (2024)

Abstract: Ensembling multiple models has always been an effective approach to push the limits of existing performance and is widely used in classification tasks by simply averaging the classification probability vectors from multiple classifiers to achieve better accuracy. However, in the thriving open-source Large Language Model (LLM) community, ensembling methods are rare and typically limited to ensembling the full-text outputs of LLMs, such as selecting the best output using a ranker, which leads to underutilization of token-level probability information. In this paper, we treat the **G**eneration of each token by LLMs **a**s a **C**lassification (**GaC**) for ensembling. This approach fully exploits the probability information at each generation step and better prevents LLMs from producing early incorrect tokens that lead to snowballing errors. In experiments, we ensemble state-of-the-art LLMs on several benchmarks, including exams, mathematics and reasoning, and observe that our method breaks the existing community performance ceiling. Furthermore, we observed that most of the tokens in the answer are simple and do not affect the correctness of the final answer. Therefore, we also experimented with ensembling only key tokens, and the results showed better performance with lower latency across benchmarks.

## introduction large language models (llms) have demonstrated remarkable capabilities in a wide range of natural language processing tasks @xcite . over time, new and more powerful llms are continually being released, pushing the boundaries of the llm community @xcite . due to the diversity of data sources, architectures and training methods, different llms have strengths and weaknesses in different tasks and contexts (jiang 1 our code: https://github.com/yaoching0/gac ## analysis and prior work in this chapter, we review previous llm ensemble studies and their features, as well as the problems our approach addresses. previous studies can be categorized as follows: output-level ensemble methods select multiple candidate models and use their complete outputs for ensembling. @xcite trained an additional ranking model (pairranker) to score each candidate output and select the best one. @xcite and @xcite trained a router to select the most appropriate candidate model given a question. however, these methods are limited to the existing candidate outputs and become ineffective if all the outputs are incorrect. other studies have trained a fusion model to blend the outputs @xcite , overcoming the limitation of selecting only existing candidate outputs and often achieving superior results. however, the generalization of the fusion model is a major challenge, and they cannot fully exploit the probability information from each generation step. weight-level ensemble methods merge the weights of multiple models and are primarily used in multi-task learning @xcite . the expectation is that the merged model will inherit capabilities across multiple tasks. however, a limitation is that the architectures of the models to be merged must be homologous, which limits the use of the capabilities of the llm community. and it is rare to observe that the merged model outperforms the original models @xcite . training-level ensemble like fusellm @xcite uses the output probability vectors of multiple models during training to ensemble as labels, rather than one-hot labels. in effect, this is a specific form of distillation that allows the model being trained to gain more information from the probability outputs of the ensembled (teacher) models. however, distillation is mainly used to improve small models, making it difficult to further improve the sota llms. our work can overcome the above limitations by ensembling at each generation step, allowing the output not to be confined to the original candidate output space and homologous architectures, while fully exploiting the probability information at each step. our experiments in sec.4 will also show that the ensemble consistently outperforms any single model, even sota llms. the main challenge, however, is that different llms typically have inconsistent vocabularies, leading to different dimensions in the probability vectors produced by different models. the most intuitive solution is to take the union of the vocabularies of the ensembled llms, denoted v u , which includes all tokens from the participating models. a potential problem with this approach is that different models may tokenize the same word differently, leading to conflicts. however, most mainstream llms use bpe or bbpe @xcite to train tokenizers on sampled corpora, which tend to have similar sources (e.g. commoncrawl) and distributions. this results in consistent tokenization for common words. for example, both qwen1.5 and llama3 @xcite tokenize the word " alphabetically" into [" ġalphabet", "ically"]. if both models intend to output this word, they will assign a higher probability to " ġalphabet" first. we selected several popular llms @xcite @xcite and tokenized 5,000 commonly used english words @xcite , and then calculated the proportion of identical tokenization results between each pair of llms, as shown in fig. 2 . the proportion is above 90% for all pairs, indicating that such conflicts can be ignored in most cases. ## proposed method in this section, we will first introduce the overall ensemble process of our gac framework, and then explain the details in the following subsections. ## conclusion in this paper, we present a token-level ensembling framework called gac, which fully exploits the probability information at each generation step. in our experiments, we have surpassed the performance ceiling of open-source sota llms available at different time periods (sec.4.5), further narrowing the gap between open-source and proprietary models. this progress promotes the democratization of llms and provides new motivations for future research, enabling better exploitation of col-lective intelligence. in addition, we experimented with ensembling just a few tokens and found that this approach can achieve better performance with lower latency (sec.4.6), opening up new avenues for accelerating inference.