C omb LM : Adapting Black-Box Language Models through Small Fine-Tuned Models (2023)

Abstract: Methods for adapting language models (LMs) to new tasks and domains have traditionally assumed white-box access to the model, and work by modifying its parameters. However, this is incompatible with a recent trend in the field, where the highest quality models are only available as black-boxes through inference APIs. Even when the model weights are available, the computational cost of fine-tuning large LMs can be prohibitive for most practitioners. In this work, we present a lightweight method for adapting large LMs to new domains and tasks, assuming no access to their weights or intermediate activations. Our approach fine-tunes a small white-box LM and combines it with the large black-box LM at the probability level through a small network, learned on a small validation set. We validate our approach by adapting a large LM (OPT-30B) to several domains and a downstream task (machine translation), observing improved performance in all cases, of up to 9%, while using a domain expert 23x smaller.

## introduction natural language processing (nlp) has witnessed remarkable progress in recent years thanks to the development of increasingly powerful lms @xcite @xcite . since these models are usually generalists, it is often of interest to adapt them to new domains, underrepresented or not found in the original training data. typically, domain adaptation techniques assume white-box access to the model parameters, for example by fine-tuning on a particular target domain @xcite . however, this approach has become increasingly infeasible given the ongoing paradigm shift in the field-state-of-the-art models like gpt-4 and palm-2 are only accessible as black-boxes through inference apis and, even when the model weights figure 1 : illustration of our approach. we leverage a large black-box lm and a small white-box lm, finetuned on a domain-specific corpus. we combine both models' outputs at the probability level, through a combination function learned on a small fitting set, requiring very little compute. the resulting model adapts the large black-box to the target domain, performing better than either of the original ones. in this work, we propose a simple and lightweight approach to adapt black-box lms to new domains, without requiring access to weights or intermediate activations. our method consists of two main steps: (1) training a small, white-box model on the desired target domain, and (2) learning a function that combines the probability distributions from the large black-box lm and the small domain expert lm, producing a new probability distribution. the combination function is a small neural network that is trained on a small validation dataset. we evaluate our method by adapting a black-box model to three distinct domains and a downstream task-machine translation (mt). in all cases, we observe that the combined model outperforms both the large black-box model and the small domain expert. this shows that it is possible to adapt blackbox lms to new domains, opening an exciting line of research. ## proposed method our approach works in two steps: (1) we train a small domain expert lm, and (2) we learn a function that combines the outputs of the domain expert lm and a large black-box lm at the probability level. more concretely, an lm defines a probability distribution over the possible continuations of any given text. that is, given a sequence of tokens @xmath0, x 2 , ..., x n ) ∈ v * , where v is the model vocabulary, an lm parametrizes p lm (y next |x), the probability that y next is the continuation of x in a text. we let p s denote our small domain expert lm, and p l denote the large black-box generalist lm. our combination function f defines a new combined probability distribution p c : ## results we next present our main results on domain adaptation ( §4.1) and mt ( §4.2). ## analysis in this section, we study the following aspects of our approach: in order to study how the performance of the adapted model varies with respect to the amount of data used to fit the combination function, we fit each combination function three times, on a varying number of tokens. we report results for the amazon reviews dataset in as expected, performance improves with more training data. however, the difference varies across methods. for example, constant-scalar, which has a very low capacity, performs equally well when trained on 100 or 1000 sequences. on the other hand, the full-scalar and full-vector functions, that take the entire probability distribution as input, benefit from more training sequences. the entropyscalar combination strikes a good balance, performing well across the board, and retaining strong performance when fit on as little as 100 sequences. ## related work we present related work on domain adaptation of lms ( §6.1), and language modeling through domain experts ( §6.2). ## conclusions in this work, we present a method for adapting black-box lms to new domains and tasks, requiring access to probability-level outputs only. we first fine-tune a small domain expert white-box lm (a) log-probability difference between the small and large lm. the small fine-tuned lm gave higher probabilities to the green tokens, while the large black-box lm gave higher probability to the red ones. figure 3 : difference between the small fine-tuned lm and the large black-box lm according to log-probability (a) and predicted weight (b). the closer the two match, the better the learned combination is at predicting which model will be "right" for a given token. the text sample was chosen randomly from the amazion reviews testset. on a domain-specific corpus, and then combine it with the large black-box through a combination function learned on a small fitting set, yielding an adapted lm. additionally, our method requires only access to probability level outputs, and thus allows to leverage powerful models optimized for inference or behind apis, without the need for white-box access to the weights. we experiment on several datasets and a downstream task, as well as performing extensive analysis of our method, reaching several conclusions: while our approach is effective, observed performance is still not close to the max prob oracle, which represents the ideal system where 100% of the weight is given to the best model at each time step. in future work, we would like to investigate the reasons for this gap, and potential ways of addressing it.