Tracking the perspectives of interacting language models (2024)

Abstract: Large language models (LLMs) are capable of producing high quality information at unprecedented rates. As these models continue to entrench themselves in society, the content they produce will become increasingly pervasive in databases that are, in turn, incorporated into the pre-training data, fine-tuning data, retrieval data, etc. of other language models. In this paper we formalize the idea of a communication network of LLMs and introduce a method for representing the perspective of individual models within a collection of LLMs. Given these tools we systematically study information diffusion in the communication network of LLMs in various simulated settings.

## introduction the success of large pre-trained models in natural language processing @xcite , computer vision @xcite , signal processing @xcite , among other domains @xcite across various computing and human benchmarks has brought them to the forefront of the technology-centric world. given their ability to produce human-expert level responses for a large set of knowledge-based questions @xcite , the content they produce is often propagated throughout forums that have influence over other models and human users @xcite . as such, it is important to develop sufficient frameworks and complementary tools to understand how information produced by these models affects the behavior of other models and human users. we refer to a system where a model can potentially influence other models as a system of interacting language models. beyond their ability to influence information on human-model forums, systems of interacting language models are interesting in their own right. in-sofar as an individual model is an intriguing proxy for an individual human foot_0 @xcite , a system of interacting language models is an intriguing proxy for human communities. systems of interacting language models are thus an alluring alternative or complement to studying human communities in the social sciences. for example, it is often infeasible or unethical to subject entire communities to different information paradigms to understand how individuals within the community -as well as the community itself -change in response to an intervention. these issues are less prominent for systems of interacting language models. further, there is potential for greater control in community membership and cross-community interactions, which may improve reproducibility and mitigate the effects of sociological confounders. in this paper, we study information diffusion in a system of interacting language models. we define information diffusion as the process by which information spreads and distorts across individuals or groups, typically through communication networks. the framework and methods that we develop can be applied to monitoring information diffusion in human-model forums and to the treatment of systems of interacting language models quantitatively as proxy human communities. the current standard @xcite for studying information diffusion in a system of interacting language models requires i) parameterizing models with different system prompts, contexts, weights, or collections of data, ii) providing an environment or template for model-to-model or model-to-dataset interactions, and iii) analyzing how the outputs of the models change after a sequence of interactions. for example, researchers include descriptions of desired model behavior or personality in the system prompt -e.g., "you have opinion a" is we introduce the perspective space of a collection of models to address the gap in quantitative methods for studying the diversity and evolution of model responses. the perspective space is an embedding-based representation of a collection of models designed to capture the relative differences in model responses for a fixed set of prompts. the method can be used to study information diffusion and general system dynamics by querying each model with the same set of queries at each time step. to demonstrate the effectiveness of the perspective space for understanding model-level diversity and for analyzing model-level and system dynamics, we formalize the system of interacting language models as a graph. the formalization enables systematic study of the effect of different communication structures on information diffusion that is otherwise not possible. our contribution is two-fold: i) we model a system of interacting language models as a graph and systematically study the effect of different communication structures on information diffusion. ii) we introduce the perspective space as a method to quantitatively analyze information diffustion in a population of language models. ## a communication network of llms consider a system that consists of a collection of language models @xmath0, . . . , f n } and databases @xmath1, . . . , d n ′ }. given a set of prompts x, systems deploying model @xmath2, context retrieval, etc. -to produce more relevant outputs with respect to x. the outputs of the updated model may be used to update a (potentially different) database d ′ ∈ d. the updated database can then be used as a fine-tuning, retrieval, etc. database for a (potentially different) model f ′ ∈ f. this set of interactions between a model and a database may occur across various models and various databases in the system. as described, this system can be modeled as a graph @xmath3, e) where @xmath4, v ′ ) is in e if vertex v has influence on vertex v ′ . for example, the edge (d, f ) exists if f has access to d for retrieval augmentation or if it can use a subset of d as fine-tuning data. conversely, the edge (f, d) exists if the output of f can influence the content of dataset d. our primary interest is the dynamics of a system of interacting llms and databases where the vertex and edge sets are indexed by a discrete variable @xmath5, . . . , t }. there are many ways components of the graph may vary in t in such a system. for example, the dataset d (t) ∈ v (t) may be updated based on the outputs of the model f (t) ∈ v (t) or the model f (t) may change after fine-tuning on the contents of the dataset d (t) . in both cases t+1) . similarly, external factors such as the terms of use for a dataset may change to disallow its use for retrieval augmentation or a model may lose write-access to a dataset. in both cases e (t) ̸ = e (t+1) . figure 1 illustrates simple examples of systems of llms as graphs, including three structures that are studied in the simulated settings in section 4. ## defining a perspective space with surrogate data kernels the system-of-llms-as-a-graph perspective provides a framework to systematically study the effect of different vertex sets and edge structures on the flow of information through the system as a function of t. the framework does not, however, provide a method to track the information flow. for this, we introduce an adaptation of the embeddingbased data kernel presented in @xcite . for our purposes, an embedding function g is a mapping to real-valued vectors. the methodology can be extended to compare the embedding spaces of multiple models f 1 , . . . , f n at once by considering the pairwise distance matrix of the corresponding data kernels. in particular, the classical multi-dimensional scaling @xcite ) of the n × n matrix m with entries m @xmath6, x) -a(g j , x) || f yields d-dimensional euclidean representations of the model f i with respect to x. after this transformation, inference methods designed for euclidean objects can be used for model-level analysis such as inferring differences in the training data mixtures. the data kernel, as defined in @xcite , requires the model f i to have an associated embedding function g i . unfortunately, for some state-of-the-art llms such as openai's gpt series, anthropic's claude series, etc., an associated embedding function is unavailable and the data kernel cannot be constructed. to rectify this, we replace a model's associated embedding function with a surrogate embedding function g : x ′ → r p that is not necessarily related to any of the llms under study. the surrogate embedding function is not a dropand-replace solution for model comparisons, however, since the embedding g(x) is independent of f i . instead, we query the model with the elements of x and embed the responses f i (x) with g: the surrogate data kernel the surrogate data kernel is a m × p matrix representation of model f i with respect to g and x. ## simulating systems of interacting llms we next simulate three different systems of interacting llms to demonstrate the effectiveness of the perspective space and its derivatives for capturing model and system dynamics for different underlying communication structures. the initial models in each system are based on an instance of the 410-million parameter model from the pythia suite @xcite that has been instructiontuned using databricks' dolly 15k @xcite . for each system we further fine-tune the base model on random question-pairs from setting specific topics from yahoo! answers (ya) dataset @xcite to promote response variation. we provide details on the instruction-tuning of the base model and the fine-tuning of the initial models in appendix a and appendix b, respectively. we use all-minilm-l6-v2, a sentence embedding function from @xcite based on @xcite hosted on the huggingface hub @xcite , as the surrogate embedding function and the implementation of cmds from graspologic @xcite . in the three case studies (c.s.) we consider, each model interacts with another model in the system at each t. an interaction consists of model i asking model j ̸ = i a random set of questions from a fixed question bank and fine-tuning model i using the resulting question-answer pairs as fine-tuning data. for a given t, the underlying communication structure e (t) determines which set of model interactions are possible for model i. in particular, the interviewed model j is randomly selected from the set of models such that (f j , f i ) ∈ e (t) . the fixed question bank is used as the evaluation set to induce the perspective space. while each system that we study technically consists of models and databases, each dataset is associated with only a single model. for convenience we discuss the systems as if the models themselves are directly connected. our setting -where models are sequentially trained on each others outputs without intervention -can be viewed as a generalization of a single model sequentially trained on its own outputs as studied in @xcite . at the end of each simulation setting we provide examples that motivated the case study. ## related work our work is closely related to simulating groups of computational agents to study sociological and cultural phenomena @xcite and to continual learning @xcite . the former has seen renewed interest with the recent successes of llms. in particular, llms are -as of this writing -the computational tool that produces language artifacts most similar to ours and, as such, are an intriguing prospect for multi-agent sociological and cultural simulations. recent work has included objectiveless behavioral studies @xcite , studying the formation of social networks @xcite , tracking opinion dynamics via classification of llm response @xcite , and analyzing document collaboration @xcite . our work extends these by introducing a framework to systematically study interventions and by introducing a quantitative method for tracking the evolution of agent perspectives. continual learning @xcite is largely concerned with how a single agent adapts to previously unseen inference tasks while avoiding "catastrophically forgetting" @xcite previous tasks. our setting is slightly different, since we have multiple agents and no explicit task -though a large movement in perspective space is likely highly correlated to change in performance on language benchmarks related to the evaluation set. indeed, large enough movements in perspective space and the emergence of model sinks when training a model recursively is related to catastrophic forgetting @xcite . ## conclusion we introduced a system-of-llms-as-a-graph to enable systematic interventions to a system of interacting llms and the perspective space to quantitatively study the corresponding evolution of the system. we used these tools to highlight differences in paired systems across three case studies. for the particular interaction mechanic and update function that we used in our simulations, the model behaviors in perspective space consistently demonstrated initial model exploration and, in most cases, the emergence and persistence of model sinks. further, we used derivatives of the perspective space such as the iso-mirror, polarization, and clustering to highlight differences in the evolution of paired systems. for example, we observed differences in the isomirror (stable versus unstable after disruption) and clustering (global sinks versus local sinks after disruption) in the first case study; differences in the sensitivity of the average perspective of nonadversarial models to an adversarial perspective across number of victims and frequency of attack in the second case study; and differences in the behavior of polarization of two classes of models in the third case study.