Package: interlab
InterLab - toolkit for multi-agent interactionsWelcome to InterLab, a research-focused toolkit created to facilitate study and experimentation in the realm of agent interactions, particularly those based on Language Learning Models (LLMs). Our primary objective is to simplify the process of crafting, deploying, and inspecting complex and structured queries within the context of agent interactions, while also providing robust support for interaction logging, UI and visualization. While we maintain a broad scope and plan to include game theoretic agents and a variety of scenarios, our main emphasis lies in the sphere of LLM interactions.InterLab is developed at theAlignment of Complex Systems Research Groupand distributed under MIT license (seeLICENSE).Current status: InterLab is in open beta.InterLab is also active development, we use it internally for our experiments, and we want to make it available to the wider research community. Although we aim to limit the breaking changes between major versions, the API may change substantially as we refine the design and gather information about usage.If you use InterLab, want to share ideas, feedback or have any questions, please email us atgavento@acsresearch.org- we'd be happy to hear from you.OverviewInterLab is composed of three main packages:interlabcontains the core functionality and common abstractions of actors, environments, memory, and language model interactions, along with a few helpful scaffolds. The main modules there are:actorprovides framework for actor interactions, including a generic LLM single-shot agent and a web console for playing as an actor, along with actor memory systems. The agents may be queried for structured (typed JSON-like) or unstuctured actions.environmentproviding abstraction over partially observable environments.queriescontains powerful helpers for querying the models, in particular querying for any dataclass or Pydantic model with robust response parsing and retries, summarization tools etc.treetraceoffers comprehensive structured logging of nestedTracingNodess, storage for traing nodes, and custom visualization of content such as Images, generic HTML, and tracking f-string-like field substitutions in larger text.interlab_zooserves as a repository for specific and opinionated implementations of actors, scenarios, actor memory systems, tracing post-processing, and other tools that enhance the InterLab project. Its mission is to gather code that is both widely applicable and useful, while maintaining a compact and focused core package.Beyond that, some less-tested and experimental code can be found inexperimentalsubmodules accross the core package.Structured interaction log browserIn-notebook or independent browser for the structured logs, with live updates, support for JSON-like structured data and inline visualizations. Captures both high-level interaction structure and the low-level API calls for easy inspection and debugging.
Example screenshots (click to zoom in):Example notebooksYou can find Jupyter notebooks with a few worked-out exampleshere. The notebookcar_negotiation.ipynbis a good starting point for a simple bargaining simulation.InstallationYou can install the packageinterlabfrom PyPI usingpipor any other package manager.pip install interlabInstallation with development toolsThis repository utilizesPoetrypackage management, which is recommended for dependency installation and is required for InterLab development. Poetry automatically generates and manages a virtual environment for you, and also installsinterlabmodule itself. If you have poetry installed, running the following command inside this repository will install InterLab:poetry installpip andrequirements.txt.Alternatively,pipcan be used to install dependencies withpip install -r requirements.txt(core requirements) orpip install -r requirements-full.txt(including development tools, Jupyter Lab, etc.; equivalent topoetry install). To use InterLab in Google Colab, we use the requirements inrequirements-colab.txtto get aroung some Colab versioning conflicts.Development and experimentsJupyter Lab provides the simplest way to interact with the code and design experiments:poetry run jupyter lab
# Or without poetry, in the project root folder:
jupyter labAfter running the command, open the provided link in your browser.notebooks/car_negotiation.ipynbis a recommended starting point.Google ColabGoogle Colab often offers a lightweight alternative to setting up InterLab locally on your computer. Interlab comes with built-in colab compatibility and we have prepared aExample InterLab Colab experimentwith common setup and a simple example experiment with two LLMs interacting on behalf of their users.Note: API KeysIn order to use LLM provider serveics and APIs, you need to generate and provide the corresponding API keys. You can provide the keys as environment variables, via.envfile, or interactively on every run (e.g. in the colab). Storing keys in the notebook is possible but not recommended (as they easily leak into git or while sharing the code).API keys can be stored in a.envfile located in the notebook directory or your home directory. (This file is ignored bygitby default, providing an additional security measure.) The file is a simple text file with key=value pairs, for example:OPENAI_API_KEY=sk-...
OPENAI_API_ORG=org-...
ANTHROPIC_API_KEY=sk-ant-...
...You can then import these variables from the.envfile into a Jupyter notebook using thedotenvpackage:importdotenvdotenv.load_dotenv()Contributions and developmentTo contribute, please submit a pull request with your proposed changes based on the latestmainbranch.By submitting a pull request for this project, you agree to license your contribution under the MIT license to this project as written in theLICENSEfile.Citing InterLabIf you use InterLab in your researh, please cite it in your work using the "Cite this repository" github gadget or with:Tomáš Gavenčiak, Ada Böhm:InterLab [Computer software].2023.https://github.com/acsresearch/interlab