Package: ai2-tango
AI2 Tango replaces messy directories and spreadsheets full of file versions by organizing experiments into discrete steps that can be cached and reused throughout the lifetime of a research project.Quick linksDocumentationPyPI PackageContributingLicenseIn this READMEQuick startInstallationInstalling with PIPInstalling with CondaInstalling from sourceChecking your installationDocker imageFAQTeamLicenseQuick startCreate a Tango step:# hello.pyfromtangoimportstep@step()defhello(name:str)->str:message=f"Hello,{name}!"print(message)returnmessageAnd create a corresponding experiment configuration file:// hello.jsonnet{steps:{hello:{type:"hello",name:"World",}}}Then run the experiment using a local workspace to cache the result:tangorunhello.jsonnet-w/tmp/workspaceYou'll see something like this in the output:Starting new run expert-llama
● Starting step "hello"...
Hello, World!
✓ Finished step "hello"
✓ Finished run expert-llamaIf you run this a second time the output will now look like this:Starting new run open-crab
✓ Found output for step "hello" in cache...
✓ Finished run open-crabYou won't see "Hello, World!" this time because the result of the step was found in the cache, so it wasn't run again.For a more detailed introduction check out theFirst Stepswalk-through.Installationai2-tangorequires Python 3.8 or later.Installing withpipai2-tangois availableon PyPI. Just runpipinstallai2-tangoTo install with a specific integration, such astorchfor example, runpipinstall'ai2-tango[torch]'To install with all integrations, runpipinstall'ai2-tango[all]'Installing withcondaai2-tangois available on conda-forge. You can install just the base package withcondainstalltango-cconda-forgeYou can pick and choose from the integrations with one of these:condainstalltango-datasets-cconda-forge
condainstalltango-torch-cconda-forge
condainstalltango-wandb-cconda-forgeYou can also install everything:condainstalltango-all-cconda-forgeEven thoughai2-tangoitself is quite small, installing everything will pull in a lot of dependencies.
Don't be surprised if this takes a while!Installing from sourceTo installai2-tangofrom source, first clonethe repository:gitclonehttps://github.com/allenai/tango.gitcdtangoThen runpipinstall-e'.[all]'To install with only a specific integration, such astorchfor example, runpipinstall-e'.[torch]'Or to install just the base tango library, you can runpipinstall-e.Checking your installationRuntangoinfoto check your installation.Docker imageYou can build a Docker image suitable for tango projects by usingthe official Dockerfileas a starting point for your own Dockerfile, or you can simply use one of ourprebuilt imagesas a base image in your Dockerfile. For example:# Start from a prebuilt tango base image.# You can choose the right tag from the available options here:# https://github.com/allenai/tango/pkgs/container/tango/versionsFROMghcr.io/allenai/tango:cuda11.3# Install your project's additional requirements.COPYrequirements.txt.RUN/opt/conda/bin/pipinstall--no-cache-dir-rrequirements.txt# Install source code.# This instruction copies EVERYTHING in the current directory (build context),# which may not be what you want. Consider using a ".dockerignore" file to# exclude files and directories that you don't want on the image.COPY..Make sure to choose the right base image for your use case depending on the version of tango you're using and the CUDA version that your host machine supports.
You can see a list of all available image tagson GitHub.FAQWhy is the library named Tango?The motivation behind this library is that we can make research easier by composing it into well-defined steps.  What happens when you choreograph a number of steps together?  Well, you get a dance.  And since ourteam's leaderis part of a tango band, "AI2 Tango" was an obvious choice!How can I debug my steps through the Tango CLI?You can run thetangocommand throughpdb. For example:python-mpdb-mtangorunconfig.jsonnetHow is Tango different fromMetaflow,Airflow, orredun?We've found that existing DAG execution engines like these tools are great for production workflows but not as well suited for messy, collaborative research projects
where code is changing constantly. AI2 Tango was builtspecificallyfor these kinds of research projects.How does Tango's caching mechanism work?AI2 Tango caches the results of steps based on theunique_idof the step. Theunique_idis essentially a hash of all of the inputs to the step along with:the step class's fully qualified name, andthe step class'sVERSIONclass variable (an arbitrary string).Unlike other workflow engines likeredun, Tango doesnottake into account the source code of the class itself (other than its fully qualified name) because we've found that using a hash of the source code bytes is way too sensitive and less transparent for users.
When you change the source code of your step in a meaningful way you can just manually change theVERSIONclass variable to indicate to Tango
that the step has been updated.Teamai2-tangois developed and maintained by the AllenNLP team, backed bythe Allen Institute for Artificial Intelligence (AI2).
AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.
To learn more about who specifically contributed to this codebase, seeour contributorspage.Licenseai2-tangois licensed underApache 2.0.
A full copy of the license can be foundon GitHub.