Package: aim
Drop a star to support Aim ⭐Join Aim discord communityAn easy-to-use & supercharged open-source experiment trackerAim logs your training runs and any AI Metadata, enables a beautiful UI to compare, observe them and an API to query them programmatically.SEAMLESSLY INTEGRATES WITH:TRUSTED BY ML TEAMS FROM:AimStack offers enterprise support that's beyond core Aim. Contact viahello@aimstack.ioe-mail.About•Demos•Ecosystem•Quick Start•Examples•Documentation•Community•Blogℹ️ AboutAim is an open-source, self-hosted ML experiment tracking tool designed to handle 10,000s of training runs.Aim provides a performant and beautiful UI for exploring and comparing training runs.
Additionally, its SDK enables programmatic access to tracked metadata — perfect for automations and Jupyter Notebook analysis.Aim's mission is to democratize AI dev tools 🎯Log Metadata Across Your ML Pipeline 💾Visualize & Compare Metadata via UI 📊ML experiments and any metadata trackingIntegration with popular ML frameworksEasy migration from other experiment trackersMetadata visualization via Aim ExplorersGrouping and aggregationQuerying using Python expressionsRun ML Trainings Effectively ⚡Organize Your Experiments 🗂️System info and resource usage trackingReal-time alerting on training progressLogging and configurable notificationsDetailed run information for easy debuggingCentralized dashboard for holistic viewRuns grouping with tags and experiments🎬 DemosCheck out live Aim demos NOW to see it in action.Machine translation experimentslightweight-GAN experimentsTraining logs of a neural translation model(from WMT'19 competition).Training logs of 'lightweight' GAN, proposed in ICLR 2021.FastSpeech 2 experimentsSimple MNISTTraining logs of Microsoft's "FastSpeech 2: Fast and High-Quality End-to-End Text to Speech".Simple MNIST training logs.🌍 EcosystemAim is not just an experiment tracker. It's a groundwork for an ecosystem.
Check out the two most famous Aim-based tools.aimlflowAim-spaCyExploring MLflow experiments with a powerful UIan Aim-based spaCy experiment tracker🏁 Quick startFollow the steps below to get started with Aim.1. Install Aim on your training environmentpip3installaim2. Integrate Aim with your codefromaimimportRun# Initialize a new runrun=Run()# Log run parametersrun["hparams"]={"learning_rate":0.001,"batch_size":32,}# Log metricsforiinrange(10):run.track(i,name='loss',step=i,context={"subset":"train"})run.track(i,name='acc',step=i,context={"subset":"train"})See the full list of supported trackable objects(e.g. images, text, etc)here.3. Run the training as usual and start Aim UIaimupLearn moreMigrate from other toolsAim has built-in converters to easily migrate logs from other tools.
These migrations cover the most common usage scenarios.
In case of custom and complex scenarios you can use Aim SDK to implement your own conversion script.TensorBoard logs converterMLFlow logs converterWeights & Biases logs converterIntegrate Aim into an existing projectAim easily integrates with a wide range of ML frameworks, providing built-in callbacks for most of them.Integration with Pytorch IgniteIntegration with Pytorch LightningIntegration with Hugging FaceIntegration with Keras & tf.KerasIntegration with Keras TunerIntegration with XGboostIntegration with CatBoostIntegration with LightGBMIntegration with fastaiIntegration with MXNetIntegration with OptunaIntegration with PaddlePaddleIntegration with Stable-Baselines3Integration with AcmeIntegration with ProphetQuery runs programmatically via SDKAim Python SDK empowers you to query and access any piece of tracked metadata with ease.fromaimimportRepomy_repo=Repo('/path/to/aim/repo')query="metric.name == 'loss'"# Example query# Get collection of metricsforrun_metrics_collectioninmy_repo.query_metrics(query).iter_runs():formetricinrun_metrics_collection:# Get run paramsparams=metric.run[...]# Get metric valuessteps,metric_values=metric.values.sparse_numpy()Set up a centralized tracking serverAim remote tracking server allows running experiments in a multi-host environment and collect tracked data in a centralized location.See the docs on how toset up the remote server.Deploy Aim on kubernetesThe official Aim docker image:https://hub.docker.com/r/aimstack/aimA guide on how to deploy Aim on kubernetes:https://aimstack.readthedocs.io/en/latest/using/k8s_deployment.htmlRead the full documentation onaimstack.readthedocs.io📖🆚 Comparisons to familiar toolsTensorBoard vs AimTraining run comparisonOrder of magnitude faster training run comparison with AimThe tracked params are first class citizens at Aim. You can search, group, aggregate via params - deeply explore all the tracked data (metrics, params, images) on the UI.With tensorboard the users are forced to record those parameters in the training run name to be able to search and compare. This causes a super-tedius comparison experience and usability issues on the UI when there are many experiments and params.TensorBoard doesn't have features to group, aggregate the metricsScalabilityAim is built to handle 1000s of training runs - both on the backend and on the UI.TensorBoard becomes really slow and hard to use when a few hundred training runs are queried / compared.Beloved TB visualizations to be added on AimEmbedding projector.Neural network visualization.MLflow vs AimMLFlow is an end-to-end ML Lifecycle tool.
Aim is focused on training tracking.
The main differences of Aim and MLflow are around the UI scalability and run comparison features.Aim and MLflow are a perfect match - check out theaimlflow- the tool that enables Aim supoerpowers on Mlflow.Run comparisonAim treats tracked parameters as first-class citizens. Users can query runs, metrics, images and filter using the params.MLFlow does have a search by tracked config, but there are no grouping, aggregation, subplotting by hyparparams and other comparison features available.UI ScalabilityAim UI can handle several thousands of metrics at the same time smoothly with 1000s of steps. It may get shaky when you explore 1000s of metrics with 10000s of steps each. But we are constantly optimizing!MLflow UI becomes slow to use when there are a few hundreds of runs.Weights and Biases vs AimHosted vs self-hostedWeights and Biases is a hosted closed-source MLOps platform.Aim is self-hosted, free and open-source experiment tracking tool.🛣️ RoadmapDetailed milestonesTheAim product roadmap:sparkle:TheBacklogcontains the issues we are going to choose from and prioritize weeklyThe issues are mainly prioritized by the highly-requested featuresHigh-level roadmapThe high-level features we are going to work on the next few months:In progressAim SDK low-level interfaceDashboards – customizable layouts with embedded explorersErgonomic UI kitText ExplorerNext-upAim UIRuns managementRuns explorer – query and visualize runs data(images, audio, distributions, ...) in a central dashboardExplorersDistributions ExplorerSDK and StorageScalabilitySmooth UI and SDK experience with over 10.000 runsRuns managementCLI commandsReporting - runs summary and run details in a CLI compatible formatManipulations – copy, move, delete runs, params and sequencesCloud storage support – store runs blob(e.g. images) data on the cloudArtifact storage – store files, model checkpoints, and beyondIntegrationsML Frameworks:Shortlist: scikit-learnResource management toolsShortlist: Kubeflow, SlurmWorkflow orchestration toolsDoneLive updates (Shipped:Oct 18 2021)Images tracking and visualization (Start:Oct 18 2021, Shipped:Nov 19 2021)Distributions tracking and visualization (Start:Nov 10 2021, Shipped:Dec 3 2021)Jupyter integration (Start:Nov 18 2021, Shipped:Dec 3 2021)Audio tracking and visualization (Start:Dec 6 2021, Shipped:Dec 17 2021)Transcripts tracking and visualization (Start:Dec 6 2021, Shipped:Dec 17 2021)Plotly integration (Start:Dec 1 2021, Shipped:Dec 17 2021)Colab integration (Start:Nov 18 2021, Shipped:Dec 17 2021)Centralized tracking server (Start:Oct 18 2021, Shipped:Jan 22 2022)Tensorboard adaptor - visualize TensorBoard logs with Aim (Start:Dec 17 2021, Shipped:Feb 3 2022)Track git info, env vars, CLI arguments, dependencies (Start:Jan 17 2022, Shipped:Feb 3 2022)MLFlow adaptor (visualize MLflow logs with Aim) (Start:Feb 14 2022, Shipped:Feb 22 2022)Activeloop Hub integration (Start:Feb 14 2022, Shipped:Feb 22 2022)PyTorch-Ignite integration (Start:Feb 14 2022, Shipped:Feb 22 2022)Run summary and overview info(system params, CLI args, git info, ...) (Start:Feb 14 2022, Shipped:Mar 9 2022)Add DVC related metadata into aim run (Start:Mar 7 2022, Shipped:Mar 26 2022)Ability to attach notes to Run from UI (Start:Mar 7 2022, Shipped:Apr 29 2022)Fairseq integration (Start:Mar 27 2022, Shipped:Mar 29 2022)LightGBM integration (Start:Apr 14 2022, Shipped:May 17 2022)CatBoost integration (Start:Apr 20 2022, Shipped:May 17 2022)Run execution details(display stdout/stderr logs) (Start:Apr 25 2022, Shipped:May 17 2022)Long sequences(up to 5M of steps) support (Start:Apr 25 2022, Shipped:Jun 22 2022)Figures Explorer (Start:Mar 1 2022, Shipped:Aug 21 2022)Notify on stuck runs (Start:Jul 22 2022, Shipped:Aug 21 2022)Integration with KerasTuner (Start:Aug 10 2022, Shipped:Aug 21 2022)Integration with WandB (Start:Aug 15 2022, Shipped:Aug 21 2022)Stable remote tracking server (Start:Jun 15 2022, Shipped:Aug 21 2022)Integration with fast.ai (Start:Aug 22 2022, Shipped:Oct 6 2022)Integration with MXNet (Start:Sep 20 2022, Shipped:Oct 6 2022)Project overview page (Start:Sep 1 2022, Shipped:Oct 6 2022)Remote tracking server scaling (Start:Sep 11 2022, Shipped:Nov 26 2022)Integration with PaddlePaddle (Start:Oct 2 2022, Shipped:Nov 26 2022)Integration with Optuna (Start:Oct 2 2022, Shipped:Nov 26 2022)Audios Explorer (Start:Oct 30 2022, Shipped:Nov 26 2022)Experiment page (Start:Nov 9 2022, Shipped:Nov 26 2022)HuggingFace datasets (Start:Dec 29 2022,Feb 3 2023)👥 CommunityAim README badgeAdd Aim badge to your README, if you've enjoyed using Aim in your work:[![Aim](https://img.shields.io/badge/powered%20by-Aim-%231473E6)](https://github.com/aimhubio/aim)Cite Aim in your papersIn case you've found Aim helpful in your research journey, we'd be thrilled if you could acknowledge Aim's contribution:@software{Arakelyan_Aim_2020,author={Arakelyan, Gor and Soghomonyan, Gevorg and {The Aim team}},doi={10.5281/zenodo.6536395},license={Apache-2.0},month={6},title={{Aim}},url={https://github.com/aimhubio/aim},version={3.9.3},year={2020}}Contributing to AimConsidering contibuting to Aim?
To get started, please take a moment to read theCONTRIBUTING.mdguide.Join Aim contributors by submitting your first pull request. Happy coding! 😊Made withcontrib.rocks.More questions?Read the docsOpen a feature request or report a bugJoin Discord community server