Package: raitracker
Responsible AI TrackerResponsible AI Tracker is a JupyterLab Extension for managing, tracking, and comparing results of machine learning experiments for model improvement. Using this extension, users can view models, code, and visualization artifacts within the same framework enabling fast model iteration and evaluation processes. The extension is a work-in-progress research prototype to test and understand tooling functionalities and visualizations that can be helpful to data scientists. If you would like to propose new ideas for improvement feel free to contact the development team atrai-toolbox@microsoft.comor create new issues in this repository.This repo is a part of theResponsible AI Toolbox, a suite of tools providing a collection of model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.Main functionalities of the tracker include:Managing and linking model improvement artifacts: the extension encourages clean and systematic data science practices by allowing users to associate the notebook used to create a model with the resulting model. These practices support careful model tracking and systematic experimentation.Disaggregated model evaluation and comparisons: the model comparison table in the extension provides an in-depth comparison between the different models registered in the extension. This comparison contrasts performance results across different data cohorts and metrics, following a disaggregated approach, which goes beyond single-score performance numbers and highlights cohorts of data for which a model may perform worse than its older versions. Read more about disaggregated analysishere.Integration with the Responsible AI Mitigations library: as data scientists experiment and ideate different steps for model improvement, theResponsible AI Mitigations Libraryhelps them implement different mitigation techniques in python that may improve model performance and can be targeted towards specified cohorts of interests.TourWatch avideo tourof the Responsible AI Tracker and follow along using the notebooks and datasethere.InstallationThe Responsible AI Tracker can be deployed on Windows or Ubuntu, using anaconda or python.The Responsible AI Tracker prerequisites:NodejsPython(versions supported 3.9to3.10.6)JupyterLabIf you use pip:pipinstalljupyterlab==3.6.1If you use conda:condainstall-cconda-forgejupyterlab==3.6.1The Responsible AI Tracker has two installation options:The default installation only installs the essential packages.pipinstallraitrackerThe installation With the [all] flag installs the essential packages plus PyTorch, and Tensorflow.pipinstallraitracker[all]Installation through the JupyterLab Extension Manager coming soon.RunningStart up JupyterLab using:jupyterlabThe extension should be available in the left vertical bar. For ideas on getting started, watch thevideo tourand follow along using the notebooks and datasethere.DependenciesjupyterlabfluentuinodejsreactreduxluminolodashbabelcodeMirrorwebpackmlflownumpypandasscikit-learnpytorchGetting helpWe encourage you to check the Responsible AI Trackerdocumentation.For Responsible AI Mitigations Library help seeResponsible AI Mitigations documentation.Seeherefor further support information.Bug reportsTo report a bug please read theguidelinesand then open aGithub issue.Feature requestsWe welcome suggestions for new features as they help make the project more useful for everyone. To request a feature please use thefeature request template.ContributingTo contribute code or documentation to the Responsible AI Tracker, please read thecontribution guidelines.Microsoft Open Source Code of conductTheMicrosoft  Code of conductoutlines expectations for participation in Microsoft-managed open source communities.TrademarksThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
trademarks or logos is subject to and must followMicrosoft's Trademark & Brand Guidelines.
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
Any use of third-party trademarks or logos are subject to those third-party's policies.Research and AcknowledgementsCurrent Maintainers:ThuVan Pham,Matheus Mendonça,Besmira Nushi,Rahee Ghosh Peshawaria,Marah Abdin,Mark Encarnación,Dany RouhanaPast Maintainers:Irina SpiridonovaResearch Contributors:Besmira Nushi,Jingya Chen,Rahee Ghosh Peshawaria,ThuVan Pham,Matheus Mendonça,Ece Kamar,Dany Rouhana