title: Systematic Error Analysis and Labeling
emoji: 🦭
colorFrom: yellow
colorTo: pink
sdk: streamlit
sdk_version: 1.10.0
app_file: app.py
pinned: false
license: apache-2.0
SEAL
Systematic Error Analysis and Labeling (SEAL) is an interactive tool for discovering systematic errors in NLP models via clustering on high-loss example groups and semantic labeling for interpretability of those error-groups. It supports fine-grained analytical visualization for interactively zooming into potential systematic bugs and features for crafting prompts to label those bugs semantically.
Table of Contents
Installation
Please use python>=3.8 since some dependencies require that for installation.
git clone https://huggingface.co/spaces/nazneen/seal
cd seal
pip install --upgrade pip
pip install -r requirements.txt
Quickstart
streamlit run app.py
Running Locally
To run seal on any text classification model and dataset, please use the notebooks provided in ./assets/notebooks/ and replace the model and datasets with any HF datasets and model on the hub https://huggingface.co/models.
If you need to run inference on a dataset, please run ./util/run_inference.py with the appropriate HF model and dataset. You can also use the same script to select the model's layer for extracting the representation of the input examples.