Giskard on Spaces

Giskard is an AI model quality testing toolkit for LLMs, tabular, and NLP models. It consists of an open-source Python library for scanning and testing AI models and an AI Model Quality Testing app, which can now be deployed using Hugging Face’s Docker Spaces. Extending the features of the open-source library, the AI Model Quality Testing app enables you to:

• Debug tests to diagnose your issues

• Create domain-specific tests thanks to automatic model insights

• Compare models to decide which model to promote

• Collect business feedback of your model results

• Share your results with your colleagues for alignment

• Store all your QA objects (tests, data slices, evaluation criteria, etc.) in one place to work more efficiently

Visit Giskard’s documentation and Quickstart Guides to learn how to use the full range of tools provided by Giskard.

In the next sections, you’ll learn to deploy your own Giskard AI Model Quality Testing app and use it right from Hugging Face Spaces. This Giskard app is a self-contained application completely hosted on Spaces using Docker.

Deploy Giskard on Spaces

You can deploy Giskard on Spaces with just a few clicks:

You need to define the Owner (your personal account or an organization), a Space name, and the Visibility. If you don’t want to publicly share your models and quality tests, set your Space to Private.

Once you have created the Space, you’ll see the Building status. Once it becomes Running, your Space is ready to go. If you don’t see a change in the screen, refresh the page.

Request a free license

Once your Giskard Space is up and running, you’ll need to request a free license to start using the app. You will then automatically receive an email with the license file.

Create a new Giskard project

Once inside the app, start by creating a new project from the welcome screen.

Generate a Hugging Face Giskard Space Token and Giskard API key

The Giskard API key is used to establish communication between the environment where your AI models are running and the Giskard app on Hugging Face Spaces. If you’ve set the Visibility of your Space to Private, you will need to provide a Hugging Face user access token to generate the Hugging Face Giskard Space Token and establish a communication for access to your private Space. To do so, follow the instructions displayed in the settings page of the Giskard app.

Start the ML worker

Giskard executes your model using a worker that runs the model directly in your Python environment, with all the dependencies required by your model. You can either execute the ML worker:

• From your local notebook within the kernel that contains all the dependencies of your model

• From Google Colab within the kernel that contains all the dependencies of your model

• Or from your terminal within the Python environment that contains all the dependencies of your model

Simply run the following command within the Python environment that contains all the dependencies of your model:

giskard worker start -d -k GISKARD-API-KEY -u https://XXX.hf.space --hf-token GISKARD-SPACE-TOKEN

Upload your test suite, models and datasets

In order to start building quality tests for a project, you will need to upload model and dataset objects, and either create or upload a test suite from the Giskard Python library.

These actions will all require a connection between your Python environment and the Giskard Space. Achieve this by initializing a Giskard Client: simply copy the “Create a Giskard Client” snippet from the settings page of the Giskard app and run it within your Python environment. This will look something like this:

from giskard import GiskardClient

url = "https://user_name-space_name.hf.space"
api_key = "gsk-xxx"
hf_token = "xxx"

# Create a giskard client to communicate with Giskard
client = GiskardClient(url, api_key, hf_token)

If you run into issues, head over to Giskard’s upload object documentation page.

Feedback and support

If you have suggestions or need specific support, please join Giskard’s Discord community or reach out on Giskard’s GitHub repository.