please help with this error:

Saved successfully!
Skip to main content
Copy of Intro to Argilla.ipynb
Copy of Intro to Argilla.ipynb_Notebook unstarred
Argilla
Why
To ensure AI solutions are safe, accurate, and efficient, we need teamwork across various roles and experiences.

The key to effective, impactful teamwork in AI is collaborating on the data we use to train and evaluate it.

What
An open-source collaborative platform to build robust language models with human and machine feedback.

For
AI/ML/NLP engineers, data scientists, and computational linguists.
Subject matter experts and business profiles
Wanting to
Fine-tune, evaluate, and maintain:

Predictive NLP models
Large Language Models
How
Argilla SDK: Python library for AI teams to build iterative and human-in-the-loop workflows.
Argilla UI: Easy to use interface for exploratory data labelling.
Quickstart
What we'll show:

Deploy a private Argilla Space on Hugging Face and connect to it
Use the new task_templates to create a text classification dataset
Fine-tune some small models
[1]
1m
%pip install argilla datasets setfit -qqq
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 420.4/420.4 kB 3.6 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 510.5/510.5 kB 6.7 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.9/75.9 kB 5.8 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.9/75.9 kB 3.0 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.1/17.1 MB 21.1 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.3/116.3 kB 7.9 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 194.1/194.1 kB 10.0 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.8/134.8 kB 10.0 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 163.3/163.3 kB 16.1 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.1/84.1 kB 6.3 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 5.9 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 5.7 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.7/23.7 MB 42.1 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 823.6/823.6 kB 38.5 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.1/14.1 MB 61.1 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 731.7/731.7 MB 2.2 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.6/410.6 MB 3.7 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.6/121.6 MB 8.4 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.5/56.5 MB 10.0 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 MB 7.6 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.0/196.0 MB 5.9 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.0/166.0 MB 7.3 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.1/99.1 kB 11.1 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.1/21.1 MB 57.9 MB/s eta 0:00:00
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
chex 0.1.86 requires numpy>=1.24.1, but you have numpy 1.23.5 which is incompatible.
[2]
0s
import os

# you need to go to https://huggingface.co/settings/tokens and paste your token here
os.environ['HF_TOKEN'] = "<TRUFFLEHOG_REDACTED_HUGGINGFACE>"
[12]
0s
import argilla as rg

rg.init(
    api_url="https://bowersjames-test2.hf.space",
    api_key="admin.apikey",
    # we need this because it's a private space
    extra_headers={"Authorization": f"Bearer {os.environ['HF_TOKEN']}"},
)

/usr/local/lib/python3.10/dist-packages/argilla/client/client.py:178: UserWarning: No workspace configuration was detected. To work with Argilla datasets, specify a valid workspace name on `rg.init` or set it up through the `rg.set_workspace` function.
  warnings.warn(
Read source dataset
[13]
0s
import pandas as pd

df = pd.read_csv(
    "https://gist.githubusercontent.com/dvsrepo/f0e81dd7b826c4430e7021006ad730d1/raw/940de40bbab27d66804445fc88e5fc2c6c178f23/gistfile1.txt"
    ).sample(frac=1.0)
df

Next steps:
Create dataset in Argilla
Here will use Task templates which simplify dataset configuration for well-known tasks

[14]
0s
dataset = rg.FeedbackDataset.for_text_classification(
    labels=["good", "nonsense"],
)
[15]
0s
dataset
FeedbackDataset(
   fields=[TextField(name='text', title='Text', required=True, type='text', use_markdown=False)]
   questions=[LabelQuestion(name='label', title='Label', description='Classify the text by selecting the correct label from the given list of labels.', required=True, type='label_selection', labels=['good', 'nonsense'], visible_labels=None)]
   guidelines=This is a text classification dataset that contains texts and labels. Given a set of texts and a predefined set of labels, the goal of text classification is to assign one label to each text based on its content. Please classify the texts by making the correct selection.)
   metadata_properties=[])
   vectors_settings=[])
)
[16]
0s
records = [
    rg.FeedbackRecord(
        fields={"text": text}
        #suggestions=[{"question_name": "label", "value": "good",}] this can be used to add pre-filled responses
    )

    for text in df.text
]
[17]
0s
records[0]
FeedbackRecord(fields={'text': 'Create a budget planning for beginners.'}, metadata={}, vectors={}, responses=[], suggestions=(), external_id=None)
[18]
0s
dataset.add_records(records)

#remote_dataset = dataset.push_to_argilla(name="good-or-nonsense-v3", workspace="admin")
Fine-tune some models
Here we will use the Argilla Trainer module

SetFit
[19]
0s
from argilla.feedback import ArgillaTrainer, TrainingTask

dataset = rg.FeedbackDataset.from_argilla(name="good-or-nonsense", workspace="admin")

[ ]
task = TrainingTask.for_text_classification(
    text=dataset.field_by_name("text"),
    label=dataset.question_by_name("label")
)
[ ]
trainer = ArgillaTrainer(
    dataset=dataset,
    task=task,
    framework="setfit"
)
[ ]
trainer.train(output_dir="setfit")

[ ]
trainer.predict("What is a plant?")
{'good': 0.6471568267896435, 'nonsense': 0.3528431732103566}
[ ]
records[0]
FeedbackRecord(fields={'text': 'Which exercises are effective for building core strength?'}, metadata={}, responses=[], suggestions=(SuggestionSchema(question_name='label', type=None, score=None, value='good', agent=None),), external_id=None)
spaCy
[ ]
dataset = rg.FeedbackDataset.from_argilla(name="good-or-nonsense", workspace="admin")

trainer_spacy = ArgillaTrainer(
    dataset=dataset,
    task=task,
    framework="spacy"
)

trainer_spacy.train(output_dir="spacy")
doc = trainer_spacy.predict("my dog feels sick")
/usr/local/lib/python3.10/dist-packages/argilla/client/feedback/dataset/remote/dataset.py:796: UserWarning: A local `FeedbackDataset` returned because `prepare_for_training` is not supported for `RemoteFeedbackDataset`. `RemoteFeedbackDataset`.pull().prepare_for_training(*args, **kwargs)` is applied.
  warnings.warn(
WARNING:argilla.client.feedback.dataset.local.dataset:spaCy `lang` is not provided. Using `en`(English) as default language.
INFO:ArgillaTrainer:            ArgillaBaseTrainer info:
            _________________________________________________________________
            These baseline params are fixed:
                dataset: RemoteFeedbackDataset(
   id=73bf03ad-258b-455e-8fbf-ba15a96f47ba
   name=good-or-nonsense
   workspace=Workspace(id=0bd9fbc6-06c5-4620-ab0d-b61a9b1945c5, name=admin, inserted_at=2023-11-09 12:36:54.164894, updated_at=2023-11-09 12:36:54.164894)
   url=https://dvilasuero-private-argilla.hf.space/dataset/73bf03ad-258b-455e-8fbf-ba15a96f47ba/annotation-mode
   fields=[RemoteTextField(id=UUID('d7572d4b-07e4-4017-8931-26f66bab3512'), client=None, name='text', title='Text', required=True, type='text', use_markdown=False)]
   questions=[RemoteLabelQuestion(id=UUID('f4465c00-ea0b-481e-bbbc-562cbdcf740d'), client=None, name='label', title='Label', description=None, required=True, type='label_selection', labels=['good', 'nonsense'], visible_labels=None)]
   guidelines=This is a text classification dataset that contains texts and labels. Given a set of texts and a predefined set of labels, the goal of text classification is to assign one label to each text based on its content. Please classify the texts by making the correct selection.)
                task: formatting_func=None text=RemoteTextField(id=UUID('d7572d4b-07e4-4017-8931-26f66bab3512'), client=None, name='text', title='Text', required=True, type='text', use_markdown=False) label=LabelQuestionUnification(question=RemoteLabelQuestion(id=UUID('f4465c00-ea0b-481e-bbbc-562cbdcf740d'), client=None, name='label', title='Label', description=None, required=True, type='label_selection', labels=['good', 'nonsense'], visible_labels=None), strategy=<LabelQuestionStrategy.MAJORITY: 'majority'>)
                train_size: None
                seed: None


            <class 'argilla.client.feedback.training.frameworks.spacy.ArgillaSpaCyTrainer'> info:
            _________________________________________________________________
            The parameters are configurable via `trainer.update_config()`:
                WARNING:`ArgillaSpaCyTrainer.update_config` only supports the update of the `training` arguments defined in the `config.yaml`.

`ArgillaSpaCyTrainer`
	dev_corpus: corpora.dev
	train_corpus: corpora.train
	seed: ${system.seed}
	gpu_allocator: ${system.gpu_allocator}
	dropout: 0.1
	accumulate_gradient: 1
	patience: 1600
	max_epochs: 0
	max_steps: 20000
	eval_frequency: 200
	frozen_components: []
	annotating_components: []
	before_to_disk: None
	before_update: None

            Using the trainer:
            _________________________________________________________________
            `trainer.train(output_dir)` to train to start training. `output_dir` is the directory to save the model automatically.
            `trainer.predict(text, as_argilla_records=True)` to make predictions.
            `trainer.save(output_dir)` to save the model manually.

WARNING:ArgillaSpaCyTrainer:Note that the spaCy training is expected to be used through the CLI rather than programmatically, so the dataset needs to be dumped into the disk and then loaded from disk. More information at https://spacy.io/usage/training#api
INFO:ArgillaSpaCyTrainer:Dumping the train dataset to ./train.spacy
ℹ Pipeline: ['textcat']
ℹ Initial learn rate: 0.001
E    #       LOSS TEXTCAT  CATS_SCORE  SCORE 
---  ------  ------------  ----------  ------
  0       0          0.25      100.00    1.00
200     200         18.26      100.00    1.00
400     400          5.30      100.00    1.00
600     600          2.61      100.00    1.00
800     800          1.54      100.00    1.00
1000    1000          1.01      100.00    1.00
1200    1200          0.72      100.00    1.00
1400    1400          0.53      100.00    1.00
1600    1600          0.41      100.00    1.00
[ ]
doc.cats
{'good': 0.5347529053688049, 'nonsense': 0.4652470648288727}
[ ]
import argilla as rg

import os

rg.init(
    api_url="https://argilla-llm-eval.hf.space",
    api_key="owner.apikey"
)
/usr/local/lib/python3.10/dist-packages/argilla/client/client.py:165: UserWarning: No workspace configuration was detected. To work with Argilla datasets, specify a valid workspace name on `rg.init` or set it up through the `rg.set_workspace` function.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/argilla/client/client.py:182: UserWarning: You're connecting to Argilla Server 1.16.0 using a different client version (1.18.0).
This may lead to potential compatibility issues during your experience.
To ensure a seamless and optimized connection, we highly recommend aligning your client version with the server version.
  warnings.warn(
Colab paid products - Cancel contracts here
  0s
completed at 10:34 PM