Repository: argilla-io/argilla

Issue:
Python Client side record validation using `pydantic.validators`
**Is your feature request related to a problem? Please describe.**
Errors only get detected after having tried to log the data.

**Describe the solution you'd like**
Detect errors from the python client when creating the record, instead of doing so only after trying to log.

**Describe alternatives you've considered**
N.A.

**Additional context**
https://docs.pydantic.dev/usage/validators/


Discussion:
tomaarsen: On a tangentially related note: The docs at https://docs.pydantic.dev/usage/model_config/#options may be useful to improve the user experience further. For example, we can supply `error_msg_templates` to override the error messages with our custom ones.
dvsrepo: > On a tangentially related note: The docs at https://docs.pydantic.dev/usage/model_config/#options may be useful to improve the user experience further. For example, we can supply `error_msg_templates` to override the default error messages on validation errors with our custom ones.

Hi @tomaarsen this is a highly important point. I think having clearer error messages can be very beneficial. Is this something you tackle/experiment with?
tomaarsen: @dvsrepo I do not have any experience with it personally. I was looking into the documentation for it on the `pydantic` side now, and I'm afraid that it seems like the `pydantic` team is planning on removing the `error_msg_templates` configuration property, as discussed [here](https://docs.pydantic.dev/blog/pydantic-v2/#removed-features-limitations):
> The following Config properties will be removed:
> * ...
> * `error_msg_templates`, it's not properly documented anyway, error messages can be customized with external logic if required

That said, there are likely still ways to improve on the error messages (as they mention, too).

codecov[bot]: # [Codecov](https://codecov.io/gh/argilla-io/argilla/pull/2337?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io) Report
Base: **91.86**% // Head: **91.47**% // Decreases project coverage by **`-0.40%`** :warning:
> Coverage data is based on head [(`69005a6`)](https://codecov.io/gh/argilla-io/argilla/pull/2337?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io) compared to base [(`5f0627c`)](https://codecov.io/gh/argilla-io/argilla/commit/5f0627cf523fdfcb2efeb7ae5663c3e9a845ecb9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##           develop    #2337      +/-   ##
===========================================
- Coverage    91.86%   91.47%   -0.40%     
===========================================
  Files          161      161              
  Lines         7895     7896       +1     
===========================================
- Hits          7253     7223      -30     
- Misses         642      673      +31     
```

| Flag | Coverage Δ | |
|---|---|---|
| pytest | `91.47% <100.00%> (-0.40%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/argilla-io/argilla/pull/2337?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io) | Coverage Δ | |
|---|---|---|
| [src/argilla/client/models.py](https://codecov.io/gh/argilla-io/argilla/pull/2337?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io#diff-c3JjL2FyZ2lsbGEvY2xpZW50L21vZGVscy5weQ==) | `93.78% <100.00%> (+0.03%)` | :arrow_up: |
| [.../server/daos/backend/client\_adapters/opensearch.py](https://codecov.io/gh/argilla-io/argilla/pull/2337?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io#diff-c3JjL2FyZ2lsbGEvc2VydmVyL2Rhb3MvYmFja2VuZC9jbGllbnRfYWRhcHRlcnMvb3BlbnNlYXJjaC5weQ==) | `83.26% <0.00%> (-8.18%)` | :arrow_down: |
| [...lla/server/daos/backend/client\_adapters/factory.py](https://codecov.io/gh/argilla-io/argilla/pull/2337?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io#diff-c3JjL2FyZ2lsbGEvc2VydmVyL2Rhb3MvYmFja2VuZC9jbGllbnRfYWRhcHRlcnMvZmFjdG9yeS5weQ==) | `76.31% <0.00%> (-5.27%)` | :arrow_down: |
| [...gilla/labeling/text\_classification/label\_errors.py](https://codecov.io/gh/argilla-io/argilla/pull/2337?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io#diff-c3JjL2FyZ2lsbGEvbGFiZWxpbmcvdGV4dF9jbGFzc2lmaWNhdGlvbi9sYWJlbF9lcnJvcnMucHk=) | `86.58% <0.00%> (-3.66%)` | :arrow_down: |
| [...rgilla/server/daos/backend/search/query\_builder.py](https://codecov.io/gh/argilla-io/argilla/pull/2337?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io#diff-c3JjL2FyZ2lsbGEvc2VydmVyL2Rhb3MvYmFja2VuZC9zZWFyY2gvcXVlcnlfYnVpbGRlci5weQ==) | `90.75% <0.00%> (-2.90%)` | :arrow_down: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/argilla-io/argilla/pull/2337?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=argilla-io).


Patch:
diff --git a/README.md b/README.md
index 87474b4e3..b034e12f1 100644
--- a/README.md
+++ b/README.md
@@ -61,7 +61,7 @@
 
 ### Advanced NLP labeling
 
-- Programmatic labeling using [weak supervision](https://docs.argilla.io/en/latest/guides/techniques/weak_supervision.html). Built-in label models (Snorkel, Flyingsquid)
+- Programmatic labeling using [rules and weak supervision](https://docs.argilla.io/en/latest/guides/programmatic_labeling_with_rules.html). Built-in label models (Snorkel, Flyingsquid)
 - [Bulk-labeling](https://docs.argilla.io/en/latest/reference/webapp/features.html#bulk-annotate) and [search-driven annotation](https://docs.argilla.io/en/latest/guides/features/queries.html)
 - Iterate on training data with any [pre-trained model](https://docs.argilla.io/en/latest/tutorials/libraries/huggingface.html) or [library](https://docs.argilla.io/en/latest/tutorials/libraries/libraries.html)
 - Efficiently review and refine annotations in the UI and with Python
@@ -71,93 +71,34 @@
 ### Monitoring
 
 - Close the gap between production data and data collection activities
-- [Auto-monitoring](https://docs.argilla.io/en/latest/guides/steps/3_deploying.html) for [major NLP libraries and pipelines](https://docs.argilla.io/en/latest/tutorials/libraries/libraries.html) (spaCy, Hugging Face, FlairNLP)
+- [Auto-monitoring](https://docs.argilla.io/en/latest/guides/log_load_and_prepare_data.html) for [major NLP libraries and pipelines](https://docs.argilla.io/en/latest/tutorials/libraries/libraries.html) (spaCy, Hugging Face, FlairNLP)
 - [ASGI middleware](https://docs.argilla.io/en/latest/tutorials/notebooks/deploying-texttokenclassification-fastapi.html) for HTTP endpoints
-- Argilla Metrics to understand data and model issues, [like entity consistency for NER models](https://docs.argilla.io/en/latest/guides/steps/4_monitoring.html)
+- Argilla Metrics to understand data and model issues, [like entity consistency for NER models](https://docs.argilla.io/en/latest/guides/measure_datasets_with_metrics.html)
 - Integrated with Kibana for custom dashboards
 
 ### Team workspaces
 
 - Bring different users and roles into the NLP data and model lifecycles
-- Organize data collection, review and monitoring into different [workspaces](https://docs.argilla.io/en/latest/getting_started/installation/user_management.html#workspace)
+- Organize data collection, review and monitoring into different [workspaces](https://docs.argilla.io/en/latest/getting_started/installation/configurations/user_management.html)
 - Manage workspace access for different users
 
 ## Quickstart
-Argilla is composed of a `Python Server` with Elasticsearch as the database layer, and a `Python Client` to create and manage datasets.
+👋 Welcome! If you have just discovered Argilla this is the best place to get started. Argilla is composed of:
 
-To get started you need to **install the client and the server** with `pip`:
-```bash
-
-pip install "argilla[server]"
-
-```
-
-Then you need to **run [Elasticsearch (ES)](https://www.elastic.co/elasticsearch)**.
-
-The simplest way is to use`Docker` by running:
-
-```bash
+* Argilla Client: a powerful Python library for reading and writing data into Argilla, using all the libraries you love (transformers, spaCy, datasets, and any other).
 
-docker run -d --name elasticsearch-for-argilla --network argilla-net  -p 9200:9200 -p 9300:9300 -e "ES_JAVA_OPTS=-Xms512m -Xmx512m" -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:8.5.3
-
-```
-> :information_source: **Check [the docs](https://docs.argilla.io/en/latest/getting_started/quickstart.html) for further options and configurations for Elasticsearch.**
-
-Finally you can **launch the server**:
-
-```bash
-
-python -m argilla
-
-```
-> :information_source:  The most common error message after this step is related to the Elasticsearch instance not running. Make sure your Elasticsearch instance is running on http://localhost:9200/. If you already have an Elasticsearch instance or cluster, you point the server to its URL by using [ENV variables](#)
+* Argilla Server and UI: the API and UI for data annotation and curation.
 
+To get started you need to:
 
-🎉 You can now access Argilla UI pointing your browser at http://localhost:6900/.
+1. Launch the Argilla Server and UI.
 
-**The default username and password are**  `argilla`  **and**  `1234`.
-
-Your workspace will contain no datasets. So let's use the `datasets` library to create our first datasets!
-
-First, you need to install `datasets`:
-```bash
-
-pip install datasets
-
-```
-
-Then go to your Python IDE of choice and run:
-```python
-
-import pandas as pd
-import argilla as rg
-from datasets import load_dataset
-
-# load dataset from the hub
-dataset = load_dataset("argilla/gutenberg_spacy-ner", split="train")
-
-# read in dataset, assuming its a dataset for text classification
-dataset_rg = rg.read_datasets(dataset, task="TokenClassification")
-
-# log the dataset to the Argilla web app
-rg.log(dataset_rg, "gutenberg_spacy-ner")
-
-# load dataset from json
-my_dataframe = pd.read_json(
-    "https://raw.githubusercontent.com/recognai/datasets/main/sst-sentimentclassification.json")
-
-# convert pandas dataframe to DatasetForTextClassification
-dataset_rg = rg.DatasetForTextClassification.from_pandas(my_dataframe)
-
-# log the dataset to the Argilla web app
-rg.log(dataset_rg, name="sst-sentimentclassification")
-```
+2. Pick a tutorial and start rocking with Argilla using Jupyter Notebooks, or Google Colab.
 
-This will create two datasets that you can use to do a quick tour of the core features of Argilla.
+To get started follow the steps [on the Quickstart docs page](https://docs.argilla.io/en/latest/getting_started/quickstart.html).
 
 > 🚒 **If you find issues, get direct support from the team and other community members on the [Slack Community](https://join.slack.com/t/rubrixworkspace/shared_invite/zt-whigkyjn-a3IUJLD7gDbTZ0rKlvcJ5g)**
 
-For getting started with your own use cases, [go to the docs](https://docs.argilla.io).
 ## Principles
 -  **Open**: Argilla is free, open-source, and 100% compatible with major NLP libraries (Hugging Face transformers, spaCy, Stanford Stanza, Flair, etc.). In fact, you can **use and combine your preferred libraries** without implementing any specific interface.
 
diff --git a/src/argilla/client/models.py b/src/argilla/client/models.py
index 609a92762..845e7f8eb 100644
--- a/src/argilla/client/models.py
+++ b/src/argilla/client/models.py
@@ -121,7 +121,9 @@ class _Validators(BaseModel):
         return values
 
     class Config:
+        # https://docs.pydantic.dev/usage/model_config/#options
         extra = "forbid"
+        validate_assignment = True
 
 
 class BulkResponse(BaseModel):


Test patch:
diff --git a/tests/client/test_models.py b/tests/client/test_models.py
index 75d37bde1..bf3d72c36 100644
--- a/tests/client/test_models.py
+++ b/tests/client/test_models.py
@@ -306,3 +306,24 @@ def test_nat_to_none_to_datetime():
 def test_text2text_prediction_validator(prediction, expected):
     record = Text2TextRecord(text="mock", prediction=prediction)
     assert record.prediction == expected
+
+
+@pytest.mark.parametrize(
+    "record",
+    [
+        TextClassificationRecord(text="This is a test"),
+        TokenClassificationRecord(
+            text="This is a test", tokens="This is a test".split()
+        ),
+        Text2TextRecord(text="This is a test"),
+    ],
+)
+def test_record_validation_on_assignment(record):
+    with pytest.raises(ValidationError):
+        record.prediction = "rubbish"
+
+    with pytest.raises(ValidationError):
+        record.annotation = [("rubbish",)]
+
+    with pytest.raises(ValidationError):
+        record.vectors = "rubbish"