---
name: dynaword-contributor
description: Use this skill when contributing to faroese-dynaword with an agent. It covers adding or updating datasets, writing dataset cards and create.py scripts, running the required make commands, checking generated files for churn, and preparing Hugging Face PRs.
---

# Dynaword Agent Skills

This file is for contributors using an AI coding agent to work on `faroese-dynaword`.

## Scope

Use this guide when:

- adding a new dataset under `data/<dataset_name>/`
- updating an existing dataset
- regenerating descriptive statistics or plots
- preparing a Hugging Face PR

## Repo Facts

- package and environment management uses `uv`
- common commands live in `Makefile`
- tests live in `src/tests/`
- dataset cards live in `data/<dataset_name>/<dataset_name>.md`
- generated dataset files are usually:
  - `<dataset_name>.parquet`
  - `descriptive_stats.json`
  - `images/dist_document_length.png`

## Required Workflow

Read `CONTRIBUTING.md` before making changes.

For dataset additions, the normal sequence is:

1. `make install`
2. add `data/<dataset_name>/`
3. add `<dataset_name>.md`
4. add `<dataset_name>.parquet`
5. add `create.py`
6. register the dataset in `README.md`
7. run `make update-descriptive-statistics`
8. run `make bump-version`
9. update `CHANGELOG.md`
10. run `make test`

## Writing `create.py`

`create.py` is a python script that allow you to recreate the dataset from the source

When you add `create.py`, include script dependencies in the file header using the repo's existing pattern. 

If `create.py` imports from `dynaword`, pin the Dynaword source revision in the script header using `[tool.uv.sources]`.

Prefer the shared processing helpers from `src/dynaword/process_dataset.py`:

```python
from dynaword.process_dataset import (
    add_token_count,
    ensure_column_order,
    remove_duplicate_text,
    remove_empty_texts,
)
```

In the script comments, explain what the script does and how to run it:

```python
"""Download and process a Faroese source for Faroese Dynaword.

To run this script, set `GIT_LFS_SKIP_SMUDGE=1` so `uv` does not download large
Git LFS files while resolving the local Dynaword environment:

    GIT_LFS_SKIP_SMUDGE=1 uv run data/<dataset_name>/create.py
"""
```

## Writing Dataset Cards

Dataset card guidance now lives in [CONTRIBUTING.md](./CONTRIBUTING.md#writing-dataset-cards).

Read that section before drafting or editing a dataset card. It applies to agents as well.
## Testing

Run:

```bash
make test
```

Notes:

- one test is intentionally skipped in `src/tests/test_quality/test_duplicates.py`

## PR Checklist

Before opening or updating a PR, verify:

- tests pass with `make test`
- descriptive statistics were updated with `make update-descriptive-statistics`
- version was bumped with `make bump-version`
- `CHANGELOG.md` was updated when appropriate
- `create.py` includes script dependencies if present
- generated files were reviewed for unrelated churn

## Agent Conduct

If you are an agent operating in a contributor's local workspace:

- do not commit unless the user explicitly asks
- do not push unless the user explicitly asks
- avoid reverting unrelated local changes
- review generated markdown and plot changes before staging them
- prefer minimal, understandable diffs over fully automated code updates

## Suggested Self-Review Order

Before finalizing a dataset PR, inspect changes in this order:

1. `data/<dataset_name>/create.py`
2. `data/<dataset_name>/<dataset_name>.md`
3. `data/<dataset_name>/<dataset_name>.parquet`
4. `data/<dataset_name>/descriptive_stats.json`
5. `README.md`
6. `descriptive_stats.json`
7. files under `images/`
8. `pyproject.toml`
9. `CHANGELOG.md`