# A phone and a web application to collect and transcribe

Mélanie Jouitteau, Alice Millour, Loïc Grobol, Jean-Yves Antoine

► To cite this version:

Mélanie Jouitteau, Alice Millour, Loïc Grobol, Jean-Yves Antoine. A phone and a web application to collect and transcribe. 7th of the Dialect Syntax in Westernmost Europe research network, Université Humboldt de Berlin, Sep 2025, Berlin, Germany. [hal-05250020](https://hal.science/hal-05250020v1)

HAL Id: hal-05250020

https://hal.science/hal-05250020v1

Submitted on 11 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# A phone and a web application to collect and transcribe

Mélanie Jouitteau, Alice Millour, Loic Grobol, Jean-Yves Antoine

IKER CNRS, Universities of Paris 8, Paris Nanterre, Tours

9 septembre 2025

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>
<figure><img src="image_9.png" /></figure>

* As linguists, do we know what is going on ?

* What's a linguist to do ?

* Valorize sub-products of syntactic research

* More we can do :

* Preservation of our raw material : linguistic diversity

* Building datasets for computational linguistics AND AI training

* Use our knowledge of the ecosystems of linguistic diversity

<figure><img src="image_10.png" /></figure>
<figure><img src="image_11.png" /><figcaption>Figure 1: We propose CVQA, a large-scale multilingual VQA benchmark, representing the cultures of 30 countries and 31 different languages across 10 diverse categories, comprising 10k samples.</figcaption></figure>

<figure><img src="image_12.png" /></figure>
<figure><img src="image_13.png" /><figcaption>(a) Using English-only question-option pairs</figcaption></figure>

<caption>(b) Using local-language question-option pairs</caption>

Figure 3: Model performance per Country-Language pair. The blue lines indicate separation by continent. All models show similar behaviour in the majority of cases, despite having different sizes.

<figure><img src="image_14.png" /></figure>
Compared NLP Inclusion of Breton, ELG dashboard

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>
# Why does it matter in bilingual ecosystems?

* Aggravation of diglossia :

* SMS dictation > usages turn to dominant language

* Automatized derush > economic advantage for filmmaking in dominant language

* Automatized meeting synthesis in dominant language > more work

* ... Whereas automatized translation could open new usage spaces (offering facilities for monolinguals)

<figure><img src="image_17.png" /></figure>
# The starting pack for AI multimodality

## A Hiking Path to the starting pack for a Multimodal Chatbot

<figure><img src="image_18.png" /></figure>

<figure><img src="image_19.png" /></figure>
# The starting pack for AI multimodality

## A Hiking Path to the starting pack for a Multimodal Chatbot

<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>
* Valorize subproducts of linguistics research

* Grammar exs. constitute useful corpora

* ARBRES wikigrammar

* https://arbres.iker.cnrs.fr

* 100 visits a day

* 5,000 pages, 12500 extractable examples

```html
{ class="prettytable"
| (1) || eur || mell || gwezenn
|-
||| [[un|un]] || [[mell|grand]] || arbres.[[-enn|SG]]
|-
||| colspan="15" | 'un grand arbre'
|}
```

Figure – wikitable of a Breton example

<figure><img src="image_22.png" /></figure>
* Evidence for significative quality improvement when small quality corpora is used, probably because of high grammatical diversity

<table><caption>Table 1: Evaluation results on the Ofis test dataset.</caption><thead><tr><th>Model</th><th>BLEU</th><th>ChrF++</th><th>TER</th></tr></thead><tbody><tr><td>Apertium</td><td>24.15</td><td>50.23</td><td>63.93</td></tr><tr><td>m2m100-418M</td><td>0.58</td><td>11.85</td><td>114.49</td></tr><tr><td>+OPUS</td><td>30.01</td><td>50.16</td><td>55.37</td></tr><tr><td>+ARBRES</td><td>37.68</td><td>56.99</td><td>48.65</td></tr></tbody></table>

Figure – BLEU = n-gram overlap, ChrF++ = / character, TER is number of edits necessary
* AI Inclusion < available data

* *Common Voice* lacks independent participation motives & naturelness sentences (< 3")

* Let's equip speech data acquisition by supporting actual community practices (low impact on uses, natural data)

<figure><img src="image_23.png" /></figure>
<figure><img src="image_24.png" /></figure>

* YAR Project (now-2026; 100.000€)

* two main tools in pilot development for Breton

* a mobile application for geolocated speech collection

* a web platform for participatory transcription

<figure><img src="image_25.png" /></figure>
<figure><img src="image_26.png" /></figure>

## Deliverable early 2026

* Records a sound and links it to a geotag

* Maps the recordings

* Allows one to listen to tracks on a route, or from home

<figure><img src="image_27.png" /></figure>
<figure><img src="image_28.png" /></figure>

## User cases

* hiking path

* treasure hunt

* tribute to a local figure

* collection of testimonies on traditional place names

* storytelling walks in the moor (Breton legends)

* geology classes

* augmented tourism (museums, touristic routes)

<figure><img src="image_29.png" /></figure>
Place names tour of Fulup Kloareg in
Douarnenez, each saturday morning

Kontañ-kaoz, gathering of Oct 29, 2024
organized by Yann-Bêr Rivalin, picture
Jaklin Pennaneac'h

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>
* about 107,000 speakers, all bilinguals

* Advantages of geolocalisation :

* Dynamic cartography of dialectal diversity

* Creation of learning paths anchored in the territory

* Visibilisation of Breton in public spaces countering clandestinisation of the language

<figure><img src="image_32.png" /></figure>

<figure><img src="image_33.png" /></figure>
<figure><img src="image_34.png" /></figure>

## Copyright laws & enlightened consent

* Explicit consent form

* CC-BY-SA, or CC-BY-SA-NC

* Both can be used for science

* Concrete incentive for the implication of companies

<figure><img src="image_35.png" /></figure>
# YAR mobile app - Under Android

<figure><img src="image_36.png" /></figure>

## Naturalness of data

* both read and natural material

* Length diversity

* Public data gives what people think is of enough quality

* Non-public data provides us with very much needed beginners material

* Users app profile provides metadata

<figure><img src="image_37.png" /></figure>
# YAR.bzh, A collaborative transcription platform

ELAN 6.0 - 350681[1].eaf

Fichier Edition Annotation Acteur Type Rechercher Affichage Options Fenêtre Aide

<figure><img src="image_38.png" /></figure>
YAR.bzh, A collaborative transcription platform

<figure><img src="image_39.png" /></figure>

Collaborative transcription with an AI
in the group

* We transcribe a starting sizeable aligned corpus (under ELAN)

* Transcribers co-build an annotation guide available online

* We train an AI "pre-transcriptor" (Kaldi or Whisper)

* As the platform is used by teachers, we incrementally fine-tune the AI ASR support with human high quality transcriptions.
YAR.bzh, A collaborative transcription platform

<figure><img src="image_40.png" /></figure>

Collaborative transcription with an AI
in the group

* Real time collaboration

* Special admin rights to teachers

* Gamification of the task (race for transcript, hidden clues)

* Humans can correct the AI and make fun of it.

* Usefulness provides beginners with symbolic legitimation in "entering the community".

<figure><img src="image_41.png" /></figure>
* General desirability of digital tools

* Plan de réappropriation des langues de Bretagne 2024-2027 (Brittany Region)

* endorsed by the Cultural Council of Brittany, 2023, Dec. 02.

* Favorable reception in conferences for a large public

* Inventory of adversatives

* Why build AI material if it hurts us?

* Fear of AI acceleration

* Research of purity

* Protection of scarce resources against competition

<figure><img src="image_42.png" /></figure>
<figure><img src="image_43.png" /></figure>

* Answers documented needs :

* ASR tools

* socializing tools in Breton

* pedagogical sound material for teaching Breton

* Co-conception with the final users :

* Historic association for Breton data collection (Dastum)

* Breton teaching structures for adults (Roudour, Stumdi, Mervent...)

<figure><img src="image_44.png" /></figure>
Check-list :

Explicit consent

Open access information on digital rights

No linguistic pollution

Speakers acculturated to the tools (low impact on digital cultures)

If we fail, the preparation of aligned data can be valorized.

On a large scale,

Reduction of digital gap between languages in contact

No financial competition with our users

Formation of local workforce

Collaboration with local organisations and schools

Tools exportable to other ecosystems

<figure><img src="image_45.png" /></figure>
Bucureşti

я

İstanbul

Antalya

5642 m
საქართველო

Türkiye

4058 m

4135 m

Azərbaycan

тебер

Türkmenistan

Oʻzbekiston

Tоо

Challenges

* Underrepresented in NLP

* Limited or no digital infrastructure

* Multiscript

* Security challenges

* (but large number of speakers wrt Breton)

العراقي
1952 m
دمشق
الاردن
القاهرة
2285 m
مصر
العوودي الجمودي
العربية
1954 m
العربية
501 m
لبنان
3927 m
لويستان
المتحدة
1961 m
لسطين

العربية

<figure><img src="image_46.png" /></figure>
[1] Cajka, S., M. Jouitteau & Eva Vetter 'A Hiking Path for Burgenland Croatian; Desirability and technical affordances of AI language learning chatbots for minoritised languages : a pilot study', oral presentation at *Eurocall*, Milano, Italy, 2025.

[2] Grobol, L. & M. Jouitteau (2024) 'ARBRES Kenstur : A Breton-French Parallel Corpus Rooted in Field Linguistics', *Proceedings of the Fourteenth Language Resources and Evaluation Conference, European Language Resource Association (ELRA)*, https://cnrs.hal.science/hal-04551941

[3] Jouitteau, M. A. Millour, J-Y Antoine, L. Grobol (2024) 'Yezh Ar Vro - The language of the country : Building the appropriation of data collection applications', https://cnrs.hal.science/hal-04790596v1.

[4] Romero & al. (2024) *CVQA : Culturally-diverse Multilingual Visual Question Answering Benchmark*, https://arxiv.org/pdf/2406.05967.

<figure><img src="image_47.png" /></figure>