<figure><img src="image_1.png" /></figure>

HAL
open science

# Multilingual content management and standards with a view on AI developments

Laurent Romary

► To cite this version:

Laurent Romary. Multilingual content management and standards with a view on AI developments. AI4EI - Conference Artificial Intelligence for European Integration, Oct 2020, Turin / Virtual, Italy. . hal-02961857

HAL Id: hal-02961857

https://inria.hal.science/hal-02961857

Submitted on 8 Oct 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Multilingual content management and standards with a view on AI developments

Laurent Romary

Directeur de Recherche, Inria, team ALMAnaCH

ISO TC 37, chair
Language and AI

* Central role of language in the revival of AI (machine-learning based models)

* Applications: document management and understanding, chatbots, machine translation

* Information sources: public (web, cultural heritage repositories) and private (Siri, Amazon Alexa) linguistic information

* European context: cf. Europe's Languages in the Digital Age, META-NET White Paper Series

* Variety of linguistic forms

* Spoken, written, chats and forums

* Multilingualism, accents, dialects, technical domains, registers, language learners

* General notion of language variety

* Classifying and referencing the relevant features

* Role of standards and standards developing organization (SDO)
A concrete example for a start

<figure><img src="image_3.png" /></figure>
# The infrastructure to create OSCAR - goclassy

<figure><img src="image_4.png" /></figure>
OSCAR

<table><thead><tr><th rowspan="2">Language</th><th colspan="2">Size</th><th colspan="2">Words</th></tr><tr><th>Orig</th><th>Dedup</th><th>Orig</th><th>Dedup</th></tr></thead><tbody><tr><td>English</td><td>2.3T</td><td>1.2T</td><td>418,187,793,408</td><td>215,841,256,971</td></tr><tr><td>Russian</td><td>1.2T</td><td>568G</td><td>92,522,407,837</td><td>46,692,691,520</td></tr><tr><td>Spanish</td><td>278G</td><td>149G</td><td>47,545,122,279</td><td>25,928,290,729</td></tr><tr><td>French</td><td>282G</td><td>138G</td><td>46,896,036,417</td><td>23,206,776,649</td></tr><tr><td>German</td><td>308G</td><td>145G</td><td>44,878,908,446</td><td>21,529,164,172</td></tr><tr><td>Italian</td><td>137G</td><td>69G</td><td>22,248,707,341</td><td>11,250,012,896</td></tr><tr><td>Portuguese</td><td>124G</td><td>64G</td><td>20,641,903,898</td><td>10,751,156,918</td></tr><tr><td>Chinese</td><td>508G</td><td>249G</td><td>14,986,424,850</td><td>6,350,215,113</td></tr><tr><td>Japanese</td><td>216G</td><td>106G</td><td>4,962,979,182</td><td>1,123,067,063</td></tr><tr><td>Polish</td><td>109G</td><td>47G</td><td>15,277,255,137</td><td>6,708,709,674</td></tr></tbody><tfoot><tr><td><strong>Total OSCAR</strong></td><td>6.3T</td><td>3.2T</td><td>844,315,434,723</td><td>425,651,344,234</td></tr></tfoot></table>
# Language and standards – why?

* A general issue of reproducibility

* Can we compare two linguistic descriptions

* E.g. two etymological assumptions in a dictionary entry

* An acuter context created by the machine learning revival

* Comparing results of AI components

* Issues related to segmentation, categorisation (noun, verbs, feminine, dual, etc.), dependencies towards other linguistic descriptions (lexica, lower linguistic levels)

* Reproducibility of results

* Can we compare two training campaigns?

* How do we deal with cross-linguistic comparison?

* Precise qualification (e.g. coverage) and identification of training corpora
# Standards

* What they are

* Reference background for the management of a technical process

* The three pillars of a standard: consensus, publicity, maintenance

* A constraint for each, a benefit for all!

* Reading it, understanding it, implementing it...

* What they aren't

* Something coming from nowhere... participating is always an option

* Mandatory documents: rather a common ground for a transaction

* Regulations: unless a state or an international organisation takes it up

* The need for standards developing organizations (SDO)

* Ensure that the three pillars of standardisation are in place

* Standards are like the EU: you need to understand how they work to appreciate them...
# Standards and AI

* A new committee has been set up

* IT context: ISO/IEC JTC 1/SC 42 Artificial intelligence

* Work program - 3 publications so far:

* ISO/IEC 20546:2019 Information technology — Big data — Overview and vocabulary

* ISO/IEC TR 20547-2:2018 Information technology — Big data reference architecture — Part 2: Use cases and derived requirements

* ISO/IEC TR 20547-5:2018 Information technology — Big data reference architecture — Part 5: Standards roadmap

* Numerous ongoing activities:

* Concepts and terminology, Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML), Risk Management, Assessment of the robustness of neural networks, Overview of ethical and societal concerns, Governance implications of the use of artificial intelligence by organizations

* Overview:

* Very high level endeavour

* Requires domain-specific standardisation activities => language
# Language and standards – where?

* Various levels of specialisation

* Horizontal vs. Vertical standards

* W3C - World Wide Web Consortium

* Development of (horizontal) standards for the Internet

* Ex. HTML, XML, XPath, RDF, OWL, SKOS

* The basic layer upon which language resources are being represented

* ISO - International Organization for Standardization

* General purpose standardisation consortium

* Organized in technical committees targeted at specific applications

* ISO TC 37: Language and terminology (language codes, language resources, translation and interpretation)

* TEI - Text Encoding Initiative

* Consortium dedicated to textual content

* Drama, Transcription of spoken language, dictionaries, manuscripts, etc.

* Based on XML – defines a vocabulary of around 600 elements
# Language and standards – what and how?

* All levels of linguistic documentation, description, representation

* Character encoding

* Structure of documents

* Linguistic annotation

* Lexical resources

* Metadata and bibliography

* The need for a variety of complementary standards

* Standards piling
A selection of some ISO/TC 37 activities

<figure><img src="image_5.png" /></figure>
# Piling up standards

<figure><img src="image_6.png" /></figure>

<entry xml:lang="fr-CA">
<form>
<orth>char</orth>
</form>

xml:lang="fr-CA"

Fr-CA

Type: language Subtag: fr
Description: French

fr, fra, fre
# Dealing with variety: lexical resources

* A variety of contexts and forms

* Legacy dictionaries, dialectological studies, NLP lexica

* Onomasiological (concept to term) vs. semasiological (word to sense) structures

* Lexical (machine processable) vs. Editorial (human readable) views

* Full forms, etymology, corpus based examples

* Word document, database, shoebox, XML...

* Why standardizing all this?

* Defining methods, models and format to facilitate

* Exchange of lexical data

* Pooling heterogeneous lexical data

* Interoperability between software components

* Search engines, layout, extraction of linguistic properties

* Comparability of results

* E.g. Linguistic coverage of lexical databases
Complementary of standardisation activities in the lexical domain

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>
# Back to machine learning: converting legacy dictionaries into TEI compliant structures

<figure><img src="image_9.png" /><figcaption>Cow (structured data)</figcaption></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /><figcaption>Hamburger (unstructured data)</figcaption></figure>

“Converting PDF to XML is a bit like converting hamburgers into cows. You may be best off printing it and then scanning the result through a decent OCR package.”

Michael Kay (http://lists.xml.org/archives/xml-dev/200607/msg00509.html)
GROBID-Dictionaries

* Automatic extraction of TEI structures from digitised dictionaries
(Khemakhem et al. 2017)

• Input: PDF (or OCR/HTR formats such as ALTO)

• Output: TEI compliant lexical resource

- Spin-off from GROBID (Romary and Lopez 2015)

• Initiated in 2007

* Automatic extraction of structural data from scholarly papers

• Metadata (author, title, affiliations, keywords, abstract), bibliography, ... full text

• And open source...

- Uses Conditional Random Fields (CRF) (Lavergne et al. 2010)

• Probabilistic models for sequence labelling tasks
Cascading CRF models,
an intuition

CON

Eugène IV à Ferrare en 1418-1439, puis à
Florence de 1419 à 1442), de Latran (1512-
1517), de Trente (1545-1563) [où fut décédée
la réforme générale de l'Eglise catholique en
 face de la Réforme protestante], de Vati-
can I (1870) [où fut défini le dogme de l'in-
faitilité postérieure], de Vatican II (1962-
1965) [où fut défini l'attitude de l'Eglise
romaine à l'égard du monde moderne]

conculiable adj. Qui peut se concilier avec
une autre chose.

conciliabule [kəsələby] n. m. (lat. conci-

concordant, e adj. Qui s'accorde : émoi-
neur, concordant, concordant

concordat [kɔ:kɔ:də] n. m. (lat. concordan-
mas). Traité entre le pape et un gouvernement
sur les affaires religieuses. | Dr. Accord entre
le commerçant qui, ayant déposé son bilan,
a été admis par le tribunal de commerce au
règlement judiciaire et ses créanciers.

— Les plus anciens concordats sont le concordat de Worms (1122), entre Calixte II et Henri V; le concordat de 1516, entre Léon X et François Ier. Le concordat entre

concupiscence n. f. (du lat. concupiscere,
désignation : le concordant est bien fait
restre, particulièrement des plaisirs sensuels.)

concupiscent [kɔ:kypisə], e [-At] adj. Qui exprime la concupiscence : regards concupiscents. Attaché aux plaisirs sensuels.

concurrence [kɔ:kju:n'se] n. f. (lat. concurrence). Par des concours mutuel, de concert : agir concurremment avec quelqu'un.

concurrance n. f. Rivalité entre plusieurs
personnes qui visent un même but : entrer

condenser [kɔ:dəs] v. t. (lat. condensare,
rendre épais). Rendre plus dense, réduire à

un moindre volume. || Liquéfier un gaz par

refroidissement ou compression : le froid
condense la vapeur d'eau. || Fig. Exprimer
d'une manière concise, en peu de mots :

conclaviste n. m. Personne qui s'enferme
au conclave avec un cardinal, pour le servir.

concluant, e adj. Qui prouve bien ce qu'on
s'avance : arguer concluant.

conclure [kɔ:klyr] v. t. (lat. concludere)
[conj. 62]. Achever, terminer : conclure une
affaire. || Trier une conséquence : conclure
une chose d'une autre. || — V. i. Donner
son avis, ses conclusions; se prononce : an
vous demande de conclure. | Etre prohant
les témoignages concluant contre lui.

conclusion n. f. (lat. conclusio). Arrange-
ment définitif : conclusion d'un traité. | Fin,
résultat final : la conclusion d'un discours.
| Conséquence d'un argument : la conclusion
d'un syllogisme ne doit pas dépasser les pré-
mises. || — Pi. Prétentions respectives de
chacune des parties dans un procès. Ecrit
exposant ces prétenlontions. | Réquisition du
ministère public. || — En conclusion loc. adv.
En conséquence, pour conclure

concocter v. t. Fam. Elaborer avec soin : concliquer une lettre de réclamation.

concombre [kɔ:kbrə] n. m. (anc. provenc-
cocombre). Plante potagère de la famille
des cucurbitacées, cultivée pour ses fruits
allongés que l'on consomme comme légume
ou en salade. | Ce fruit.

concomitamment adv. De façon concor-
mitante.

concomitance [kɔ:komətəs] n. f. Coexis-
tence, simultanéité de deux ou de plusieurs
faits.

concomitant, e adj. (lat. concomitant). Qui concapagne, qui se produit en même temps des faits concomitants. • Variations concomitantes, variations simultanées et proportionnelles de certains phénomènes.

concordance n. f. Conformité, accord : concordance de témoignages. || Géol. Dispo-
sition parallèle des couches sédimentaires. •
Concordance de phases (Phys.), état de plu-
sieurs vibrations sinusoidales de même nature
et de même période, dont la différence de
phases est nulle. | Concordance des temps
régles de syntaxe d'après lesquelles le temps
du verbe d'une subordonnée varie selon celui
du verbe de la principale

concour. || Lutte sportive : concours hip-
ique. • Concours général, concours annuel
entre les premiers élèves des classes supé-
rieures des lycées, écoles d'écoles normales

concret [kɔ:kres] e [-et] adj. [lat. concrē-
nus]. Epais, condensé : huile concrète
(vieilli). | Qui exprime quelque chose de réel,
de positif : obtenir des avantages concrets. |
Qui a le sens des réalités précises : exprِi
d'un être, d'un acte ou d'une contribution qui
désigne un être ou un objet pouvant être
perçu par les sens. • Musique concrète,
téchnique de composition qui utilise les bruits
produits par divers objets somores enregistrés
des instruments magnétique et susceptibles de
transformation.

concret n. m. Qualité de ce qui est concret.

concretement adv. De façon concrète.

concrétiser v. t. (conj. 3). Rendre concret,
solidé.

concrétion [kɔ:kres] n. f. (de concrét)
Action de s'épahir : la concrétion de l'huile,
du zang. | Réunion de parties en un corps
solide : concrétion solide. || Agrégation so-
lide dans les tissus vivants : concrétions bl-
laires. || Gramme. Se dit d'un terme de

concrétiser v. t. Rendre concret ce qui est
abstrait : concrétiser une idée, un avantage.

concubinage [kɔ:kbynaʒ] n. f. Etat d'un
homme et d'une femme qui vivent ensemble
sans être mariés. (On dit aussi UNION LUXEMBOURG.)

<figure><img src="image_12.png" /></figure>

condensation.

condamnable adj. Qui mérite d'être
condamné : acte condamnable.

condamnation [kɔ:kjən'si:ə] n. f. (lat.
condemnato). Décision d'un tribunal impo-
sant à l'un des plaideurs de s'incliner au
moins partiellement devant les prétenlontions
de son adversaire. || Décision d'une juridic-
tion prononçant une peine contre l'auteur
d'un être, d'un acte ou d'une contribution
(En cour d'assises, le jury juge la culpabilité
de l'accusé, et le cour prononce la condam-
nation.) | La peine infligée : une condamna-
tion à la réclusion criminelle. || Blâme,
désapprivoion : la condamnation des obvetés
des prétenlontions.

condamnatoire adj. Qui porte condamna-
tion.

condamné, e n. Personne qui a subi une
condamnation. — Adj. Qui ne peut échap-
ser d'un sort rénové ou modifié.

condamner (kɔ:dəm) v. t. (lat. condam-
nare). Prononcer un jugement contre un pla-
deur ou un inculpé : condamner un criminel. |
Astreindre, réduire à : condamner au si-
lence, à l'immobilité. | Désapprouver, blâ-
mer : condamner une opinion, un usage. |
Interdire : la loi condonne la bigamie. |
Déclarer perdu, incurable : les médecins l'ont
condamné. | Battrer, muer : condamner une
porte.

condensable adj. Qui peut être condensé,
reduit à un moindre volume.

condensateur n. m. Phys. Appareil ser-
vant à emmagasiner une charge électrique
la bouteille de Leyde est un condensateur
electrique. | L'esselle servant à éclairer un
objet dont on veut former une image.

condensation n. f. Action de condenser ou
effet qui en résulte. | Liquéfaction d'un gaz.
|| Soudure de plusieurs molécules chimiques.
avec élimination d'eau.

condensé n. m. Résumé d'une œuvre litté-
raire.

condenser [kɔ:dəs] v. t. (lat. condensare
rendre épais). Rendre plus dense, réduire à
un moindre volume. | Liquéfier un gaz par
refroidissement ou compression : le froid
condense la vapeur d'eau. || Fig. Exprimer
concrètement concours.
# Cascading CRF models

aA

a. A 1. Türk alfabetinin ilk surlarında yer alan ve 4 ad verilen bu harf, ses bilimi baki- mından kalın unplerinin düz ve geniş olanını gösterir. 2. müz. Nota işaretlerini harflerle gösterme yönteminde la sesini bilirler. a'dan z'ye (kadar) baştan aşağı, tamamen, tamamıyla, bütüntüyle: Evini a'dan z'ye de- ğistirir.

a ünl. (a) Şası, hatırlama, sevinme, acima, üzülme, kızma vb. duyuların anlatımına güç kazandıran söz: A, ne güzel! A, sen bu- rada myydin?

a / e ünl. Dilek kipinin ikinci teklik ve çokkuk şahısının çekiminden sonra gelerek an- lami pekistiren ve güçlendiren bir söz: "Az- cık darsana oğum, dedi." -A. Kabaklı. "O- nu bana getiriversene!" -N. Hikmet. "Başka gazetelere boksanma! Onlar da yazılar. -" - N. F. Kısakürek. "Şimdi de başka çıkmaz- dayışı desenlen." -N. Uygar.

ab a. (a:bi Far, ab: esk. Su
→ abihayat, abkeveser, abuhava

aba (I) a. hlk. 1. Abla. 2. Anne.

aba (II) a. dr. 'abâ 1. Yünnün dövülmesiyle yapılan kalın ve kaba kumaş. 2. Bu kumaş- tan yapılmış yakasız ve uzun üstütlük. 3. sf. Bu kumaştan yapılan. 4. esk. Bu kumaştan yapılan ve devriğezeleyiğin hırka. 5. Kö- pencek (I), aba altında er yatar bir insanının değeri giyimiyle kuşamıyla ölçülemez" anlamında kullanılan bir söz. aba altından sopla (veya değnek) göstermek birini imali bir biçimde tehdit etmek. aba gibi kaba ve kalın (kumaş). aba vakti yaha, yaha vakti aba "gereksinimler vaktinden önce ve uzucu olduğu zaman karşılanmamıdır" anlamında kullanılan bir söz. abanın kadri yağmurdı bilinir "bir şeyin gerçek değeri ona gereksinim duyulduğunda anlatılır" anlamında kullanılan bir söz. (bir yere) abayı sermek 1) istenilmediği hâlde teklifszice yerleşmek; 2) uzun süre yerleşip kalmak. abayı yak- mak iz: birine aşırı bir biçimde gönül ver- mek, tutulmak, aşık olmak. "Sen mi verin ona gönül yoksa o mu yakıza sana daha önce

abayy?" -O. C. Kaygılı.
→ aba güreşti

abacı a. 1. Aba yapan veya satan kimse. 2. Abadan giyeyecek yapan veya satan kimse. 3. sf. mec. Asalak. 4. hlk. Bedavacı. abacı ke- beci (ara yerde) sen neci? hlk. "sen ilgi- lendirmeyen işe için karşıyorsun" anlam- unda kullanılan bir söz.

abacık, kâ a. Abacımını yaptığı iş.
abadi a. (a:ba:di) j. Far. abadî esk. Açık sa- man renginde, ipekten yapılan, yan mat, kalınca bir yazı kâğıdı türü.

aba güreşti a sp. Aba giyilerek ve bele kuşak bağlanarak yapılan bir güreş türü.
abajur a. Fr. abat-jour 1. Işığı bir yere top- lamak, doğrudan doğruya gözlere varmasını önlemek için kullanılan, kâğıt, kumaş, mas- den veya renkli camdan yapılmış lamba si- peri. 2. Genellikle üzeri siperli masa lamba- si veya ayaklı lamba: "Etejerin sağında bir telefon, ortasında bir heykel, solunda bir a- bajur." -N. F. Kısakürek.

abajurcu a. Abajur yapan veya satan kimse.

abajurlu a. Abajurun olan: "Östüme lacivert abajurulu, parlak bir madenden lamba." -P. Safa.

abajursuz sf. Abajuru olmayan: "Tavandaki abajursuz, çelik elektrik lambasını yakmış okuyordu." -S. F. Abasıyanık.

abaküsa a. Fr. abacus mai. 1. Sayı boncuğu. 2. mim. Sütün başlanmış üstüne yatay olarak konan ve kenarlarından biraz dışarı taşan taş blok.

abalı sf. Aba giymiş olan.

Abana öz. a. (aba'na) Kastamonu iline bağlı içlərdən biri.

abanabilme a. Abanabilmek işi.

abanabilmek (~e) 1. Abanma ihtimali veya imkını bulunmak: "Sana, yeni eserimən, ta- kat getirilməyən çoklədə abanabilməyim?" - N. F. Kısakürek. 2. Abanmaya gücü yet- mek.

1

<figure><img src="image_13.png" /></figure>

https://github.com/MedKhem/grobid-dictionaries

<entry>
  <form type="lemma" xml:lang="center_form">
    <orth>center</orth>
    <pron>sent</pron>
    <gramGrp>
      <pos>noun</pos>
    </gramGrp>
    <usg type="geo">U.S./usg</usg>
    <form type="variant">
      <orth>centre</orth>
      <usg type="geo">U.K./usg</usg>
      <pron>sent</pron>
    </gramGrp>
  </form>
</entry>

<entry>
  <form type="inflected">
    <orth>centers</orth>
    <usg type="geo">U.S./usg</usg>
    <gramGrp>
      <number>plural</number>
    </gramGrp>
  </form>
</entry>

<entry>
  <form type="inflected">
    <orth>centres</orth>
    <usg type="geo">U.K./usg</usg>
    <gram type="number">plural</gram>
  </form>
</entry>

<entry>
  <sense>
    <def>the point around which a circle or sphere is described</def>
    <cit type="example">
      <quote>earth center</quote>
    </cit>
  </sense>
</entry>

<entry>
  <sense>
    <gramGrp>
      <pos>verb</pos>
    </gramGrp>
    <def>place in the middle</def>
    <cit type="example">
      <quote>center the picture on the wall</quote>
    </cit>
  </sense>
</entry>

<entry>
  <re type="multiWordExpression">
    <form>
      <seg corrres>"#dead_form" n="1">dead</seg>
      <seg corrres>#center_form" n="2">center</seg>
    </form>
  </re>
</entry>
Experiments and results - 2

DLF

<figure><img src="image_14.png" /></figure>

EEBD

<figure><img src="image_15.png" /></figure>

MxSp

<figure><img src="image_16.png" /></figure>

FrFang

<figure><img src="image_17.png" /></figure>

FangFr

ALL
# Further steps in language resources management

* Training

* Basic digital literacy

* Even for very simple issues related to online presence (authorship, affiliation)

* Concrete knowledge of existing standards for language resources

* Preventing the development of (any) new formats, unless strongly supported by evidence

* Active contributions

* Participation to standardisation activities

* Building up standards-based infrastructures

* General policies

* A strong relation to open science principles (the I in FAIR, which leads to the R)

* Hence a necessary stance for any research project
Merci pour votre attention