# A Neural Approach for Detecting Morphological Analogies

Safa Alsaidi, Amandine Decker, Puthineath Lay, Esteban Marquer, Pierre-Alexandre Murena, Miguel Couceiro

► To cite this version:

Safa Alsaidi, Amandine Decker, Puthineath Lay, Esteban Marquer, Pierre-Alexandre Murena, et al.. A Neural Approach for Detecting Morphological Analogies. IEEE DSAA 2021 - The 8th IEEE International Conference on Data Science and Advanced Analytics, Oct 2021, Porto / Online, Portugal. , IEEE DSAA 2021. [https://hal-03328841](https://hal-03328841)

HAL Id: hal-03328841

https://inria.hal.science/hal-03328841

Submitted on 30 Aug 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# A Neural Approach for Detecting Morphological Analogies

Safa Alsaidi*, Amandine Decker*, Puthineath Lay*, Esteban Marquer*, Pierre-Alexandre Murena**, Miguel Couceiro*

*Université de Lorraine, CNRS, LORIA, F-54000, Nancy, France; **HIIT, Aalto University, Helsinki, Finland

## Proportional Analogy

**Notation:**

► A : B :: C : D (read "A is to B as C is to D")

**3 Core properties:**

1. A : B :: C : D → C : D :: A : B (symmetry)

2. A : B :: C : D → A : C :: B : D (central permutation)

3. A : B :: A : B (reflexively)

**Morphology:**

► "read is to readable as count is to countable"

► *aalto* : *aalloksi* :: *spirituaali* : *spirituaaliksi* (Finnish)

## Training and Evaluation Procedure

Analogies obtained by aligning morphological transformations:

► *dog*, to NUM=PLURAL, *dogs*

► *cat*, to NUM=PLURAL, *cats*

► resulting analogy: *dog* : *dogs* :: *cat* : *cats*

For each analogy A : B :: C : D extracted from the data (use max 50000 of the available analogies):

1. embed A, B, C, and D

2. permute embeddings using properties of proportional analogy:

*   **Train. & eval.:** 8 valid analogies

*   **Training:** 3 invalid analogies

*   **Evaluation:** 3 × 8 invalid analogies and their equivalent forms

3. aggregate over all permutations:

*   **Training:** loss (Binary Cross-Entropy)

*   **Evaluation:** accuracy

A : B :: C : D (base form) C : D :: A : B

A : C :: B : D B : A :: D : C D : B :: C : A

D : C :: B : A C : A :: D : B B : D :: A : C

<caption>Table: 8 equivalent forms per analogy</caption>

B : A :: C : D C : B :: A : D A : A :: C : D

<caption>Table: 3 invalid analogies per analogy</caption>

## Proposed Approach: Morphological Embedding + CNN Classifier

<figure><img src="image_2.png" /></figure>

## Classification Results

<table><thead><tr><th rowspan="2">Language</th><th colspan="2">CNN (ours)</th><th colspan="2">Best Baseline</th><th colspan="2">Number of analogies in the data</th></tr><tr><th>Valid</th><th>Invalid</th><th>Valid</th><th>Invalid</th><th>Training</th><th>Test</th></tr></thead><tbody><tr><td colspan="7" style="text-align: center;">Sigmorphon2016 task 1 (inflexion) [2]</td></tr><tr><td>Arabic</td><td><strong>99.89</strong></td><td><strong>97.52</strong></td><td>34.21 (Alea@10)</td><td><strong>97.79</strong></td><td>(Kolmo@1)</td><td>373240</td><td>555312</td></tr><tr><td>Finnish</td><td><strong>99.44</strong></td><td>82.62</td><td>25.60 (Lepage)</td><td><strong>98.78*</strong></td><td>(Alea@1)</td><td>1342639</td><td>4691453</td></tr><tr><td>Georgian</td><td><strong>99.83</strong></td><td>91.71</td><td>93.20 (Kolmo@10)</td><td><strong>95.21</strong></td><td>(Alea@1)</td><td>3553763</td><td>8368323</td></tr><tr><td>German</td><td><strong>99.48</strong></td><td>89.01</td><td>(Alea@10)</td><td><strong>97.19</strong></td><td>(Alea@1)</td><td>994740</td><td>1480256</td></tr><tr><td>Hungarian</td><td><strong>99.99</strong></td><td><strong>98.81</strong></td><td>36.80 (Kolmo@10)</td><td><strong>98.40</strong></td><td>(Kolmo@1)</td><td>3280891</td><td>66195</td></tr><tr><td>Maltese</td><td><strong>99.96</strong></td><td><strong>77.83</strong></td><td>78.05 (Alea@10)</td><td>69.29 (Kolmo@1)</td><td></td><td>104883</td><td>3707</td></tr><tr><td>Navajo</td><td><strong>99.53</strong></td><td>90.82</td><td>21.45 (Kolmo@10)</td><td><strong>94.93</strong></td><td>(Kolmo@1)</td><td>502637</td><td>4843</td></tr><tr><td>Russian</td><td><strong>97.95</strong></td><td>79.85</td><td>42.37 (Alea@10)</td><td><strong>93.88</strong></td><td>(Lepage)</td><td>1965533</td><td>6421514</td></tr><tr><td>Spanish</td><td><strong>99.94</strong></td><td>78.33</td><td>85.90 (Alea@10)</td><td><strong>86.62</strong></td><td>(Lepage)</td><td>1425838</td><td>4794504</td></tr><tr><td>Turkish</td><td><strong>99.48</strong></td><td><strong>92.63</strong></td><td>44.76 (Alea@10)</td><td>91.40 (Kolmo@1)</td><td></td><td>606873</td><td>11360</td></tr><tr><td colspan="7" style="text-align: center;">Japanese Bigger Analogy Test Set [4]</td></tr><tr><td>Japanese</td><td><strong>99.99</strong></td><td><strong>98.65</strong></td><td>19.20 (Kolmo@10)</td><td><strong>98.13</strong></td><td>(Lepage)</td><td>18487</td><td>7923</td></tr></tbody></table>

123 no significant difference

123 best result, significant difference between baselines and ours

* obtained on 4000 analogies (too slow on 50000)

## Transferability

<figure><img src="image_3.png" /></figure>

## Baselines

### 1 Classifier:

► Lepage Classifier [3]

► matrix based approach to align characters between words and find common sub-words

### 2 Solvers:

► find missing x making A : B :: C : x valid; if expected D in top 1 or 10 solutions, valid analogy

► Kolmogorov Complexity [6]:

► minimize complexity of f such that $B = f(A)$ and $f(C)$ is computable; $x = f(C)$

► Alea [5]:

► random permutations of the characters of B not in A and those of C (Monte-Carlo method)

## Perspectives

✓ explore transferability [1]

✓ balancing data

✓ regression (analogy solving)

✓ qualitative analysis of embedding model

✓ other domains (images, text, explanations, etc.)

## Bibliography

[1] Safa Alsaidi et al. "On the Transferability of Neural Models of Morphological Analogies". In: *AIMLAI*, ECML PKDD 2021. 2021.

[2] Ryan Cotterell et al. "The SIGMORPHON 2016 Shared Task—Morphological Reinflection". In: *SIGMORPHON*, 2016. ACL, Aug. 2016.

[3] Rashel Fam and Yves Lepage. "Tools for The Production of Analogical Grids and a Resource of N-gram Analogical Grids in 11 Languages". In: *11th LREC. ELRA*, 2018.

[4] Marzena Karpinska et al. "Subcharacter Information in Japanese embeddings: when is it worth it?" In: *Workshop on the Relevance of Linguistic Structure in Neural Architectures for NLP*. ACL, 2018.

[5] Philippe Langlais et al. "Improvements in Analogical Learning: Application to Translating Multi-Terms of the Medical Domain". In: *12th EACL*. ACL, 2009.

[6] Pierre-Alexandre Murena et al. "Solving Analogies on Words based on Minimal Complexity Transformation". In: *29th IJCAI*. 2020.