# TAYSIR Competition: Transformer+rnn: Algorithms to Yield Simple and Interpretable Representations

Rémi Eyraud, Dakotah Lambert, B Tahri, A Gaffarov, Mathias Cabanne, J

Heinz, C Shibata

► To cite this version:

Rémi Eyraud, Dakotah Lambert, B Tahri, A Gaffarov, Mathias Cabanne, et al. TAYSIR Competition: Transformer+rnn: Algorithms to Yield Simple and Interpretable Representations. International Conference on Grammatical Inference, Jul 2023, Rabat, Morocco. pp.275-290. hal-04237436

HAL Id: hal-04237436

https://hal.science/hal-04237436v1

Submitted on 11 Oct 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# TAYSIR Competition: Transformer+RNN: Algorithms to Yield Simple and Interpretable Representations*

Rémi Eyraud

Dakotah Lambert

Badr Tahri Joutei

Aidar Gaffarov

Université Jean Monnet Saint-Etienne, CNRS, Laboratoire Hubert Curien UMR 5516, 42023 Saint-Etienne, France

Mathias Cabanne

Euranova France

Jeffrey Heinz

Department of Linguistics & Institute of Advanced Computational Science, Stony Brook University

Chihiro Shibata

Department of Advanced Sciences, Faculty of Science and Engineering, Hosei University

REMI.EYRAUD@UNIV-ST-ETIENNE.FR

DAKOTAHLAMBERT@ACM.ORG

TAHRIBADR@GMAIL.COM

AIDAR.GAFFAROFF@GMAIL.COM

MATHIAS.CABANNE@EURANOVA.EU

JEFFREY.HEINZ@STONYBROOK.EDU

CHIHIRO@HOSEI.AC.JP

**Editor:** François Coste, Faissal Ouardi, Guillaume Rabusseau

## Abstract

This article presents the content of the competition Transformers+RNN: Algorithms to Yield Simple and Interpretable Representations (TAYSIR, the Arabic word for 'simple'), which was an on-line challenge on extracting simpler models from already trained neural networks held in Spring 2023. These neural nets were trained on sequential categorical/symbolic data. Some of these data were artificial, some came from real world problems (such as Natural Language Processing, Bioinformatics, and Software Engineering). The trained models covered a large spectrum of architectures, from Simple Recurrent Neural Network (SRN) to Transformers, including Gated Recurrent Unit (GRU) and Long Short Term Memory (LSTM). No constraint was given on the surrogate models submitted by the participants: any model working on sequential data was accepted. Two tracks were proposed: neural networks trained on Binary Classification tasks, and on Language Modeling tasks. The evaluation of the surrogate models took into account both the simplicity of the extracted model and the quality of the approximation of the original model.

**Keywords:** Recurrent Neural Networks, Transformers, Knowledge Distillation, Surrogate Model, Benchmark

## 1. Introduction

The main shadow over the rise of deep learning is the lack of understanding of the models it produces. Indeed, most trained models are black-boxes that escape human understanding, which causes important scientific (Räz and Beisbart, 2022) and legal (GDPR, 2016) problems. To tackle the issue, a whole field is quickly developing, commonly referred as eXplainable AI (XAI). A wide spectrum of approaches are being investigated, with one of the most studied ones being local interpretability, where the decision of a trained model on a given datum is analyzed (Guidotti et al., 2018).

<footnote>* This work is supported in part by the ANR TAUDoS (ANR-20-CE23-0020).</footnote>
Another promising, though more challenging, line of research focuses on providing a gen-
eral explanation of the trained Neural Network by providing a simpler model that mimics the
trained one: this approach is usually referred as Knowledge Distillation (Hinton et al., 2015).
Actually, the surrogate model does not have to be completely explainable to be of interest:
a simple model requiring less computation power than a deep neural network, and whose
decisions are similar or close to the original one, is already making a difference (Harmon
and Auseklis, 2009).

It is in this context that the TAYSIR competition was held during the months of March and April 2023 as a satellite event of the 16th International Conference on Grammatical Inference. The aim of this challenge was to regroup researchers interested in extracting surrogate models from Neural Nets trained on finite sequences of symbols (Giles et al., 1992; Deoras et al., 2011; Wang et al., 2017; Eyraud and Ayache, 2021; Dong et al., 2021; Lacroce et al., 2021; Barbot et al., 2021; Panchapagesan et al., 2021; Muškardin et al., 2022; Hong et al., 2022; Mayr et al., 2022). Two tracks were proposed. For the first one, the models were trained on binary classification tasks (language membership in a formal language theory sense). For the second one, the models were trained on language modeling tasks. A classical use case of this latter task is in Natural Language Processing where the model calculates the probability of the next symbol given a prefix. Each track provided 11 and 10 models, respectively, carefully selected among the 10 000+ models we trained specifically for the competition. The competitors then had to submit a simpler model extracted from the already-trained ones.

The training datasets covered a wide spectrum of tasks, from artificially generated ones to ones from Natural Language Processing, Bio-informatics and Software Engineering. For each task, the participant only received the part of the data used as a validation set during the training phase together with the trained Neural Network model.

More than 40 teams registered to the challenge website to access the trained Neural Nets. Among them, only 7 teams submitted at least one surrogate model. However, the total number of surrogate models submitted exceed 700 (each participant was allowed to submit 20 models per day at most).

This article is organized as follows. First, it describes the data used for the training phase in Section 2. It then details the training approach and the chosen Neural Nets in Section 3. Section 4 depicts the baselines that were made available in order to show the participants how to use the trained models and in order to help the organizers select the models used in the challenge. The competition framework, including the metrics used to evaluate the quality and the simplicity of the surrogate models, is detailed in Section 5. The results of the competition are briefly described in Section 6 while Section 7 concludes.

## 2. Training Data

This section describes the data used for training the Neural Nets (NN) in TAYSIR. Two separate tracks were created. The first one corresponds to NN trained on a binary classification task, that is, a language membership task in Formal Language Theory. The second track provided a NN trained on a Language Modeling task, a framework where the models are required to provide a probability distribution over the set of potential next symbols given any prefix.
It is important to note that the participants of the competition had only access to the part of the data used as a validation set in training the NNs: in most cases, we divided each dataset in three, using 70% for training, 10% for validation and 20% for testing.

In some cases, the same dataset was used to generate two different TAYSIR tasks. This can happen because each TAYSIR task corresponds to a trained Neural Net. Since we trained different architectures on all datasets, we sometimes picked two different models trained on the same dataset if we thought they were interesting enough to deserve their own distillation task.

Notice that more datasets were initially considered but were not used *in fine*, mainly for two reasons: either we did not manage to train a NN with good performances, or the distillation problem was solved by our simple baselines (see Section 4 for details on the latter).

## 2.1. Track 1: Binary Classification Tasks

The data for this Track came from 3 different sources. The first one is the new ML-RegTest (van der Poel et al., 2023) benchmark. This benchmark provides 1 800 artificially generated datasets covering a large hierarchy of subregular formal languages. We chose 6 languages in this set corresponding to different classes of complexity and used the large datasets for training, validation and testing.

The second source of data is the Enzyme/Not-Enzyme (ENE) dataset developed by Nicolas Buton. It is generated using the UniProtKB/Swiss-Prot protein knowledgebase release 2022_01 (The UniProt Consortium, 2022). Firstly, mmseq2 (Steinegger and Söding, 2017) is employed to form clusters of sequences having 40% or higher identity, and these clusters are then distributed randomly across three sets. The training set contains 90% of the clusters, while the validation and test sets each contain 5%. In the validation and test sets, only the representative sequence for each cluster is included, whereas all sequences are retained in the training set. This strategy avoids overemphasizing highly represented sequences. The enzyme/non-enzyme labels are obtained from the “recommendedName_ecNumber” field of the UniProtKB/Swiss-Prot XML dump. Sequences possessing this field are labeled as enzymes, while those without it are classified as non-enzymes.

The last source of data is the previously organized Omphalos Competition (Starkie et al., 2005). The Omphalos data was generated from artificially created Context-Free Grammars.

Table 1 summarizes the different datasets used to train the provided NNs.

## 2.2. Track 2: Language Modeling Tasks

Two sources were used to train NNs for this Track: the SPiCe benchmark (Balle et al., 2017) and the PAutomaC one (Verwer et al., 2014).

The PAutomaC competition provides artificial problems generated by randomly generated finite state machines: Deterministic Probabilistic Finite Automata (DPFA), non-deterministic Probabilistic Finite Automata (PFA), and Hidden Markov's Models (HMM). A large range of machine sizes, alphabet sizes, and sparsity indicators values are covered by the different instances and the competition results proved it is covering a wide range of difficulty levels. For each of these datasets, we have access to a training set (20 000 or 100 000 sequences).
<table><caption>Table 1: Track 1 training data. $|\Sigma|$ is the number of distinct symbols.</caption><thead><tr><th>TAYSIR Number</th><th>Dataset</th><th>|Σ|</th><th>validation set|</th></tr></thead><tbody><tr><td>1.0</td><td>MLRegTest - 16.04.TLT.2.1.4</td><td>16</td><td>10 000</td></tr><tr><td>1.1</td><td>MLRegTest - 04.04.Reg.0.0.9</td><td>4</td><td>10 000</td></tr><tr><td>1.2</td><td>MLRegTest - 16.16.LT.4.1.5</td><td>16</td><td>10 000</td></tr><tr><td>1.3</td><td>MLRegTest - 16.04.TLT.2.1.4</td><td>16</td><td>10 000</td></tr><tr><td>1.4</td><td>MLRegTest - 16.16.LT.4.1.5</td><td>16</td><td>10 000</td></tr><tr><td>1.5</td><td>MLRegTest - 16.16.SP.2.1.0</td><td>16</td><td>10 000</td></tr><tr><td>1.6</td><td>MLRegTest - 64.64.SF.0.0.0</td><td>64</td><td>10 000</td></tr><tr><td>1.7</td><td>MLRegTest - 64.64.SL.4.1.0</td><td>64</td><td>10 000</td></tr><tr><td>1.8</td><td>ENE</td><td>25</td><td>4 775</td></tr><tr><td>1.9</td><td>Omphalos 1</td><td>5</td><td>968</td></tr><tr><td>1.10</td><td>Omphalos 1</td><td>5</td><td>968</td></tr><tr><td>1.11</td><td>ENE</td><td>25</td><td>4 775</td></tr></tbody></table>

The SPiCe benchmark is made of 15 problems covering a large variety of contexts, from various natural languages processing data to software engineering data, including bioinformatics and well-chosen synthetic ones. The original test sets from SPiCe consisting only of prefixes, they do not correspond to what we need for our task, since they do not provide complete information on whole sequences. We thus split randomly the available learning samples into a training, a validation, and a test sample, with proportion 70%, 10%, and 20%, respectively.

Table 2 summarizes the datasets used for training the models for this Track.

<table><caption>Table 2: Track 2 training data</caption><thead><tr><th>TAYSIR Number</th><th>Dataset</th><th>|Σ|</th><th>validation set|</th></tr></thead><tbody><tr><td>2.0</td><td>Spice 4 (NLP)</td><td>33</td><td>489</td></tr><tr><td>2.1</td><td>Pautomac 18 (DPFA)</td><td>20</td><td>9 090</td></tr><tr><td>2.2</td><td>Pautomac 23 (HMM)</td><td>7</td><td>9 090</td></tr><tr><td>2.3</td><td>PAutomac 47 (DPFA)</td><td>15</td><td>9 090</td></tr><tr><td>2.4</td><td>Spice 1 (HMM)</td><td>20</td><td>1 643</td></tr><tr><td>2.5</td><td>Spice 4 (NLP)</td><td>33</td><td>489</td></tr><tr><td>2.6</td><td>Pautomac 18 (DPFA)</td><td>20</td><td>9 090</td></tr><tr><td>2.7</td><td>Spice 10 (Biological)</td><td>20</td><td>4 491</td></tr><tr><td>2.8</td><td>Spice 6 (Software)</td><td>66</td><td>411</td></tr><tr><td>2.9</td><td>Spice 10 (Biological)</td><td>20</td><td>4 491</td></tr><tr><td>2.10</td><td>Spice 4 (NLP)</td><td>33</td><td>489</td></tr></tbody></table>

# 3. Trained Models

## 3.1. Recurrent Neural Networks training

For each task, we trained several neural networks of different architectures using Torch library version 1.11.0. These included simple recurrent neural networks (SRN) (Elman, 1990), gated recurrent units (GRU) (Cho et al., 2014), and long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997) networks. Models were created as a stack of one or two recurrent layers of the specified type fed into a dense region of one or two hidden linear layers followed by a sigmoidal activation. For consistency and ease of use, we created a custom class that
arranges the PyTorch implementations of these layers and manages the transfer of data between and among them.

In order to accommodate distillation methods that require inspection of the model's internal state, we made sure to expose this state in forward calls. For SRN and GRU this is trivial, as the hidden state for each input step is already provided. However, for LSTM the internal state consists of two parts: a hidden state *h* and a cell (or carry) state *c*. While the PyTorch implementation emits *h* for each step, only the final *c* is available. We built a compatible replacement module over the LSTMCell class in order to make both parts available at each step.

In addition to cell type, we varied several other parameters. As previously stated, the number of recurrent layers (Nb layers) could be either one or two, as could the number of dense layers (Nb Dense). The number of neurons per layer (Neurons/Layer) covered powers of two from 32 through 512. Patience, the number of consecutive iterations allowed without improvement, was set to either five or twenty. Another parameter (Bidirectional) determined whether the recurrent layer was bidirectional or unidirectional. And finally, the batch size was varied between 32 and 128. For small datasets, we performed an exhaustive grid search over this space. In the interest of time, only a few combinations were tested for the larger datasets.

For each dataset, the best model was selected per cell type. In some cases, no trained model could achieve satisfactory performance against the test data. No models were included in the competition for these datasets. On the other extreme, our baseline extraction method performed at ceiling on some models. Those were also excluded.

## 3.2. Transformers training

As a transformer model, we employed Hugging Face's DistillBERT (Sanh et al., 2019). DistillBERT is one of the standard implementations of BERT-type models, maintaining a network structure nearly identical to the original BERT (Devlin et al., 2019), with the exception of halving the number of layers. For binary classification, the model was trained to predict the binary label from the output vector corresponding to the classification token, which was a special token inserted at the beginning of the sentence. To develop a language model, the loss function was designed to predict subsequent characters from output vectors associated with characters in the input word. In this context, future masking was essential. Since it is lacking in BERT, we modified that specific part.

Competition datasets had different characteristics compared to natural language, the usual target. For example, the maximum number of symbols was about 64, which is much smaller than the vocabulary of a natural language. Also, the relatively small sample size suggests that a more compact model may be more appropriate. We searched for the hyper-parameters: number of transformer blocks, the vector size, the number of heads, and the size of the middle layer, which is the union of all transformer blocks.

Table 3 details the parameter sets explored for each track. We trained models on each dataset by selecting 100 uniformly randomized combinations of hyperparameters from the search space.

In Track 1, the percentage of correct answers was close to 1.0 for the majority of problems. Therefore, we selected the models with the lowest computational cost among those with
<table><caption>Table 3: Search spaces for hyperparameters.</caption><thead><tr><th>Track</th><th>Type</th><th>Search Space</th></tr></thead><tbody><tr><td rowspan="4">1</td><td>Num. of T. Blocks</td><td>{1, 2, 3, 4, 5, 6}</td></tr><tr><td>Dimension</td><td>{4, 6, 8, 12, 16, 24, 32, 48, 64, 96, 128, 192, 256, 384, 512, 768}</td></tr><tr><td>Num. of Heads</td><td>{2, 3, 4, 6, 8, 12, 16, 24, 32, 48, 64, 96, 128, 192}</td></tr><tr><td>Ratio of Interm. Dim.</td><td>{1, 2, 4}</td></tr><tr><td rowspan="5">2</td><td>Num. of T. Blocks</td><td>{2, 4, 6, 8, 10, 12}</td></tr><tr><td>Dimension</td><td>{16, 32, 64, 128, 256}</td></tr><tr><td>Num. of Heads</td><td>{2, 4, 8, 16}</td></tr><tr><td>Ratio of Interm. Dim.</td><td>{1, 2, 4}</td></tr><tr><td>Positional Embedding</td><td>{ trainable, sinusoidal }</td></tr></tbody></table>

error rates less than $\epsilon$. For all problem sets, $\epsilon$ was set to 0.02. The computational cost is an approximation of the total number of sums and products in the Transformer block: $m(3Ld^2+2L^2d+2rd^2)$, where $m, L, d$, and $r$ are the number of transformer blocks, the maximum length of input words, the embedding dimension, and the scale factor for the intermediate layer in the fully connected layer (FCN) each transformer block has, respectively. Figure 1 shows the relationship between the computational cost and accuracy, illustrated only for problems among all MLRegTest ones (1.0-1.7 in Table 1) where accuracy reaches close to 1.0. For those problems, we see that the accuracy quickly reaches around 1 while the computational cost is very low.

<figure><img src="image_2.png" /><figcaption>Figure 1: Accuracy (correct answer rate) as a function of approximate computational cost. Results for other TAYSIR tasks are not reported because their test accuracy never statistically outperformed the random classifier.</figcaption></figure>

In Track 2, since the upper bound on quality was usually not reached, i.e., Word Error Rate (WER) was not zero, the one with the lowest WER was selected.
### 3.3. Provided Models

Table 4 and Table 5 describes the architectures of the trained models provided for Track 1 and Track 2, respectively. The last column of these two tables gives an evaluation of the quality of the Neural Nets on the test set: its accuracy for the first track and 1 minus the Word Error Rate (WER) for the second one (for both metrics, the higher the better).

<table><caption>Table 4: Track 1 model architectures</caption><thead><tr><th>Number</th><th>Type</th><th>Nb layers</th><th>Neurons/Layer</th><th>Bidirectional</th><th>Patience</th><th>Nb Dense</th><th>Accuracy</th></tr></thead><tbody><tr><td>1.0.</td><td>SRN</td><td>1</td><td>32</td><td>false</td><td>20</td><td>1</td><td>1</td></tr><tr><td>1.1</td><td>SRN</td><td>1</td><td>32</td><td>true</td><td>5</td><td>1</td><td>0.998</td></tr><tr><td>1.2</td><td>SRN</td><td>1</td><td>32</td><td>true</td><td>5</td><td>2</td><td>0.998</td></tr><tr><td>1.3</td><td>GRU</td><td>2</td><td>512</td><td>false</td><td>5</td><td>1</td><td>1.0</td></tr><tr><td>1.4</td><td>GRU</td><td>1</td><td>512</td><td>true</td><td>5</td><td>1</td><td>0.999</td></tr><tr><td>1.5</td><td>GRU</td><td>1</td><td>512</td><td>false</td><td>5</td><td>2</td><td>1.0</td></tr><tr><td>1.6</td><td>LSTM</td><td>1</td><td>64</td><td>true</td><td>5</td><td>2</td><td>0.997</td></tr><tr><td>1.7</td><td>Transformer</td><td>1*</td><td>12*</td><td></td><td></td><td></td><td>1.0</td></tr><tr><td>1.8</td><td>GRU</td><td>1</td><td>64</td><td>false</td><td>5</td><td>1</td><td>0.730</td></tr><tr><td>1.9</td><td>SRN</td><td>2</td><td>32</td><td>true</td><td>20</td><td>1</td><td>0.639</td></tr><tr><td>1.10.</td><td>GRU</td><td>2</td><td>32</td><td>false</td><td>20</td><td>2</td><td>0.647</td></tr><tr><td>1.11</td><td>LSTM</td><td>1</td><td>64</td><td>false</td><td>5</td><td>1</td><td>0.598</td></tr></tbody></table>

The transformer (TAYSIR 1.7) for Track 1 has 1 transformer block, 6 heads, a dimension of 12, and a scaling factor for FCN of 1.

<table><caption>Table 5: Track 2 model architectures</caption><thead><tr><th>Number</th><th>Type</th><th>Nb layers</th><th>Neurons/Layer</th><th>Bidirectional</th><th>Nb Dense</th><th>1-WER</th></tr></thead><tbody><tr><td>2.0</td><td>GRU</td><td>2</td><td>512</td><td>false</td><td>1</td><td>0.237</td></tr><tr><td>2.1</td><td>GRU</td><td>1</td><td>32</td><td>true</td><td>2</td><td>0.348</td></tr><tr><td>2.2</td><td>LSTM</td><td>2</td><td>256</td><td>false</td><td>2</td><td>0.364</td></tr><tr><td>2.3</td><td>LSTM</td><td>2</td><td>64</td><td>false</td><td>2</td><td>0.393</td></tr><tr><td>2.4</td><td>GRU</td><td>1</td><td>32</td><td>false</td><td>2</td><td>0.154</td></tr><tr><td>2.5</td><td>LSTM</td><td>1</td><td>512</td><td>false</td><td>2</td><td>0.252</td></tr><tr><td>2.6</td><td>SRN</td><td>2</td><td>64</td><td>false</td><td>2</td><td>0.372</td></tr><tr><td>2.7</td><td>SRN</td><td>2</td><td>64</td><td>false</td><td>2</td><td>0.339</td></tr><tr><td>2.8</td><td>SRN</td><td>2</td><td>128</td><td>false</td><td>1</td><td>0.201</td></tr><tr><td>2.9</td><td>GRU</td><td>1</td><td>32</td><td>true</td><td>2</td><td>0.361</td></tr><tr><td>2.10</td><td>Transformer</td><td>8*</td><td>256*</td><td></td><td></td><td>0.452</td></tr></tbody></table>

The transformer (TAYSIR 2.10) for Track 2 has 8 transformer blocks, 8 heads, a dimension of 256, and a scaling factor for FCN of 2.

Finally, note that models 1.0 and 2.0 were used for beta-testing our framework and the one of the participants, and were thus not part of the competition *per se*.

### 4. Baselines

In order to evaluate the complexity of the task of extracting a model from a trained NN, and to provide examples to the participants, we developed and provided two baseline algorithms, one per task.

Both the baselines produced surrogate models which were finite state machines: the one for Track 1 outputted a Deterministic Finite State Automaton (DFA) while the one for
Track 2 extracted a Weighted Automaton (WA) from an already-trained NN. To provide diversity in the type of approaches exemplified, the core of the two baselines were different. However they both return an automaton in linear form (Arrivault et al., 2017): A *linear representation* of an automaton *A* is a triplet $\langle I, (E_\sigma)_{\sigma \in \Sigma}, F \rangle$ where: the vector *I* provides the initial weights; the vector *F* is the terminal weights; each matrix $E_\sigma$ corresponds to the $\sigma$-labeled transition weights, with $\sigma$ a symbol and $\Sigma$ the set of all symbols. The dimension of each element is the number of states of the automaton. Figure 2 shows the same automaton using the usual state diagram as well as its linear form.

<figure><img src="image_3.png" /><figcaption>Figure 2: An automaton in its graphical form and its equivalent linear representation.</figcaption></figure>

$$ 
\begin{align*}
I &= \begin{bmatrix} 1 \\ 0 \end{bmatrix} & F &= \begin{bmatrix} 0 \\ 1/4 \end{bmatrix} \\
E_a &= \begin{bmatrix} 1/2 & 1/6 \\ 0 & 1/4 \end{bmatrix} & E_b &= \begin{bmatrix} 0 & 1/3 \\ 1/4 & 1/4 \end{bmatrix}
\end{align*}
$$

The first baseline relied on the ability to open the black-box (Giles et al., 1992): it parsed the dataset inside the trained model, kept trace of the hidden states encountered, and then used a clustering algorithm on this set of latent vectors to define the states of the DFA (one per cluster). Finally, the transitions were constructed by initializing the hidden state with the centroid of each cluster and feeding the network with each symbols of the alphabet $\Sigma$. The pseudo-code is given in Algorithm 1.

The baseline for Track 2 treated the NN as a pure black-box and used it as an oracle. This baseline was based on the spectral distillation algorithm (Eyraud and Ayache, 2021): the oracle was first used to generate prefixes and suffixes in order to define a finite block of an object called the Hankel matrix, then the trained model was used to fill this matrix, and finally the equations for spectral learning (Balle et al., 2014) were used to obtain a WA in linear form. Its pseudo-code is given in Algorithm 2.

# 5. Competition framework

The manual of the competition, available on its website<fnref n="1" />, provided a large amount of technical information. We detail some of them here.

## 5.1. Technical implementation

The Neural Nets were trained using python 3.9.15, cuda 11.6, cudnn 8.63.8, and torch 1.13.1. They were provided to participants in the form of a MLflow Model (Chen et al., 2020) version 1.25.1 which offers the ability to run the models on any system while at the same time allowing the participants to have access to their inner functions.

The participants had to submit their surrogate models as a MLflow pyfunc, which is the basic generic interface for MLflow Python models. This forbade us to open their models

<footnote n="1">1. https://remieyraud.github.io/TAYSIR/</footnote>
<caption>**Algorithm 1:** TAYSIR baseline for Track 1 – Extraction of a DFA from a RNN</caption>

**Inputs**: $R(\sigma, h)$: trained RNN, $X$: set of sequences, $\Sigma$: set of symbols, $n$: number of states

**Output**: An extracted DFA $(I, F, E_{\Sigma})$ in linear form

$H = []$ // To store the latent states

for $w \in X$ do
    $h = h_0$
    for $i = 0: |w| - 1$ do
        // loop to get all hidden states
        $h \leftarrow \text{RNN}(w[i], h).\text{latent}$
        $H.add(h)$

$k_{\text{means}} = k_{\text{means\_algorithm}}(H, n)$
$Q = k_{\text{means\_centroids}}$ // get centroids as the DFA states

$I, F \leftarrow \text{zeros}(|Q|)$ // initialize final and initial state matrices
$E_{\Sigma} \leftarrow [E_{\sigma} = \text{zeros}(|Q|, |Q|), \forall \sigma \in \Sigma]$ // initialize the transition matrices

for $q \in Q$ do
    // Constructing the transitions
    for $\sigma \in \Sigma$ do
        $\text{out}, h \leftarrow \text{RNN}(\sigma, q)$
        $q' \leftarrow k_{\text{means\_classify}}(h)$
        $E_{\Sigma}[\sigma][q, q'] \leftarrow 1$
        if $\text{out} \ge 0.5$ then
            // Checking if current state is final
            $F[q] \leftarrow 1$

$q_0 \leftarrow k_{\text{means\_classify}}(h_0)$ // Finding the initial state
$I[q_0] \leftarrow 1$

return $(I, F, E_{\Sigma})$

<caption>**Algorithm 2:** TAYSIR baseline for Track 2 – Extraction of a WA from any LM-NN</caption>

**Input**: LM-NN model $\mathcal{M}$, numbers of prefixes $p$ and of suffixes $s$, number of states $r$

**Output**: $A$, a Weighted Automaton

$(\mathcal{P}, \mathcal{S}) \leftarrow \text{Generate\_Basis}(\mathcal{M}, p, s)$
$H_B, (H_{\sigma})_{\sigma \in \Sigma} \leftarrow \text{Fill\_Hankels}(\mathcal{M}, \mathcal{P}, \mathcal{S})$
$A \leftarrow \text{Spectral\_Extraction}(H_B, (H_{\sigma})_{\sigma \in \Sigma}, r)$

return $A$
but provided an easy to use 'pickle' that we could run on a test set. A toolbox was made available to help the participants to create their pyfunc archive.

We used the Codalab platform (Pavao et al., 2022) for the front-end: participants had to create an account there to have access to the trained Neural Nets and to submit their surrogate models.

The submitted models were tested on a dedicated back-end server hosted in our lab. Only a CPU (i9-12900k) was allowed for running the surrogate models and we controlled its use to be sure that only one process ran at any given time.

## 5.2. Evaluation Metrics

The submissions were evaluated on 3 criteria:

1. **Approximation quality**: We used the error rate for the binary classification and the Mean Square Error (MSE) for language modeling. Even when true targets were available, these metrics were computed with the output of the trained neural net output as ground truth.

2. **Memory usage**: The memory footprint of the surrogate model during a prediction on the test set, in mebibytes.

3. **CPU time**: The CPU run time spent by the surrogate model during a prediction on the test set, in milliseconds.

The goal of the participants was to reach the smallest score possible on all these criteria. To evaluate the overall quality of a submission, we computed a global score:

$$ \text{Score} = \frac{1}{2}\text{Quality} \times C + \frac{1}{4}\text{Memory\_ratio} + \frac{1}{4}\text{Time\_ratio} $$

where

$$ \text{Memory\_ratio} = \frac{\text{surrogate memory usage}}{\text{Neural Net memory usage}} $$ $$ \text{Time\_ratio} = \frac{\text{surrogate CPU time}}{\text{Neural Net CPU time}} $$

The value of C was always 1 for Track 1, but was the magnitude order of the mean square error of the baseline on the test set for Track 2. This was done because MSE values are extremely small compare to the other metrics, which implies that without this constant C MSE would have had small impact in the global score.

The memory usage was tracked using the psutil (process and system utilities) package (version 5.9.4). This had the drawback to come with a fixed offset which was due to the overhead of using MLflow to run the surrogate models: even an empty model that just always returned a fixed value, for instance True for Track 1, had a memory usage of 120mb in our local setting (python 3.8 and MLflow 1.25.1). It also forced the participants to be very cautious with their variable usage since any declared variable was serialized by MLflow. Unfortunately, it appeared during the competition that the offset size changes with the
version of the libraries used, minimizing the interest of this metric – though finding the versions that requires the smallest memory usage is of interest of its own, it was not the aim of TAYSIR.

To deal with the CPU time computation, we created a virtual environment for each participant that was sourced for all their submissions. This means that if their code required the installation of specific packages to run, the first submission CPU time would be biased by the installation of the packages. However, this would happen only once and the participants were invited to resubmit the same model to have a fair evaluation.

Finally, each run of a surrogate model was limited to 300 seconds.

# 6. Results

At the time we are writing this article, we do not know about the approaches followed by the participants, which makes it hard to give relevant insights about the results.

Table 6 provides the detailed results for Track 1 of the top 3 participants (and of the baseline, for comparison) for each task.

The first element to notice is that tasks 1.2 to 1.7 were solved from a knowledge distillation perspective: the error rate of the submitted surrogate models is 0 or very close to. On these tasks, the memory usage is the metric that gave the final ranking of participants.

Secondly, on all the last 3 problems, it is worth noticing that the leader of the task almost always have an error rate an order of magnitude lower than their opponents.

Finally, on problem 1 and 8, the competition finished with extremely tight results on almost all metrics, making these problems the most disputed ones.

Table 7 details the results of Track 2. One team (named EdiMuskardin) managed to win every tasks which made it the clear winner of the Track. As the same team did also great on Track 1, it is the obvious winner of the competition.

# 7. Conclusion

Despite the lower than expected participation on the second track, the success of the first one validated the interest of this event for the community.

Given the diversity of the architectures and of the tasks on which the Neural Nets were trained, it is likely that TAYSIR will become a reference benchmark for knowledge distillation of models trained on sequences of symbols. All the elements, including the test sets and different scripts used for evaluation, are available on the competition website. In this benchmark archive, we also included as a bonus the transformers trained on TAYSIR datasets but not used during the competition.

# Acknowledgments

We are deeply grateful to Nicolas Buton for the ENE dataset. We also want to thank the members of our scientific committee for the amazing discussions and the work realized for this competition.
<table><caption>Table 6: Track 1 results. Detailed score of the Top 3 on each task and of the baseline.</caption><thead><tr><th>Task Number</th><th>Team Name</th><th>Error Rate</th><th>Memory Usage</th><th>CPU time</th><th>Score</th></tr></thead><tbody><tr><td rowspan="4">1.1</td><td>EdiMuskardin</td><td>0.0750300 (1)</td><td>122.00 (3)</td><td>0.045 (1)</td><td>0.1286291 (1)</td></tr><tr><td>neuralchecker</td><td>0.0844100 (2)</td><td>139.00 (4)</td><td>0.068 (2)</td><td>0.1478028 (2)</td></tr><tr><td>kawa yo</td><td>0.2030100 (3)</td><td>99.00 (1)</td><td>0.090 (3)</td><td>0.1812771 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.4478500 (4)</td><td>150.00 (5)</td><td>0.167 (4)</td><td>0.3482493 (4)</td></tr><tr><td rowspan="4">1.2</td><td>kawa yo</td><td>0.0000100 (2)</td><td>89.00 (1)</td><td>0.032 (1)</td><td>0.0664160 (1)</td></tr><tr><td>EdiMuskardin</td><td>0.0000000 (1)</td><td>120.00 (2)</td><td>0.039 (2)</td><td>0.0891704 (2)</td></tr><tr><td>neuralchecker</td><td>0.0000000 (1)</td><td>121.00 (3)</td><td>0.072 (3)</td><td>0.0934730 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.5001800 (3)</td><td>150.00 (4)</td><td>0.166 (4)</td><td>0.3740798 (4)</td></tr><tr><td rowspan="4">1.3</td><td>kawa yo</td><td>0.0000000 (1)</td><td>93.00 (1)</td><td>0.085 (3)</td><td>0.0639508 (1)</td></tr><tr><td>EdiMuskardin</td><td>0.0000000 (1)</td><td>120.00 (2)</td><td>0.038 (1)</td><td>0.0824521 (2)</td></tr><tr><td>neuralchecker</td><td>0.0000000 (1)</td><td>121.00 (3)</td><td>0.074 (2)</td><td>0.0831711 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.5000000 (2)</td><td>150.00 (5)</td><td>0.168 (5)</td><td>0.3531740 (5)</td></tr><tr><td rowspan="4">1.4</td><td>kawa yo</td><td>0.0000000 (1)</td><td>90.00 (1)</td><td>0.032 (1)</td><td>0.0627416 (1)</td></tr><tr><td>neuralchecker</td><td>0.0000000 (1)</td><td>96.00 (2)</td><td>0.057 (3)</td><td>0.0669726 (2)</td></tr><tr><td>EdiMuskardin</td><td>0.0000000 (1)</td><td>119.00 (3)</td><td>0.038 (2)</td><td>0.0829482 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.5001700 (2)</td><td>150.00 (4)</td><td>0.164 (4)</td><td>0.3548888 (4)</td></tr><tr><td rowspan="4">1.5</td><td>kawa yo</td><td>0.0000000 (1)</td><td>93.00 (1)</td><td>0.086 (3)</td><td>0.0654267 (1)</td></tr><tr><td>neuralchecker</td><td>0.0000000 (1)</td><td>96.00 (2)</td><td>0.056 (2)</td><td>0.0674238 (2)</td></tr><tr><td>EdiMuskardin</td><td>0.0000000 (1)</td><td>120.00 (3)</td><td>0.038 (1)</td><td>0.0841690 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.5207700 (2)</td><td>150.00 (5)</td><td>0.168 (5)</td><td>0.3660161 (5)</td></tr><tr><td rowspan="4">1.6</td><td>kawa yo</td><td>0.0000100 (1)</td><td>93.00 (1)</td><td>0.086 (3)</td><td>0.0663661 (1)</td></tr><tr><td>neuralchecker</td><td>0.0000100 (1)</td><td>96.00 (2)</td><td>0.054 (2)</td><td>0.0683040 (2)</td></tr><tr><td>EdiMuskardin</td><td>0.0000200 (2)</td><td>120.00 (3)</td><td>0.037 (1)</td><td>0.0852084 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.4977400 (3)</td><td>149.00 (4)</td><td>0.166 (4)</td><td>0.3553508 (4)</td></tr><tr><td rowspan="3">1.7</td><td>kawa yo</td><td>0.0000000 (1)</td><td>92.00 (1)</td><td>0.086 (3)</td><td>0.0621229 (1)</td></tr><tr><td>neuralchecker</td><td>0.0000000 (1)</td><td>96.00 (2)</td><td>0.055 (2)</td><td>0.0646378 (2)</td></tr><tr><td>EdiMuskardin</td><td>0.0000000 (1)</td><td>120.00 (3)</td><td>0.037 (1)</td><td>0.0806272 (3)</td></tr><tr><td rowspan="4">1.8</td><td>kawa yo</td><td>0.3269795 (2)</td><td>105.00 (1)</td><td>0.025 (1)</td><td>0.2256924 (1)</td></tr><tr><td>neuralchecker</td><td>0.3269795 (2)</td><td>108.00 (2)</td><td>0.030 (2)</td><td>0.2275360 (2)</td></tr><tr><td>EdiMuskardin</td><td>0.3267700 (1)</td><td>132.00 (3)</td><td>0.038 (3)</td><td>0.2416978 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.3269795 (2)</td><td>163.00 (4)</td><td>0.166 (4)</td><td>0.2622803 (4)</td></tr><tr><td rowspan="4">1.9</td><td>EdiMuskardin</td><td>0.0074658 (1)</td><td>118.00 (2)</td><td>0.040 (2)</td><td>0.0917099 (1)</td></tr><tr><td>kawa yo</td><td>0.0725840 (3)</td><td>90.00 (1)</td><td>0.024 (1)</td><td>0.1027667 (2)</td></tr><tr><td>neuralchecker</td><td>0.0306927 (2)</td><td>121.00 (3)</td><td>0.057 (3)</td><td>0.1072263 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.9274160 (4)</td><td>149.00 (4)</td><td>0.154 (4)</td><td>0.5855869 (4)</td></tr><tr><td rowspan="4">1.10</td><td>EdiMuskardin</td><td>0.0149316 (1)</td><td>119.00 (2)</td><td>0.042 (2)</td><td>0.0944037 (1)</td></tr><tr><td>kawa yo</td><td>0.0680216 (3)</td><td>91.00 (1)</td><td>0.023 (1)</td><td>0.0999327 (2)</td></tr><tr><td>neuralchecker</td><td>0.0522605 (2)</td><td>126.00 (3)</td><td>0.059 (3)</td><td>0.1191474 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.0680216 (3)</td><td>149.00 (4)</td><td>0.172 (5)</td><td>0.1505605 (4)</td></tr><tr><td rowspan="4">1.11</td><td>EdiMuskardin</td><td>0.0083787 (1)</td><td>133.00 (2)</td><td>0.059 (3)</td><td>0.0854677 (1)</td></tr><tr><td>neuralchecker</td><td>0.0219941 (2)</td><td>186.00 (6)</td><td>0.086 (4)</td><td>0.1246985 (2)</td></tr><tr><td>kawa yo</td><td>0.2899036 (3)</td><td>104.00 (1)</td><td>0.025 (1)</td><td>0.2083020 (3)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.7100964 (4)</td><td>164.00 (5)</td><td>0.164 (6)</td><td>0.4561645 (6)</td></tr></tbody></table>
<table><caption>Table 7: Track 2 results. Detailed score of the Top 2 on each task and of the baseline.</caption><thead><tr><th>Task Number</th><th>Team Name</th><th>Score</th><th>MSE×C</th><th>Memory Usage</th><th>CPU Time</th></tr></thead><tbody><tr><td rowspan="3">2.1</td><td>EdiMuskardin</td><td>0.1750089 (1)</td><td>0.1562607 (1)</td><td>118.0 (1)</td><td>0.04500 (1)</td></tr><tr><td>neuralchecker</td><td>0.2858219 (2)</td><td>0.3767927 (2)</td><td>118.0 (1)</td><td>0.09700 (2)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.4090530 (3)</td><td>0.5728363 (3)</td><td>148.0 (2)</td><td>0.16350 (4)</td></tr><tr><td rowspan="3">2.2</td><td>EdiMuskardin</td><td>0.0097656 (1)</td><td>0.0012267 (1)</td><td>118.0 (1)</td><td>0.04350 (1)</td></tr><tr><td>neuralchecker</td><td>0.0134414 (2)</td><td>0.0085731 (2)</td><td>118.0 (1)</td><td>0.09700 (2)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.1665942 (3)</td><td>0.3102189 (3)</td><td>148.0 (2)</td><td>0.17100 (3)</td></tr><tr><td rowspan="3">2.3</td><td>EdiMuskardin</td><td>0.0959499 (1)</td><td>0.0000366 (1)</td><td>117.0 (1)</td><td>0.04450 (1)</td></tr><tr><td>neuralchecker</td><td>0.0970022 (2)</td><td>0.0004323 (2)</td><td>118.0 (2)</td><td>0.09600 (2)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.1897990 (3)</td><td>0.1367553 (3)</td><td>148.0 (3)</td><td>0.16300 (3)</td></tr><tr><td rowspan="3">2.4</td><td>EdiMuskardin</td><td>0.0966079 (1)</td><td>0.0000064 (2)</td><td>118.0 (1)</td><td>0.04300 (1)</td></tr><tr><td>neuralchecker</td><td>0.0972557 (2)</td><td>0.0000001 (1)</td><td>118.0 (1)</td><td>0.09750 (2)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.3531379 (3)</td><td>0.4612565 (3)</td><td>148.0 (2)</td><td>0.16650 (3)</td></tr><tr><td rowspan="3">2.5</td><td>EdiMuskardin</td><td>0.0943649 (1)</td><td>0.0000001 (1)</td><td>117.0 (1)</td><td>0.04350 (1)</td></tr><tr><td>neuralchecker</td><td>0.0951833 (2)</td><td>0.0000001 (2)</td><td>118.0 (2)</td><td>0.09550 (2)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.3684454 (3)</td><td>0.4981066 (3)</td><td>148.0 (3)</td><td>0.16200 (3)</td></tr><tr><td rowspan="3">2.6</td><td>EdiMuskardin</td><td>0.1956771 (1)</td><td>0.1971876 (1)</td><td>118.0 (1)</td><td>0.04550 (1)</td></tr><tr><td>neuralchecker</td><td>0.3246898 (2)</td><td>0.4544016 (2)</td><td>118.0 (1)</td><td>0.09650 (2)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.4473630 (3)</td><td>0.6495335 (3)</td><td>148.0 (2)</td><td>0.16150 (3)</td></tr><tr><td rowspan="3">2.7</td><td>EdiMuskardin</td><td>0.0959622 (1)</td><td>0.0000000 (1)</td><td>118.0 (1)</td><td>0.04400 (1)</td></tr><tr><td>neuralchecker</td><td>0.0973649 (2)</td><td>0.0000000 (1)</td><td>119.0 (2)</td><td>0.09700 (2)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.1942414 (3)</td><td>0.1435849 (2)</td><td>149.0 (3)</td><td>0.16950 (3)</td></tr><tr><td rowspan="3">2.8</td><td>neuralchecker</td><td>0.0569728 (1)</td><td>0.0053492 (1)</td><td>118.0 (1)</td><td>0.09700 (2)</td></tr><tr><td>EdiMuskardin</td><td>0.0601653 (2)</td><td>0.0118116 (2)</td><td>118.0 (1)</td><td>0.04350 (1)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.1949540 (3)</td><td>0.2536402 (3)</td><td>148.0 (2)</td><td>0.16450 (3)</td></tr><tr><td rowspan="3">2.9</td><td>EdiMuskardin</td><td>0.0975634 (1)</td><td>0.0000000 (1)</td><td>119.0 (1)</td><td>0.04350 (1)</td></tr><tr><td>neuralchecker</td><td>0.0977556 (2)</td><td>0.0000000 (1)</td><td>119.0 (1)</td><td>0.06800 (2)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.2016953 (3)</td><td>0.1572833 (2)</td><td>149.0 (2)</td><td>0.16850 (3)</td></tr><tr><td rowspan="3">2.10</td><td>EdiMuskardin</td><td>0.1554995 (1)</td><td>0.1442694 (1)</td><td>119.0 (1)</td><td>0.04250 (1)</td></tr><tr><td>neuralchecker</td><td>0.1757673 (2)</td><td>0.1832887 (2)</td><td>120.0 (2)</td><td>0.12050 (2)</td></tr><tr><td>TAYSIR-Baseline</td><td>0.2038741 (3)</td><td>0.2002162 (3)</td><td>148.0 (3)</td><td>0.16800 (3)</td></tr></tbody></table>
References

D. Arrivault, D. Benielli, F. Denis, and R. Eyraud. Scikit-SpLearn: a toolbox for the spectral learning of weighted automata compatible with scikit-learn. In *Conférence francophone en Apprentissage*, 2017.

B. Balle, X. Carreras, F. Luque, and A. Quattoni. Spectral learning of weighted automata. *Machine Learning*, 96(1-2):33–63, 2014.

B. Balle, R. Eyraud, F. M. Luque, A. Quattoni, and S. Verwer. Results of the sequence prediction challenge (SPiCe): a competition on learning the next symbol in a sequence. In *Proc. of the International Conference on Grammatical Inference*, volume 57 of *PMLR*, pages 132–136, 2017.

B. Barbot, B. Bollig, A. Finkel, S. Haddad, I. Khmelnitsky, M. Leucker, D. Neider, R. Roy, and L. Ye. Extracting context-free grammars from recurrent neural networks using tree-automata learning and a* search. In *Proc. of ICGI*, volume 153, pages 113–129. PMLR, 2021.

A. Chen, A. Chow, A. Davidson, A. DCunha, A. Ghodsi, Sue Ann Hong, A. Konwinski, C. Mewald, S. Murching, T. Nykodym, P. Ogilvie, M. Parkhe, A. Singh, F. Xie, M. Zaharia, R. Zang, J. Zheng, and C. Zumar. Developments in mlflow: A system to accelerate the machine learning lifecycle. In *DEEM'20*, 2020.

K. Cho, B. van Merrienboer, Ç. Gülçehre, F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. *CoRR*, abs/1406.1078, 2014.

A. Deoras, T. Mikolov, S. Kombrink, M. Karafiát, and S. Khudanpur. Variational approximation of long-span language models for lvCSR. *Proc. of the International Conference on Acoustics, Speech and Signal Processing*, pages 5532–5535, 2011.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In *Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)*, pages 4171–4186. Association for Computational Linguistics, 2019.

G. Dong, J. Wang, J. Sun, Y. Zhang, X. Wang, T. Dai, J. S. Dong, and X. Wang. Towards interpreting recurrent neural networks through probabilistic abstraction. In *Proc. of ASE*, page 499–510. Association for Computing Machinery, 2021.

J. L. Elman. Finding structure in time. *Cognitive Science*, 14(2):179–211, 1990.

R. Eyraud and S. Ayache. Distillation of Weighted Automata from Recurrent Neural Networks using a Spectral Approach. *Machine Learning*, 2021.

GDPR. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal
data and on the free movement of such data (General Data Protection Regulation). *Official Journal of the European Union*, L119:1–88, May 2016.

C. L. Giles, C. B. Miller, D. Chen, H. H. Chen, G. Z. Sun, and Y. C. Lee. Learning and extracting finite state automata with second-order recurrent neural networks. *Neural Computation*, 4(3):393–405, 1992.

R. Guidotti, A. Monreale, F. Turini, D. Pedreschi, and F. Giannotti. A survey of methods for explaining black box models. CoRR, abs/1802.01933, 2018.

R. R. Harmon and N. Auseklis. Sustainable it services: Assessing the impact of green computing practices. In *PICMET '09 - 2009 Portland International Conference on Management of Engineering Technology*, pages 1707–1717, 2009.

G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. In *NIPS Deep Learning and Representation Learning Workshop*, 2015.

S. Hochreiter and J. Schmidhuber. Long short-term memory. *Neural Computation*, 9(8): 1735–1780, 1997.

D. Hong, A. M. Segre, and T. Wang. Adaax: Explaining recurrent neural networks by learning automata with adaptive states. In *Proc. of KDD*, page 574–584. Association for Computing Machinery, 2022.

C. Lacroce, P. Panangaden, and G. Rabusseau. Extracting weighted automata for approximate minimization in language modelling. In *Proc. of ICGI*, volume 153 of *Proceedings of Machine Learning Research*, pages 92–112. PMLR, 2021.

F. Mayr, S. Yovine, F. Pan, N. Basset, and T. Dang. TOWARDS EFFICIENT ACTIVE LEARNING OF PDFA. In *LearnAut 2022*, Paris, France, 2022.

E. Muškardin, B. K. Aichernig, I. Pill, and M. Tappler. Learning finite state models from-recurrent neural networks. In *Proc. of IFM*, page 229–248. Springer-Verlag, 2022.

S. Panchapagesan, D. S. Park, C.-C. Chiu, Y. Shangguan, Q. Liang, and A. Gruenstein. Efficient knowledge distillation for rnn-transducer models. In *IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pages 5639–5643, 2021.

A. Pavao, I. Guyon, A.-C. Letournel, X. Baró, H. Escalante, S. Escalera, T. Thomas, and Z. Xu. Codalab competitions: An open source platform to organize scientific challenges. *Technical report*, 2022.

Tim Räz and Claus Beisbart. The importance of understanding deep learning. *Erkenntnis*, 2022.

V. Sanh, L. Debut, J. Chaumond, and T. Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. *arXiv preprint arXiv:1910.01108*, 2019.

B. Starkie, F. Coste, and M. van Zaanen. Progressing the state-of-the-art in grammatical inference by competition: The omphalos context-free language learning competition. *AI Commun.*, 18(2):93–115, 2005.
M. Steinegger and J. Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. *Nat Biotechnol*, 35:1026-1028, 2017.

The UniProt Consortium. UniProt: the Universal Protein Knowledgebase in 2023. *Nucleic Acids Research*, 51(D1):D523-D531, 11 2022.

S. van der Poel, D. Lambert, K. Kostyszyn, T. Gao, R. Verma, D. Andersen, J. Chau, E. Peterson, C. St. Clair, P. Fodor, C. Shibata, and J. Heinz. Mlregtest: A benchmark for the machine learning of regular languages, 2023.

S. Verwer, R. Eyraud, and C. de la Higuera. PAutomaC: a probabilistic automata and hidden markov models learning competition. *Machine Learning*, 96(1-2):129-154, 2014.

Q. Wang, K. Zhang, A. Ororbia, X. Xing, X. Liu, and C. Lee Giles. An empirical evaluation of recurrent neural network rule extraction. *Neural Computation*, 30, 2017.