dataesr/scientific-paragraphs-categorization huggingface dataset:
The precision of the fixedpoint value is a configurable parameter. The activation functions and the recurrent activation functions other than the ReLU are implemented as LUT. The LUTs are generated with Python scripts. A configurable parameter allows using full precision mathematical functions instead of LUTs. Two variants of the recurrent functions are implemented to support the single-cell and the sliding-window architectures. The single-cell function uses one instance of the LSTM cell implementation and allows linking the output of this cell at a given BC to its input at the subsequent BC. A continuous output flow is achieved with data entering through recursive calls of the logic, however requiring an input frequency no larger than the cell computation time. In the sliding-window, the function invokes for each window five instances of either the LSTM cell or the vanilla RNN cell, one for each BC.
The output of each cell serves as an input to the next. The algorithm requires one such chain of five RNN cells for each BC to predict the deposited energy. To be able to process data in real time without using multiple RNN chains for multiple BCs, a fully pipelined design is needed. The implemented design ensures that the initiation interval, i. e. the number of clock cycles between two inputs in HLS, is equal to one. Every loop is fully unrolled: each of the loop iterations has its own logic resources. The memory needed is implemented as registers to optimize the latency. A comparison of the energy computation in software, as given by Keras, and in firmware simulation with Quartus 21. 1 and Questa Sim 10. 7c is shown in Fig. 11. The fixed point values are chosen to ensure a resolution of the order of 1%.
Raw data reduction was done using GlitterTM software. The analyses of pentlandite were quantified with S as an internal standard, using the  mean S concentration measured by EMPA  (Lorand, 1985). Apart from a few analyses having sampled "pure" minerals, most are mixtures of several phases, BMS and/or alloys, as indicated by their large range of Ni and Cu contents (see below). The compositions of such mixtures were quantified using either S, Cu and/or Ni as internal standards and linear mixing between pentlandite/native copper and pentlandite/awaruite volume ratio calculated from Cu/S and Ni/S count rates in timeintegrated count diagrams. The detection limits in each quantification were calculated for each analysis from the gas blank and the sensitivity of the reference materials following the equations in  Longerich et al.
8 ). If incorporated in the calculation of PGE concentrations, such Te-rich phases produce nearly flat CI-chondrite normalized PGE pattern (Fig. 7 C, D ). Marchesi et al. (2010)  also detected Pt-rich micronuggets up to several microns across in their LA-ICPMS analyses. Pentlandite compositions vary according to the coexisting mineral assemblage intercepted by the laser beam and to a lesser extent, the tectonometamorphic domain they come from. Marchesi et al. (2010)  interpreted their extreme Os tenors (up to 908 ppm) as likely caused by Os-rich minerals (alloys or sulfides) inside the targets of the laser spots. We did not find such extreme Os contents in our LA-ICPMS data set. However, several pentlandite analyses in DR-7 (analyses Qa-11 and Qa-16 in Table  S1 ), a spinel tectonite lherzolite, produced laurite-type chondrite-normalized HSE patterns characterized by positive Os, Ru and (Rh) anomalies with respect to Ir (Fig.
Acknowledgements The authors thanks  Monique Marot  for the polished thick sections and  Olivier Alard  and  Olivier Bruggier  for their help with the LA-ICPMS analyses. Financial funding was provided by  CNRS  ( UMR 7160  and  6112 ), the  University of Granada  ( Junta de Andalucia ), a  French-Spanish  integrated action ( HF1997-0082 ) and the  Spanish DGES  (Project  PB97-1211 ). The final version was greatly improved thanks to comments from  Laurie Reisberg  and two anonymous reviews, and editorial suggestions from associate editor  Michael Roden.
Linguistic Features We enriched the original textual material with three types of linguistic annotations (obtained automatically): part of speech tags, morphological tags and syntactic dependencies. Theses three types of annotations were realized using an augmented (neural network based) version our software MACAON  (Nasr et al. , 2011) , where words in a sentence are discovered then annotated one at a time (from left to right). Annotation is based on classifiers that take as input features about current word and its context and produce as output a probability distribution over a set of actions. Such actions posit word boundaries on the raw text, associate part of speech and morphological tags to words or link words of a sentence with syntactic dependencies. The actions that perform the prediction of syntactic dependencies are based on the transition based parsing framework  (Nivre, 2003) , which makes use of a stack that stores words that should be connected to words not yet discovered.
The Decision Layer The decision layer is simply a linear layer (with bias) of input size 1600 and output size 1. There are 5 different decision layers (one for each value to predict). Parameter of this linear layer are the only one that were not shared between the 5 predictions tasks. predict the challenge's five oculometric measures, but with different models. The software used to train our model is available online: https://gitlab. lis-lab. fr/franck. dary/macaon/ 3 See section 4 about hyperparameters selection. 4 Implementation: https://github. com/stanfordnlp/GloVe. Words with less than 2 occurrences were treated as unknown words, thus producing an unknown words embedding.
Ground truth creation Training and testing data have been annotated with a layout analysis and a baseline model, and manually proofread via eScriptorium  [14, 12]. The dataset is composed of 1. 861 lines of text (1,524 reserved for training, 168 for validation and 169 for testing). We built a total of four datasets: 1. D-exp, consisting of a transcription with full expansion of the abbreviations. We consider two variants, D-exp1 with inter-word spaces restored (separation of words according to Latin grammar and not the spacing present in the manuscript), and D-exp2 without spaces. These datasets are respectively composed of 34 and 33 classes. The number of classes has been limited to include enough samples in each. This can lead to strong intra-class inertia, due in particular to the unsystematic grouping of upper-and lower-case letters for the less endowed classes.
2. D-abb, composed of transcriptions registering the abbreviation system used in this manuscript (see supra 1. 1). We also consider two variants, D-abb1 with inter-word spaces, and D-abb2 without spaces. These datasets are respectively composed of 60 and 59 classes. In detail, we have 36 classes representing alphanumeric characters and punctuation marks, 10 classes specific to Latin paleography to represent certain abbreviations according to the same scheme as the Oriflamms project (e. g. ƀ and ꝙ; see infra), and 13 classes made up of combining signs (written above or below one or more letters). These last 13 classes can be difficult to identify for an HTR engine. Moreover, additional data was used for some of the trainings: 1. Oriflamms diplomatic and allographetic transcriptions provided by D.
HTR on abbreviated and expanded data Common parameters have been chosen for training steps of HTR-CB and HTR-WB. Line images provided as input are resized to 64px in height and have varying widths. No Unicode normalisation or data augmentation is applied. A repolygonisation has been performed to equalise polygons of first and last rows of each text columns  [24]. Due to the small size of the dataset and to avoid overfitting, training steps are carried out with a dynamic learning rate starting at 0. 001 -to which a coefficient of 0. 75 is applied every 10 epochs -, and we use a batch of one image for each iteration. We have limited training to 30 epochs. First results are summarised in Table  1. The two architectures give equivalent results on the two variants of the Dabb dataset. At identical initial parameters, we do not observe any significant difference, except that HTR-CB converges twice as fast as HTR-WB.
Text normalisation approach The modular text normalisation approach uses several consecutive steps, with specialised tools each necessitating a training of its own  [6] , as follows (fig. 2 ): 1. HTR (see above). 2. word segmentation using a deep-learning word segmentor, Boudams. Word segmentation A neural word segmentor, Boudams, was trained using the best known configuration: convolutional encoder without position embeddings (Embedding 256, KernelSize 5, Dropout 0. 25, Layers 10)  [8]. Three datasets were used: for abbreviated text, D-abb (with word separation and line hyphenations normalised) alone, and with the addition of Oriflamms data; for normalised version, D-exp (with word separation and line endings normalised) with the addition of PL data. Three models were trained for each setting, and the best of the three (based on F-statistic) was selected.
Discussion and further research Results seem to confirm the importance of the word level rather than the character level, with word-based overperforming character-based HTR, and with character-based HTR results being improved by adding tools dealing with wordlevel tokens and context, in particular for a small dataset. In this regard, artificial intelligence can be compared to human intelligence and seems to confirm the practice of global reading and global perception of the words rather than individual letters, reflected in the use of abbreviations in Latin (especially scholastic) manuscripts. Future research should investigate differences with vernacular manuscripts, for instance literature in Old French, where reading and the use of abbreviations are supposed to have remained mostly syllabic.
A problem arises when a domain expert user has to study the behavior of particular dataset instances over a predictive model. For example, a physician wanting to perform a cohort study of his patients may also need an explanation of prediction for every single patient, rather than just a global one. In this case, a global explanation is not enough to give the information needed by the study. To fulfill that need, past researches studied the possibility of explaining single instance prediction of a model, e. g. [3]  and  [4]. However, these methods may be specifically designed for a single model type as in  [5] , which limits their use. Some methods are designed to be applicable for all types of models and are more and more used in the industry  1  but suffer from a lack of precision  [6]. Yet overcoming this lack of precision can make their algorithms very long to apply  [7].
Related works 2. 1 How to explain a model by using its predictions Explaining the influence of each attribute of a dataset on the output of a predictive model has been explored largely. A few of the works related to global attribute importance on a model can be seen in  [11]   [12]. The most recent methods are based on swapping the values of attributes in the dataset and analyzing which swaps affect the trained model predictions the most. The more modifying the values of the attributes affect the predictions, the more this attribute is considered important for the model, as a whole. These methods are often used during feature selection, allowing to opt-out attributes useless for the model. Another approach for understanding a model is described by Helenius et al. in  [13]. The authors seek to understand which attributes of the datasets are "linked" to each other, according to the model.
In particular, they proceed by randomizing the values of potential groups of attributes: when the predictions vary more than a predetermined threshold, the attributes are linked together. The problem with a fixed global influence is that predictive models are often not consistent with the whole dataset on which they are trained. These global influences give an insight into the general working of the studied model, but there will often be particular regions of the dataset where the model deviates from this general influence. Thus, there also exists a need for a single instance prediction explanation, showing the user how a particular instance is classified, independently to the rest of the dataset. These methods aim to provide insight into the global influence of each attribute, rather than on their influence on a single prediction.
In the field of single prediction, these global influence methods have served as a basis to  [4] , using the same randomization technique. Given a single instance, the importance of each attribute is obtained by looking at the evolution of the prediction performance of the model on the instance when all of its values are swapped with other values of the dataset except the value of the attribute being studied. The more the prediction varies with the values swapping, the less the fixed attribute is important for the prediction. This method has the interest of relying on the same principle as the global technique which is largely recognized, but the main caveat is its computational cost needed for explaining the prediction of only one instance, as a large number of new predictions has to be generated for each single instance prediction to explain.
Another caveat is this prediction explanation is realized from the point of view of model performance. Meaning that their metric shows which feature improves the performance of the model, rather than which feature the model consider as important for its prediction. If this line of reasoning is interesting for the model explanation field, it does not correspond to our scope, as we are aiming to help users understand how a model works, and not how to improve it. Another approach can be found in  [14] , which inspired many of the functions of  [15]. The principle here is to determine the smallest change needed in order to change the classification of an instance by the model. It has been integrated into the What-if tool. This tool allows the user to select a particular data point in the dataset, and visualize the nearest point classified differently.
This absence of an attribute is simulated by a weighted mean of the predictions of the model with all the possible values of the attribute, weighted by their probability of appearing in the dataset. This is faster than the method of  [4]  as only one value is randomized. Later, in  [3] , the possibility of retraining the model entirely without the considered attribute in the dataset is proposed, which consists of an interesting trade-off: the time needed for retraining a model for each attribute of a dataset can be considerable, but once this training has been done, the prediction comparison can become near-instantaneous. These methods are named SHapley Additive exPlanations by Lundberg et al. [10] , who regrouped a large number of similar methods of explanation, and detailed in Section 3.
Thus, it is more relevant to focus on the works as  [16]  or  [17] , cited as additive methods, as they generate a simple set of importance weights for each attribute. This set of weights is easy to interpret, even for someone without expertise in machine learning. Yet, these methods have a major deterrent: their complexity makes them difficult to use for the average user. That is why  [10]  explored methods to generate explanations faster, but at the cost of very restricting hypotheses, as the independence of each attribute of the dataset, or the linearity of the model, which is not always the case. LIME is another popular additive interpretability method  [18]. However, it suffers from previously described restricting hypotheses as it relies on a surrogate linear model to locally approximate influence of each variable, through multiple random perturbations of the instance to explain.
Grouping attributes In our work, we want to facilitate the generation of prediction explanation, without having to restrict to a given set of models. This paper is the continuity of  [8]  in which we already established possible methods of simplification and rely on the automatic detection of groups of attributes (especially the K-complete method, detailed in Section 3. 2. ). For this work, we aim to identify and compare additional methods detecting groups, in order to compare their influence on the efficiency of the simplification method. The selection of relevant attributes to be grouped can take inspiration from the works in the field of feature selection  [20] [21]. In particular, the methods proposed in a dimensional reduction goal seem to reach our scope. Indeed, these methods have to automatically detect interactions between attributes for reducing a potential high dimensionality in a dataset.
Thus, two main approaches, feature extraction (mainly the principal component analysis) and filter methods (which measure the relevance of features by their correlations) can be considered. The fact that the principal component analysis (PCA) and the filter methods rely only on information provided by a dataset (independent of the model used in analysis) is a great advantage for our work, in contrast with techniques such as SVM-RFE  [22]  or FS-P  [23] , based on a specific model. Indeed, different predictive models can classify differently the same instance. Thus, an explanation on this instance can be different from one model to another, and cannot depend on a selection of influence attributes made by a unique predictive model, such as SVM. The PCA is a largely recognized method to provide new features from sets of correlated attributes.
We then look at not only the influence of the attribute but also its use in all subsets of attributes. We thus define the complete influence of an attribute a i ∈ A on the classification of an instance x : given a dataset of instances described along the attributes of A, the complete influence of the attribute a i on the classification of an instance x by the classifier confidence function f on the class C is dependant on the influence of all the possibles subgroups A ⊆ A which does not contain a i. Thus, the complete influence of a i is : I C ai (x) = A ⊆A\ai p(A , A) * (inf C f,(A ∪ai) (x) -inf C f,A (x)) (1) With p(A , A) a penalty function accounting for the size of the subset A. Indeed, if an attribute changes a lot the result of a classifier, in a large group of attributes, it can be considered as very influential compared to the others.
On the opposite, an attribute changing the result of a classifier, whereas this classifier is based on a few attributes, cannot be considered to have a decisive influence. The Shapley value  [25]  is a promising candidate and defines this penalty as: p(A , A) = |A |! * (|A| -|A | -1)! |A|! (2) This complete influence of an attribute now takes into consideration its importance among all the possible attribute configurations, which is closer to the original intuition behind attributes' influence. However, computing the complete influence of a single instance is extremely computationally expensive, with complexity in (2 n * l(n, x)), with n the number of attributes, x the number of instances in the dataset, and l(n, x) the complexity of training the model to be explained. It is then not practical to use the complete influence.
They are summarized in  [10]  as methods giving for a particular prediction a weight to each attribute of the dataset. This creates a very simple "predictive model", locally mimicking the original model's behavior. Thus, we have a simple interpretable linear model that gives information on the original model's inner working in a small vicinity of the predicted instance. The methods, from which these weights are affected to each attribute, vary between the different additive methods, but the end result is always this vector of weights. In particular, this unified approach is based on a SHAP (SHapley Additive exPlanation) value giving the importance of a feature depending on the predictive model used (for a specific prediction): LinearSHAP for linear models, TreeSHAP for tree-based models and DeepSHAP for neural networks.
Model-based coalition Regarding the Model-based coalition approach, the groups of attributes are created by using the model itself to detect interacting attributes. In this approach, no correlation is detected, but only interaction in the sense of the model usage of the attributes. This is done by randomizing the values of the dataset and studying the evolution of the model predictions. It consists of measuring the differences of predictions on the whole dataset before and after the randomization. When attributes are considered to be part of the same group, their values are swapped together with the values of another instance, classified by the model as the same class as the starting instance. Each attribute outside of the group has its value swapped completely randomly. Once this has been done, the new instances are classified by the model.
The ratio of differences between the old and the new classification is called fidelity. A higher fidelity meaning a lower variation of the predictions. At each iteration, the attribute which removal lowers the less the fidelity is removed until it is not possible to keep the fidelity above a fixed threshold. Then the group is considered as fixed. This attribute grouping algorithm has been developed in  [34]  and is detailed in Algorithm 1. Example 1. Given the same dataset with 4 attributes, we apply the algorithm from  [34]  on the dataset. It aims to build groups as small as possible such as when: • The values of the grouped attributes are randomized inside of their original instance class ; • The values of the non-grouped attributes are randomized completely ; • This randomization is applied to the whole dataset.
The predictions of the model for the whole dataset do not vary more than a threshold percentage of δ. In the first iteration, the algorithm finds that the smallest group of attributes that makes the predictions varies less than the threshold δ is {B, C}. Removing C or B makes the predictions vary more than the threshold and as such, the algorithm stores {B, C} as a first group. σ ← σ L(R) R ← {} A ← {} else Find an attribute j whose removal from R decreases the fidelity least j ← argmax j∈R f id({{R\{j}} {{j}} {A} F (σ)) if |R| = 1 or f id({{R\{j}} {{j}} {A} F (σ)) then If the fidelity drops below ∆ add the group of attributes to the results and look for the next group of attributes σ ← σ {R} R ← A A ← {} else If the fidelity stays above ∆ continue removing the grouping R R ← R\{j} A ← A {j} end if end if end while return σ Now the algorithm tries to build another such group with the remaining non grouped attributes and find that {A, D} makes the predictions vary too much.
PCA based coalition The main principle of a Principal Component Analysis (PCA) is to reduce a dataset to its simplest expression in terms of attributes. In other words, if the dataset is considered a multidimensional matrix, the PCA aims to reduce its dimensionality as much as possible. To do that, the different attributes of the dataset are combined linearly, the result being a new set of attributes, each new attribute being a linear combination of the previous ones. Our reasoning, for this approach, is to consider the set of combined attributes (summarized by the new attribute of the PCA) as a group of influence. Given a dataset D = (A, X) composed of a set of n attributes A = {a 1 ,. , a n }, and a set of instances X where x ∈ X, x = {x 1 ,. , x n }∀i ∈ [1. n], x i ∈ a i. We can apply a principal component analysis which produces a new dataset D = (A , X ) such as A = {a 1 ,.