dataesr/scientific-paragraphs-categorization huggingface dataset:
Classification on SAT For classification, we have chosen the following option: we consider only the 367 valid analogies, and we apply our dataset extension getting 367 × 8 positive examples and 367×16 negative examples (obtained by permutation). Then we use Method 2 and 10-fold cross-validation to train and test as usual. In that context, we have a dataset of total size 367×24 = 8, 808 (positive and negative) examples. As we have a ratio of 1 valid analogy for 2 non-valid ones, a baseline algorithm classifying all quadruples a : b :: c : d as non-valid would have 66% accuracy. The results are in Table  10. We observe that the best performer is CNN with an accuracy larger than 77% for 10 epochs whatever the GloVe dimension. This could indicate that CNN is able to capture more complex relationships between words than a simple NN.
The datasets were generally limited in terms of size with regard to the current standard. For instance  [55]  uses SAT corpus: the highest accuracy on this set is 56. 1% knowing that a random guess yields an accuracy of 20% and senior high school students get 57%. These results are quite interesting as the dataset mainly consists of semantic analogies like mason:stone::carpenter:wood. More recently, researchers were more interested in working on the analogy completion problem, as with word embedding techniques, numerical tools became widely available. Instead of dealing with discrete space of words, they deal with continuous space language models. From another perspective, the BART (standing for Bayesian Analogy with Relational Transformations) model described in  [34] , is also based on neural-network but more dedicated to relation induction i.
e. finding pairs (c, d) when a pair (a, b) is given. From another viewpoint, some researchers have focused on the task of transforming a query image according to an example pair of related images  [47]. They have taken their inspiration from language modeling techniques. They focus on generating an appropriate image to make a valid analogy between images. Their approach is a two-step learning process not far from what we did: • firstly, learn a deep encoder function f from R d to R k that maps images to an embedding space suitable for reasoning about analogies. In some sense, it is the counterpart of word2vec or GloVe for images. In the target space, they solve the analogical equation using f (b) -f (a) + f (c) • secondly, learn a deep decoder function g from R k to R d to ultimately solve the equation a : b :: c : x with g(f (b) -f (a) + f (c)).
In fact, we have no such understanding. Our methodology is quite different: it takes advantage of the theoretical analysis of the structural properties of analogical proportions for augmenting the training set and of recent word embedding systems in order to learn the unknown relationship amongst the words in an analogy. Embedding the words using GloVe into a multi-dimensional vector space R n , the matrix constituted of 4 vectors can be viewed as an image of dimension n × 4. From a machine learning perspective, the permutation properties of analogical proportion lead to an increase of the dataset size by 8 for positive examples and by 16 for negative examples. Using analogies from Google, BATS and Dif-fVec datasets, we have implemented several machine learning classifiers (SVM, Random Forest, NN and CNN) with an average (on all categories) accuracy higher than 96% for Google, higher than 90% for BATS and higher than 79% for DiffVec whatever the tested machine learning methods (note that if we ignore SVM and CNN on DiffVec, the average accuracy is higher that 95%).
However, the accuracies on the SAT dataset were lower, but it is still up to 77% for the most successful method, namely CNN. SAT is indeed much more "semantic" than Google, BATS and DiffVec, which probably makes it more challenging. Regarding the analogy completion task, we also depart from the classical views derived from the arithmetic analogy using a formula. Instead, we try to learn such a definition via machine learning techniques. Experiment results show that the machine learning methods outperform the formula-inspired methods for classification task (all four datasets) and analogy completion task (three out of four datasets). Machine learning methods might be useful for solving more semantic analogies. Due to the success of the machine learning approaches with GloVe embedding, one could ask what we could achieve with a complete end-to-end learning approach, where the initial embedding is built via a neural network.
are two well-known datasets dedicated to RI experiments. Let us consider 3 examples obtained from DiffVec [59] by combining two pairs from the same class: • (accident, damage) and (bath, cleanliness) are two instances of a relation expressing a link CAU SE -P U RP OSE Cause : Ef f ect. Joining these two instances lead to a debatable analogical proportion: accident : damage :: bath : cleanliness.
If a : b :: c : d is considered as a valid analogy, then embed(a) -embed(b) embed(c) -embed(d) or equivalently ||embed(a) -embed(b) -embed(c) + embed(d)|| 0 So, as proposed by Miclet et al. [35], the so called analogical dissimilarity AD(a, b, c, d) = ||embed(a) -embed(b) -embed(c) + embed(d)|| could be considered as a good measure of how "far" a : b :: c : d is from being an analogical proportion, given the embedding. In other words, when AD(a, b, c, d) is close to 0, a : b :: c : d should be read "a is to b almost as c is to d". We have then computed this analogical dissimilarity for each quadruple of the 4 datasets we have used, then getting the average value per dataset. The results are shown in.
Either we start from the full dataset T S at hand, extend this set then split into train and test subsets, i. e. we have to implement split(Ext(T S)) = (train(Ext(T S)), test(Ext(T S))) Method Or we first split the dataset T S at hand into two subsets and extend them leading to the pair Ext(split(T S)) = (Ext(train(T S)), Ext(test(T S))) Method Or we split the dataset T S but extend only the training set. This means that for classification, the test data has only positive class:.
[15]15]37]-fromGoogle (questions-words file), containing exactly 19,544 analogies, each line has four distinct words. These analogies are classified in 14 categories such as 'common-capitals, countries' like Athens : Greece : Ottawa : Canada, 'country-currency', or 'opposite' like acceptable : unacceptable :: aware : unaware, etc. It appears that some analogy can appear in different categories and some of the categories may contain several occurrences of the same quadruple of words (but in a different order). When we removed these redundancies, we got a final dataset of 12,771 distinct analogies. Table2describes the classes with examples, their cardinality with and without redundancy. 2. The one proposed by Gladkova et al. [18,15], BATS. In fact, BATS is a set However, for the interested reader, this complete table can be found in[15].
3. The one proposed by Vylomova et al. in [59], DiffVec, initially built of pairs (a, b) with their corresponding class that we have transformed into a set of analogical proportions just by considering all options (a : b :: c : d) as soon as (a, b) and(c, d) belong to the same class (among 36). 4. A dataset of 374 analogies coming from the SAT college entrance test12. The analogies in this dataset are entirely semantic and are not syntactic (i. e. not morphological). That is why dealing with this dataset is highly challenging. As some analogies make use of words without GloVe embedding, we ultimately end up with 367 analogies having their GloVe counterpart.
Accuracy of regression (analogy completion task) using NN, 3CosM ul and LRCos for the BATS dataset. This dataset has four categories: E-Encyclopedia, L-Lexicography, I-Inflectional Morphology, D-Derivational Morphology. Each category has ten subcategories. The table above shows the average result of each category. The best results are in bold (unless the accuracy is 100%).
12: Classification results for CNN on Google categories per GloVe dimension. Appendix B. Detailed results on BATS dataset for classificationTaking into account our experiments with Google dataset, we have experimented per BATS subcategory (40 subcategories in total) with the following parameters:• CNN classifier only (as CNN is the most successful on Google)• Glove dimension 50 (as it is enough for the CNN to get good results on Google) • Number of epochs 5 (as it is still enough for the CNN to get good results on Google). As we know, the most challenging category is L, with subcategory L10 very tough to capture. But the CNN provides acceptable results. On subcategory L10, CNN still leads to an accuracy close to 70%.
1. Method 1 : 2. 2 : 3. 3 : 12233 Either we start from the full dataset T S at hand, extend this set then split into train and test subsets, i. e. we have to implement split(Ext(T S)) = (train(Ext(T S)), test(Ext(T S))) Method Or we first split the dataset T S at hand into two subsets and extend them leading to the pair Ext(split(T S)) = (Ext(train(T S)), Ext(test(T S))) Method Or we split the dataset T S but extend only the training set. This means that for classification, the test data has only positive class:.
3. The one proposed by Vylomova et al. in [59], DiffVec, initially built of pairs (a, b) with their corresponding class that we have transformed into a set of analogical proportions just by considering all options (a : b :: c : d) as soon as (a, b) and(c, d) belong to the same class (among 36). 4. A dataset of 374 analogies coming from the SAT college entrance test12. The analogies in this dataset are entirely semantic and are not syntactic (i. e. not morphological). That is why dealing with this dataset is highly challenging. As some analogies make use of words without GloVe embedding, we ultimately end up with 367 analogies having their GloVe counterpart. a b n no redundancy Common capitals athens greece 506 253 All capitals abuja nigeria 4, 524 4, 353 U S cities chicago illinois 2, 467 2121 Currencies algeria dinar 866 433 N ationalities albania albanian 1, 599 820 Gender boy girl 506 253 P lurals banana bananas 1, 332 666 Base to gerund code coding 1, 056 528 Gerund to past dancing danced 1, 560 780 Base to 3 rd person decrease decreases 870 435 Adj.
Since its version 3. 3, the Modelica language offers the possibility to model multimode systems having different DAE-based dynamics in each mode, thanks to the introduction of state machines. When the differentiation index and structure varies with mode changes, compilers generate erroneous simulation code, often resulting in runtime exceptions. We propose in this paper a multimode structural analysis for both multiple modes and mode change events and we show how correct code for restarts can be generated. Our approach is illustrated on two simple but representative mechanical systems.
Introduction Since version 3. 3, the Modelica language offers the possibility of specifying multimode dynamics, by describing state machines with different DAE dynamics in each different state  (Elmqvist et al. , 2012). This feature enables describing large complex cyber-physical systems with different behaviors in different modes. While being undoubtedly valuable, multimode modeling has been the source of serious difficulties for non-expert users of the current generation of Modelica tools. Indeed, while many large-scale Modelica models are properly handled, some physically meaningful models do not result in correct simulations with most Modelica tools. As such problematic models are actually easy to construct, the likelihood of such bad cases occurring in large models is significant. It is unfortunately unclear which multimode Modelica models will be properly handled, and which ones will fail.
Some years ago, we started a project aiming at addressing all the above issues. In this paper we explain our approach, by illustrating it on two simple yet physically meaningful examples that current Modelica tools fail to properly simulate. The use of nonstandard analysis allows us to perform the analysis of both modes and mode changes in a unified framework, including the handling of transient modes and that of impulsive mode changes. Standardization techniques are then used in order to generate effective code for restarts at mode changes. As an efficient implementation of such methods in Modelica compilers would greatly expand the class of multimode models amenable to reliable numerical simulation, we hint at possible mechanizations towards the end of the paper; this aspect is developed in both the companion paper  (Benveniste et al.
When γ = T, equations (e 3 , e 4 ) are active and equations (e 5 , e 6 ) are disabled, and vice-versa when γ = F. If the clutch is initially released, then, at the instant of contact, the relative speed of the two rotating shafts jumps to zero; as a consequence, an impulse is expected on the torques. The model yields an ODE system when the clutch is released, and a DAE system of index 1 when the clutch is engaged (see Section 5. 1). The clutch in Modelica: Figure  2  details the Modelica model of the Ideal Clutch system. It is a faithful translation in the Modelica language of the two-mode DAE (1), except that the two differential equations have been linearized. Also, the trajectory of the input guard γ (here called g) has been fully specified: it takes the value T between t 1 and t 2 and F otherwise.
This model is deemed structurally nonsingular by the two Modelica tools we had the opportunity to test: Open-Modelica 1. 17. 0  (Fritzson et al. , 2020)  and Dymola 2021 (Dassault Systèmes AB). However, none of these tools generates correct simulation code from this model. Indeed, simulations fail precisely at the instant when the clutch switches from the uncoupled mode (g=false) to the coupled one (g=true). This is evidenced by a division by zero exception, as shown in Figure  3. The cause of this exception is that none of these tools performs a multimode structural analysis. Instead, the structure of the model is assumed invariant, and a Dummy Derivatives method  (Mattsson and Söderlind, 1993 ) is implemented, which is correct on single-mode DAE systems, whereas it may fail on multimode systems unless the model structure is independent of the mode.
(Trenn, 2009a,b)  are important works as they point out the difficulty in defining piecewise smooth distributions. Liberzon and Trenn were able to define complete solutions for a class of switched DAE systems in which each mode is in quasi-linear form  (Liberzon and Trenn, 2012) : notably, switching conditions are time-based only. In  (Benveniste et al. , 2019) , an interesting subclass of multimode DAE systems was identified, which possibly exhibit impulsive variables at mode changes. They extend the "quasi-linear systems" proposed by  Trenn et al. ;  in particular, switching conditions are no longer restricted to be time-based, but can be state-based. Nevertheless, the analysis and discretization schemes proposed in  (Benveniste et al. , 2019)  are mathematically sound. Building on this work, Elmqvist and Otter have developed the ModiaSim 1  Julia packages for semi-linear multimode DAE systems.
Conclusion Through the case study of two examples of multimode DAEs that are currently not handled by the existing Modelica tools (with the notable exception of ModiaMath), we presented a mathematically sound and physics-agnostic compilation process for DAE-based physical systems modeling languages. This method relies, in particular, on an extension of structural analysis to multimode systems, that allows the handling of both modes and mode changes in a unified framework. Both examples studied in this paper are multimode models with mode-dependent differentiation index and impulsive behaviour at mode changes, which is not well supported by existing Modelica compilers. This paper showed how our approach handles such models: not only is the structural analysis correctly performed in all continuous modes, but the computation of restart values at mode changes is also handled at compile time, unless an under/over-determination at a mode change event causes the model to be rejected with proper diagnostics.
Acknowledgements The authors are indebted to  Hilding Elmqvist  and  Martin Otter ,  John Pryce , and  Vincent Acary. Khalil Ghorbal participated to the first version of this approach. This work was supported by the  FUI ModeliScale   DOS0066450/00  French national grant ( 2018-2021 ) and the  Inria IPL ModeliScale large scale initiative  ( 2017-2021 , https://team. inria. fr/modeliscale/). Dymola licences were provided to the authors by  Dassault Systèmes  in the context of the  FUI ModeliScale  project.
Palabos-npFEM is a computational framework for the simulation of blood flow with fully resolved constituents. The software resolves the trajectories and deformed state of blood cells, such as red blood cells and platelets, and the complex interaction between them. The tool combines the lattice Boltzmann solver Palabos for the simulation of blood plasma (fluid phase), a finite element method (FEM) solver for the resolution of blood cells (solid phase), and an immersed boundary method (IBM) for the coupling of the two phases. Palabos-npFEM provides, on top of a CPU-only version, the option to simulate the deformable bodies on GPUs, thus the code is tailored for the fastest supercomputers. The software is integrated in the Palabos core library, and is available on the Git repository https://gitlab.
Palabos-npFEM solves equations (  1 ) & (2) in a modular way, and performs the FSI through the Immersed Boundary Method (IBM). By modularity, we mean the complete decoupling of the solvers, i. e. the resolution of fluid phase is "unaware" of the resolution of the solid phase and vice-versa. Our software framework takes care of communication whenever needed in relationship with the FSI. Figure  1   2 ) is solved by the nodal projective finite elements method (npFEM)  [7]. The npFEM is a mass-lumped linear FE solver that resolves both the trajectories and deformations of blood cells with high accuracy. The solver has the capability of capturing the rich and non-linear viscoelastic behaviour of any type of blood cells as shown and validated in  Kotsalos et al. [7]. The IBM  [14, 13]  for the coupling of the solid & fluid phases, i.
e. the computation of term b in equations (  1 ) & (2), is implemented in the Palabos library. The modular system allows different spatial discretisations, i. e. the fluid domain is discretised into a regular grid with spacing ∆x in all directions, and the solid bodies are discretised into triangular surface meshes. The data exchange between solvers of different discretisation is handled through interpolation kernels, as dictated by the immersed boundary method. Moreover, the temporal discretisation is solver dependent. Given the explicit nature of LBM and the implicit/semi-implicit nature of npFEM, the latter can in principle handle larger time steps.
Implementation and architecture Palabos-npFEM is build on top of two independent solvers, i. e. fluid and solid solvers, and couples them through the FSI module (see Figure  1 ). The choice of the particular solvers, namely Palabos and npFEM, is crucial for high fidelity and performant simulations. However, other users of the code could extend it by replacing the particular choices by alternatives, e. g. opting for a mass-spring-system solid solver instead of a FEM one. The alternative solvers need to be similarly parallelisable through domain decomposition and allow interaction with solid particles through an immersed boundary method. Palabos  [10]  is an open-source library for general-purpose computational fluid dynamics based on the lattice Boltzmann method. Palabos is written in C++ with extensive use of the Message Passing Interface (MPI).
Our approach follows the redesigned projective dynamics approach as described in  Liu et al. [11]. • In Computer Graphics, the solvers are approximating Newton's equations (conservation of linear and angular momenta), reducing the computational cost. Our solver is not approximating Newton's equations, but we provide a converged solution, focusing on accuracy and physically correct states. For legacy reasons, we have decided to keep the ShapeOp code structure and file naming. This approach allows the users to better understand our extensions, perform their own, and most importantly to benefit from both communities, namely the computational science and computer graphics ones. Figure  2  presents the realisation of this modular design of Palabos-npFEM. Indeed, the solvers possess independent execution streams (their implementation details are black-boxes to the end user).
Quality control Palabos-npFEM uses the default quality control tools integrated in Palabos. Palabos is hosted in GitLab, which provides continuous integration tools. From these tools, we have a test checking that the latest Palabos version compiles successfully. Currently, no unit testing framework is implemented. Nevertheless, we provide a number of example applications that any user can go through, and check if the various solvers are operational. Palabos-npFEM is extensively documented, and there are instructions for installing the software in any supported system, but also instructions for testing various example applications. See Example Applications sections below for more details on testing Palabos-npFEM library, and verifying its operational status. Furthermore, an extensive validation and verification of our framework has been performed in  Kotsalos et al.
In more details, npFEM is an implicit solver, which makes it capable of resolving very high deformations/interpenetrations with unconditional stability for arbitrary time steps. Taking advantage of this property, we run the framework for few thousand time steps, which indeed resolves the overlappings. Figure  5  presents the cell packing application as performed in a box at 35% hematocrit. The cell packing application deactivates the branches of the code that deal with the fluid and the FSI (since there is no need for them), and instead uses the framework for efficiently detecting colliding neighbours and for executing the npFEM solver. The cellPacking params. xml (found in bloodFlowDefoBodies folder) provides options to perform the cell packing under various geometries and at different hematocrit.
Rhino-Grasshopper environment for setting up new materials The framework/environment presented in this section is intended for Windows only. Setting up new materials, in a platform-agnostic way, is presented in the next section. The npFEM library (derived from ShapeOp) can be used as a stand-alone library independently of Palabos. The library can be found by cloning Palabos from the GitLab repository *. The source code is located in the folder coupledSimulators → npFEM. In- side this folder, one can navigate further to npFEM StandAlone RhinoGH folder, where we provide the option to compile npFEM as a stand-alone dynamic library. The produced dll can later be used in Rhino † -Grasshopper † † environment. Rhino is a computer aided design software, and Grasshopper is a plug-in for Rhino intended for parametric design.
Grasshopper uses the Python language for scripting various operations, and by using the Python standard foreign function library ctypes, one can call npFEM from within Rhino-Grasshopper. The locally provided README. md file helps the user setup the framework in a step-by-step manner. The ShapeOp library provides a complete description in its documentation ‡ on how to setup the Rhino-Grasshopper framework, but instead of ShapeOp, we will be using the compiled npFEM dynamic library. For legacy reasons with ShapeOp, we are using Rhino version 5, but the same principles apply to newer versions. After setting up the environment, one can open the Rhino & Grasshopper files provided in npFEM StandAlone RhinoGH → npFEM RhinoGH. Figure  8  presents the environment with a setup that shows a stretched RBC in the left pane (Rhino window), and in the right pane there is the Grasshopper window.