dataesr/scientific-paragraphs-categorization huggingface dataset:
• It should be able to learn additional knowledge from new data; These four points apply to any general incremental learning problem. In our application, we want to use the principle of incremental learning in a classical transfer learning context. In classical transfer learning, a pre-trained model is re-trained once on a new dataset with a predefined number of classes. Our approach based on incremental learning trains a model progressively while adding new classes at each transfer of knowledge. The common point between this learning and classical incremental learning is that the model learns new data without destroying the knowledge acquired from the old data. On the other hand, the difference between them is that our approach requires re-training the system on both old and new data. We can distinguish mainly three types of incremental learning algorithms: • Architectural strategy  [56] : this algorithm modifies the architecture of the model in order to mitigate forgetting, e.
• Repetition strategy  [59] : old data are periodically replayed in the model to strengthen the connections associated with the learned knowledge. A simple approach is to store some of the previous training data and interleave it with new data for future training. In our approach, we use the regularization strategy by modifying the loss function of the system. For validation purposes, we carried out experiments on the LifeClef 2015 Fish (LCF-15) dataset. This underwater benchmark dataset is captured by underwater cameras in the open sea.
Total Loss Function The total loss function (L total ) of the network is the sum of the knowledge distillation (L distillation ) and the loss function used by the network to learn the classes (L loss ). L total = λ o L distillation (P o , Po ) + L loss (Y, Ŷ) (3) where λ o is a loss balance weight between old and new classes. By increasing its value, we favor training the old images over the new images. Y and Ŷ are the vectors of truth and calculated labels, respectively, corresponding to the training image set X = X o ∪ X n. This approach has advantages over classic transfer learning. Indeed, the strategy of feature extraction without fine-tuning generally performs worse on a new dataset because the shared parameters θ s are related to the old classes, and they have not learned to extract discriminative features related to new classes.
Experiments We used the LCF-15 benchmark underwater dataset to evaluate the effectiveness of the proposed approach for fish species classification. The dataset contains images of fish of different colors, textures, positions, scales, and orientations. It was issued from the European project Fish4Knowledge (F4k) (www. fish4knowledge. eu accessed on 7 July 2022)  [61]. During this project of five years, a large dataset of over 700,000 unconstrained underwater videos with more than 3000 fish species was collected in Taiwan, the largest fish biodiversity environment in the world.
LifeClef 2015 Fish (LCF-15) Benchmark Dataset The LCF-15 is an underwater live fish dataset. The training set consists of 20 annotated videos and more than 22,000 annotated sample images. In this dataset, we have 15 different fish species. Figure  3  shows examples of the 15 fish species, and Table  1  gives the distribution of the fish species in the dataset. Each video is manually labeled and agreed on by two specialist experts. The dataset is imbalanced in the number of instances of different species; for example, the number of the species 'Dascyllus reticulates' is about 40 times more than the species 'Chaetodon speculum'. The fish images have various sizes ranging from about 20 × 20 to about 200 × 200 pixels. The test set has 73 annotated videos. We note that for three fish species, there are no occurrences in the test set (Table  1 ).
This is to evaluate the method's capability of rejecting false positives. Compared with other underwater live fish datasets, the LCF-15 dataset provides challenging underwater images and videos marked by more noisy and blurry environments, complex and dynamic backgrounds and poor lighting conditions  [31]. We choose this dataset because it contains two categories of species: hard and easy species. Finally, we note that fish images can be extracted from videos using the available ground truth of available fish bounding boxes.
• Construction of two groups: in order to separate the species into two subsets, difficult and easy, we train the pre-trained network ResNet50 on all species of the LCF-15 dataset with transfer learning. Figure  4  illustrates the confusion matrix. From this confusion matrix, we can group the species into two main groups: group of species with low precision, difficult species, (AN, AV, CC, CT, MK, NN, PD, ZS) and group of species with high precision, easy species, (AC, CL, CS, DA, DR, HM, PV).
i. Optimization technique The choice of the optimizer is a crucial step that greatly influences performance. We first evaluate our approach using different optimizers for loss function (Equation (  3 )). For this, we set λ o to 0, and we test different optimization techniques (SGD, Adam, Adamax or RMSprop). Table  2  compares the accuracies with different optimization techniques on the LCF-15 dataset. We can observe that Adam shows the best results for our problem statement. Adam is recommended as the default optimizer for most of the deep learning applications. It inherits the good features of RMSProp and other algorithms. Unlike maintaining a single learning rate through training in SGD, the Adam optimizer updates the learning rate for each network weight individually. We use this optimizer for the rest of the work, and we add the distillation loss term.
Comparative Study Table  3  shows the comparison performances of our proposed approach with the state-of-the-art methods on the LCF-15 benchmark dataset. We note that we implemented modified AlexNet  [24]  and FishResNet  [28]  approaches by using the same provided parameters as in their papers. We also implemented the approach of Yolov3  [19]  based on the spatial information using RGB images. From Table  3 , we observe that our proposed approach based on incremental learning for live fish species classification outperforms the-state-of-the-art methods. We note that Salman et al. [31]  reached an accuracy of 93. 65% by testing their model on 7500 fish images issued from the LCF-15 dataset, but these fish images are not from the original test set provided in the dataset. When they tested their method, they did not use the original training and test split provided in the LCF-15 dataset.
Conclusions In this paper, we proposed a new CNN approach based on incremental learning for a live fish classification task in an unconstrained underwater environment. We proposed to first train ResNet50 on a hard fish species and then add an easy fish species by using incremental learning to reduce the forgotten problem. For this, we modified the loss function by adding knowledge distillation loss. Experiments on the LifeClef 2015 Fish benchmark dataset demonstrated that incremental precisions are higher than non-incremental preci-sions, and our proposed approach outperforms various state-of-the-art methods for fish species identification. Our future work will aim to keep the old and new precision of each species higher in order to improve the performance of the system.
Adding parameters to a LTS is quite standard but enabling holes inside LTSs is not a well-defined notion. We thus define open automata that can be seen as LTSs with parameters and holes. The transitions of open automata are much more complex than transitions of an LTS as the firing on a transition depends on parameters and actions that are symbolic. This article defines the notion of open transition a transition that is symbolic in terms of parameters and coordinated actions. Contrarily to pNets, open automata are not hierarchical structures, thus they are more convenient for formal reasoning but not adapted to the definition of a complex and structured system like pNets. Additionally, open transitions are expressed in terms of logics more than in terms of synchronised actions, synchronisation vectors on the contrary make it easy to express synchronisations that exist in process algebra or in specification and high-level languages for distributed systems.
This can be seen as "conditional transitions" of the pNet, or alternatively, as a syntax to encode structural operational semantics (SOS rules) of the system: each vector expresses not only the actions emitted by the pNet but also what transitions of the composed pLTSs must occur to trigger this global transition. Synchronisation vectors can also express the exportation of an action of a sub-pNet to the next level, or to hide an interaction and make it non-observable. Finally, a pNet can leave sub-pNets undefined and instead declare holes with a well-defined signature. Holes can then be filled with a sub-pNet. This is defined as follows.
Definition 2. 4 (pNet composition ). An open pNet: P = P i∈I i , Sort j∈J j , SV can be (partially) filled by providing a pNet Q to fill one of its holes. Suppose j 0 ∈ J and Sort(Q) ⊆ Sort j 0 , then: P [Q] j 0 = P i∈I i {j 0 →Q}, Sort j∈J\{j 0 } j , SV pNets are composition entities equipped with a rich synchronisation mechanism: synchronisation vectors allow the expression of synchronisation between any number of entities and at the same time the passing of data between processes. Their strongest feature is that the data emitted by processes can be used inside the synchronisation vector to do addressing: it is easy to synchronise a process indexed by n with the action a(v, n) of another process. This is very convenient to model systems and encode futures or message routing. pNets have been used to model GCM distributed component systems, illustrating the expressiveness of the model [ABHK + 17].
These works clearly highlight the interest of incremental and compositional verification in a very general setting. In our own work on open pNets, adding more structure to the composition model, we show how to enforce a compositional proof system that is more powerful than independent sets of variables. Our theory has also been encoded into an SMT solver and it would be interesting to investigate how the examples of evolving systems studied by the authors could be encoded into pNet and verified by our framework. However, the models of Johnson et al. are quite different from ours, in particular they are much less structured, and translating them is clearly outside the scope of this article. In previous work  [GHM13] , we also have shown how (closed) pNet models could be used to encode and verify finite instances of reconfigurable component systems.
Experimental data The experimental data used in this paper are Sougou news data captured by crawlers for a period of 6 months. Based on these data, the training, verification, and test sets required by the training model are constructed. In this paper, the BIOES labeling rules are used to label data sets. There are five kinds of entities in the data set: person (PER), location (LOC), organization (ORG), time (TIM), and number (NUM). From these, 12 kinds of labels are generated: "O", "B-ORG", "I-ORG", "B-PER", "I-PER", "B-LOC", "I-LOC", "B-TIM", "S-NUM", "E-LOC", "E-ORG" , and "E-PER".
Result discussion The experimental environment is shown in Table  1. This paper adopts the Centos operating system and uses Python and TensorFlow for research and development. As shown in Table  2 , we adopt 3 605 398 tokens with 341 968 phrases to evaluate the metrics for each type of entity using CNIL-BiLSTM. Specifically, there are five types of entities in opendomain Chinese NER tasks, including LOC, NUM, ORG, PER, and time TIM. To estimate the performance of the CNIL-BiLSTM, four methods are used for comparison, namely, CRF, BiLSTM-CRF, GRU-CRF, and CASCADED HMM. The experimental results are shown in Table  3. As can be seen, the results indicate that CNIL-BiLSTM is superior to other models in terms of different experimental indexes. In particular, compared to the BiLSTM-CRF model, the proposed integrated learning algorithm in this paper improved precision, recall, and F1 value by 4.
Microscopy and image stabilization Bacteria confined within the microfluidic traps were imaged on a custom inverted microscope 14 using a long working distance 40x objective lens (NA=0. 6, air, Nikon) and an sCMOS camera (2560×2160 pixels; Zyla, Andor Technology). The sample was illuminated with white light by an LED illumination system pE-4000 (CoolLED Ltd, Andover, UK). Images were captured with Andor iQ3 software at a pixel size of 111 nm/pixel, and saved as a 16-bit tiff stack. Each stack consisted of 2,000 slices per experiment, with 20 ms time step between slices. The microscope was equipped with 3D-axis Helmholtz coils with a controller (C-SpinCoil-XYZ, Micro Magnetics Inc. ). These coils were used to apply DC magnetic fields with a precision of ±2. 5 µT (5% of the Earth's magnetic field).
A photo of the microscope platform and the 3D-axis Helmholtz coils is shown in Supplementary Fig. 16. A 3-axis magnetic sensor (Micro Magnetics Inc. ) was used to measure the precise value of the Earth's magnetic field in sample location. The magnetic field was controlled using a LabVIEW (National Instruments) based program and the ambient magnetic field was canceled out before each experiment. The magnetic field was then set to 0 µT, 50 µT or 500 µT parallel to the microfluidic trap diameter (inset of Supplementary Fig. 16 ). The microscope was equipped with three motorized linear stages (Z825B, Thorlabs) to provide sample positioning in 3D. Due to the micron-size of the microfluidic traps and the fact that the bacteria tend to swim near the walls, even a small amount of vibration hinders correct background subtraction and consequently the bacteria tracking process.
Therefore, a custom-made script was developed in MATLAB (MathWorks, Inc. ) for image stabilization and background subtraction. The image stabilization was based on comparing two successive frames and detecting the corners and features that were sharp and changed drastically (the green crosses in Supplementary Fig. 17c, d ). Then matched points were detected and connected, and transformation (including scaling, rotation, translation, and shearing) was applied to the image to overlay the points (Supplementary Fig. 17e, f ). An effective background subtraction, and hence tracking of the bacteria, could be achieved after image stabilization (Supplementary Fig. 17g, h ).
Bacterial tracking Bacteria tracking in 2D was performed using custom-made scripts in MATLAB, after image stabilization and background subtraction (see Supplementary Fig. 17 ). The bacteria were identified based on an intensity threshold and their locations were determined by finding a centroid of contiguous pixels in a region with intensity above this level. The trajectories were formed by connecting centroid positions in subsequent frames. Resulting trajectories were smoothed with a five-point moving average (Supplementary Fig. 4 , and Supplementary Video 1). The points resulting from the incorrect tracking of the wall were discarded by applying a cutoff circle with radius of 1. 1 times the radius of the trap according to the thickness of trap wall shadow in the recorded images (see Supplementary Fig.
Bacterial magnetosome counting The number and diameter of the magnetosomes were computed from the TEM 2D images using a semi-automatic custom-made application in MATLAB. For each image, the user had to manually draw a region of interest (ROI) to delimit the boundary of each bacteria. These ROIs were used to count the magnetosomes separately for each bacterium and to avoid artefacts that could be found outside the cell. A build-in MATLAB function based on Circular Hough Transform 63 was used to automatically find the number of magnetosomes and their sizes. Manual filtering of artefacts was possible when the automatic segmentation failed.
Magnetic moment measurement The magnetic moments of the bacteria were determined by examining the number and diameter of magnetosomes inside the bacteria by transmission electron microscopy (TEM). TEM grids were prepared by adding 20 μL of bacterial suspension and letting them settle down during 30 min before washing them twice with milliQ water and drying them with paper. TEM images were taken with a transmission electron microscope (EM912, Zeiss) at an accelerating voltage of 120 kV with a magnification of 4,000-10,000x. The variability of the magnetic moment can be estimated from the distribution of the numbers and sizes of magnetosomes in the TEM images (see Supplementary Figures 11, 12 and 13). According to the data of TEM images (Supplementary Figure  11 ), the mean value of the number of magnetosomes inside bacteria is N = 23.
Acknowledgements The authors thank  Heike Runge  for taking the TEM images. This work was supported by the  Deutsche Forschungsgemeinschaft  within the  Priority Program 1726 on microswimmers  (grant No. FA 835/7-1  and  KL 818/2-1  to D. F. and S. K. ). T. R. acknowledges support from the  MaxSynBio  consortium which is jointly funded by the  Federal Ministry of Education and Research of Germany  and the  Max Planck.
Indeed STMs are derived from observations and are designed to cope with the non-deterministic and irreversible nature of observed dynamics  [5]. STMs are also intended to be userfriendly, enabling participatory model development and collaborative management  [6]. The main goal of STMs is to assist managers and scientists in collectively proposing policies driving the ecosystem through some desired pathways while avoiding others. In order to remain userfriendly, STMs sizes usually do not exceed a few dozens of states. While STMs originally stem from rangeland management [5], they are now used in many fields such as natural park management  [34]  (see for example the "EDIT" database housing a large catalogue of STMs  [35] ), geomorphology  [36] , or agroecology  [37]. In addition, STGs are found in the field of community assembly under the concept of assembly graphs.
Implementation of a scalable model-checker. We instantiated the general modelchecking methodology from Fig 3 (in black) as indicated by the blue italic annotations, and implemented it within toolkit ecco. ecco is a Python  [50]  library intended to be used within Jupyter notebooks  [51]  and aimed at providing tools for the formal modelling and analysis of ecosystems. ecco has been developed and used for years to model and analyse varied ecosystems  [39, 44, [52] [53] [54] [55] [56]. ecco is available as a free software released under the GNU LGPL and is hosted at http://github. com/fpom/ecco. In particular, it features an implementation of the ifthen rules language presented in Section 1. 1. 3, as well as an efficient STG structure implemented on the top of ITS-tools and libDDD  [57, 58].
At its core, ecco allows to compute a symbolic representation of the states of an if-then model: individual states are not explicitly enumerated, but instead a compact data structure (based on Data Decision Diagram, DDD  [59] ) gathers sets of states from which sets of successor (or predecessor) states can be efficiently Table  2. Catalogue mapping query patterns to their translations in CTL. Dynamical properties relevant to ecological systems are gathered into patterns. The patterns are written in English and translated into CTL formulas. x and y are place-holders for state properties. (Adapted from  [49] ).
An example of this implementation of the model-checking methodology in systems biology is given in  [17] , associating a system description based on Boolean network and dynamical properties written as CTL formulas to analyse and validate a model of the mammalian cell. When the duration of the transitions between states are available and of interest  [72] , they may be incorporated into the STG by labelling the transitions with their durations. Timed automata  [19, 73, 74]  is a modelling formalism computing such STGs, that can be implemented with the software Uppaal  [75]  incorporating a model-checker. A complete example of this implementation of the model-checking methodology in ecology is given in  [19] , associating a system description based on timed automata and dynamical properties written as Timed CTL formulas (an extension of CTL handling quantitative time) to analyse scenarios of a coral reef ecosystem.
This stage gives also the opportunity for the designer to refer to each constituant if any changes are required. At the end of the designers work, the plans of each part and all the dimensions that characterize them are modeled. The exploded view of the speed reducer is represented in Figure  7  and the bill of materials in Table  2. The software FreeCAD were used for the design of the treated example. The XML-MBSE data generation file for the speed reducer example is represented in Figure  6.
If their values are in line with those of the initially defined requirements, then the product performance is deemed satisfactory. If not, an analysis report should be sent to the designer to make the appropriate changes. The next step is to analyse the process performance indicators for each assembly sequence separately, and to test their response to the specification requirements. Based on the results of the analysis, the SE can select the best assembly sequence that meets all the requirements and offer a higher quality. Given the predefined criteria, it is very rare to have two assembly sequences with the same score. However, if the system engineer ends up with several acceptable ASPs then the designers team will choose the most optimal solution. For the speed reducer example, the analysis of the design solution is carried out using Modelcenter MBSE connection tool, which connects the requirements stored in Magicdraw software in the requirement diagram to the analysis indicators values for the proposed solution.
Data DP emission line galaxies are a type of galaxies where the emission lines of the central ionised gas have a double peak shape. This shape can be better modelled using a double Gaussian than a single Gaussian function. As discussed in Sect. 1, this emission line shape has been used in multiple studies to find galaxy mergers or dual AGNs. But as discussed in detail in M20, since we are only probing the central part of a galaxy, we cannot exclude a rotating gaseous disc as the origin of the DP structure. In order to probe the merging scenario resulting in DP emission lines, we build up a galaxy sample of DP galaxies with CO observations (hereafter DP sample). We, therefore, use new CO observations and also gather existing CO measurements from the literature. Offset from the MS for all CO samples as a function of stellar mass (M * ) using the parametrisation of the MS found by  Speagle et al.
Other literature data In order to enlarge our sample, we further select DP galaxies from published CO observations, which match the criteria described in Sect. 2. 1. To select DP galaxies, we perform a simplified DP selection procedure as described in M20 but with no emission line stacking and multiple selection stages. Our present algorithm consists of a simple emission line fitting algorithm and selection criteria but is finally relying on a visual inspection since noisy spectra tend to be falsely selected to show DP emission lines. This enables us to also select DP galaxies with strong perturbed gas kinematics, which deviate from double Gaussian profiles and would be sorted out by automated robust DP selection algorithms as discussed in M20. We start from the SDSS spectra and subtract the best fitting stellar continuum template provided by  Chilingarian et al.