dataesr/scientific-paragraphs-categorization huggingface dataset:
Comparison The abstracted models (Figure  7 ) are exported from the yEd tool to the NetworkX tool based on the GraphML format. We used the code depicted in Listing 1 to compare abstracted graphs (e. g. , comparing SysML and Modelica graphs). We were able to assess the consistency between models within five iterations. We depict the comparison method as follows: As a first step, we constitute a first version of the label repository, where we list the set of vertices representing the inter-model correspondences. For each iteration, we import the different abstracted graphs (e. g. , we imported SysML and Modelica graphs in Listing 1, lines 7-8). Then, we represent graphically the common subgraph that exists between SysML and Modelica graphs in lines 17-23 of Listing 1. After that, we compose the two graphs in a graph g4 in lines 26-48 of Listing 1.
In this graph, we represent the common subgraph between SysML and Modelica graphs, and the extra nodes and edges are represented in different colors. The extra nodes and edges represent specific entities for each model. Finally, the code in lines 43-48 of Listing 1 print a graph represented as a chord diagram as shown in Figure  8. The final iteration enables the visualization of all correspondences and differences between the two modeling languages. As an example, we represent in Figure  8  successively the inter-model connections between SysML and Modelica abstracted models (Figure  8a ) and the inter-model connections between the SysML and Altarica abstracted models (Figure  8b ) in the final iteration. In this stage, the project manager diagnoses the differences detected in the comparison process.
Concretization The updated graphs are implemented in the Eclipse platform as discussed in Section 4 to update entry models using model transformation techniques. The updated graphs are exported from yEd tool to Elipse platform using the XMI format. Then, the project manager implements a code that executes the transformation rules (i. e. , indirect transformation rules). Thereafter, they obtain three textual XMI files that represent the updated models based on the concepts of each language. Then, each engineer collects the appropriate XMI file that corresponds to their language and imports it into their proper tool. For example, the systems engineer retrieves an XMI file of their SysML model that contains the corrections proposed by the project manager. They import the XMI file in the Magicdraw tool (SysML tool).
Discussion In this section, we will conduct a discussion of the main contributions of our method compared to the methods of literature. The first improvement of our approach compared to the integration approach and the model federative approach is to be able to support specific modeling language for each domain. The existing model transformation approaches are criticized since there is an epistemic gap between pragmatic models (e. g. , SysML) and formal models (e. g. , Modelica and Altarica). This make the transformation not fully automatic from pragmatic models to formal ones. Furthermore, implied explicit knowledge in target models can lead to disappointing results where the source models are blurred and overloaded and the generated formal models are incomplete and uselessly complex. Also, the method proposed by Shah et al.
e. , the project manager). Also, the classification and resolution of differences are manual processes that depend on the project manager activities. In future works, we will explore how our method can be extended to support the consistency management of behavioral models. As a first step, we will use behavioral models (e. g. , the SysML state machine diagrams and the Altarica GTS model that represent states/events formalism for reliability study). Therefore, we can represent the behavior models using graph representations and implement our techniques to analyze the consistency between behavioral models. This is a preliminary solution to ensure semantic consistency based on the model synchronization approach. Afterward, we plan to study the fundamental concepts of semantic consistency management during the development of mechatronic systems and we will investigate how other techniques for multi-domain system development, such as DEVSML (Discrete EVent Systems Modeling Language) stack by Mittal et al.
Acknowledgments: A part of the case study was supported by  IRT SystemX (Altarica) ,  University of Naples Federico II (Modelica) , and SUPMECA (SysML). The authors would like to thank  IRT SystemX , especially to  Antoine B. Rauzy ,  Michel Batteux , and  Tatiana Prosvirnova , for the many valuable discussions and for allowing the dissemination of the case study.
This phenomenon is amplified by the difficulty to obtain many specialized documents in a language other than English  (Morin and Hazem, 2014). This aspect, which is gradually changing with the deployment of open access data, is nevertheless a major difficulty. One way to overcome the limited size of specialized (in-domain) comparable corpora for bilingual lexicon induction (BLI) is to associate them with out-of-domain resources as lexical databases  (Bouamor et al. , 2013)  or large general domain corpora  (Hazem and Morin, 2018). For instance, by combining a specialized comparable corpus with a general domain corpus, methods based on distributional analysis are boosted  (Hazem and Morin, 2018). General domain corpora greatly enhance the representation of specialized vocabulary by adding new contexts.
The main drawbacks of this data augmentation approach is the introduction of polysemous information as well as the tremendous increase of computation time. Consider for instance, a French/English comparable corpus in the medical domain and the French terms os (bone) and sein (breast). When looking for additional contexts in general corpora, os is very likely to recover contexts dedicated to an operating system since OS is the abbreviation used in French. Similarly, when studying the French term sein, many contexts related to the French preposition au sein de (within) will be added. In the same way, the English medical term breast can be associated with food just as in chicken breast to mention only the most obvious ones. Associating out-of-domain data with a specialized corpus is not the mainstream research direction in bilingual lexicon induction from specialized comparable corpora.
The source and target sentences are then evaluated by H LM I,b (W O ) -H LM O,b (W O ) (where b ∈ {s, t} refers to the side of the corpus) and ranked accordingly. The cross entropy is computed using the xenC tool  (Rousseau, 2013). BERT is a supervised learning model that has proven to be efficient in many downstream NLP tasks  (Devlin et al. , 2019). It has been trained on the Masked Language Model (MLM) and Next Sentence Prediction (NSP) objectives  (Devlin et al. , 2019). Hence, it can be used whether for word representation or for sentence classification. In order to perform data selection, we use BERT as a binary classifier to predict if a given input sentence is in-domain or out-of-domain. The intuition behind this strategy is that BERT can learn shared features between in-domain sentences.
Data We conducted our experiments on two French/English specialized data sets: breast cancer and wind energy. The Breast Cancer corpus (BC) is composed of scientific documents published between 2001 and 2015 available in open access on the ScienceDirect portal 1 where the title or the keywords contain the term breast cancer in English and its translation in French. The Wind Energy corpus (WE) has been released within the TTC project 2 and is composed of documents harvested from the Web using a focused crawler based on several keywords such as wind, energy, rotor in English and their translation in French. We considered two separate out-of-domain data sets in English and French: i) JRC acquis corpus (JRC) composed of legislative texts of the European Union 3 (we used the French/English version at OPUS which is based on the paragraph-aligned corpus provided by JRC  (Tiedemann, 2012) ) and ii) a fraction of Wikipedia corpus (WIKI) 4.
Methodology Hereafter, we specify the methodology of our experiments: • For a given specialized corpus, we first use our data selection methods to find the most similar sentences (or documents for the Tf-Idf approach) of our out-of-domain corpora on both source and target languages. For fine-tuning BERT, we used bert-base-cased for English  (Devlin et al. , 2019  and Camembert  (Martin et al. , 2020)  for French with their default parameters. To build the training data set, we used the news commentary 6  corpus as out-of-domain data set. • Then, we create our training data for both languages by concatenating our full specialized corpus with sub-parts of our general corpus. We build several training data sets by incrementally adding samples of 10%, from the most similar to the least similar data (but also from the least similar to the most similar to study the impact of data selection).
• We use fastText  (Bojanowski et al. , 2016)  to train our source and target word embeddings on the previously created training data. (our parameters are as follows: method: skipgram; minCount: 5; dim: 300; ws: 5; minn: 3; maxn: 6). • To project our word embeddings of source and target languages in the same space, we use the VecMap tool  (Artetxe et al. , 2018a). • Finally, for a word to translate, we measure its similarity with every word of the target language. The target candidates are ranked using the Cross-domain Similarity Local Scaling which is an improvement of the cosine similarity that takes into account nearest neighbors  (Conneau et al. , 2017). The results are measured in terms of the Mean Average Precision (MAP)  (Manning et al. , 2008) : M AP (Ref ) = 1 |Ref | |Ref | i=1 1 r i (3) where |Ref | is the number of terms of the reference list and r i the rank of the correct translation i.
We first note that for every corpus and configuration, adding general data improves the results. If we look at the trends of the CrossEntropy+ curves, we see a great improvement at 10% and then a slower improvement for JRC and a slow degradation for WIKI. These results are particularly interesting because they show the usefulness of a good data selection, but they also show the need of a corpus that is large enough to see a real impact on the results. WIKI being larger than JRC, it has more various contexts and 10% of it corresponds to 30M words while it is only 6M words for JRC. Conversely, thepresents the opposite trend and shows slow improvements. The best results are obtained with the full corpora. The curves + for T f -Idf and BERT are not as interesting as the CrossEntropy+ (with the exception of the BC+WIKI for BERT ) as there is not a great improvement followed by a slow degradation.
Also, we do note observe much differences between theand Random curves. Table  2  presents some major points of our experiments. First, we present the results on the general and specialized corpora only. Then we show three results for each combination of specialized and general corpora. The first result (column 100%) is the concatenation of the specialized corpora with the full   general corpora, and illustrates a strategy of data augmentation. The two other columns illustrate the optimal percentage for the CrossEntropy data selection approach. The + (resp. -) columns represent the most (least) similar documents of the general corpora. We clearly see improvements for both specialized corpora with the WIKI corpus when going from a full data augmentation (100%) to data selection (10%). In terms of MAP, this makes us gain 3.
7 points for BC and 5 points for WE. However, for the smallest JRC, the gain is less important. If we still have an interesting MAP improvement of 2. 2% and 3. 2% for BC and WE respectively, it needs way more data to be effective (70%). These observations confirm the usefulness of data selection, but under the condition of using large enough corpora with various content. CrossEntropy+ CrossEntropy- T f -Idf T f -Idf - BERT + BERT - Random (d) (WE + WIKI) Based on the results of CrossEntropy+ at 10% of data for WIKI, we conducted one more experi- ment on smaller percentages to get better insights on smaller data selection effect. This experiment is illustrated in Figure  2  where the evolution of data quantity is represented in terms of number of words, to allow the comparison with the JRC corpus (3M words represent 1% of WIKI and 5% of JRC).
Computation Time In this section, we report the computation time for all the configurations. All our experiments have been carried out with an Intel Core i9-9900K and a GeForce RTX 2080. Table  3  shows the computation time needed to perform data selection and to train the word embeddings for BC and WIKI. The embeddings training time corresponds to the computation time of the source and target embeddings. We see that using in-domain corpus only (No Selection) is really fast (180s) but does not present good results (MAP of 50. 6%) while a data augmentation approach improves the results (83. 9%) but also drastically increase the computation time (65, 521s). Finally, with a good data selection approach (CrossEntropy), the augmentation of computation time is smaller (6, 972s) and even improves the MAP score (87.
If the main issue is to better characterize specialized terms while preserving the domain characteristics, we assume that the representations of common (i. e. from a general domain) words present in the general corpus are also enriched. In order to measure the augmentation impact on BLI, we selected several translation pairs that were affected either positively or negatively by data increase and discuss each case. Table  4  shows for each data selection amount, the rank of a given translation term as well as the occurrence of the translation pair after data augmentation for BC. To each rank we assign a color according to the impact of the selection on the results. Hence, green means that the obtained rank after adding n% of data equalized or improved the translation rank while red means that the rank was degraded.
This table presents the results for the CrossEntropy+ lists where we had optimal results at 10% of data selected. First, we see that the term breast is -almost-correctly translated to sein (Rank = 2) when no data is added. However, with 100% of the general corpus, the rank is hugely degraded (Rank ≥ 1000). A closer look at the added contexts revealed that the majority of documents contained the French expression au sein de (appeared 47, 025 times) which can be translated as within or at the heart of. This pair is one of the few that doesn't improve with the addition of out-of-domain data, but we can see that taking only 10% (Rank = 3) leads to a significant improvement compared to the full general corpus (Rank ≥ 1000). Contrariwise, calcium showed the benefits of adding correct contexts on both sides.
If its correct translation calcium was ranked at 140 when only specialized corpus was used, the rank is immediately improved by the addition of out-of-domain data to rank 1. It is also important to note that the source and the target words remains close in terms of frequency. This pair also shows that the selection succeeded to find most of the interesting occurrences of the words in the first 10%. The pair back -dos shows the same trend as breast -sein, mainly due to the frequent use of back in English. However, adding out-of-domain data didn't decrease the rank. On the contrary, it had a quite positive impact, mostly because at first, there were few occurrences of dos in BC. Finally, our last pair shows that the addition of out-of-domain data remains interesting even if the words pair are -almost-not found in the general corpora.
However, we find real sentences among the highest ranked (see line 115), but the CrossEntropy seems to favor phrases. As intended, for the least interesting sentences (CrossEntropy-), we can't find any relation between the breast cancer domain and the proposed sentences. Finally, Table  6  illustrates the evolution of the candidate translations over three stages of data selection. The words in bold are the correct translations. The first pair pressure-pression illustrates the problem of having only a small specialized corpus, where the words do not have enough context to be precisely represented. For the second pair breast-sein, we already have two words with a lot of occurrences, so we do not have this problem of bad representation of the words, even if some candidates should not be here (année, der.
Conclusion In order to improve BLI from specialized comparable corpora, we have studied in this paper the impact of common data selection techniques such as CrossEntropy, T f -Idf and also applied BERT for the first time as a data selection technique. The experiments on two specialized comparable corpora and two general corpora revealed that selecting a small and adapted amount of out-of-domain data is sufficient to obtain better results (MAP of 87. 6% for BC and 80. 9% for WE) than a high amount of data (resp. 83. 9% and 75. 9%) while reducing computation time by a factor 10. The data selection strategy also allowed to reduce the impact of polysemy compared to data augmentation. This strategy shows that bringing new occurrences for both the word to translate and the general vocabulary improves their mutual discrimination.
The result algorithm is validated thereafter using another test database and/or by prospective in vivo clinical trials. Different types of AI computer systems exist to fulfill so many functions. The main two AI systems categories are computer-assisted detection (CADe) for lesion detection and computer-assisted diagnosis (CADx) for optical biopsy and lesion characterization. Other AI systems offer therapeutic assistance such as lesion delineation for complete endoscopic resection. Furthermore, other AI systems also exist to offer technical assistance for better GI endoscopy performance (such as scope insertion guidance), disease prediction based on patient data and more. The main AI systems under development are summarized in Table  1. The main objective of this article is to introduce this new technology to every GI endoscopist to become familiar with as it will bring major changes to the way we perform endoscopy in the very near future.
Nearly half of the false-positive lesions were gastritis with changes in color tone or an irregular mucosal surface. The authors concluded that their system could process numerous stored endoscopic images in a very short time with a clinically relevant diagnostic ability. Another CADx was developed by Kubota et al  [35]  for automated diagnosis of depth of wall-invasion of gastric cancer. The primary database included 902 endoscopic images of 344 patients who underwent gastrectomy or endoscopic resection for gastric cancer. The images were divided into ten groups with almost equal number of patients and T staging. Computer learning was performed using about 800 images from all but one group which was used for accuracy testing (90 images). The overall diagnostic accuracy rate of the CADx was 64.
[42, 43] ther CADe systems are based on textural content of bleeding and non-bleeding images to perform the classification. [44] Combination use of both color and texture descriptors demonstrated more accurate bleeding detection. [45] o examples of CAD in VCE Hassan et al  [5]  aimed to draw inferences (bleeding or nonbleeding) using CAD system based on the spatial domain of an image by extracting features in the frequency domain using complex DL to achieve sensitivities and specificities as high as 99% for detection of GI bleeding. Xiao et al  [6]  developed another CAD system to reach >99% F1 score or performance score for GI bleeding detection in wireless capsule endoscopy (WCE). F1 score is calculated using precision and recall scores. They used DL to build this CAD using a dataset consisted of 10,000 WCE images, including 2850 GI bleeding frames and 7150 normal frames.
991. Another CADe developed by Wang et al  [53]  using WL endoscopy, and tested in a retrospective study on multiple image and video datasets, showed a sensitivity between 88. 2% and 100% and specificities around 95% for polyps' detection. Finally and most recently Klare et al,  [54]  published in 2019 their results for another CADe developed using WL endoscopy and tested in a small prospective study including 55 patients, this CADe showed a polyp detection rate of 50. 9% and an adenoma detection rate of 29. 1%. As a summary of the available published data, AI is a very promising tool to increase colon polyps' detection. In other The system presented the probability of the presence of polyps as a percentage in the upper left corner of the endoscopic image. When the probability exceeded the cut-off set at 75%, the CAD system warned of the possibility of the presence of polyps by changing the color in the four corners of the endoscopic image to red.
Misawa et al  [12]  developed a CADx system based on endocytoscopy (EC). This system automatically provides highly accurate diagnosis in real-time as endoscopists concurrently take EC images. Vital stain such as methylene blue is required to visualize glandular structural and cellular atypia. In contrast, the endocytoscopic vascular pattern can effectively evaluate microvessel findings using EC with NBI (EC-NBI) without using any dye. EC-NBI has shown a highly accurate diagnostic ability, similar to other modalities allowing endoscopists to use it more easily without the cumbersome dye staining during colonoscopy. [55] e custom software is called EndoBRAIN (Cybernet Systems Co. , Ltd. , Tokyo, Japan). To develop this CAD system, an image-database was created using 1079 EC-NBI images (431 non-neoplasms and 648 neoplasms) from 85 lesions colorectal lesions that had been detected during colonoscopy using EC and subsequently resected between December 2014 and April 2015.
From this image-database, 979 images (381 non-neoplasms, 598 neoplasms) were used for ML and the remaining randomly selected 100 images were used for ML. The CAD system provided diagnosis of 100% (100/100) of the validation samples with a diagnosis time of 0. 3 s per image. The major limitation of this system was that it cannot diagnose cancers and sessile serrated adenomas/polyps (SSA/Ps), because there were few EC-NBI images of invasive cancers and SSA/Ps for training. If there were more suitable images, better system that provides 4-class diagnosis (non-neoplastic, adenoma, invasive cancer, and SSA/P) could be developed. [12] An example of a corresponding video-output is shown in Figure  2. In this category of AI systems, or CADx for real-time optical biopsy, several other studies were published by different teams, showed very promising results for different types of CADx.
[41] [42] [43] , whereas jet-veto logarithms were resummed at NNLL in Ref. [44] [45] [46]. All these resummed results were matched to the inclusive NLO (or NNLO) total cross sections. The most accurate predictions were obtained in Ref. [47] : the transverse momentum of the W + W -system was resummed at NNLO+N3LL accuracy, the jet-veto logarithms were resummed at NNLO+NNLL, and a joint resummation for the W + W -p T spectrum in presence of a jet veto was performed at NNLO+NNLL. The matching of fixed-order computations in QCD with parton showers (PS) algorithms (NLO-QCD+PS) is well established with the MC@NLO  [48]  and the POWHEG  [49, 50]  matching algorithms, that are implemented in the public software MadGraph5_aMC@NLO  [51]  and POWHEG-BOX  [52]. Variants of these algorithms are also available within the Sherpa generator  [53, 54]  and in Herwig7  [55] , through the MatchBox framework  [56].