dataesr/scientific-paragraphs-categorization huggingface dataset:
A1 ). The other galaxies are unresolved or it was impossible to obtain kinematic maps for the whole galaxies due to very high magnification. We also note that the kinematics is not significantly affected by the shear caused by the lensing in seeing-limited observations for galaxies with a low magnification (µ 5). We find a median differential amplification of only ∼ 0. 1 in our galaxies. To obtain the kinematic properties, we model the kinematics of the galaxies using GalPaK 3D 4  , a three-dimensional galaxy disk modeling code that directly fits the datacubes  (Bouché et al. 2015). The model is convolved with the PSF and the line spread function (LSF) to derive kinematic and morphological properties. The beam smearing is therefore taken into account when modelling the kinematics. The PSF has been determined from a star observed in one of the IFUs during the observations and the LSF has been measured from the skylines.
This could also indicate that clumps can form in different physical conditions. The galaxies with no clump also show a wide range of dispersion (10 < σ 0 [ km s -1 ] < 75). Several of them also have a low υ rot /σ 0 ratio (< 3). This could mean that the turbulence in these galaxies is either due to different mechanisms than in the clumpy galaxies or that these galaxies are about to form clumps. through the NASA Hubble Fellowship grant HST-HF2-51413. 001-A awarded by the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Inc. , for NASA, under contract NAS5-26555. GLASS was supported by NASA through STSCI grant HST-GO-13459. This research made use of astrodendro, a Python package to compute dendrograms of Astronomical data (http://www.
The general planning of the demonstrator is seen in Figure  3 , which shows the relationship between modules. Reliable external apparatus are connected to the ATLAS demonstrator: -A scheduler to generate visualization of tasks scheduling; -A dedicated module for "messages" management, especially alerts; -Constraint propagation engine (COFIADE, proposed by the Centre Génie Industriel from the Ecole des Mines d'Albi-Carmaux) for managing rules coupling system constraints and project constraints; -And a "feedback" management module for storing models and experience of a project and then reusing it (T-REX, proposed by the École nationale d'ingénieurs de Tarbes -ENI of Tarbes). Competence management is integrated into the "project management" module, and specific parameters are used to define specific constraints that are managed with COFIADE through the "coupling" infrastructure.
These scenarios have been built with information captured during interviews with the industrial companies before their validation by the same companies. For each example, a specific database has been filled with the initial inputs, and then each company has been invited to apply the scenarios with the TALAS demonstrator. Then a new interview has been made to formulate the following feedback: What bugs have been identified? What are some useful improvements with the use of the demonstrator? What are the improvements of the methodology underlined by the scenarios? The problems encountered when using the demonstrator are operational problems that are not linked to our research proposals. As the demonstrator was not dedicated to being sold, they have not been considered. Considering improvements of the software, operational ones such as modifications of the Man-Machine Interface have been archived for a future tool.
Knowledge bases are increasingly exploited as gold standard data sources which benefit various knowledge-driven NLP tasks. In this paper, we explore a new research direction to perform knowledge base (KB) representation learning grounded with the recent theoretical framework of knowledge distillation over neural networks. Given a set of KBs, our proposed approach KD-MKB, learns KB embeddings by mutually and jointly distilling knowledge within a dynamic teacher-student setting. Experimental results on two standard datasets show that knowledge distillation between KBs through entity and relation inference is actually observed. We also show that cooperative learning significantly outperforms the two proposed baselines, namely traditional and sequential distillation.
Concretely our set up is the following: the representation learning model of each KB is equipped with two losses which are jointly optimized: 1) a classic KB supervised margin based ranking loss whose objective is to make the scores of positive triples lower than those of negative ones; and 2) a mimicry cooperative distillation loss that makes the posterior class predictions of aligned entities and aligned relations close to the entity and relation class probabilities of its peer respectively. Through joint optimization, the knowledge is also naturally transferred from the seed aligned information to unaligned information. In summary, the main contributions of the paper are the following: 1) a first attempt to ground multigraph representation learning by a knowledge distillation theoretical framework; 2) a novel KB representation learning model called KD-MKB, based on a cooperative knowledge distillation strategy; 3) experiments on two standard datasets, WN18RR and FB15K-237, that empirically validate the rationale of knowledge distillation across KBs and show the effectiveness of the cooperative knowledge distillation as proposed in KD-MKB.
Representation learning across multiple KBs Learning KB embeddings has drawn a huge attention in recent years. The embedding models are mainly categorized into: 1) translational models such as TransE  (Bordes et al. , 2013) , TransH  (Wang et al. , 2014) , and TransR  (Lin et al. , 2015)  or complex based models such as ComplEx  (Trouillon et al. , 2016)  and RotatE  (Sun et al. , 2019) , which learn vector embeddings of both entities and relations by interpreting a relation as a translation operation from a head entity to a tail; 2) deep neural models based for instance on graph convolutional networks  (Schlichtkrull et al. , 2018). More recently, representation learning across multiple graphs has gained an increasing attention. The general objective is to encode different graphs into a unified embedding space, such that the alignment likelihood between entities can be directly measured via their embeddings.
Settings Datasets and splits. We perform our experiments on two standard real-world WN18RR and FB15K-237 KBs 1. We simulate the multiple KBs setting by randomly splitting each of the WN18RR and FB15K-237 KB train triples into 2 and 3 partitions (n = 2, 3 in the KD-MKB setting, see Section 3. 2). Our motivation behind this evaluation setting is sustained by two reasons: 1) our goal with KD-MKB is to learn empowered KB embeddings instead of multi-graph embeddings; 2) evaluate the intrinsic effect of the KD-MKB model without any bias induced by uncontrolled effect of knowledge alignment quality. More precisely, two FB15K-237 partitions usually share 95% of the entities but drops to 64% for WN18RR. The setting n = 1 allows reporting the traditional KB model on the entire set of triples. Table  1  provides statistics for both datasets used in our experiments.
Thus, the predictions and parameters of the KB models are jointly updated. Implementation details. We implemented all baselines and our model using PyTorch 3. The loss function is minimized using the Adam stochastic method with a learning rate of 10 -5. The maximum number of iterations is set to 8 × 10 4. Parameters of the TransE model were fixed by selecting the best configuration in the validation partition of each dataset and by following recommendations from  (Sun et al. , 2019). Thus, embedding size is fixed to 1000 (resp. 500), batch size to 512 (resp. 256), negative sampling size to 128 (resp. 512), the α al adversarial loss to 1 (resp. 0. 5), and the margin hyperparameter γ to 9 (resp. 6) for FB15K-237 (resp. WN18RR). Top-k entities are found using faiss  (Johnson et al. , 2017)  with k set to 10.
KD-MKB model analysis Distillation model. To highlight the benefit behind cooperatively distilling knowledge across KBs, we report in Table 3 link prediction results using the three knowledge distillation strategies Independent, 5587 WN18RR FB15K-237 Teacher Student Teacher Student n H@1 H@3 H@10 MRR H@1 H@3 H@10 MRR H@1 H@3 H@10 MRR H@1 H@3 H@10 MRR Xdistills∼X and KD-MKB by using the same splits than those presented in Table  2. We report best and worst results of M i and M j models for each dataset partition. The main observation that can be drawn from Table  3  is that the KD-MKB model outperforms the Independent model (e. g. , between 24. 8% and 455. 7% improvement based on HITS@1, between 17. 9% and 85. 7% improvement based on MRR) over all the partitions and datasets and w. r. t. to all the metrics.
Additionally, we can also see that the Xdistills∼X model outperforms the Independent model. For example, when n = 2, the HITS@3 performance reaches a level around 27. 93 (resp. 32. 60) using Xdistills∼X for the WN18RR (resp. FB15K-237) dataset vs. a lower value around 19. 95 (resp. 31. 92) using the Independent model. This can be easily explained as the Independent model is only trained over the soft-labels in just one partition while the Xdistills∼X model uses first hard labels from its own partition (when it plays the role of a teacher) and then uses soft labels from the other partitions (when it plays the role of a student). We can also interestingly observe that the KD-MKB model outperforms the Xdistills∼X model though by a lower percent change (e. g. , between 1. 0% and 11. 0% improvement for HITS@10 using WN18RR dataset, between 10.
6% and 18. 6% improvement for HITS@10 using FB15K-237 dataset). It is worth mentioning that the KD-MKB model uses the same number of soft-and hard-labels than the Xdistills∼X model but our proposed cooperative strategy takes more advantage of both kind of labels. This result highlights a clear benefit of both dynamically switching between the teacher and student roles for each of the M i models empowered by the mimicry cooperative learning and joint update of their parameters. Distillation with larger alignments. We further analyze the effect of the size of entity alignments on KD-MKB performance. To be coherent with the results presented in the previous experiments, we keep the same partitions. Figure  3  plots the performance variations w. r. t. to all the metrics using the Figure  3 : HITS@1, HITS@3, HITS@10 and MRR link prediction results for KD-MKB using WN18RR when different sizes of the alignment set (Ie(i, j)) are used.
Our best performances are highlighted by a circle and values were included. Figure  4 : HITS@1, HITS@3, HITS@10 and MRR link prediction results for KD-MKB using FB15K-237 when different sizes of the alignment set (Ie(i, j)) are used. Our best performances are highlighted by a circle and values were included. WN18RR dataset. It is worth mentioning, that unlike the FB15K-237 which exhibits an overlap of 95%, the WN18RR actually allows to simulate an increasing overlap of entities by adding extra information to I e (i, j) until reaching the 100% overlap. The additional mapping information between entities does not increase the number of triplets to train but it allows a larger number of entities to be distilled from teachers. Figure  3  indicates that in average, 20% of extra aligned entities leads to an absolute gain of 5.
2 points in HITS@10 and 0. 06 points in MRR. As expected, higher numbers of soft-labels improves the mutual knowledge inference from a KB to its peers. Results for FB15K-237 datasets are presented in Figure  4. In this case the increasing is beneficial being more stable when n = 3 because of the larger increase of the overlapped entities. Multi-KB learning vs. single-KB learning. We depict in Figures  3  and 4  the performances of KD-MKB as well as for TransE using both datasets. Both models were trained using full train data, hard labels only for the latter, and hard-labels and soft-labels for the former. Results are constant for TransE as its performances do not depend on the number of aligned entities. When the number of aligned entities is 100% (highlighted by a circle), we can verify from these figures the improvements of using KD-MKB on multiple KBs instead of the classical TransE embedding model on an individual KB.
This paper presents an interactive approach for multi-class segmentation of aerial images. Precisely, it is based on a deep neural network which exploits both RGB images and annotations. Starting from an initial output based on the image only, our network then interactively refines this segmentation map using a concatenation of the image and user annotations. Importantly, user annotations modify the inputs of the network -not its weights -enabling a fast and smooth process. Through experiments on two public aerial datasets, we show that user annotations are extremely rewarding: each click corrects roughly 5000 pixels. We analyze the impact of different aspects of our framework such as the representation of the annotations, the volume of training data or the network architecture.
, 2015)  and is now efficiently tackled with powerful convolutional neural networks (CNNs) such as Deeplabv3+  (Chen et al. , 2018). Under appropriate conditions (e. g. when a large enough training dataset is available), one might say that semantic segmentation is nearly achieved. Indeed, these segmentation algorithms lack only a few percents of precision to reach perfect scores on public benchmarks. However, these few percents can visually make a big difference and therefore not be tolerable in practice. Besides, it often gets worse in real-life datasets due to a variety of factors (complex data, lack of wellannotated ground-truth, various usage domains,. ). This paper proposes a fast procedure to iteratively refine the segmentation maps with a human in the loop. It consists in a neural network pre-trained with simulated human annotations and which does not require any retraining during the interactive process.
In order to concretely motivate our approach, let us consider two famous aerial image datasets in remote sensing. On the INRIA Aerial Image Labelling Dataset  (Maggiori et al. , 2017) , a building segmentation dataset, the current best networks reach an Intersection over Union (IoU) around 0. 8 and a pixel accuracy around 97% on the test set. On the ISPRS Potsdam multi-class segmentation dataset  (Rottensteiner et al. , 2012) , the state-of-the-art approaches almost reach a pixel accuracy of 92% on the test set. While these performances are incredibly high, there might still remain some misclassified areas unacceptable for an end-user. Besides, these optimal results are obtained using top notch neural 1 -Initial segmentation 2 -Annotation phase.
-Refined segmentation Ground-truth Figure  1. Example of the proposed interactive semantic segmentation approach on the ISPRS Postdam multi-class dataset  (Rottensteiner et al. , 2012)  networks which have required many specific refinements  (Yue et al. , 2019). An off-the-shelf neural network still yields good results but, as the baselines show, a drop of performance between 5 and 10% can be expected. Moreover, these performances decrease quickly when the networks are faced to the domain shift issues inherent to machine learning. Therefore, the segmentation masks output by these neural networks have to be manually reviewed to meet the expectations of a potential end-user. Let us also consider a practical application for which current approaches still yield imperfect results. Drones are increasingly used to monitor different environments like crop fields, railroads or quarries.
In this context, semantic segmentation can be extremely useful for different tasks such as defects detection, volumes computation or crop monitoring. However, due to the complexity and the high variety of the acquisitions, results are usually not as good as on public datasets while a high precision is necessary for these tasks. Therefore, the operators often have to manually refine the segmentation maps which is a slow process. To address these issues, we propose to adopt an interactive semantic segmentation approach, as sketched in Figure  1. Indeed, a human in the loop can easily spot the misclassified areas and correct them thanks to a more complex yet intuitive analysis. The difficulty then is to reach optimal classification while keeping the whole process swift and engaging enough. Our present contribution is as follows.
A particular challenge is to get enough useful annotations. For this purpose,  (Mahadevan et al. , 2018)  use a hard-sample mining strategy at training by selecting annotations among erroneous predictions. Alternatively,  (Jang and Kim, 2019)  iteratively optimize the annotation maps given as inputs by back-propagating the errors between predictions and annotations. Finally, DEXTR  (Maninis et al. , 2018)  and  (Wang et al. , 2019b)  both ask the user to click points on the borders and corners of the objects. Recently,  (Benenson et al. , 2019)  assess these various strategies in the first large scale study of interactive instance segmentation with human annotators. Their experiments hint that center annotation clicks are the most robust and that distance transform to encode the annotation points can be replaced by binary disks.
Experimental setup Datasets. We have tested our approach on the two standard remote sensing datasets mentionned in the Introduction. We split the initial training sets into a training and a validation sets with a 80%-20% ratio and use the validation sets for our experiments. The INRIA Aerial Image Labelling dataset  (Maggiori et al. , 2017)  is composed of two classes (buildings and not buildings) and covers more than 800 km 2 with a spatial resolution of 0. 3m. The size of each image is 5000 × 5000 pixels. The training set is composed of 144 images and the validation set of 36 images. The ISPRS Potsdam dataset  (Rottensteiner et al. , 2012)  is composed of 6 classes (impervious surface, buildings, low vegetation, tree, car and clutter). The class car is sub-represented compared to the other classes.
This dataset covers around 3 km 2 with a spatial resolution of 0. 05m. The size of each image is 6000 × 6000 pixels. The training set is composed of 19 images and the validation set of 5 images. Neural Network. Except in the backbone comparison, we use a LinkNet  (Chaurasia and Culurciello, 2017)  architecture. It is a classic encoder/decoder architecture relying on a ResNet encoder  (He et al. , 2016). The networks are trained using stochastic gradient descent (SGD) and cross-entropy loss for 50 epochs with a batch of size 8, seeing during each epoch 10000 samples randomly chosen and cropped (size 512 × 512). The initial learning rate is fixed at 0. 05 and is divided by 10 after 15, 30 and 45 epochs. Only basic data augmentation is performed: horizontal and vertical flips. The implementation is done using Pytorch.
Approach assessment and automatic evaluation strategy Here we compare the two proposed automatic evaluation strategies. Let us recall that, for the first strategy, the clicks are sampled iteratively in the biggest erroneous areas independently of the class while they are sampled equally in each class in the second one. As we can see on Figure  4 , the class independent evaluation strategy allows to reach a higher overall IoU but to the detriment of the sub-represented car class in the Potsdam dataset. This is due to the fact that the biggest erroneously predicted areas inherently belong to larger instances than cars such as buildings. Therefore, even though the overall metric gain is not as good as with the class-independent evaluation process, we choose the class-dependent one for our further evaluations on Potsdam.
Influence of the annotation strategy We compare here the different annotation strategies: inside clicks or border clicks, binary encoding or distance transform, and single or multiple channels. As we can see on Figure  6 , the distance transform clearly increases the benefits of the annotations compared to the binary encoding. While  (Benenson et al. , 2019)  conclude that the binary encoding leads to better performances, our opposite conclusion might be inherent to the large size and  scale of aerial images which dilute the annotations localized over very small areas. Both the contours and the inside points are efficiently used by the network to enhance its predictions but it is still noticeably better with the inside points. We can also notice that the last 20 added points considerably boost the performances of the inside point strategies for the Potsdam dataset: this is due to the fact that these points belong to the class clutter, an under-represented class.
Influence of the volume of training data On the Potsdam dataset, even though the initial training size is lower than in the INRIA dataset, the network is still initially quite accurate with little training data. Indeed, according to the results of  (Castillo-Navarro et al. , 2019) , since there are pictures from only one city, few training images are enough to learn the general semantic of the dataset even if the full training set provides better performances. However, the accuracy gain is really low with little training data. For example, the IoU gain is less than 1% with 20% of the initial volume of data while it is slightly over 4% with the full training set. We believe that this lack of performance in low-data regime is due to over-fitting. Indeed, since there are only a few training images in this scenario, there are also less possible annotations and they might not fully reflect the reality of the test set.
Manual analysis In this experiment, the images from the Potsdam validation set have been manually refined by a human annotator. If the number of clicks exceeds 120, we threshold it at 120 in order to make a fair comparison with the the automatic process. Local insights. On one hand, as shown on Figure  9  For example, in the Potsdam dataset, there is only one outside car park considered as building which means only one place with the semantic "car" surrounded by "building" in the dataset. We kept the associated image in the validation set to study the impact of the annotations in this scenario. Figure  10  shows the outcome of our approach on this car park. Since it also looks like a road, it is initially difficult for the network to segment it correctly. Nonetheless, it succeeds to recognize the cars parked there.
2014) , and for the Meisei Electric Co. , Ltd. , radiosondes (RS-11G and IMS-100:  Kizu et al. 2018). In the framework of the Global Climate Observing System (GCOS) Reference Upper-Air Network (GRUAN),  Dirksen et al. (2014)  propose methods to correct Vaisala RS92 relative humidity measurements and to provide measurement uncertainty estimates. Such methods are applied to improve data quality at all GRUAN stations that use RS92 radiosondes. Similar investigations should be carried out on other relative humidity sensors of radiosondes used in the GRUAN network. At least two stations of the GRUAN network (Paris, France and Réunion in the southern Indian Ocean) use Meteomodem Co. M10 radiosondes for regular temperature and relative humidity profiling. The aim of this paper is to identify error sources that can lead to dry or wet bias and uncertainties.
Observational dataset To test correction methods and evaluate their effects on measurement accuracies, we looked for a dataset that contains a significant number of both nighttime and daytime dual-radiosonde profiles that include M10 radiosondes and another well characterized radiosonde that can serve as reference, both under the same balloon. This well characterized radiosonde is here defined as a radiosonde for which there exist published studies on its accuracy and on its measurement uncertainties. This is the case for Vaisala RS92 radiosondes, which have been extensively tested  (Dirksen et al. 2014). For this study, the data from two field experiments, which are presented in the following subsections, are considered.