dataesr/scientific-paragraphs-categorization huggingface dataset:
End User AR Application Regarding AR deployment, we use a similar approach as the one described in  (Soedji et al. , 2020). The output of the VR editor is an XML file that describes the different AR augmentation layers and that is interpreted by a dedicated AR player at runtime. Similarly, we also propose a VR player that can simulate AR devices in order to assess an application before its deployment in the real world. We enhanced this solution by including our adaptation mechanisms. The architecture of this process is inspired by the AARCon framework  (Krings et al. , 2020). It is composed of the same three main components. First, a context monitoring module constantly collects the data from the pre-developed context sensors. It also solves the values of the context parameters visually programmed in the VR editor according to data coming from the connected objects.
Moreover, they also highlight that a single tool in VR could not satisfy every user. Therefore, as a first future work we could imagine developing a full software suite with a dedicated framework including our VR design tool but also a 3D editor with a 2D user interface and an AR design tool. Content creators would then choose which one to use depending on the operation they have to perform, their preferences, their needs, and their skills. As detailed in Section 2, such an idea was partly implemented in ACARS  (Zhu et al. , 2013). As future work, we also plan to consider adaptation to the target environment topology. Indeed, for now, the AR application will fit only one real environment. Regarding our case study, we would need a solution for retargeting our prototyped application for another building.
Experimental Results We performed experiments on the proposed algorithm using MATLAB x64 on a system with Core (TM) i7, 3. 6 GHz, 8 GRAM, Windows 10 as a platform. A code was developed to implement the algorithm, and selected simulation cases have been executed to demonstrate the efficiency of the proposal. The algorithm automatically runs several times, changing the radius of the sensors or not and calculating the corresponding coverage inside the area of interest with the connectivity constraint. In the end, the algorithm chooses the best result. We have performed several simulations in several cases that show the algorithm's effectiveness in both deploying and maintaining connectivity. All cases are based on actual maps of cities or parts of cities taken from Google Maps. For the cases illustrated and detailed below, we use the parameters shown in Table  1.
In this paper, our work consists in exploiting the potential of existing annotated LPIS in France and efficient deep learning pre-trained models to produce crop type maps in Tunisia from Sentinel-2 satellite image time series (STIS) while having little training data i. e. in a few shot context. Few-shots classification aims to learn a classifier to recognize unseen classes during training with limited labelled examples. The contribution of this project is to propose innovative solutions based on deep learning to automate producing land cover maps for agricultural landscapes with better accuracy, especially on cultivated areas with a fine nomenclature while reducing in-field observations that are time and cost-consuming. Moreover, this project is a key to a wide range of applications beyond crop monitoring, both for public and private entities, the large-scale monitoring of agricultural parcels is a matter of major political and economic importance and may provide a sustainable promising solution for land cover mapping in countries that lack geographical information systems and especially LPIS.
Each head extracts highly specialised temporal features which are in turn concatenated into a single representation. This PSE + LTAE approach outperforms other advanced time-series classification algorithms on an open-access satellite image dataset while using far fewer parameters and with reduced computational complexity ( table 1)(Sainte-Fare  Garnot et Landrieu, 2020). This result was confirmed recently by different independent works using different datasets  (Kondmann et al. , 2021)  and PSE-LTAE is used as a backbone by many works  (Schneider and Körner, 2020).
, 2020). The Temporal Attention Encoder is an attention-based network achieving equal or better performance than RNNs, while being completely parallelizable and thus faster. In 2020, (Sainte-Fare  Garnot et al. , 2020)  presented a new lightweight network for embedding sequences of observations such as satellite time-series. Thanks to a channel grouping strategy and the definition of the master query as a trainable parameter, his proposed approach is more compact and computationally efficient than other attention-based architectures. Evaluated on an open-access satellite dataset, the L-TAE performs better than state-of-the-art approaches, with significantly fewer parameters and a reduced computational load, opening the way for continent-scale automated analysis of Earth observation. In this study, results of both architectures will be compared in a few shots context to test their generalisation capacity.
Dataset We used a Sentinel-2 multispectral image sequence at level 2A, in canopy top reflectance. We excluded the atmospheric bands (bands 1, 9, and 10), allowing us to retain C = 10 spectral bands. The six 20m resolution bands are resampled to the maximum spatial resolution of 10m. The Area of Interest (AOI) corresponds to a single tile in the Sentinel-2 tile grid (T32SNE) in central Tunisia. This tile offers a difficult use case with a monitor wide variety of crop types and different terrain conditions. For our theme and depending on the availability of images we have chosen 24 dates from November 2020 to April 2011. Parcel plots were digitised manually using a high resolution spatial imagery leading to a geo-referenced LPIS with class label for each parcel (figure  4 ). We have digitised 10111 parcels, covering an area of 197 km².
Topological validation was processed. The parcel delimitations were checked in a field mission. A detailed nomenclature of 17 crop type classes was used. The most observed class of parcels is the olive tree with 411 parcels and wheat with 180 parcels. On the other hand there were very minor classes such as vines and prickly pear which leads to a highly imbalanced dataset as shown in the figure  8. Satellite images are cropped using digitised polygons to constitute the image time series.
Experiments We used cloud computing tools to address the concern of large data volumes, and to implement our models we used the Google Colab platform, which is designed for training models in machine learning. In addition, Colab provides us with the opportunity to train and test with a GPU that can be 60 times faster than a classical learning on a CPU. Indeed Colab offers us a Tesla T4 graphics card with 16 GB of RAM. In addition, we used the following tools and libraries: ArcGIS, QGIS, Pytorch, Cuda, Envi, Rasterio. For data augmentation, we used an efficient Python library Imgaug (imgaug Development Team, 2020). Several parameters and criteria were tested in order to choose the most efficient and effective ones. We present below an explanatory table of the models architecture, their modules, hyperparameters and the number of parameters (table  3  and table 2 ).
For the optimizer, we use Adam  (Kingma et al. , 2015)  with the following values: • Lr = 10 -3 ; • β = (0. 9, 0,999). In addition, we train the models with the focal loss (γ = 1) to calculate the losses. We applied the "K-fold" cross-validation with K=5. For each file, the dataset is divided into training, validation and test sets with a ratio of 3:1:1 respectively. The validation step allows us to select the best performing epoch and evaluate it on the test set.
3 Transfer learning with pre-trained transformer models It should be noted that, we have a reduced dataset (4548 plots with data augmentation) and different from the French dataset: since we have 17 classes in Tunisia with a more detailed annotation than that of RPG. Based on these two criteria (reduced and different dataset), we need to keep (freeze) only the layers of "feature extractions" (Sainte-FareGarnot et  Landrieu, 2020)  which are in our case the layers of PSE. We finally obtained the results presented in the tables below. Table 7. Results of transfer learning using pre-trained models (PSE+LTAE) and (PSE+TAE) Model Micro F_ OA Recall Precisi IoU score on PSE+LTAE 0. 90 0. 93 0. 93 0. 93 0. 94 PSE+TAE 0. 84 0. 9 0. 92 0. 91 0. 91 6. DISCUSSION.
All of them are supported by the ICTs (Information and Communication Technologies) as the enabler for digital cities. mySMARTLife project (grant agreement #731297)  [6] , which is H2020-funded, aims at the sustainable and smart transformation of the three lighthouse cities of Nantes, Hamburg and Helsinki by applying the aforementioned concepts. More than 150 actions are contributing to the pillars of energy, mobility, ICT, citizens, economy and governance to the new urban sustainability concepts. mySMARTLife targets are: (a) renewable share of 54%; (b) reduction of 55% of greenhouse emissions due to buildings, city infrastructure and mobility actions. Furthermore, mySMARTLife fosters both smart economy and smart people supported by local economy growth and entrepreneurship. The main driver for designing and implementing this innovative concept of smart city is integrated urban planning.
Thus, the evaluation framework is set to measure the progress obtained in the cities under this approach: (1) to achieve an economic growth decoupled from resource use to face the current pollution and CO 2 emissions; (2) to improve the life quality of citizens; and (3) cities operate in a more efficient way. As result of this step, the "human" language objectives are obtained, which need to be mapped into dimensions and quantifiable indicators included in steps 2 and 3 of the methodology. (2)  Step 2 identifies the dimensions to evaluate the evaluation framework. These dimensions are extracted from the well-established ones from CITYKeys  [16] , complemented with experiences from mySMARTLife on how to allocate the impacts to be reached to the project technologies. Initially, a document is defined by the technical partners involved in the project, which are updated and validated by city partners during diverse iterations and dedicated workshops.
The output of this work is shown in Table  1  below, which is depicted to understand the next steps of the methodology. (3) Step 3 maps the available indicators to the objectives and dimensions defined within steps 1 and 2. mySMARTLife made the exercise of collecting the list of indicators from literature and reference projects, as summarized in Table  2  and as available in  [21] , in order to create a dataset of indicators to be applied to the demonstrated solutions. From this list, the suitable indicators for each dimension are selected so as to quantify the final impacts and results of the actions, i. e. , each city or project does not need to define its own indicators, but needs to choose the appropriate ones from the list. Conversely, to guarantee a proper evaluation of impacts, some considerations have been also included below:.
These are, as mentioned beforehand, energy and environment, mobility, urban infrastructures (including the digitalisation of the city through the ICTs and urban data platforms deployment), citizens, economy and governance. All of them are driven by a set of indicators  [21, 22] , which are summarised in Table Within each of the stated levels, a set of categories (named fields in the city level and pillars at the project level) are established. These are, as mentioned beforehand, energy and environment, mobility, urban infrastructures (including the digitalisation of the city through the ICTs and urban data platforms deployment), citizens, economy and governance. All of them are driven by a set of indicators  [21, 22] , which are summarised in Table  3. City level indicators are calculated with the data compiled from public databases, mainly from city statistics.
• Reduction of air quality emissions due to replacement of ICE (internal combustion engine) by electricity powered vehicles (EVs); • Amount of travel, energy consumption and journey quality of e-vehicles; • Amount of use and pattern of the charging infrastructure installed; • Degree of energy managed and supplied to EVs by renewable sources; • Willingness to use multimodality actions and investment in urban freight. Data collected from transport facilities are used for the calculation of KPIs, identified by each mobility action with the exception of the impacts in the air quality emissions that need of a specific methodology to quantify the avoided air emissions. Thus, the evaluation approach in mySMARTLife establishes two measurement periods: baseline with ICE vehicles as reference for comparison and reporting period with e-vehicles.
The topic of summarization evaluation has recently attracted a surge of attention due to the rapid development of abstractive summarization systems. However, the formulation of the task is rather ambiguous, neither the linguistic nor the natural language processing community has succeeded in giving a mutually agreed-upon definition. Due to this lack of well-defined formulation, a large number of popular abstractive summarization datasets are constructed in a manner that neither guarantees validity nor meets one of the most essential criteria of summarization: factual consistency. In this paper, we address this issue by combining state-of-the-art factual consistency models to identify the problematic instances present in popular summarization datasets. We release SummFC, a filtered summarization dataset with improved factual consistency, and demonstrate that models trained on this dataset achieve improved performance in nearly all quality aspects.
Introduction While the revolutionary success of the Transformer  (Vaswani et al. , 2017)  architecture has drawn a surge of attention to automatic summarization, most research has been focused on improving performance metrics of summarization models on a set of popular datasets. It is often taken for granted that these datasets provide representative examples of high quality summaries, and that models capable of producing summaries that are similar to the ones of the dataset are superior in the task of automatic summarization. But is this really the case? Numerous works  (Fabbri et al. , 2021; Huang et al. , 2020)  have already pointed out deficien-cies in the most widely employed summarization datasets (e. g. CNN/DailyMail (Hermann et al. , 2015; Nallapati et al. , 2016)  and XSUM  (Narayan et al.
, 2018) ). These datasets are typically composed of news articles automatically extracted from news websites, paired together with a highlight or introduction sentence which serves as the summary. However, the qualities we seek in highlights and introductions are fundamentally different from the ones we seek in summaries. It is thus time for the NLP community to take a step back, revisit the formulation of the summarization task and reconsider the appropriateness of currently employed datasets. The goal of automatic summarization is to take an information source, extract content from it and present the most important content to the user in a concise form and in a manner sensitive to the user's or application's needs  (Mani, 2001). In the scope of this paper, we solely focus on generic summarization  (Nenkova and McKeown, 2011) , where we make few assumptions about the audience or the goal for generating the summary.
While the importance of the selected content and the conciseness of its form are rather subjective, there is one criterion for summarization that is certain: the summary content should be extracted from the information source. In other words, the summary should be factually consistent with the source document. This is however not the case with summaries constructed from highlights or introductions. It is common for highlights and introductions to contain information that are not mentioned in the main article or even exaggerate certain facts to achieve the goal of click baiting. Therefore, the models trained on these datasets perform the task of "pitch generation" rather than the intended summary generation. An example of such a reference summary is given in Table  1. In this paper, we aim to identify these erroneous data samples by scoring them with factual consistency models.
We push towards answering two main research questions : Q1 What kind of factual inconsistency can different factuality models capture and how can they be combined for better detection of erroneous samples? This question is answered in Sections 3. 2 and 5. 1. Q2 Does having more reliable datasets with more factually consistent reference summaries lead to better performing summarization models? And do the results only improve in factual consistency or also in other quality aspects such as informativeness and saliency? We address this question in Section 5. 4. Our research aims to answer these questions by filtering out samples with problematic reference summaries. We achieve this by leveraging state-of-the-art factual consistency models. We test the performance of different models on different categories of factuality errors  (Pagnoni et al.
, 2021)  and eventually combine them for an optimized filtration methodology. Our contributions are summarized below: 1. We prove the effectiveness of three stateof-the-art factual consistency models in detecting factuality errors. We use these models to affirm the factual consistency issue present in three of the most popular summmarization datasets. We devise a filtration methodology that combines different state-of-the-art factuality models and achieves better detection of misleading reference summary samples. 2. We release SummFC1, a Summarization dataset with improved Factual Consistency. We prove that finetuning summarization models on this dataset leads to better performance on not only factuality but also other quality aspects. We believe that regularly amending issues in widely employed benchmark datasets should become a common practice in NLP.
Related Work Summarization Benchmark Datasets Summarization benchmark datasets are typically composed of a large number of news documents paired together with "gold-standard" human reference summaries. Unfortunately, the human reference summaries in these datasets are often constructed in a suboptimal way. Gehrmann et al. (2022)  analyze a sample of 20 papers proposing summarization approaches published in 2021; they find 27 datasets that models were being evaluated on. The most popular ones, CNN/DM  (Nallapati et al. , 2016)  and XSum  (Narayan et al. , 2018) , were used five and four times respectively. However, both of these datasets have multiple issues. Taking the CNN/DM dataset for example, the construction is done by pairing an article with the bullet points written for it on the CNN and Daily Mail websites.
This design works well for its initial use as a Question Answering dataset  (Hermann et al. , 2015) , but does not function at the same level after its adaptation for summarization. Regarding the XSUM dataset, the main issue concerns its factuality. The reference summaries were never meant to be a real summary, there is thus no requirement for it to be faithful to the source article. An analysis of XSum finds that over 70% of reference summaries contain factual inconsistencies  (Maynez et al. , 2020). Factual Consistency Models Thanks to the development of reference-free factual consistency metrics for summarization  (Huang et al. , 2021) , the evaluation of factual consistency can now be performed automatically on a large scale. Since such metrics are generally modelbased, we refer to them as factual consistency models, in order to distinguish them from the evaluation metrics in Section 5.
3. Popular factual consistency models can generally be categorized into two different paradigms: Question Answering  (Durmus et al. , 2020; Wang et al. , 2020)  and Entailment Classification  (Kryscinski et al. , 2020; Goyal and Durrett, 2020). Question Answering models are based on the intuition that if we ask the same question to a summary and its source article, they should provide similar answers if the summary is factual. Entailment classification models rely on the idea that a factually consistent summary should be semantically entailed by the source article. They are usually trained models finetuned on either synthetic or human-annotated datasets. According to the FRANK benchmark  (Pagnoni et al. , 2021) , Entailment classification models perform significantly better than the Question Answering ones.
Improving Factual Consistency The idea defended in this paper is that factual consistency models allow us to discover issues in current datasets and eventually release improved versions. Dataset quality in summarization has only started receiving attention recently and only a few papers have attempted to make efforts in this direction. Gehrmann et al. (2021)  release an improved version of XSUM that filters the dataset with a BERT-based classifier fine-tuned on 500 document-summary pairs, manually annotated for faithfulness  (Maynez et al. , 2020). However, the classifier is fairly naive with only a single classification layer added after the BERT model and they did not compare the performance of models trained respectively on the original and improved datasets. Along similar lines,  Matsumaru et al.
(2020)  build a binary classifier for de-tecting untruthful article-headline pairs and filter a headline generation dataset. Nan et al. (2021)  also perform filtration for summarization datasets but both the filtration and evaluation methodologies are limited to the entity level. Goyal and Durrett (2021)  identify non factual tokens in the XSUM training data with the Dependency Arc Entailment (DAE) model. They mask these tokens corresponding to unsupported facts and ignore them during the training step. While this approach does allow for improved summarization quality, we see potential issues arising when isolated ignored tokens are confronted with the contextual nature of the Transformer architecture. Filtering out entire samples with detected inconsistency is thus the preferred solution.
Analyzing Factual Consistency Models In this section, we present the factual consistency models we employ for detecting factual errors in the reference summaries. We introduce three state-of-the-art models that each rely on a distinctive mechanism. We rely on human annotations from the FRANK benchmark  (Pagnoni et al. , 2021)  to validate the effectiveness of these models and analyze the variations in their performance when shifting dataset domain or error category. The FRANK benchmark is the most recent benchmark proposed for evaluating factual consistency models, it is the only benchmark to date that provides a categorization of factual errors.
BARTScore is shown to produce stateof-the-art results for factuality evaluation on the SummEval Dataset  (Fabbri et al. , 2021). the entailment model on synthetic datasets. However, they later extend their method by training it on human annotations and achieve improved performance  (Goyal and Durrett, 2021). This extended version of DAE is proven to be the best performing entailment-based model tested on the factuality dataset introduced in  Falke et al. (2019).