dataesr/scientific-paragraphs-categorization huggingface dataset:
The download component enables developers to check the available datasets and the annotation progress state (both raw and annotated datasets may be downloaded). The authentication component manages user accounts and login, only authorized people can enter the system and read data. For more explanations, see the demo https: //lawbot. unimes. fr/annotateur. Data. A record in MongoDB is a document, which is a data structure composed of field and value pairs, documents are similar to JSON objects  [2]. MongoDB simplifies annotation saving process and getting datasets in a standard format. It is tolerant about document keys, we can omit some in the documents and place them in others. Figure  3  represents the diagram of the database. There is a collection of system tasks, each task can have multiple sub-tasks (a dataset may be associated with each sub-task).
In this paper, we develop a procedure for selective imaging of highly variable sources. In Sections 2 and 3, we summarize the synthetic data generation and imaging methods used herein. In Section 4, we show the limitations of imaging in sparse and uneven coverage. In Section 5, we survey several metrics capable of ranking (u, v)-coverage quality. Additionally, we derive a novel isotropy-based metric that addresses the limitations described in Section 4. In Section 6, we apply these metrics to the 2017 EHT coverage of Sgr A * , validate their ability to predict reconstruction quality from (u, v)coverage geometry, and make recommendations for selective dynamical imaging of the 2017 EHT Sgr A * data set. In Section 7, we briefly discuss the utility of coverage metrics in ranking and selecting between different available observing periods.
Synthetic Data Generation Synthetic data were generated based on April 7 of the 2017 EHT coverage using the eht-imaging library  (Chael et al. 2018). A simulated observation corresponded to approximately 11. 5 hr of observing on all available baselines. Snapshot images from the simulated sources and the resulting amplitudes and closure phases can be seen in Figure  1  and Appendix A. Observation parameters (e. g. , R. A. and decl. of the source, observing frequency, bandwidth) were duplicated from the 2017 EHT survey of Sgr A *. The stations included in the simulated observations were the Atacama Large Millimeter/ submillimeter Array (ALMA), the Atacama Pathfinder Experiment (APEX), the Large Millimeter Telescope (LMT), the James Clerk Maxwell Telescope (JCMT), the Submillimeter Array (SMA), the IRAM 30 m telescope on Pico Veleta (PV), and the South Pole Telescope (SPT).
In this paper we use the Difmap software (e. g. ,  Shepherd 1997 and M. Shepherd 2011, private communication)  for CLEAN imaging. Once the imaging procedure converges based on a specified stopping criterion into an average static image, CLEAN dynamic imaging is performed by first dividing the data set into smaller portions with a time duration similar to that of the expected source timescale variability (i. e. , "snapshots"). Under the assumption of small structural changes over time, the model corresponding to the static image is used as an initial model, upon which we look for structural changes by cleaning the residual map corresponding to each data snapshot. To guide the deconvolution with such a limited (u, v)-coverage, we limit the extra cleaning to the imaging regions in which we have emission in the averaged image by placing tight cleaning windows.
StarWarps allows for the exact inference of the video by computing a mean and covariance of the image, which provides a complete description under Gaussian approximation. By contrast, the RML dynamic reconstruction derives only a MAP estimation. StarWarps requires an initial static image, which can be either data driven (e. g. , a best-fitting static image of the entire data set being dynamically reconstructed) or prior driven (e. g. , a synthetic image of a ring). In this paper, we use the RML dynamic imaging algorithms implemented in eht-imaging (also referred to as ehtim) and SMILI (Sparse Modeling Imaging Library for Interferometry) and the StarWarps algorithm in eht-imaging. See Event Horizon Telescope Collaboration et al. 2022c for more details of regularization functions and other imaging parameters used in the reconstructions.
This ranking is consistent with the predictions of the coverage metrics described in Section 5 and Appendix B. A wide array of factors impact reconstructions, and the tests presented here do not compose a realistic synthetic data suite for assessing whether or not Region II can accurately recover dynamical variability for Sgr A *. The tests provided here solely demonstrate that Region II is the best time region for performing dynamical reconstructions based on coverage considerations. Additional testing on more complex source types with realistic data corruptions can be found in Event Horizon Telescope Collaboration et al. (2022c).
We also select a period from ∼19:45 GMST to ∼21:00 GMST ("Region I") with reconstruction capability. Reconstructions of time-variable sources allowed successful recovery of the characteristic source variability in Region II. In contrast, reconstructions in Region I were unable to recover the characteristic motion. The ranking determined by the suite of reconstructions performed on synthetic data verifies the predictions made by the examined coverage metrics. We expect that attempts to recover variability in real EHT observations of the Galactic center will produce the most robust and accurate recoveries in Region II of the 2017 April 7 data set, and therefore we recommend performing dynamical imaging procedures in that time region. Coverage metrics have additional utility for ranking interobservation comparisons based on their ability to recover dynamical variability, which has a variety of applications to the broader field of interferometery.
We thank the  National Science Foundation  (awards  OISE-1743747 ,  AST-1816420 ,  AST-1716536 ,  AST-1440254 ,  AST-1935980 ) and the  Gordon and Betty Moore Foundation  ( GBMF-5278 ) for financial support of this work. This work was supported in part by the  Black Hole Initiative , which is funded by grants from the  John Templeton Foundation  and the  Gordon and Betty Moore Foundation  to  Harvard University. Support for this work was also provided by the  NASA   Hubble Fellowship  grant  HST-HF2-51431. 001-A  awarded by the  Space Telescope Science Institute , which is operated by the  Association of Universities for Research in Astronomy, Inc. , for  NASA , under contract  NAS5-26555. The Event Horizon Telescope Collaboration thanks the following organizations and programs: the  Academy of Finland  (projects  274477 ,  284495 ,  312496 ,  315721 ); the  Agencia Nacional de Investigacin y Desarrollo (ANID) ,  Chile  via  NCN19 058  (TITANs) and  Fondecyt 3190878 , the  Alexander von Humboldt Stiftung ; an Alfred P.
884631 ); the  Gordon and Betty Moore Foundation  (grant  GBMF-3561 ); the  Istituto Nazionale di Fisica Nucleare (INFN) sezione di Napoli , iniziative specifiche TEONGRAV; the two  Dutch National Supercomputers, Cartesius and Snellius (NWO  grant  2021. 013 ); the  International Max Planck Research School for Astronomy and Astrophysics at the Universities of Bonn and Cologne ;  Joint Princeton/Flatiron and Joint Columbia/ Flatiron Postdoctoral Fellowships , with research at the  Flatiron Institute  being supported by the  Simons Foundation ; the  Japanese Government  (Monbukagakusho: MEXT) Scholarship; the  Japan Society for the Promotion of Science (JSPS)   Grant-in-Aid for JSPS Research Fellowship  ( JP17J08829 ); the  Key Research Program of Frontier Sciences, Chinese Academy of Sciences  (CAS, grants  QYZDJ-SSW-SLH057 ,  QYZDJ-SSW-SYS008 ,  ZDBS-LY-SLH011 ); the  Leverhulme Trust Early Career Research Fellowship ; the  Max-Planck-Gesellschaft (MPG) ; the  Max Planck Partner Group of the MPG  and the CAS; the  MEXT/JSPS KAKENHI  (grants  18KK0090 ,  JP21H01137 ,  JP18H03721 ,  18K03709 ,  18H01245 , 25120007); the  Malaysian Fundamental Research Grant Scheme  ( FRGS) FRGS/1/2019/STG02/UM/02/6 ; the  MIT International Science and Technology Initiatives (MISTI) Funds ; the  Ministry of Science andTechnology (MOST) of Taiwan  ( 103-2119-M-001-010-MY2 ,  105-2112-M-001-025-MY3 ,  105-2119-M-001-042 ,  106-2112-M-001-011 ,  106-2119-M-001-013 ,  106-2119-M-001-027 ,  106-2923-M-001-005 ,  107-2119-M-001-017 ,  107-2119-M-001-020 ,  107-2119-M-001-041 ,  107-2119-M-110-005 ,  107-2923-M-001-009 ,  108-2112-M-001-048 ,  108-2112-M-001-051 ,  108-2923-M-001-002 ,  109-2112-M-001-025 ,  109-2124-M-001-005 ,  109-2923-M-001-001 ,  110-2112-M-003-007-MY2 ,  110-2112-M-001-033 ,  110-2124-M-001-007 , and  110-2923-M-001-001 ); the  Ministry of Education (MoE) of Taiwan Yushan Young Scholar Program ; the  Physics Division, National Center for Theoretical Sciences of Taiwan ; the  National Aeronautics and Space Administration  ( NASA, Fermi Guest Investigator  grant  80NSSC20K1567 ,  NASA   Astrophysics Theory Program  grant  80NSSC20K0527 ,  NASA   NuSTAR award   80NSSC20K0645 ); the  National Institute of Natural Sciences (NINS) of Japan ; the  National Key Research and Development Program of China  (grant  2016YFA0400704 ,  2016YFA0400702 ); the  National Science Foundation (NSF , grants  AST-0096454 ,  AST-0352953 ,  AST-0521233 ,  AST-0705062 ,  AST-0905844 ,  AST-0922984 ,  AST-1126433 ,  AST-1140030 ,  DGE-1144085 ,  AST-1207704 ,  AST-1207730 ,  AST-1207752 ,  MRI-1228509 ,  OPP-1248097 ,  AST-1310896 ,  AST-1555365 ,  AST-1615796 ,  AST-1715061 ,  AST-1716327 ,  AST-1903847 ,  AST-2034306 ); the  Natural Science Foundation of China  (grants  11650110427 ,  10625314 ,  11721303 ,  11725312 ,  11933007 ,  11991052 ,  11991053 );  NWO  grant No.
OCENW. KLEIN. 113 ; a fellowship of  China Postdoctoral Science Foundation  ( 2020M671266 ); the  Natural Sciences and Engineering Research Council of Canada (NSERC , including a  Discovery Grant  and the  NSERC   Alexander Graham Bell Canada Graduate Scholarships-Doctoral Program ); the  National Youth Thousand Talents Program of China ; the  National Research Foundation of Korea  (the Global PhD Fellowship Grant: grants  NRF-2015H1A2A1033752 ,  2015-R1D1A1A01056807 ; the  Korea Research Fellowship Program :  NRF-2015H1D3A1066561 ,  Basic Research Support Grant   2019R1F1A1059721 ); the  Netherlands Organization for Scientific Research (NWO)   VICI award  (grant  639. 043. 513 ) and  Spinoza Prize SPI   78-409 ; the  New Scientific Frontiers  with  Precision Radio Interferometry Fellowship  awarded by the  South African Radio Astronomy Observatory (SARAO) , which is a facility of the  National Research Foundation (NRF) , an agency of the  Department of Science and Technology (DST) of South Africa ; the  Onsala Space Observatory (OSO)  national infrastructure, for the provisioning of its facilities/observational support (OSO receives funding through the  Swedish Research Council  under grant  2017-00648 ); the  Perimeter Institute for Theoretical Physics  (research at the  Perimeter Institute  is supported by the  Government of Canada  through the  Department of Innovation, Science and Economic Development  and by the  Province of Ontario  through the  Ministry of Research, Innovation and Science) ; the  Spanish Ministerio de Ciencia e Innovacin  (grants  PGC2018-098915-B-C21 ,  AYA2016-80889-P ,  PID2019-108995GB-C21 ,  PID2020-117404GB-C21 ); the  State Agency for Research of the Spanish MCIU through the "Center of Excellence Severo Ochoa " award for the  Instituto de Astrofsica de Andaluca  ( SEV-2017-0709 ); the  Toray Science Foundation ; the  Consejera de Economa, Conocimiento ,  Empresas y Universidad of the Junta de Andaluca  (grant  P18-FR-1769 ), the  Consejo Superior de Investigaciones Cientficas  (grant  2019AEP112 ); the  M2FINDERS  project, which has received funding by the  European Research Council (ERC)  under the European Unions  Horizon 2020 Research and Innovation Programme  (grant agreement No.
101018682 ); the  US Department of Energy (USDOE)  through the  Los Alamos National Laboratory  (operated by  Triad National Security, LLC , for the  National Nuclear Security Administration of the USDOE  (contract  89233218CNA000001 ); the European Unions  Horizon 2020 research and innovation program  under grant agreement No. 730562  RadioNet;  Shanghai Pilot Program for Basic Research, Chinese Academy of Science ,  Shanghai Branch  ( JCYJ-SHFY-2021-013 );  ALMA North America Development Fund ; the  Academia Sinica ; Chandra  DD7-18089X  and  TM6-17006X ; and the  GenT Program (Generalitat Valenciana)  Project  CIDEGENT/2018/021. This work used the  Extreme Science and Engineering Discovery Environment (XSEDE) , supported by  NSF  grant  ACI-1548562 , and CyVerse, supported by  NSF  grants  DBI-0735191 ,  DBI-1265383 , and  DBI-1743442.
0 and  Siyuan Mark -I at  Shanghai Jiao Tong University. The computer resources of the  Finnish IT Center for Science (CSC)  and the  Finnish Computing Competence Infrastructure (FCCI)  project are acknowledged. Junghwan Oh  was supported by the  Basic Science Research Program  through the  National Research Foundation of Korea (NRF)  funded by the  Ministry of Education  ( NRF-2021R1A6A3A01086420 ). We thank  Martin Shepherd  for the addition of extra features in the Difmap software that were used for the CLEAN imaging results presented in this paper. The computing cluster of Shanghai VLBI correlator supported by the  Special Fund for Astronomy  from the  Ministry of Finance  in China is acknowledged. The LMT is a project operated by the  Instituto Nacional de Astrofisica, Optica, y Electronica (Mexico)  and the  University of Massachusetts at Amherst (USA).
The IRAM 30 m telescope on Pico Veleta, Spain, is operated by IRAM and supported by  CNRS  ( Centre National de la Recherche Scientifique, France ),  MPG  ( Max-Planck-Gesellschaft, Germany ), and  IGN (Instituto Geogrfico Nacional, Spain). The SMT is operated by the  Arizona Radio Observatory , a part of the  Steward Observatory of the University of Arizona , with financial support of operations from the  State of Arizona  and financial support for instrumentation development from the  NSF. Support for SPT participation in the EHT is provided by the  National Science Foundation  through award  OPP-1852617  to the  University of Chicago. Partial support is also provided by the  Kavli Institute of Cosmological Physics at the University of Chicago. The SPT hydrogen maser was provided on loan from the GLT, courtesy of ASIAA.
The Internet of Things (IoT) is a fact today where a high number of nodes are used for various applications. From small home networks to large-scale networks, the aim is the same: transmitting data from the sensors to the base station. However, these data are susceptible to different factors that may affect the collected data efficiency or the network functioning, and therefore the desired quality of service (QoS). In this context, one of the main issues requiring more research and adapted solutions is the outlier detection problem. The challenge is to detect outliers and classify them as either errors to be ignored, or important events requiring actions to prevent further service degradation. In this paper, we propose a comprehensive literature review of recent outlier detection techniques used in the IoTs context.
Introduction The IoT can be seen as a collection of technologies that work together and provide Internet-based services and applications. With the aid of electronic devices connected to physical objects, heterogeneous sensors can collect data for process control  [1]. The IoT involves many resource-constrained nodes that are deployed to sense, collect, and transfer data to a base station or a data center. In such a way, the appropriate decision can be taken in a controlled environment. IoT is utilized in various sectors, including environment monitoring, medical, agriculture, disaster warning, smart city, and manufacturing. However, the collected data in the real world are vulnerable to outliers, which are data samples that are considerably different from normal data. In IoT, an outlier may occur due to inherent characteristics of the sensor device itself, or because of the harsh environment where the nodes are deployed.
Data quality could be affected by a sensor failure, noise, malfunction, missing or duplicated data values, etc. These outliers can also concern the exchanged network information relative to the network operation (i. e. , sending and receiving messages). So, it is essential to detect these outliers before any in-network processing, like fusion or aggregated data, to limit the propagation of these incorrect data and conserve the collected data's effectiveness to perform a suitable decision. To ensure network QoS, the IoTs must detect the outliers and take the necessary action to eliminate the degradation of the service. Moreover, IoT comes with the aspect of a high number of heterogeneous embedded devices that generate big data, which makes the outlier detection more complex against such massive data  [2].
It provides real-time information and warnings to control water pollution  [9]. In the financial domain, it is crucial to monitor the audit logs for the financial transactions stored in a database, then to report and verify the abnormal behaviour detected in data  [10]. Therefore, studying the outlier detection approaches of sensing data in IoTs is both scientifically and practically valuable. The aim of this work is to provide a comprehensive literature review of previous and current outlier detection techniques used in IoT. Our methodology to establish this review is as follows: • We select papers between 2010 and 2021 (80 papers) from the leading scientific online databases (MDPI, Science Direct, SpringerLink, IEEE Explorer, ACM Digital Library, etc. ). Moreover, we consider some required and original papers before 2010 (13 papers).
Outlier Detection Fundamentals in IoT Context In this section, we will provide some definitions of an outlier in wireless sensor networks (WSN) and IoT and differentiate the three types of outlier sources. Then, we will talk about the outlier detection approach, and finally, we will talk about how we can evaluate outlier detection techniques. Firstly, let us give some common definitions of an outlier. In WSNs, outliers can be defined as: "measurements that vary considerably from the typical pattern of sensed data"  [11]. In the context of IoTs "an outlier is data value that is considerably different from other data values, or does not correspond to the predicted normal behaviour, or conforms well to a defined abnormal behaviour"  [12]. Another definition is provided by  [13]  where spatial-temporal outliers are defined as "a spatial-temporal data value whose non-spatial attribute values are considerably different from those of other spatially and temporally referenced data values in its spatial or/and temporal neighbours".
Supervised and unsupervised detection The supervised techniques characterize all the outliers and the non-outliers depending on the model built using the pre-labelled data in the training phase. These pre-labelled data mean the need for specific predefined data in the training phase to create a normal or abnormal data model before detecting outliers. These supervised techniques may be helpful for intrusion detection applications. However, it is crucial to notice that these pre-labelled data are not easy to obtain or unavailable in many real-life IoT applications. In addition to the enormous online streaming data coming, the supervised model may be valid for specific time instances, but invalid for other time instances. The unsupervised techniques do not require pre-labelled data, but they utilize specialized metrics to identify outliers.
For example, some techniques use distance measures, while others use a familiar statistical distribution model. e. Multivariate Data and univariate In the univariate data, the data value has a single attribute, whereas, in the multivariate data, the sensor data value has many attributes. Thus, we have an outlier if the data value has abnormal values in its attributes. The outlier detection technique must be able to deal with the multivariate data and consider their correlations. It must also consider the complexity of computations of these multivariate data. Otherwise, the technique will not be suitable for IoT applications. f.
Sensor Failure Detection Strategies in IoT As described in Section 2. 1, an outlier can be due to a faulty node. There are three main strategies to identify sensor failure, as showed in Figure  4:  • Network-level strategy: with this strategy, the aim is to monitor the data at the network level to detect any failure in the sensors. The Markov models can be used to characterize the behaviour of the normal and the abnormal sensors, where the sensors can monitor each other in the IoT situation. • Homogeneous strategy: the aim here is to use many spatially correlated similar sensors to detect any abnormal sensor behaviours. We use the Auto-Regressive Integrated Moving Average (ARIMA) time-series model  [18] , which compares the data value measured by the sensor with predicted measured. • Heterogeneous strategy: the aim here is to group different sensors to detect the sensor's failure by classifying the sensors data values output.
Evaluation of Outlier Detection Techniques in IoTs Outlier detection for IoT aims to detect outliers and differentiate with high detection rate (DR) and low false alarm rate (FAR) between errors and events. The efficient outlier detection technique must respect the IoT resource constraint devices, such as the computational cost, memory usage, and communication overhead. The DR is the percentage of abnormal data values that are considered as outliers correctly. The FAR is the percentage of normal data values that are considered as outliers incorrectly. Another way to measure the efficiency of an outlier detection technique is to use the receiver operating characteristic (ROC) curve  [16, 17, 19] , which is a two-dimensional graph that represents the trade-off between the DR and the FAR. Figure  5  shows a generic illustration of how to interpret this curve by different fictional approaches.
Outlier Detection Challenges in IoT Context Developing an effective outlier detection technique is more difficult due to the IoT context and the sensor data nature. The IoT ecosystem encompasses the traditional WSN, which allows more analysis and processing of the collected data  [20]. However, as shown in Figure  6 , there are numerous challenges encountered while designing an outlier detection system for IoT. Due to the nature of sensor devices, they are subjected to failure, missing, duplicated data or may be affected by noise. So, it is crucial to detect outliers and distinguish between errors, events, and malicious attacks, while not losing important events by considering them as errors.
They classified the existing techniques into four categories: clustering, classification, dimensionality reduction, and hybrid algorithms. In  [26] , the authors proposed a global classification review about the progress of outlier detection techniques. They provide the performance, the pros, the cons, and the challenges which face these techniques. However, it is a general survey about outlier detection techniques and not specified for IoTs or WSN. In addition to the classification of the outlier detection techniques, authors in [27] proposed a methodology for outlier detection in IoT through a systematic analysis of the data set based on five stages. The first stage is to define a scenario of generating a labelled dataset using mathematical modelling on a real IoT system. Then, in the second stage, the aim is to capture data packets and inject anomalous packets by modifying some of them.