dataesr/scientific-paragraphs-categorization huggingface dataset:
Dataset Generation One of the challenges of this research is to efficiently and effectively transform (often large) remotely-sensed images into a specific format usable by a Deep Learning approach. Careful choices have to be made in order to have the correct balance of quantity versus quality of examples in the dataset used to train a CNN-based approach, as this is of direct influence to the performance of the resulting model  (Kumar & Manash 2019). In order to correctly train the model, a sufficient amount of images that contain one or more examples of a class of object, are needed. Therefore, the intent is to have as many examples and images as possible, although each has to be sufficiently different from the others to prevent overfitting -where the model memorises the examples instead of learning to generalise  (Goodfellow, Bengio & Courville 2016).
The original dataset consists of geospatial data, i. e. , a vector shape file per object class (with the bounding boxes or annotations encoded as X,Y coordinates) and the LiDAR images in GeoTIFF format (see  Verschoof-van der Vaart et al. 2020). In order to use the data for the developed approach, it first has to be converted into the format of the Microsoft COCO dataset  (Lin et al. 2014). Therefore, the bounding boxes need to be associated with the correct LiDAR image and subsequently the coordinates of the bounding boxes need to be converted from real-world coordinates to image-based (pixel) coordinates. This information can then be used to make input images, i. e. , cropped subtiles of the large LiDAR images. Therefore, a script written in Python (Van Rossum & Drake 2009) was developed to automate this task (see Figure  3 ).
In this script, the gdalinfo command line utility (GDAL/OGR contributors 2021) is used to retrieve the coordinates of the LiDAR images. Subsequently, the bounding box of every object in the dataset is associated with one of these images, based on their coordinates and it is checked whether the entire bounding box of the object falls inside the image. Using this newly acquired information, a new database is constructed (Figure  3 ), called CSV of polygons positions, which contains all necessary information (i. e. , the position of the bounding box, the type of object, and the associated image) to construct the final dataset in Microsoft COCO format. To create cropped LiDAR subtiles, the coordinates of the bounding boxes are first converted to pixel coordinates (between 0 and the length/width of the LiDAR image), using a simple rescaling and translation (Equation  1 ).
( ) ( ) - - - min max min min max min x x f x x x x x x ¢ ¢ ¢ = + (1) (where x min , x max are the original range of the value, and   is the desired range) Subsequently, the LiDAR images are cropped into subtiles of N pixels, with N being a fixed value, centred around a particular archaeological object. However, having an object in the exact centre of every subtile can introduce a bias in the object detection model. Therefore, a random jitter value on the x and y coordinates, taken uniformly between [100, 100] is added to the coordinates of every object before cropping. This guarantees that the subtile is not perfectly centred around an object, reducing bias and creating a more difficult example. After cropping, the script iterates through every entry in the CSV of polygons positions database and if an object falls completely within one of the cropped images, the annotation is added to a.
txt file associated with that particular image. The resulting dataset, in Microsoft COCO format  (Lin et al. 2014) , consists of the cropped subtiles, which are all accompanied with. txt files containing the pixel coordinates of the bounding box(es) and class information. Various elements of the dataset were varied, i. e. , different image input sizes, the re-use of objects, and the addition of negative examples, to generate a dataset that resulted in the best performance of our approach. Increasing the input size of the images can reduce the information loss that occurs during the downsampling in the CNN. As archaeological objects are often cluttered together, the cropped subtiles generally contain multiple examples. It is an option to only use images with unique examples, i. e. , exclude images with objects that already appeared in another image.
However, it might be more beneficial to re-use objects, which already appeared in other images, to increase variability in the dataset. Finally, the use of negative examples, i. e. , images with only background and no objects of interest, can teach the CNN the specific texture features of the (nonarchaeological) background, which are generally more intricate in remotely-sensed images than in natural images  (Gao et al. 2019 ). This will potentially reduce the number of False Positives. The final version of the dataset, which produced the best overall performance, consists of images of 1000 × 1000 pixels, incorporates images with re-used objects, and includes circa 20% negatives examples -created by cropping images centred around random coordinates. The total dataset contains around 4500 images, which were randomly split 80/20 in a training and test dataset (Table  1 ).
e. , 'Out Of Memory' errors). Presumably, an even larger input size (e. g. , 608 × 608) could even further improve detection performance at the cost of increased memory needed. Another issue with (archaeological) objects in LiDAR images is the fact that these are often occluded, fragmented, and/or do not appear with the same clarity as examples seen in other (remotely-sensed) imagery. This means that we need to construct and train a network that is robust to changes in context and occlusion. Fortunately, research has focused on developing data augmentation techniques  (Goodfellow, Bengio & Courville 2016 ) that attempt to solve these issues. These augmentation techniques generally 'mix' different training images by covering parts of one another. This makes it harder for the model to correctly detect objects but also improves performance.
An example of this type of augmentation is CutMix  (Yun et al. 2019). Another example, Mosaic  (Bochkovskiy, Wang & Liao 2020) , creates a new image out of four, by creating a sort of 'mosaic', where the four images can take varying portions of the new image. Both augmentation techniques were used in the modified YOLOv4 model. A further modification, regularisation, was implemented to deal with possible overfitting (and increases complexity of the CNN). Overfitting is a common issue in Deep Learning, especially when small training datasets are used, as is often the case in archaeological automated detection (see Section 1. 1). Regularisation usually involves 'dropping' random activations in a CNN and train without those connections, through a technique known as DropOut  (Srivastava et al. 2014).
POST-PROCESSING As archaeology relies heavily on GIS, such as QGIS (QGIS Development Team 2017), to manage, analyse, and visualise archaeological information  (Gillings, Hacıigüzeller & Lock 2020) , it was deemed important to develop a post-processing step to integrate the detection results into this kind of software, i. e. , turn the results of the object detection into geospatial data (see also  Verschoof-van der Vaart & Lambers 2019). A simple solution to incorporate the results into GIS is to convert the detections into a CSV file, which can be imported as a shape file in most GIS software. Therefore, the detectionsToCSV script was developed (see Figure  4 ), which allows users to convert the results of the object detection into three CSV databases, one for each class. The detection results per subtile, i.
e. , a class, confidence score, and location of the bounding box, are saved in a. txt file. The script first connects the subtiles to the LiDAR images from which they originally derive. Then it uses the gdalinfo command line utility (GDAL/OGR contributors 2021) to get the coordinates of the main image, and based on that compute the real-world coordinates of the bounding boxes, which are then converted into Well Know Text (WKT) format and written into the corresponding database. Additional post-processing is performed to remove erroneous detections, in particular for Celtic fields, consisting of bounding boxes with a very small width and height but with a high confidence score. These detections are simply discarded by thresholding the ratio between the width and height of all bounding boxes.
IMPLEMENTATION DETAILS The two versions of the YOLOv4 detection framework (vanilla and modified; see Table  2 ) were trained and tested on the developed datasets (see Table  1 ). Both models use a Darknet53 CNN as backbone network, pretrained on the Microsoft COCO dataset  (Lin et al. 2014) , and are implemented in the C programming language. The models were transfer-learned  (Razavian et al. 2014 ) on the training dataset for 10,000 epochs on a Nvidia GTX 1660 GPU. Training times varied between 20 to 25 hours. Subsequently, the fine-tuned models were used to detect archaeological objects in the test dataset of cropped subtiles of 1000 × 1000 pixels (see Figure  4 ). To test the speed of our model, the interference or testing time was measured. Testing with the YOLOv4 framework went at a rate of 18 images per second.
In comparison the testing time of Faster R-CNN on comparable LiDAR images was about one second per image (Verschoofvan der Vaart & Lambers 2019). This high speed of the former is due to YOLO's design as a fast and accurate object detector. While speed is not the focus of this research, a shorter inference time (without a loss in performance) is beneficial, especially when considering the exceptional size of most remotely-sensed datasets. The results of the object detection were post-processed and evaluated (Figure  5 ).
GENERAL PERFORMANCE Table  3  shows the results, averaged on all object classes, on the testing dataset of the vanilla and modified YOLOv4 models (see Table  5  for a breakdown of the metrics per class). As can be seen, both models perform fairly well (Figure  5 ) with an average F1-score of 0. 69 and 0. 76 respectively. Both models obtain a high mAP@50 score between 0. 75-0. 87, and a good average IoU between 0. 45-0. 58. The results show that the modified version of YOLOv4 significantly outperforms the vanilla version of YOLOv4 (Table  3 ). A more detailed analysis shows that of the different modifications implemented, the highest performance boost is gained by using the CutMix and Mosaic data augmentations. However, the use of these augmentation techniques does have a negative impact on the training time of the model, which increases up to 25%.
Interestingly, while most classes are predicted accurately (see Table  4 ), there seems to be confusion between barrows and Celtic fields, i. e. , barrows are often (circa 10%) confused with Celtic fields. It might be due to a class imbalance present in the dataset, i. e. , there are more Celtic Field examples than there are of any other class. Another possible explanation could be the similarities in appearance in LiDAR data between barrows and small fragments of Celtic fields, i. e. , both are relatively strong positive elevations. Also the LiDAR data, especially in this format, might not retain enough information to allow the model to correctly distinguish between the two classes. On the other hand, this inter-class confusion does not occur in other research on the detection of both these classes, using the same LiDAR data but a different Deep Learning architecture (see  Verschoof-van der Vaart et al.
In the research of Verschoof-van der Vaart et al. (  2020 ) the same LiDAR dataset and classes were used as in this research. However, the division of the data into a training, validation, and test dataset was different, and therefore the density of archaeological objects in the datasets, i. e. , a different ratio of positive and negative examples (see also  Soroush et al. 2020) , varied with this research. The same is the case for the test dataset used by  Trier, Reksten & Løseth (2021, Table 7) , in which all images contained at least one archaeological object of interest, i. e. , the test dataset contained only positive examples. As noted by the researchers, this results in a bias towards less False Positives. Such differences in the density of objects in the test dataset can be of significant influence on the performance of a Deep Learning approach  (Verschoofvan der Vaart et al.
76 versus 0. 75-0. 82 F1-score), while the performance on charcoal kilns is comparable (circa 0. 85 F1-score). However, we again urge a certain caution in comparing these results as the testing methodology of  Trier, Reksten & Løseth (2021)  might not accurately represent the performance of the model 'in the wild' (see  Verschoof-van der Vaart et al. 2020). Indeed on when tested on another dataset, where negative examples were included, Trier's model obtained a 14% Accuracy on barrows, with 86% False Positives. This highlights the need for a robust and unbiased testing methodology when the performance of those models (see Verschoof-van der Vaart in press). This comparison does show the benefit of employing state-of-the-art approaches and the implementation of additional modifications geared towards a specific task.
Within such a collaboration archaeologists can define the exact problems and goals of a project, ensure the implementation of the results in digital archaeological practice (e. g. , GIS), and can provide domain knowledge that is generally out of reach of computer scientists  (Bennett, Cowley & De Laet 2014). This can inform and help the latter to build more accurate detection methods, which are well-adapted to the specific task. In turn, computer scientists can use their up-to-date knowledge of data science and software engineering to streamline the process of model creation and training, and pinpoint and explain more accurately the issues that can arise during this process (see  Trier, Cowley & Waldeland 2019). Especially in Deep Learning, a specialist is needed to correctly create a dataset, select the best CNN architecture, evaluation methods and metrics, and training regime.
CONCLUSIONS In this paper, an automated detection workflow, based on the state-of-the-art YOLOv4 detection framework, was presented. The results of the experimental evaluation show that this model is able to accurately detect and categorise archaeological objects in challenging data, such as remotely-sensed LiDAR data. While multiple issues remain to be solved, such as hard to differentiate objects (e. g. , roundabouts and barrows), the results are promising. The Python scripts to generate the datasets (see Section 2. 2. 1) and configuration files for the modified YOLOv4 model (see Section 3) are available on the Github repository: https://github. com/epsln/YOLOv4LiDAR. The archaeological data used in this study are available from the authors upon reasonable request. It is also shown that modifying an existing model to a specific task, using the latest developments in Deep Learning object detection, can increase performance over an 'vanilla' off-the-shelf method.
FURTHER IMPROVEMENTS One promising path for future research is the use of multiple types of data to train and test automated approaches. For example, one could use LiDAR and aerial photography data to train a more robust model. Combining such data might require modifications to the detection model and the use of modified file formats, e. g. , images containing four instead of three channels: one for the LiDAR grayscale values, and three for the RGB values of the photographs. Alternatively, multiple models or more elaborate feature fusion could be used. Another potential avenue for research is the use of semantic segmentation models that would be able to segment objects on a pixel level (see for instance  Bundzel et al. 2020; Kazimi, Thiemann & Sester 2019). While bounding boxes are adequate for the localisation of objects, segmentation might offer additional information concerning their size, coverage, etc.
Tools for uncertainty quantification The ensemble forecasting approach allows us to distinguish the part of uncertainty that comes from different sources. A specific source of uncertainty can be tracked through the modeling chain and along lead times  (Boelee et al. , 2019). Following  Thiboult et al. (2016) , we created four ensemble prediction systems that differ in how hydrometeorological uncertainties are quantified (Table  1 ). Ensembles were built from the HOOPLA (HydrOlOgical Prediction LAboratory; https://github. com/AntoineThiboult/HOOPLA, last access: 23 January 2021) modular framework  (Thiboult et al. , 2018) , an automatic software that allows us to carry out model calibration and obtain hydrological simulations and forecasts at different time steps. They consider uncertainty coming from system A, forcing, system B, forcing and initial conditions, system C, forcing and model structure, and system D, forcing, initial conditions, and model structure.
Precipitation forecast uncertainty: ensemble forecast Forcing uncertainty is characterized by precipitation ensemble forecasts issued by the European Centre for Medium-Range Weather Forecasts (ECMWF)  (Buizza and Palmer, 1995; Buizza et al. , 2008) , downloaded through the TIGGE database for the 2011-2016 period. In this study, the set consists of 50 exchangeable members issued at 12:00 UTC, for a maximum forecast horizon of 7 d at a 6 h time step. The database was originally provided with a 0. 25 • spatial resolution and was reduced to 0. 1 • by bilinear interpolation  (Gaborit et al. , 2013)  to ensure that several grid points are situated within each catchment boundary. Additionally, when downscaling, we considered the contribution of the points close to the catchment boundaries, which allows us to have a better description of the meteorological conditions of the catchments and implicitly account for position uncertainty  (Thiboult et al.
Acknowledgements. Funding for this work was provided to the first and second authors by FloodNet, an  NSERC Canadian Strategic Network  (grant no. NETGP 451456-13 ) and by  NSERC   Discovery Grant   RGPIN-2020-04286. The authors thank the  Direction d'Expertise Hydrique du Québec  for providing hydrometeorological data and ECMWF for maintaining the TIGGE data portal and providing free access to archived meteorological ensemble forecasts. Special thanks go to  Michael Scheuerer  for sharing the R codes of the CSGD processor and offering many insights into CSGD. The authors also would like to thank the reviewers for their thoughtful and constructive comments towards improving the manuscript. Financial support. This research has been supported by the  Natural Sciences and Engineering Research Council of Canada  (grant nos.
This study aims at introducing two conservative thermodynamic variables (moist-air entropy potential temperature and total water content) into a one-dimensional variational data assimilation system (1D-Var) to demonstrate the benefit for future operational assimilation schemes. This system is assessed using microwave brightness temperatures from a ground-based radiometer installed during the field campaign SOFGO3D dedicated to fog forecast improvement. An underlying objective is to ease the specification of background error covariance matrices that are currently highly dependent on weather conditions making difficult the optimal retrievals of cloud and thermodynamic properties during fog conditions. Background error covariance matrices for these new conservative variables have thus been computed by an ensemble approach based on the French convective scale model AROME, for both all-weather and fog conditions.
The second experimental framework, where the control vector is z = ((θ s ) a , q t ), corresponding to the conservative variables, is named "EXP". The numerical aspects of the 1D-Var minimisation are kept the same as in "REF". Then, a set of reference matrices B x (T, q v ) has been estimated every hour using the Ensemble Data Assimilation (EDA) system of the AROME model on 9 February 2020. These matrices were obtained by computing statistics from a set of 25 members providing 3 h forecasts for a subset of 5000 points randomly selected in the AROME domain to obtain a sufficiently large statistical sample. Then, matrices associated with fog areas, and noted B x (T, q v ) f og , were computed every hour by applying a fog mask (defined by areas where q l is above 10 -6 kg kg -1 for the three lowest model levels), in order to select only model grid points for which fog is forecast in the majority of the 25 AROME members.
Contrary to the classical variables, which are rather independent in clearsky atmospheres as previously shown by  Ménétrier and Montmerle (2011) , the B z matrix reflects the physical link between the two new variables as diagnosed from the AROME model. The correlations are positive with and without fog mask. This result shows that the matrix B z (θ s , q t ) is less sensitive to fog conditions than the B x matrix. It could therefore be possible to compute a B z (θ s , q t ) matrix without any profile selection criteria that would be nevertheless suitable for fog situations, resulting in a more robust estimate. This result is key for 1D-Var retrievals which are commonly used in the community of ground-based remote sensing instruments to provide databases of vertical profiles for the scientific community.
The use of the θ s variable in data assimilation can thus be considered as an additional and natural step that continues these series of studies. The 1D-Var system has been adapted to consider these new quantities as control variables. Since the radiative transfer model needs profiles of temperature, water vapour and cloud liquid water for the simulation of T B, an adjustment process has been defined to obtain these quantities from (θ s ) a and q t. The adjoint version of this conversion has been developed for an efficient estimation of the gradient of the cost-function. Dedicated background error covariance matrices have been estimated from the Ensemble Data Assimilation system of AROME. We firstly demonstrated that the matrices for the new variables are less dependent on the meteorological situation (all-weather conditions vs.
g. using a temperature threshold criteria). Indeed, the variable (θ s ) a can easily be generalized to the case of the ice phase and mixed phases by taking advantage of the general definition of θ s and (θ s ) 1 , where L vap q l is simply replaced by L vap q l + L sub q i. Code and data availability. The numerical code of the radiative transfer model RTTOV-gb together with the associated resources (coefficient files) can be downloaded from http://cetemps. aquila. infn. it/rttovgb/rttovgb. html and from https://nwp-saf. eumetsat. int/site/software/ rttov-gb/. The 1D-Var software has been adapted from the NWP SAF 1D-Var provided here: https://nwp-saf. eumetsat. int/site/software/.
Our volcanological dataset includes: (1) the location of past vents, (2) the mapping of the main fault structures, and (3) the eruption styles of past events, obtained from critical analysis of the literature and/or inferred from volcanic deposits and morphological features observed remotely and in the field. To illustrate the effects of considering the expected eruption style in the construction of vent opening maps, we focus on the analysis of small-scale pyroclastic density currents derived from phreatomagmatic activity or from low-intensity magmatic volcanism. For the numerical simulation of these phenomena we adopted the recently developed branching energy cone model by using the program ECMapProb. Our results show that the implementation of thematic vent opening maps can produce significantly different hazard levels from those estimated with traditional, non-thematic maps.