dataesr/scientific-paragraphs-categorization huggingface dataset:
Relatively lower values of DTB were mainly located in hilly and mountainous areas, such as Sichuan province, Chongqing city, Guangxi province, and the mountainous areas of Northeast China. The spatial pattern of the DTB map of this study is similar to those of the maps produced by Pelletier et al. 10  and Shangguan et al. 11. In addition to predictions by ensemble model, another dataset including median predictions and uncertainty was produced. Estimations of three percentiles (0. 1 (Fig. 7     The uncertainty map of the prediction of DTB is shown in Fig. 8. The uncertainty in the predictions in part depends on the density of sampling  38. In our study, it was low in deserts, sandy areas, the North China Plain, and the Northeast China Plain, where the topography is relatively simple and sampling was relatively dense.
Code availability All code used to generate the predictions is available from the Github (https://github. com/yanfp/DTB100China). The map products were calculated using R version 3. 4. 1 and packages "randomForest (4. 6-14)", "xgboost (0. 71. 2)", "quantregForest (1. 3-7)", "sp (1. 3-1)", "rgdal (1. 2-16)", "raster (2. 7-15)", "pkgmaker (0. 27)", "lattice (0. 20-35)", "plotKML (0. 5-8)", "hexbin (1. 27. 2)" and "ggplot (3. 1. 0)".
The first task of this automation lies in the possibility of identifying objects of interest during the excavation. In order to make this recognition of archaeological entities possible it is necessary to ask when they are easily identifiable: in the excavation report. The hypothesis formulated here is that excavation reports can be used as a source for creating learning data sets of neural networks dedicated to the recognition of archaeological objects on site. Two important steps in automating the integration of archaeological data are presented here, the extraction of images and their semantics from excavation reports and the learning process of a neural network for the recognition of archaeological entities at the site of their discovery. The extraction of images and the identification of what they contain allows to enrich neural network learning datasets.
INTRODUCTION The practice of archeology generates knowledge. They are accumulated, ordered and recorded for each iteration that a search represents. Throughout this process, from the updating to the publication of scientific works, the stages of documentation follow one after the other, methodically and painstakingly. Today, the data created is intended to be found in digital media to facilitate the dissemination and reuse of the knowledge created. How to allow an effective integration of archaeological knowledge in a contemporary work and dissemination media? The question of capturing information arises as soon as artefacts are discovered on the excavation site. Digital integration at the earliest contradicts existing methods. The organization of the data entry is still mainly in paper form, while the cameras are also present and already record certain data in digital format.
The study presented here is preliminary to the use of videos during the excavation to automate a large number of data entry steps as well as the production, ultimately, of 3D models. To use a video made on an archaeological site, you must be able to offer automatic extraction methods. The information obtained would correspond to the list of artifacts present and the ranges of the video on which they are found. This information would allow the selection of video ranges and the production of a 3D model with the extracted images. Object tracking can also be used to prepare pre-filled data sheets. Do images from an archaeological site allow us to use a neural network? Can we easily retrieve learning datasets for these tools? and finally, can we analyze a sequence of images from excavations and draw up a chronology of the objects? Before trying to integrate video cameras on an archaeological site it is important to be able to answer these first questions.
3D documentation of archaeology This question already arises when the time comes to acquire data sets to produce 3D models by photogrammetry during excavation. 3D acquisition replaces drawing during excavation to document and record entire archaeological sites. The condition of a site can be preserved regardless of its evolution over time  (Alby et al. , 2013). The photogrammetric method saves time at the excavation site and reduces time-consuming activities while retaining quality information for the drawing  (Alby, 2015). It is even possible to give on-site historical stakeholders the opportunity to manage the creation of their dataset themselves and to link them to data already produced according to reliable photogrammetric principles (Šedina, J. et al 2016). The practice of excavation according to archaeologists' methods hybridizes with technological solutions and provides flexibility in 3D documentation to take into account the hazards of the excavation and saves time on site  (Alby et al.
The quality of the data integrated in an automatic learning system is the basis of our project. The use of excavation reports is, for us, the basis of our approach to guarantee the quality of the information we want to produce during this project. Figure  3  shows the relationship between the two stages of the study presented here. The automation of recognition is closely related to the automation of dataset generation. Figure  4  shows the next phase of this study. Showing that neural networks can be used for the recognition of archaeological artifacts is interesting if it is possible to automate the creation of neural networks for a maximum number of artifacts in order to extract a maximum of information from the videos of archaeological excavations.
EXCAVATION REPORT ANALYSIS The need to create learning datasets on a field of knowledge generates the need for labelled images as close as possible to the picture data produced by archaeologists. The excavation report is a very good support of knowledge of the archaeological excavation. Each excavation has its report. It is a concentrate of knowledge in the field applied specifically to the excavation. It tends to contain exhaustive information specifically related to an archaeological excavation. Even if initially only images and semantic descriptive information are important, it should be kept in mind that the excavation report can provide much more information. There are several levels of information extraction sought: pictures and their semantics and then everything related to the objects identified on the pictures.
Pictures captions extraction The purpose of this part is to produce a training data set for a neural network. On the one hand, it requires relevant pictures that are extracted from the excavation report, but on the other hand, it also requires that they are labelled, which in such a document corresponds to the legend. The extraction of legends for the pictures is also a means of filtering the obtained pictures because a large part of the irrelevant pictures described above are not associated with legends. Figure  5  shows a page from an excavation report  (Lefis, 2011)  and the relationship between the pictures and their captions. The extracted pictures are extracted with their layout parameters, which makes it possible to know how they occupy the page, alone or grouped together for example.
NEURAL NETWORKS AND ARCHEOLOGY This preliminary study should make it possible to validate the use of neural networks as a means of extracting archaeological knowledge from documents and to be able to process videos from archaeological sites. The ability to identify objects on pictures from archaeological excavations must therefore be evaluated. As the two studies were conducted in parallel, a dataset was compiled based on pictures of an archaeological site. To simulate what an excavation report can produce as a set of labelled pictures, the learning dataset for this part was made on the basis of pictures taken at a given archaeological site. A neural network, once trained and functional, is a black box. It is not possible to know the precise role of each of its constituent elements. In order to implement the convolutional neural network, the analogy with the perception of vertebrates was used.
The amount of input data must be pre-processed to reduce the resource requirements for such processing. Then it is a sequence of repeated convolution and pooling (Figure  2 ) that makes it possible to both reduce the volume of data and adapt it to the shape of the neural network input while maintaining the specificity of the information to be processed. The goal of neural network training is to specialize the neural network. In the end, it must perform only one specific task on data that it will never have processed. The specialization of the neural network is linked to the training that is applied to it on the basis of the chosen data set. The resulting performance must allow it to be effective when applied to data other than the training set. The risk is that it only works correctly on the dataset in question.
It must be made robust. This is the role of the dropout layer. This step aims to randomly add error in the behavior of some of the components of the network and thus give it the ability to adapt to the different data of the training dataset. The implementation of a neural network for a specific use does not require the creation of a virgin tool. Indeed it has been proven that a pre-trained network even with pictures that have nothing to do with the study is faster to train. This transfer learning technique is presented in  (Shin et al. 2016 ). After various tests, the chosen architecture allows to recognize precisely the corresponding objects. The pictures are sub-sampled before being processed by the neural network. It remains to adjust the parameters to optimize the results. Learning is done by successive modifications of the network parameters by retropropagation.
CONCLUSION The acquisition of pictures by video on an archaeological site can be envisaged as a support to assist in the collection of archaeological data, on the condition that effective treatments are put in place to extract artefacts from these video sequences. The same videos can also provide the opportunity to generate 3D models from identified sequences. The evolution of image processing techniques shows that neural networks are very efficient for object recognition and our study shows that archaeological artifacts can be recognized using these tools. In order to do so, they need to be trained, which implies generating specific training datasets for each type of artefact. The excavation reports, as concentrates of archaeological knowledge, contain the information sought: pictures and legends.
The study presented also shows that the automatic extraction of artifact pictures and their captions is possible despite the structure of the report files. In order to be able to create sufficiently large datasets for neural networks, it is necessary to be able to do this on a large number of excavation reports. The use of a neural network has proven to be effective on pictures from archaeological excavations. The results obtained are reliable. The management of artifacts of the same type during excavation was also presented, and initial results are promising for the monitoring of objects over time.
The selection of a single "best performing" pharmacophore model was also pursued by means of clustering  (Sohn et al. , 2013; Spyrakis et al. , 2015) , providing better VS results than "classical" x-ray crystallographic derived structure-based pharmacophore models. However, to determine the "best performing" model requires datasets of known active and inactive compounds to assess the performance of the models. In cases with new targets during early hit finding stages, this information may be yet not be available and prioritizing pharmacophore models for VS campaigns can be challenging. To overcome the need to select one unique representative set of pharmacophore models,  Wieder et al. (2017)  developed the "Common Hits Approach" (CHA) in which multiple 3D pharmacophore models derived from a MD simulation were partitioned according to their feature compositions and used for subsequent VS runs.
Protein-Ligand Complex Preparation Two crystal structures of the human glucokinase in complex with activators were downloaded from the RCSB PDB databank  (Berman, 2000) , with PDB IDs 1v4s  (Kamata et al. , 2004 ) and 4no7  (Petit et al. , 2011). The sequences of the proteins were aligned and the amino acid subsequently renumbered, using the RCSB PDB comparison tool  (Prlić et al. , 2010)  and the jFATCAT_flexible algorithm  (Ye and Godzik, 2003). Amino acids 92-99 were not present in the 4no7 complex. Since they did not impact the protein stability during the simulations and no interactions with the ligand in the 1v4s system could be observed, they were not modeled. A table containing the alignment block is available in Supplementary Figure  1. The Maestro software  (Schrodinger, 2010)  was used to remove water molecules, add hydrogens and minimize the structures.
Library Generation Compounds with experimental activities measured on human glucokinase were taken from the ChEMBL database  (Gaulton et al. , 2017). In total, 756 unique molecules with activity toward the target protein expressed in EC50 were extracted. This set was split based on the activity value threshold of 1. 5 µM, resulting in 601 molecules labeled as actives and 155 as decoys. The KNIME Analytics platform  (Berthold et al. , 2009)  was used in combination with the InteLigand Expert KNIME LigandScout Diversity Picker node (InteLigand Expert KNIME Extensions) to extract the 20,000 diverse molecules from the ChEMBL library based on extended connectivity fingerprint (ECFP) similarity, also labeled as decoys. Finally, a library for virtual screening was calculated using the idbgen algorithm in LigandScout 4.
4 Expert (LigandScout 4. 4 Expert). The procedure included the generation of a maximum of 25 conformations for each of the 20,756 molecules using the icon Fast settings  (Poli et al. , 2018). The active molecules were clustered in 5 groups based on ECFP similarity. Examples of the molecules present in each cluster is shown in Supplementary Table  2. The ligands from the x-ray derived structures PDB codes 1v4s and 4no7 were in cluster numbers 4, and 2, respectively.
Pharmacophore Generation and Virtual Screening Structure-based pharmacophore models were generated for each frame output from the MD simulations using LigandScout 4. 4 Expert  (Wolber and Langer, 2005). Models generated by LigandScout support the following chemical feature types: hydrophobic interactions, hydrogen bonds donor/acceptor, positive/negative ionizable area, aromatic ring and halogen bond donor features. In addition, pharmacophore models from the xray derived crystallographic structures of 1v4s and 4no7 were created. Water molecules were discarded before the generation of the models. The LigandScout activity profiling KNIME node was used to perform all virtual screening runs of models against the dedicated database of 20,756 molecules. Receiver operating characteristic (ROC) curves were generated for the virtual screening runs and the performance of the models was assessed by the calculation of area under the curve (AUC) values at specific percentages of the number of screened database molecules.
RESULTS AND DISCUSSION Case Study: Glucokinase The hexokinase IV, or glucokinase (GK) is an isoenzyme responsible for glucose phosphorylation  (Beck and Miller, 2013). The concentration of glucose in the plasma determines the conformational switch of GK between its active and inactive states. The glucose level impact on GK activity makes this enzyme act as a sensor responsible for the glucose homeostasis in the human body  (Bell and Polonsky, 2001). Therefore, GK has been a primary target for the development of antidiabetic drugs  (Kamata et al. , 2004; Osbak et al. , 2009; Petit et al. , 2011). Two crystallographic structures of the active conformation of GK with bound activators have been selected for this study (PDB codes: 1v4s and 4no7). The structures of the ligands, their position in the binding pocket and the pharmacophore features derived from the independent x-ray experiments are depicted in Figure  4.
, 2017) , we used a filtering criteria on the unique pharmacophore model appearance count in order to reduce the noise by removing pharmacophore models which appeared only once during the simulation. In this study, we investigated the impact of several values for the filtering criteria, discarding models appearing <2 frames up to <1% of the number of frames, in order to both reduce the noise and improve the readability of the graph. Source code for the generation and visualization of the graphs is available online (Source code for the HGPM Implementation, 2020). Source code for the generation and processing of LigandScout pharmacophore models has been excluded due to intellectual property reasons. Output data for an interactive demonstration of the Hierarchical Graph representation of Pharmacophore Models generated from the first MD run of 4no7 can also be accessed online (Demonstration of HGPM, 2020).
Analysis of the Unique Feature Vectors To visualize the composition of the pharmacophore models that were obtained for each performed MD simulation of the systems, the individual partitioning of the unique observed features is represented as Venn diagrams shown in Figure  5. For the 1v4s system, 37 out of 81 (46%) of all unique observed features are present in every of the 3 MDs runs if the appearance count filtering criteria is set to 2 frames or more. This ratio stays similar when the appearance count criteria is set to 10 frames or more, FIGURE 3 | Visualization of the Hierarchical graph representation of pharmacophores models derived from a molecular dynamics simulation of human glucokinase in complex with an activator (PDB code: 1v4s). The feature vector is represented on the top of the graph and each box represents a unique pharmacophore feature.
The node associated with the hydrogen bond donor and acceptor features that interact with Arginine 63 (CF node) is located on the third column of the graph, since it possesses a low number of features and is therefore not specialized. The PDB and HF nodes were used to perform VS runs against the active and decoy database as described in the Methods section. In addition, a selection of all pharmacophore models which represent a superset of the HF and CF nodes were used for a consensus screening run. The CHA  (Wieder et al. , 2017) , that is a consensus approach considering all observed pharmacophores was also carried out. A summary of the VS results can be found in Table  2. We observed that the PDB pharmacophore for 1v4s did not retrieve hits during the VS, which might be due to the number of defined and specific directional hydrogen bond vectors defining its specific binding mode, with 5 of the 7 pharmacophore features containing defined directions for hydrogen bonding.
The 4 other pharmacophore selections for this system performed well, with area under the curve (AUC) values above 0. 96 at 1% of the number of database molecules. The HF pharmacophore model performed as well as the other models regarding the AUC at 1% but slightly below for the other and retrieved 65 hits out of 20,756 molecules. The selection of all superset models of the CF node provided the most stable and best results regarding the AUC, even outperforming the CHA. Those good results can be linked to the high amount of pharmacophore models which were used for screening, 236 for the CF+ selection and 346 for the CHA. Lastly, the set of all pharmacophore models which are supersets of the FIGURE 5 | Venn diagrams showing the unique pharmacophore feature similarities between the MD simulation runs performed for the 1v4s and 4no7 systems, respectively.
It is interesting FIGURE 6 | Pie charts of the unique pharmacophore feature composition of the feature vectors generated from all 3 MD simulations, based on the applied appearance count filtering criterion for both the 1v4s and 4no7 systems. Hydrophobic interactions are colored yellow, aromatic interactions blue, hydrogen bond donors green, hydrogen bond acceptors red and the halogen bond donors gray. to point out that the "Arbitrary" selected node and the HF node are of disparate feature composition, but still deliver high AUC values. To scrutinize this observation, we looked at the structural clusters of the active molecules retrieved by these two approaches, as detailed in Supplementary Figure  5. We observed that for a hitlist truncated at 2,076 molecules (10% of the database), the HF node retrieves a consequent number of molecules from all 5 clusters, when the arbitrary selected node selection mostly retrieves molecules from the cluster numbers 2, 3, and 5.
The feature vectors of the highlighted models are shown above of the corresponding hierarchical graphs. features involved in correctly distinguishing active and inactive molecules, all pharmacophore models associated with the feature vectors of the hierarchical graphs were used for VS. Then, each node was colored according to its AUC value at 10% of the database. For both systems, the corresponding results are shown in Figure  8. In the shown hierarchical graphs, the greener the node the closer its AUC value is to 1, and the redder its color becomes, the closer its AUC value is to 0. The "Artificial" nodes were not used for VS runs and are colored gray. The graph of the 1v4s system shows a clear separation of colors. All nodes in the area of the HF node, as well as all its superset nodes are depicted in green.
Despite these common interactions, no "Observed" pharmacophore models were present in both the 1v4s and 4no7 MD simulation runs. We can nonetheless clearly observe a distinction between the two colored clusters, depicting the initial node affiliation. The top half of the graph with cyan colored nodes represent all pharmacophore models from the 1v4s system, where we can recognize a single branch leading to the specialized pharmacophore model of the PDB structure. For the bottom half of the graph with the orange nodes from the 4no7 system, we observe a more homogeneous spacing. The two branches previously observed are not easily distinguishable. It can be pointed out that the 4no7 PDB node is located near the nodes originating from 1v4s, and the custom selection node tends to fork more to the bottom part of the graph as more detailed FIGURE 8 | The hierarchical graph of pharmacophore models derived from human glucokinase PDB codes of the 1v4s system is shown on the top, and of the 4no7 system on the bottom.
The nodes are colored according to their AUC values at 10% of the database molecules. The greener the node, the closer is its AUC value to 1, and the redder the closer to 0. Nodes with "Artificial" pharmacophore models are colored gray. The nodes corresponding to the pharmacophore models obtained from the PDB structure, from the highest frequency of appearance and with the two common features between the systems are labeled accordingly. Associated feature vectors are shown above the corresponding graphs. on Supplementary Figure  6. As the feature vector differs from the individual graphs, the pharmacophore selection for the CHA and for the subsets of pharmacophore models with the same feature as the CF nodes were different than previously and their virtual screening results details for this graph are presented on Supplementary Table  4.