dataesr/scientific-paragraphs-categorization huggingface dataset:
Intergraph Intergraph offers a novel approach to exploring digital humanities corpora by means of an iterative search and discovery workflow. The demonstrator has been designed to meet all user requirements specified in the previous section. Written in javascript, Intergraph runs in a web browser and communicates with a node. js server which queries the data from a Neo4j database. The front-end client renders the graphs using the Three. js graphics library. Given the size of the BLIZAAR dataset, an overall visualization of the corpus is neither suitable nor desirable for exploration. Instead, users are rather interested in creating and inspecting subnetworks with entities relevant to their current research. The main idea of Intergraph is therefore to begin the exploration from one or more known start nodes.
2005). Intergraph applies the citation chaining principle to documents and the entities mentioned in With regard to data imperfections, one of the most frequently encountered flaws with entity disambiguation in the CVCE dataset are entity duplicates, i. e. multiple recognized entities where in reality only one was actually mentioned. For example, named entity recognition yielded three separate nodes for "East Berlin". If the user wants to consider these three nodes as one in order to create an ego-network, it is possible to multi-select a number of nodes and to query "All entities co-appearing with a given entity (union)", meaning that the result will be the list of nodes co-appearing with at least one of the selected nodes. The user can then define a unique group node for "East Berlin" and draw a meaningful graph (user requirement 6).
User test results To assess the usability of Intergraph, two user evaluation sessions were held. The first to validate the user stories and to provide feedback on an earlier version of Intergraph, the second to evaluate the current version. The tests were conducted with a group of four scholars, all of whom were former CVCE employees. The selection criteria were familiarity with the underlying corpus on European integration and with the application of digital tools and methods. These criteria were applied to ensure that users could turn their attention to the interaction with the prototype with only minimal reminders of the underlying data model and content, and also that they were qualified to judge the pertinence of the output. For the second evaluation, evaluators were asked to submit in advance a list of five persons or intuitions and optionally up to three time periods between 1945 and 2009 they wished to explore in greater depth under the precondition that they had expert knowledge about their presence in the CVCE corpus.
To compensate for the relative unfamiliarity with the software they were to test, the scholars were reminded of BLIZAAR's research objectives and received a circa 10-min demonstration of the Intergraph core functionalities. After the presentation of the platform, they were invited to use Intergraph for themselves and to begin their session with an elementary keyword search for an entity they knew was mentioned in the corpus. From this starting point, they were free to perform more synoptic tasks, such as finding relevant collections and resources, searching for co-appearing entities and comparing their corresponding networks, in order to obtain a comprehensive overview of how the investigated element is represented, positioned and linked in the corpus. Throughout the session, users were encouraged to continuously verbalize their train of thoughts and actions, in line with the thinking aloud approach  (Boren and Ramey 2000).
Users require significant time to learn how to operate especially prototype software and to adjust long established research workflows. Most crucially however, automated data extraction and visualization-based exploration need to be understood well and their potential can only be explored by extensive experimentation and adoption to a new domain which was beyond the scope of this project. Given the exploratory nature of Intergraph, future work will concentrate on additional ways of suggesting related nodes and creating pertinent graphs out of existing ones, for example by applying recommendation algorithms  (Bobadilla et al. 2013). The multilayer character of the data should be leveraged by adding other types of interlayer edges, such as those indicating the "mentions" relationship between collections and entities.
Intergraph's functionality to merge duplicate nodes is a step in this direction. Another promising direction is the critical assessment of the composition of datasets and the potential of visualization to reveal its inherent biases. These requirements present interesting challenges and opportunities for the design of innovative tools for visual analytics. Finally, it is important to observe that the used data model is highly generic. Entities identified in collections of time-stamped resources are likely to be found in a huge number of digital corpora. It is intended to open the existing platform to other datasets, and for this purpose the authors are currently working on extending Intergraph and its functionalities to two additional historical databases: • Regesta Imperii (Kuczera 2019) (http://www.
regesta-imperii. de): a collection of records of the Roman-German kings and emperors, as well as of the popes from the Middle Ages; • Romans 1by1  (Varga 2017 ) (http://romans1by1. com): a population database registering the people which are attested in Greek and Roman epigraphic sources. By this means, we hope that Intergraph can evolve into a valuable visualization and exploration tool for many scholars working in the field of digital humanities.
a b s t r a c t Proteins are natural modular objects usually composed of several domains, each domain bearing a specific function that is mediated through its surface, which is accessible to vicinal molecules. This draws attention to an understudied characteristic of protein structures: surface, that is mostly unexploited by protein structure comparison methods. In the present work, we evaluated the performance of six shape comparison methods, among which three are based on machine learning, to distinguish between 588 multi-domain proteins and to recreate the evolutionary relationships at the protein and species levels of the SCOPe database. The six groups that participated in the challenge submitted a total of 15 sets of results. We observed that the performance of all the methods significantly decreases at the species level, suggesting that shape-only protein comparison is challenging for closely related proteins.
The SHape REtrieval Challenges (SHREC) are time-restricted challenges, which aim to evaluate the effectiveness of 3D-shape retrieval algorithms. Typically, a challenge is opened by proposing a dataset of related shapes to participants while retaining the class membership. In the SHape REtrieval Challenge 2020 (SHREC2020) track on multi-domain protein shapes, the participants had 7 weeks from the dataset publication to send their results with a description of the methods used to generate the results (see  Section 4 ). This SHREC2020 track on multi-domain protein shapes evaluates the current ability of shape comparison methods proposed by 6 different groups to tackle the protein surface comparison problem. The participants were asked to send their results in the form of matrices containing all-to-all dissimilarity scores.
The results were analyzed and the overall retrieval performances are presented here. The dataset includes 588 proteins consisting of two domains (the functional units of the proteins); only the corresponding triangulated meshes of their solvent-excluded surfaces (SES)  [2]  were provided as input to the participants. We then evaluated the retrieval performance of each method to retrieve the evolutionary relationships between orthologous proteins (proteins that have the same function in different organisms), and to retrieve the different conformations of an individual protein. Here, we present the results of all the participants and methods, and briefly discuss the trade-off between performance in retrieval and computational cost of each method.
Dataset Proteins are linear polymers (the so-called protein chains) made of amino-acid residues (up to several hundreds), which fold into a specific, well-defined 3D structure. Furthermore, many proteins need to form a complex of several chains to become functional. For instance, the human heamoglobin requires two α-globin and two β-globin chains to be fully functional. Domains define the functional units of the proteins, and are usually associated with a specific function and/or interaction; it is thus commonplace for two proteins to share one domain while their other respective domains differ. This characteristic led to the development of databases classifying proteins according to both their structure and the functions of their domains. The SHREC2020 track on multi-domain protein shapes dataset is devoted to the analysis of protein shapes generated from protein chains that comprise two domains.
Dataset creation The SCOPe database  [3] [4] [5]  organizes the protein domains according to their structural (in the 2 top levels of the SCOPe tree) and evolutionary (for the 4 bottom levels) relationships. Protein domains in the SCOPe database originate from Protein Data Bank (PDB) experimental structures  [6]  , and are characterized by their PDBId and chainId, allowing for filtering based on these parameters. From all entries implemented in the SCOPe tree (excluding entries from the 'Artifacts' and 'Low resolution protein structures' classes), we kept only the entries from X-ray crystallography PDB structures composed of two domains. When multiple copies of the same protein chain was present in the same PDB structure, we only kept one of those copies to limit redundancies. Finally, all proteins were required to have at least one orthologous protein, and classes with less than 10 members were discarded.
off files were provided to the participants on the track website ( http://shrec2020. drugdesign. fr ). At the end of the track, the ground truths were published online as well. As the participants were not provided some important details about the dataset creation (two-domains proteins only, protonation and SES calculation parameters,. ), the reverse engineering of the memberships from the surfaces (. off files) would require to compare all the PDB entries of the SCOPe database to the dataset. While feasible in principle, this approach in practice would be difficult to carry out. Compared to other known protein shapes datasets, this dataset is exclusively composed by two-domains proteins while only onedomain proteins were included in  [10, 11]. As multi-domain proteins are commonplace at the cellular level, the impact of additional domains on the protein shape retrieval performances need to be evaluated.
3D Characterization of prOteins by Deep analysis of 2D view SEQuence (CODSEQ) -Halim Benhabiles, Karim Hammoudi, Adnane Cabani, Feryal Windal & Mahmoud Melkemi The CODSEQ method is a deep learning based framework for indexing proteins. The approach consists of capturing surface details of the 3D proteins under the form of a set of 2D views. To this end, a classification architecture was tailored by exploiting a transfer learning strategy to extract relevant features from the considered views of proteins. The SHREC 2018 dataset  [10]  was exploited for the training stage as these protein surfaces share similar silhouettes with the protein surfaces proposed in the current contest (connected stretched shapes). The protein surfaces from SHREC 2019 dataset have not been exploited since their shapes visually seemed too different (com pact shapes).
Only protein classes represented by at least 19 (18 train and 1 test) different proteins (dominant classes) were considered, resulting in selecting 79 classes among 107 classes of the SHREC 2018 dataset. Noteworthy, the training stage has only been performed at the protein level since the SHREC 2018 dataset does not include the species level. The train/test methodology has been adopted thanks to the availability of the ground-truths. Descriptors calculation Extraction of protein 2D views. In this stage, 3D meshes representing protein surfaces are simplified using the Quadric Error Metric Decimation  [18]. By this way, the number of facets of each 3D mesh has been reduced to 20,0 0 0 facets (about 10% of the original surface) while maintaining the surface details. In the considered coordinate system related to the processing, each 3D mesh has its own position and size.
More precisely, an Inception ResNet architecture  [16]  pre-trained on ImageNet dataset has been fine-tuned and trained on SHREC2018 dataset  [10]  in order to learn 79 protein classes. The trained model is used to return a 512-dimensional feature vector for each 2D view by getting the output of a penultimate layer (the one before the classes output). Protein characterization based on a sequence of 2D views (Compact 3D descriptor). A bidirectional LSTM-RNN architecture  [17]  was trained on the SHREC2018 dataset to learn, as in the previous stage, 79 classes. The architecture has been fed with sequences of feature vectors obtained in the previous stage; each sequence is composed of 9 feature vectors associated to 9 views of a given protein. One of the strengths of RNN-based models is their ability to analyze data sequences (sequences of views in the current case) while keeping the most significant views to characterize protein classes.
Dissimilarity distance calculation & runtimes Dissimilarity matrices were generated by calculating the Euclidean distance between each pair of proteins using their associated 1024-dimensional feature vectors. Two matrices have been generated based on two training runs performed in the previous stage, namely CODSEQ1 with 0. 96 and 0. 18 of accuracy and loss, respectively, and CODSEQ2 with 0. 94 and 0. 14 of accuracy and loss, respectively. This framework has been developed in Python 3. 7. 6 using Open3D 0. 8. 0. 0, OpenCV 4. 2. 0 and Keras 2. 2. 4-tf on a TensorFlow-GPU 2. 1. 0 backend. The experiments have been conducted on an Intel Core i7-6700HQ CPU@2. 60 GHz with 32 GB of memory and NVIDIA GeForce GTX 1070 GPU with 24 GB of memory. The running times in seconds of each stage performed on CPU are reported in Table  2  for one protein.
In total, the number of the input features of the feature comparator is 2 * 121 (or 1771 for 3DZM) + 2 * (250 + 200 + 150) + 2 * 4 + 2 = 1452 features (4752 features for 3DZM). The first term is the 3DZDs of order 20 (n = 20), which is a 121 element vector, of the two protein shapes. The third term, 2 * 4 comes from the four comparison metrics applied to two representations of the two proteins, the original 3DZDs (or 3DZMs) and encodings, which concatenate the output of the input layer and the three intermediate layers of the encoder. The feature comparator outputs a score between 0 and 1 using a sigmoid activation function, which is the probability that the two proteins are in the same protein level classification in the SCOPe database  [3]. The feature comparator network has an input layer of a 1452-dimensional feature vector, two intermediate layer of 100 and 50 neurons respectively, and one output neuron.
The network was trained on a dataset of 247,521 protein structures from the SCOPe 2. 07 database. Proteins in Class I (Artifacts) were not included. To augment data for training the network for 3DZM, which is not rotation invariant, each protein was rotated with different random orientations. For each protein,EDTSurf  [7]  was used to generate the solvent excluded surface, which was then fed into the EM-Surfer pipeline  [22]  to compute 3DZM and 3DZD. The network was trained to correctly distinguish proteins in the same protein level category in SCOPe from the rest.
Wave Kernel Signature and SGWS based Shape Descriptor for Protein Retrieval (WKS/SGWS) -Yuxu Peng, Yu-Kun Lai & Paul L. Rosin To reach robust and improved performance, a hybrid spectral feature descriptor is used which combines the benefits of features of wave kernel signature (WKS)  [23]  and spectral graph wavelet transform (SGWS)  [24]. WKS is an isometric invariant descriptor that has been found to be effective for deformable 3D shape retrieval such as those of the dataset; in contrast to HKS, it focusses on the high-frequency information. SGWS is a generalisation of HK S and WK S, and provides a multiresolution local descriptor that is compact, easy to compute and combines the advantages of both band-pass and low-pass filters. Data pre-processing Meshes were simplified to reduce the number of faces to approximately 60 0 0 using Qslim  [18]  which provides an effective compromise between the fastest algorithms and the highest-quality algorithms to reduce computing time.
Then the mesh is fixed using the open source software meshfix  [25]  to convert a raw digitized polygon mesh to a clean mesh where all the occurrences of a specific set of "defects" are corrected. Holes, selfintersections, degenerate and non-manifold elements are all replaced with valid configurations. WKS descriptor calculation The WKS feature vectors are computed from the eigenvalues and the eigenvectors of each protein mesh. Then the vocabulary is calculated using an improved vectorbased k-means over 10% feature vectors of all proteins  [26]. Finally, the WKS descriptor is normalized for the bag-of-features (BoF) for each protein using hard vector quantization. The lengths of the WKS feature vector and the descriptor are 50 and 10 0 0 respectively. SGWS descriptor calculation The process of the SGWS descriptor is similar to that of the WKS descriptor.
Dissimilarity matrices computation & runtimes The procedure for model comparison consists in concatenating the MAPT histograms computed at the different scales and measuring distances between shapes by evaluating the Jeffrey divergence  [28]  of the corresponding concatenated vectors. The estimation of the descriptors took 112 s on average for the run HAPT1, 47 s on average for the run HAPT2, and 17 s on average for the run HAPT3 on a laptop with an Intel CoreTM i7-9750H CPU running Ubuntu Linux 18. 04. The descriptor comparison time was negligible.
Graph-based CNN (GraphCNN) for 3D shape retrieval -Stelios Mylonas, Apostolos Axenopoulos & Petros Daras Following the recent tendency of addressing many scientific tasks by exploiting the existing vast amount of data, a data-driven approach was applied for the problem of 3D protein shape retrieval. Based on the fact that the provided input proteins are in the form of triangulated meshes, a transfer learning approach was applied. A method originally designed for the task of 3D point cloud classification and segmentation was adapted to the needs of the protein shape retrieval task, and trained on a relevant dataset of protein 3D point clouds in order to learn appropriate features (descriptors) for the representation of 3D molecular shapes. Descriptors calculation SPH3D-GCN  [29]  , a graph-based CNN method equipped with a novel spherical convolution kernel, was employed as it has achieved state-of-the-art results on numerous computer vision tasks.
At the end of each decoder, a pooling operation is applied, which reduces gradually the number of considered points. In Encoder4, a modified spherical convolution is applied in order to obtain a global representation of the whole point cloud. Finally, the output features of all the four encoders are concatenated and imported to a sequence of three fully connected (FC) layers. The proposed scheme was trained on the dataset from last years competition (SHREC2019  [11]  ), which comprises 5298 structures from 17 protein classes. The network was trained on a classification task aiming to assign each structure to each corresponding protein class. During the feature extraction step, the FC layers were dropped and the concatenated output of the four encoders were used as descriptors. Therefore, for each previously unseen input, a feature vector of 832 values is extracted.
Augmented Point Pair Feature Descriptor (APPFD) The Augmented Point Pair Feature Descriptor (APPFD) is a 3D shape descriptor, which describes the local geometry around a point, p = [ p x , p y , p z ] or vertex, v = [ v x , v y , v z ] for 3D pointcloud or mesh datasets, respectively. Here, the pointcloud shape representation for this task was used, instead. Computing this descriptor involves the following stages: (i) pointcloud sampling and normals estimation, (ii) keypoint, p k i determination, (iii) local surface region (i. e. , LSP), P i selection, (iv) Augmented Point Pair Feature (APPF) extraction per LSP, and (v) final descriptor computation. The algorithms for stages (iv) and (v) are described in this section, and the reader is referred to the literature in  [31]  for more details on the other stages.
Shape Similarity Measurement Overall, the L 2 or cosine distance metric between FV s are expected to give good approximations of the similarity between shapes in the SHREC2020 Protein dataset. The cosine metric was adpted in Eq. (  8 ) , due to a slightly more improvement over the L 2 metric. cos ( FV 1 , FV 2 ) = FV 1 • FV 2 FV 1 FV 2 = n i =1 FV 1 i FV 2 i n i =1 ( FV 1 i ) 2 n i =1 ( FV 2 i ) 2 (8).
Protein level At the protein level, the 588 shapes were gathered into 7 classes of multi-domain orthologous proteins; among each class, all members share at least one common domain while the other domains are different. This feature allows the methods for having nearest-neighbor (NN) over the whole dataset ranging from 66. 6 up to 98. 5%, meaning that for a given query, the shape comparison algorithms were able to retrieve a query of the same class in at least two thirds of the cases. The performances vary largely between methods, as three methods (3DZM/D, WKS/SGWS and HAPPS1-3) achieve successful nearest-neighbor retrieval in more than 97. 5% of the cases. For all the methods, the performances decrease as we consider further results, but the performance drops are different for each method, as illustrated by the differences in the precision-recall (PR) curves profiles ( Fig.
Species level At the species level, the dataset contains 26 classes. Within each protein class, the species classes were evolutionary-related proteins. For instance, the protein class "T-cell antigen receptor" has two species child classes, the human and the murine orthologs, which  Therefore, and similarly to the last two SHREC tracks on protein shape retrieval  [10, 11]  , the performances of the shape comparison methods are significantly lowered at the species level compared to the protein level. The NN values range from 43. 8 to 84. 4%, while no Machine learning approaches have recently been applied to protein surface patches  [12]. In the present track, three of the six methods make use of learning approaches in their workflows. Their performances are comparable to the performances of the other methods, showing no improvement in the retrieval results.
Interestingly, the two learning-based methods trained on SHREC2018 and SHREC2019 tracks on protein shape retrieval (CODSEQ and GraphCNN, respectively) were outperformed by the learning-based method trained on the whole SCOPe dataset (3DZ). This latter training dataset encompasses multi-domains protein shapes while datasets from the last two SHREC tracks on protein shape retrieval only encompass one-domain protein shapes. Particularly, the CODSEQ method showed lower performances on the SHREC2020 multi-domain protein dataset compared to their training dataset (see Section 4. 1 ) for which the CODSEQ approach shows relatively high performance. This may originate from the specificites of these two sets (one-domain versus multi-domain proteins shapes); besides, the CODSEQ was only trained at the protein level.