dataesr/scientific-paragraphs-categorization huggingface dataset:
Crowd-based Visual Annotation The collection of unintentional user feedback from crowds for visual analysis has been approached in different ways. The most popular approach is to design a collaborative effort aimed at a high quality annotation of a dataset. For example, LabelMe  [11]  has collected a large amount of local annotations by asking volunteers to draw a polygon around the object. Solutions in this family normally vary depending on the incentive, which can go from an abstract call to help science, to a very accurate pricing policy. These systems tend to produce high-quality segmentations, but may result in a tedious and boring tasks for the user. The introduction of a crowd-based effort for image analysis has also been explored in the past. Previous works  [3] , have used CAPTCHA human verification systems to actually annotate images.
These systems combine data with available ground truth with other data aiming at being analyzed. In these cases, users will unintentionally annotate the unlabeled images while the verification step is based on the dataset with available ground truth. However, existing works have focused on textual annotation applied at the image global scale. In this work, we have focused on the user feedback based on clicks to generate accurate object segmentations at a local scale. A popular strategy for obtaining crowd-sourced annotations is through on-line Games With a Purpose (GWAP), which exploit the high motivational levels achieved by games in such a way that the user interaction produces some type of valuable outcome. The first game used for object detection at a local scale was Peekaboom  [14]. The game is played in pairs, where one player reveals parts of an image so that the other can guess the textual label representing the object being discovered.
CROWDSOURCED OBJECT SEGMEN-TATION This work focuses on the potential of Ask'nSeek traces to help generating pixel-wise segmentation of the relevant objects within an image. In particular, the goal of this work is to estimate the minimum amount and type of user interaction necessary to achieve segmentation results of reasonable quality, comparable to the results of state-of-the-art segmentation algorithms. This study is based on traces obtained by Ask'nSeek. These traces are combined with a ranked list of object hypotheses to estimate an accurate segmentation of the object. We also propose a "traces simulator" to enable assessing the proposed solution on a large dataset.
g. , good alignment with image edges), and a second one that ranks these regions according to mid-level features, learned from a training dataset of segmented objects. As a result, the algorithm generates a ranked list of feasible segments within an image sorted according to their probability of being 'object-like'. CPMC has been adopted in our work because it was a basic block in the pipeline that obtained the best performance in Pascal VOC2009 and VOC2010  [4]. The ranked list of object hypothesis is compared with the collected foreground and background traces to select the best candidate. The proposed algorithm selects the first region in the ranked list of candidates with the maximum amount of coherent clicks. A click is considered coherent with respect to an object candidate when its label (fg/bg) matches its corresponding location in the image describing the object candidate.
Simulated Traces The assessment of object segmentation techniques requires an extensive experimentation on large image datasets. On the other hand, one of the most limiting factors for research on crowdsourcing is the limitation on accessing large pools of users. These antagonistic scenarios can be managed by developing a simulation process to generate as many traces as desired, and validate the goodness of such simulator by collecting crowdsourcing data from a reduced portion of the whole dataset. This same approach has been adopted in previous works assessing algorithms for interactive segmentation  [10] ,  [6]. The goal of the simulator is to generate foreground and background clicks on the image. The simulator is supposed to have access to a segmentation ground truth of each test object, which codes the semantic interpretation that a human would infer by looking at it.
Protocol We deployed a web-based version of the Ask'nSeek game, where players first encounter a tutorial video explaining them how to play the game. Players are then asked to log in, and wait to be paired with another player. The pairing algorithm is random. For every game, we record the login of both the master and the seeker, along with the image they are playing over. The region hidden by the master is persisted, as well as every indication given by the master and every try of the seeker. At the end of the game, the final score and the remaining time (if any) are stored. A total of 50 users (17 females and 33 males) played 255 games on 24 different images. The images were selected from the PASCAL VOC dataset. The players' age range between 18 and 62.
Object Segmentation The potential of the presented framework in terms of object segmentation was assessed by comparing the obtained results with a ground truth. The Pascal VOC benchmark  [4]  provides a large collection of images that contain objects that belong to a diversity of semantic classes. We focused on the dataset from year 2010, to be consistent with the one used in the paper describing CPMC object candidates  [2]. Every Pascal VOC image is provided with its corresponding pixel-wise ground truth, which allows the assessment of the object masks generated with our approach. The results are provided according to the Jaccard Index, sometimes referred as overlap. Two types of experiments have been run. The first one aims at validating the simulator of traces by comparing its results to the ones obtained from real traces on a reduced dataset, while the second one uses this simulator to predict the behaviour of the system on a large volume on images.
Real vs Simulated Clicks The simulated traces described in Section 3. 3 have been validated on a reduced set of 10 objects extracted from the VOC2010 train and validation dataset. These 10 objects were selected as the ones with more clicks available on the game logs. The images have been actually used for playing Ask'nSeek and the collected traces contain at least 15 clicks for each of the considered objects. The final goal of this experiment is to assess that the simulator provides data that will generate similar segmentation results than real traces. The real traces generated a mean Jaccard index for the 10 considered objects of 0. 4878. The comparison has been performed according to the Jaccard indexes obtained for the real traces and an average of 5 simulated runs. The first feature to assess from the simulator corresponds to the location of the clicks.
Amount of clicks The main question addressed in this paper is an estimation of the amount of necessary clicks on a GWAP like Ask'nSeek to achieve a certain quality. Once the clicks simulator has been validated with real traces, it is possible to run extensive experimentation on a large dataset. The goal of these experiments is to study the evolution of the segmentation quality as the amount of clicks increase, as well as assess the influence that the semantic class of the object in the expected accuracy for a given amount of clicks. This experiment was run on the validation dataset of the Pascal VOC2010, which contains 964 images with object from 20 different classes. Figure  4  presents the averaged Jaccard Index obtained for each of the 20 considered classes in the 5 runs with respect to the amount of simulated clicks.
The plotted Jaccard index is an averaged value, which offers an estimation of the necessary clicks to obtain a certain quality. As expected, this averaged curve presents a growing trend with the amount of clicks, with a faster increase during the first 15 clicks. The last simulated value, for which 30 clicks have been used, corresponds to an average Jaccard Index of 0. 5836. This value outperforms the winners of the Pascal VOC 2012 challenge who obtained 0. 473, but on a different test dataset with the same 20 classes. However, a detailed study of the curves by class points shows that each of them has a particular behavior with faster or slower growths. The dependency of the results from the object classes is pictured in Figure  5. The average Jaccard index has been computed individually for each class for four amounts of clicks: 1, 10, 20 and 30.
CONCLUSION In this paper we have shown how we could use game logs from players' interactions in the Ask'nSeek game to feed a segmentation algorithm and improve the quality of state-ofthe-art unsupervised segmentation results. We have also presented how to design and validate a simulator to generate such game logs. Our experiments point that a total of 20 clicks related to an object will provide an accuracy quality that will not increase significantly on further clicks. However, this is an average estimation, with a high dependency on the object category. In our future work we want to conduct a larger-scale user study in order to obtain results on a full dataset, namely the BSDS dataset which has been used in other interactive segmentation works. In addition, we also plan to introduce additional factors (eg.
In order to get a comprehensive repertoire of foldable domains within whole proteomes, including orphan domains, we developed a novel procedure, called SEG-HCA. From only the information of a single amino acid sequence, SEG-HCA automatically delineates segments possessing high densities in hydrophobic clusters, as defined by Hydrophobic Cluster Analysis (HCA). These hydrophobic clusters mainly correspond to regular secondary structures, which together form structured or foldable regions. Genome-wide analyses revealed that SEG-HCA is opposite of disorder predictors, both addressing distinct structural states. Interestingly, there is however an overlap between the two predictions, including small segments of disordered sequences, which undergo coupled folding and binding. SEG-HCA thus gives access to these specific domains, which are generally poorly represented in domain databases.
Comparison of the whole set of SEG-HCA predictions with the Conserved Domain Database (CDD) also highlighted a wide proportion of predicted large (length. 50 amino acids) segments, which are CDD orphan. These orphan sequences may either correspond to highly divergent members of already known families or belong to new families of domains. Their comprehensive description thus opens new avenues to investigate new functional and/or structural features, which remained so far uncovered. Altogether, the data described here provide new insights into the protein architecture and organization throughout the three kingdoms of life.
The recognition of domain family membership for uncharacterized proteins is often a first step towards the understanding of their biological roles. Information about protein domains is stored in dedicated databases, in the form of profiles or hidden Markov models (HMMs), which are constructed through sequence similarity searches. These profiles and HMMs can be searched for detecting the domain composition of proteins, starting from their amino acid sequences  [5]. By this way, approximately half of the residues of proteomes can be assigned to well-classified domains, such as those stored in the PfamA classification  [2]. The percentage of assigned residues increases when less wellcharacterized domain databases, such as PfamB, are searched. The remaining residues, representing 10-20% of the proteomes and referred to as ''orphan'' domains, do not match any known domains  [2].
The information provided by SEG-HCA can then be compared with that included into structural databases, in order to support the structural meaning of the predictions. It can also be compared with that provided by the NCBI's conserved domain database (CDD)  [24]  in order to highlight «orphan» domains, i. e. predicted globular-like domains that don't match any conserved domain (CD). Finally, this information also merits consideration with regard to protein disorder. Intrinsically disordered proteins (IDPs) or Intrinsically unstructured proteins (IUPs) do not, by themselves, assume any stable 3D structures, under physiological conditions  [25] [26] [27] [28] [29]. IUPs however cover different forms of disorder, from totally unfolded chains (''coil''-like or natively unfolded), to coupled folding and binding (i.
SEG-HCA H2CD predictions are more than the converse of disorder predictions and may be used for defining categories in disorder The prediction of structured regions might be considered as the simple converse of disorder predictions. We thus compared the SEG-HCA predictions to the disorder predictions performed by IUPRED  [30, 31]. IUPRED is an available well-recognized method, which considers interaction energies for predicting stretches of amino acids that should not contribute to stable structures. According to the D 2 P 2 database  [32] , IUPRED provides among the lowest estimation of global percentage disorder. We achieved this comparison according two different, but parallel routes. First, we compared the total number of amino acids predicted in H2CD segments to those predicted as disordered by IUPRED (IUPREDdis).
The developed procedure, combined with disorder predictors, may facilitate the specific identification of small segments that undergo coupled folding and binding. Combined with the analysis of specific domain databases, it also highlights orphan foldable segments, which remain yet uncharacterized. reported in the D 2 P 2 database  [32]. According to the current view  [32, 33] , Eukarya have a higher content in disorder than the two species chosen here for illustrating Bacteria and Archaea (Table  2 ). However, Bacteria and Archaea show wide disorder distribution, with very low level of predicted disorder for some species, such here observed with the A. fulgidus proteome. We show (Fig. 3A  and Table  2 ) that to a large extent, there is a clear relationship between the two predictions, which are opposite.
However, SEG-HCA H2CD predictions are not the simple converse of disorder predictions, as the overlap between the two sets constitutes 13. 8% of the total number of amino acids in the human proteome (Fig. 3A ). Similar overlap percentages are observed for other eukarya (10. 9% (S. cerevisiae), 13. 8% (P. falciparum), Table  2 ). For E. coli and especially for A. fuldigus, the overlaps are much lower (3% and 0. 9%, respectively), but represent similar ratios of the total number of amino acids predicted as disordered by IUPRED. We made several additional statistics, especially in order to clarify the structural meaning of these three distinct datasets (H2CD, IUPREDdis and H2CD > IUPREDdis). We first evaluated the overall match of protein sequences included in the three datasets with PDB information.
Most of the regions (96. 1%) covered by PDB (14. 4% of the total number of amino acids) are included in the H2CD sets (H2CD and H2CD > IUPREDdis), indicating that most of the 3D structures included in PDB well cover H2CD predictions (Fig. 3B  and Table  2 ). However, PDB files may include some disordered regions. Then, in order to select only regions with defined 3D structures, we filtered the PDB assignments for disorder using MobiDB, a recent comprehensive centralized database on different flavors of disorder  [34] , including the well-known DisProt database  [35]. We also considered classes A to F of the SCOP database  [36] , covering globular as well as transmembrane domains. Although these procedures are likely not sufficient to completely remove all disordered regions, the results clearly show that amino acids from PDB-filtered files and SCOP A to F classes are well covered by amino acids in H2CD (97.
4F )). The two complementary interaction domains of mouse CBP and human ACTR, which undergo synergistic folding, were also detected in the H2CD > IUPREDdis set (pdb: 1kbh  [40] ). Some rare examples were also found of small stable globular domains, as illustrated in Fig. 4A. Worth noting is that small sequence segments of the same ET domain structural family  [41] , also falling in this H2CD > IUPREDdis category, behave either as stable domains (BRD4 ET domain, pdb 2jns(A)  [42] ) or as IDP undergoing coupled folding and binding (AF9 ETlike domain 2lm0(A)  [43] ). This indicates that for small domains, the distinction between stable and unstable 3D structures may be tenuous. Of note is that the H2CD > IUPREDdis set principally contains small H2CD (mean length 28 amino acids, versus 159 amino acids for H2CD not covered by IUPREDdis predictions (70% coverage), also see below).
As shown above, known 3D structures in the H2CD > IUPREDdis set include several segments that undergo coupled folding and binding. We thus wondered if the overlap between H2CD and IUPREDdis may actually correspond to regions predominantly predicted by ANCHOR, a predictor of disordered regions that undergo binding transitions during protein-protein   [14, 15]. These form hydrophobic clusters, which mainly correspond to regular secondary structures  [17]. This is illustrated here with the chromo and chromo-shadow domains of the mouse chromobox homolog 1 (CBX1_MOUSE, UniProt P83917), for which the observed secondary and tertiary structures are shown below the HCA plot (pdb identifiers: 1AP0 and 1DZ1)  [66]. Globular domains (boxed), containing approximately one third of hydrophobic amino acids gathered into clusters, are separated by a hinge, which is clearly less hydrophobic.
For the sequence segment included in the hexagon (IKTQQHILPEVPPVENF), the HCP percentage is 47% (8/17). A minimum is identified when this percentage is below 10%. Here, two segments are identified, as a minimum is reached at amino acid 150 (labeled 1). A distance tree is then calculated between hydrophobic clusters. This allows the definition of the limits of regions having high density in hydrophobic clusters (H2CD, after high hydrophobic cluster density), by comparing the different nodes with the segments defined before. Here, the best correspondence is observed between segment 1 and the node including clusters 1 to 16 (labeled 3 brown) and segment 2 and the node including clusters 17 to 21 (labeled 3 green). (C) The predicted H2CD are then compared to the domains assigned by RPS-BLAST from the Conserved Domain Database (CDD).
Comparison with domains assigned from the Conserved Domain Database gives access to orphan sequences We collected for each protein the CDD assignments (as found using RPS-BLAST  [24] ) and discarded multi-domains, as these are already counted with domains. 69348, 7251, 5161, 5497 and 2679 CD were identified in the proteomes of H. sapiens, S. cerevisiae, P. falciparum E. coli and A. fulgidus, respectively (Table  1 ). The number of predicted H2CD is approximately twice higher than the number of CD: 137665 and 18555 for H. sapiens and S. cerevisiae, respectively, whereas it is nearly similar (6745 and 3075) for E. coli and A. fulgidus, respectively (Table  1 ). Interestingly, the number of H2CD (  22939 ) is more than 4 four times higher than the number of CD (5161) for the P. falciparum proteome.
Most (97%) of the CD amino acids (40% of the total number of amino acids) are included in the H2CD set (human proteome H2CD and H2CD > IUPREDdis, Fig. 3D  and Table  2 ). As regards to the highlighted relationship between H2CD and foldable regions (see above), this indicates that CD mainly include foldable domains. The similarity in hydrophobic cluster composition between the CD and H2CD databases, as well as with the SCOP database (classes A to F) further supports their relationship (Fig. S1 ). We wondered whether CD, as assigned from CDD, are well detected and covered by H2CD, as predicted by SEG-HCA (Table  1 , Fig. 5 ). First, we calculated the percentages of CD positions, which are predicted as H2CD (Fig. 5A  and Table  1 ). 75% of the CD have up to 95% of their length covered by H2CD in all the proteomes, except for P.
Lengths of H2CD segments: Emphasis on small, likely foldable segments, which are not described in CDD We calculated the distributions of CD and H2CD lengths (Fig. 6  and Fig. S5 ). Above 50 amino acids, the distributions are quite similar. However, SEG-HCA predicts a lot of small H2CD (length #50 amino acids), which are not observed in the CD distribution. Indeed, 64804 (47%), 7823 (42%) and 7591 (33%) of the H2CD from the H. sapiens, S. cerevisiae and P. falciparum proteomes, respectively, have less than 50 amino acids. By comparison, only 27%, 9% and 31% of CD are small-sized. Our results thus also showed that, in three distinct eukaryotic proteomes, it exists a considerable number of short segments, rich in hydrophobic clusters. An in-depth examination of PDB entries matching these segments revealed that they constitute small stable domains (e.
This information is in particular absent from or poorly represented in domain databases. Some of these small H2CD segments correspond to isolated stable domains (as exemplified in Fig. 4A  and 4B ). However, as observed from several case studies (Fig. 4D  to 4F), small H2CD segments may also be intrinsically disordered and undergo coupled folding and binding. These segments are generally predicted as IUPs or IDPs by current disorder predictors, and the overlap between disorder and H2CD predictions may thus provide a new interesting way to highlight disorder-to-order transitions, which play key roles for molecular recognition. These segments, which are characterized by structural plasticity and make weak and transient binding, play important roles in regulatory and signaling process  [29]. Tools, such as ANCHOR  [45]  and MorFPred  [49] , have been developed for detecting such segments, but are based on different principles.
MoRFPred is based on a machine learning classifier, based on a comprehensive datasets of MoRFs (Molecular Recognition Features,  [49] ), whereas ANCHOR relies on pairwise energy estimation, which is also the basis of the disorder predictor IUPred  [45]. ANCHOR segments are likely to gain stabilizing energy by interacting with a globular partner. As shown here, ANCHOR predictions are especially found in the overlap between IUPRED (a current disorder predictor) and SEG-HCA predictions. SEG-HCA appears thus well adapted to detect such foldable segments, which have been also named protean segments (ProS,  [50] ), as it highlights specific features of their interface, enriched in hydrophobic amino acids  [45, [51] [52] [53]. The binding of some foldable segments, such as a beta-strand of MCAF1 (Fig.
A ''folding propensity'' may thus be deduced from the consideration of small H2CD, especially included in the H2CD > IUPRED set, provided that these can be distinguished from artifacts (partial domains, as illustrated in Fig. 4G ). However, such cases could be solved using evolutionary information. The residues undergoing coupled folding and binding and participating in the interaction with the partner can usually be mapped in a single continuous segment, and hence, have been connected through common examples to linear motifs  [55]. These are collected in the ELM database  [56] , which captures sequence features shared by common interacting partners. An example is shown here with a peptide of the Apollo (SNM1B) protein (Fig. 4C ), a member of the beta-CASP family  [57] , which form small alpha helices upon binding to the telomere repeat binding factor TRF2  [58, 59].