dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgments The investigations by  Y Rapoport  are based upon work supported by the project  16BF051-02  of the  Taras Shevchenko National University of Kyiv. A Alberucci. and  G Assanto  relied upon support from the  Academy of Finland  through the  Finland Distinguished Professor  grant no. 282858. B Kibler  had support from the  French  project  PIA2/ ISITE-BFC. J McNiff  received strong support from  Original Perspectives Ltd. A D Boardman  received strong IT support from the  University of Salford. Vincent Uzomah ,  University of Salford , provided some printing technical assistance.
Speech synthesis of written text lines in combination with the spotting of the talking characters would led to a very immersive experience that could help small children to apprehend comic books. Those pieces of text could be automatically translated in any required language if needed. The recognition of what is going on in a given panel, when and where the story takes place, what real or fictional characters are involved in it, could bring to the reader additional information that would help her of him to grasp the whole meaning of what she or he is reading. Some issues about old comic books preservation tend to rise as well, as the pieces produced at the early stages of the art's history are getting older every passing day. The world produced a large amount of heterogeneous comic books that now needs to be archived in some sort of database.
We organized this article as follows. The first section presents several comic books formalisms from the literature. We present as well the state of the art on the use of formalized knowledge for image analysis, especially when it comes to discover what is the meaning of the extracted content. The second section introduces, through a complete and detailed presentation, our own proposition, including two ontologies, as a framework to handle the content of comic books' images. Then we discuss our conceptualization choices and test them over a public dataset. We also present some experimental results on two applications: a new panels ordering method and an iterative process to discover the content of a page. We discuss the results in the last section that concludes this article.
Multiple formalisms for comic books If one wants to fully extract and analyze a comic books' content, it is necessary to know how that content is organized and the nature of the different relations that exist between its elements. While a growing number of studies came up lately in linguistics  [2]  and psychology  [12]  communities, the computer science community, as we know of, conducted a very few works on the comic books language conceptualization. Besides the Periodical Comics web-schema proposal  [49]  and the Grand Comics Database platform  [25] , both focusing on the publishing metadata dimension, we could only find a very small amount of endeavors on the content formalization. Not all of them have been the subject of an academic publication. In 2001, ComicsML, an XML formalism to describe the content of a comic books plate has been proposed  [29].
The TEI is a framework, proposed by the homonym consortium, to encode textual documents and illustrated documents especially. Its purpose is to make information retrieval in large encoded datasets easier. CBML extends the TEI vocabulary with some comic book's notions (panels, balloons, etc. ), while reusing, as much as possible, the existing encoding. Pastierovic proposed a third formalism, called Advanced Comic Book Format or ACBF  [37]. It focuses on the encoding of digital comic books. Also XML-based, it aims to enhance the reading experience on computers. Its authors define it as the comic books archive file formats, such as CBR and CBZ, evolution, as it adds metadata on several elements. It includes the pages and panels succession concept, as well as the distinction between the several kinds of balloons and the language in which the text is written.
Description of our framework This section introduces the ontology we propose, made of two smaller ontologies. We start by presenting a first ontology that formalizes the concepts related to image analysis, the different types of data, input, output and evaluations. Then, we explain the conceptualization process that led to create a second ontology on comic books domain knowledge. Finally, we show how we can use these ontologies together. We test our conceptualization choices over the eBDtheque ground truth dataset in the next section.
Image ontology 3. 1. 1. Image and regions of interest The purpose of the first ontology was to formalize the very basic concepts of the image processing domain, from the extraction to the evaluation that is lacking in the state of the art. We put a special effort in making it as generic as possible, excluding any concepts that would confine it in a specific application domain. We intended it to be usable in any application based on image processing (e. g. document analysis, computer vision, etc. ). It gathers the data produced by image analysis algorithms in a predefined structure. Then we use this structure to formalize the semantic of each of those elements. The data produced by those kinds of algorithms is often of a spatial nature. It can be regions of different shapes, lines or points taken from an image.
They can be of different definitions, in other words, be made of a different amount of pixels. Their resolution (the number of dots per inch) or the way they encode and compress the information, with or without loss, may vary as well. We had to add this information into the Image concept of our ontology as it can help to understand the output of an algorithm. The resolution is formalized as a data property named hasResolution. The definition is split in two attributes, hasWidth and hasHeight. Those three properties take integers as data type. One can use them to compute the original size of the digitized document, so they are sufficient to retrieve the initial conditions of the digitization process. The hasFormat and isLossLess data properties represent the storage format and the possible image compression.
A keyword, placed at the beginning of the character string, describes the nature of the object, e. g. POINT, LINESTRING or POLYGON. Last but not least, WKT is open, well documented and supported by many API in a lot of different programming languages. It is also supported by GeoSPARQL. The hasWKT attribute enriches the ROI concept. It takes string as values and describes both the type (polygon, line, dot, etc. ) and the coordinates of ROI instances. Additional hasArea, hasX and hasY attributes can be used to specify the area and the center position of a region of interest. The object property hasROI was created to link the Image and ROI concepts. The former being its domain, the latter its range. It represents the fact that an image can contain some regions of interest. However, a ROI is part of one and only one image.
), a simple keyword would be enough as a ground truth. But if we want to analyze the very content of a document, how it is organized internally, we need to make a deeper annotation. For the analysis of a comic book page, it implies to spatially define the position of each element we want to extract, such as panels and characters. Semantic annotations can also be added to these elements. As it is meant to be used as a reference, a ground truth cannot be generated automatically. A human, who can be considered as an expert in the application domain, has to build it from scratch. Depending on the level of details, the building of such a dataset can become time-consuming. The expert's knowledge, formalized through his annotations, stands for the gold standard, the limit the algorithm needs to reach to pretend having a perfect behavior.
We defined them as equivalent to the individuals in Evaluation with a higher, respectively lower, hasFScore value than a predefined threshold s (see Equation 3 and 4). One can tune this rule based on the specificities of the application to use the recall, the precision, or even other metrics such as the one introduced by VOC  [15]. Those concepts are disjoint, an evaluation being either valid or invalid. Validation ≡ Evaluation and hasFScore some double[≥s] (3) Error ≡ Evaluation and hasFScore some double[<s] (4) The choice to introduce a specific concept to conceptualize the evaluation and not a simple attribute to ROI is justified when you deal with multiple evaluations. Especially when a proposition has several intersections with different references or when more than one ground truth dataset is loaded into the A-Box.
Experiments We present in this section the experiments that used the framework we presented in Section 3. 1 or were made possible by it. We start by evaluating our conceptualization choices with regards to an annotated dataset that we used to populate the A-Box of our ontologies. Then, we present a first experiment on the progressive discovery of a comic books' page's content. We go through the protocol very briefly as we fully described it in a previous publication  [41]. The second presented experiment defines a proper way to browse a digitized comic books' content, that is respectful of the original author's intentions.
So even if a spatial inclusion of an element to another is matter of debate, their semantic belonging is formal and unquestionable. The chosen term belongs reflects the situation where a graphical element overlaps another element, (a fortiori when it includes the other), on a significant proportion of its area. When it intersects several other elements, we only keep the largest intersection. This way, we consider that a line of text belongs to the balloon that includes it, not to the panel that includes this balloon. Confronted to the lack of reliable information on what could be a significant intersection of comic books' elements, we estimated its value using the eBDtheque dataset  [19]. This dataset is made of a hundred images from various comic books styles. For each element of the dataset, we determined what would be the type of its direct container for a given value of intersection.
Similarly, the 18. 4% of text lines that are outside of any balloon are explained by the presence of a few web-comics with bibliographical annotations on the page. The dataset also includes some comics that are older than the invention of speech balloons  [43]. The authors positioned the text below each panel then, i. e. outside of any balloon. The drop that we see for an overlapping value of 100% is caused by an understandable side effect. Indeed, each element having to be entirely contained into a bigger one, a single outside pixel makes it belong to the bigger one. The effect does not affect panels that need to belong to the page, the biggest element of all.
Knowledge base Figure  13 : The interaction loop between low and high level systems. 81. 6% of text lines and 94. 9% of characters. So we can take those figures as the top values we aim to reach. The A-Box of O cb was populated with the elements from the eBDtheque ground truth dataset and the elements from the image processing algorithms. For each spatial intersection between i t , an automatically extracted element of type t, and i t an element from the ground truth of the same type, we created a new instance of Evaluation and linked it to them with the properties hasCandidate(e,i t ) and hasReference(e,i t ). We added to each of these instances a hasVOCScore attribute, representing the measure introduced by Everingham et al. [15]. This measure provides a quality indicator q, based on the spatial intersection and union of the two compared sets.
Evaluation We evaluated how our method performs on the eBDtheque dataset. Let r i = r(C i ) and r i = r (C i ) the respective functions giving to a panel C i a rank r i from the ground truth and r i from our ordering method. We computed the value of Equation  23 for each couple of panels {C i , C j } with r i = r j + 1. r(C i ) -r(C j ) = r (C i ) -r (C j ) = 1 (23) If C i is right next C j (or the other way around) by our ordering method but not according to the ground truth, the equality is not true and neither is the calculated relation. This measure is more accurate than the direct comparison of r i and r i or each panel C i. Indeed, it evaluates the relative position of panels from each other and does not penalize the whole set for one shift at the beginning of the sequence. The eBDtheque dataset is made of 100 pages, containing 849 panels.
Therefore, 749 relations have to be evaluated. The results of evaluation are given in Table  1 , along with the results of our implementation 5  of a state of the art method  [39]. Both algorithms not being strictly transitive, the given results are the best we obtained out of ten tries (results may be lower up to 2%). The presented results on the full dataset are relatively good with an accuracy over 94%. It outperforms the state of the art in every tested configuration. We explain the reasons for the 6% errors below.
Dataset Firstly, the annotated ground truth has only total reading orders and does not handle the simultaneity of included panels. Our algorithm giving the same rank value to those panels, Equation 23 is not verified in those cases. Nevertheless, the reading order asserted in the ground truth dataset is indeed one of the possible topological sorts of our own reading order. This causes 0. 5% of the errors. Secondly, five images of the dataset represent two distinct pages that are not meant to be read as one. The left page having to be completely read before the right page. We designed our algorithm to handle single pages, or actual double pages. It is irrelevant to test it over such images. By getting rid of those five images, the results rise up over 97%. The third and last reason lies in the fact that one special page (see Figure  20 ) has a lot of panels with an unusual layout.
While it is quite efficient for our own study context, it can easily be used to any other domain, such as documents or natural scenes analysis. The conceptualization choices that led to the proposed comic books ontology were driven by the idea of an image analysis use case. While the layout conventions and vocabulary are widely shared through the worldwide literature, comic books remain a form of art. We expect from artistic pieces to break the established conventions in order to make the art evolve and experiment new things. Thus, the worldwide comic books legacy presents a very heterogeneous face. There are no formal rules that can be true for all instances of comics, beyond the idea of "sequential drawings". Such a conceptualization would be too flexible to be of any practical use. We evaluated the relevance of our choices on an annotated dataset, resulting in a solid consistency of our model over 90% of panels, balloons, lines of text and characters.
Acknowledgements The authors would like to thank  Jean-Christophe Burie  and  Jean-Marc Ogier  for their help with the low-level processing. This work was supported by a  Doctorate scholarship  of the  University of La Rochelle , the  Region Poitou-Charentes (France)  and the  Future Investment Program iiBD (French government ). We are grateful to all authors and publishers of comics and manga from the eBDtheque dataset for having allowed us to use and share their works.
, 2016; Edwards and Rohwer, 2005). As a drawback, the identity of the hosts remains unknown for these newly discovered viruses, limiting our ecological understanding of the microbiome. Different methods exist to predict prokaryotic hosts for phage sequences in metagenomes, based either on coabundance, sequence homology, similarity to other phages  (Villarroel et al. , 2016)  or sequence composition similarity between viruses and their hosts  (Edwards et al. , 2016). Among tools taking this last approach, VirHostMatcher  (Ahlgren et al. , 2016)  has reported the best accuracy (proportion of correct predictions) on full-length viral genomes: between 33 and 64% at the genus level depending on the dataset. But its performance drops notably for shorter sequences, falling by 36% at 5 kbp length.
However, contigs of a few kbp length are common in viral metagenomic data due to shallow coverage and intra-population variation  (Smits et al. , 2014). In addition, the running time of VirHostMatcher hinders its use on large datasets (Supplementary Table  S5 ). Here we introduce WIsH, a tool to predict the prokaryotic host of viral contigs with good accuracy for contigs as short as 3 kbp that runs several hundred times faster than VirHostMatcher.
3113 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons. org/licenses/by/4. 0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. Bioinformatics, 33(19), 2017, 3113-3114  doi: 10. 1093/bioinformatics/btx383 Advance Access Publication Date: 13 July 2017 Applications Note To evaluate the performance of WIsH an VirHostMatcher, we used the 3780 full prokaryotic genomes of the KEGG database  (Kanehisa et al. , 2017)  and the 1420 phages in the RefSeq Virus database  (Brister et al. , 2015)  for which a host was annotated in this database. WIsH can compute P-values when provided with the parameters of the Gaussian null-distributions of each Markov model (option -n KeggGaussianFits.
Data entry time (62%), clinical utility (22%) and ease of communicating BWA v3 risks (13-17%) received additional negative appraisals. In multivariate analyses, controlling for gender and country, data entry time was perceived as longer by genetic counsellors than clinical geneticists (p < 0. 05). Respondents who (1) considered hormonal BC risk factors as more important (p < 0. 01), and (2) communicated numerical risk estimates more frequently (p < 0. 001), judged BWA v3 of lower clinical utility. Respondents who carried out less frequent clinical activity (p < 0. 01) and respondents with '11 to 15 years' seniority (p < 0. 01) had less favourable opinions of BWA v3 risk presentations. Seniority of '6 to 10 years' (p < 0. 05) and more frequent numerical risk communication (p < 0. 05) were associated with higher fear of communicating the BWA v3 risks to patients.
Methods This was a cross-sectional study. The overall survey content and example of questions are provided in Table  1. The survey included four sections addressing: (1) practice in genetic counselling and testing for cancer predisposition (14 questions); (2) importance attributed to BC risk factors, including modifying BC risk factors; BWA v3 frequency of use and data entry time; ways of communicating risk (i. e. , in relative, absolute, absolute over 5, 10 or 15 year forms) (26 questions); (3) BWA v3 aspects' assessment (13 questions); and (4) sociodemographic and professional background (8 questions) (Supplementary material).
Data analysis As shown in Fig. 1 , 525 and 203 respondents' data were extracted from the BWA and NGS survey sources respectively. The response rate obtained from the BWA survey source could not be estimated as the survey was sent to registered individuals who might no longer use the tool, and tracking BWA v3 use is not legally permitted. Among respondents contacted through NGS, the response rate was 43. 7% (203 respondents out of 464 NGS members, excluding those who responded through the BWA website). As indicated in Table  1 , we developed indicators of genetic clinical activity level, importance attributed to modifying BC risk factors (i. e. , reproductive, lifestyle, hormonal and body mass index), and for the BWA v3 appraisals. Higher scores on the three BWA v3 appraisal variables reflect favourable opinions.
BOADICEA Web Application appraisals and data entry time Less than 10% of respondents expressed negative opinions on 5 out of 9 items addressing BWA v3 aspects. Some expressed negative opinions about the scientific validity of BOADICEA (9%) and BWA v3 risk presentations (7-9%). Data entry time (62%) and estimates of clinical utility (22%) received additional negative appraisals. Moreover, a significant minority of respondents expressed fear of patients' misunderstanding (17%) and fear of upsetting patients (13%) using BWA v3 in clinical practice. About half of respondents indicated that the risk estimates would not change their clinical judgement about breast or ovarian cancer risk management (Table  3 ). The overall mean time (standard deviation) taken for data entry using BWA v3 was 15. 5 (10.
(55) You think patient would NOT be eligible for risk reduction salpingo-oophorectomy but after looking at BOADICEA numerical and graphical results you think that she is eligible for risk reduction salpingo-oophorectomy 226 (58) that increased frequency of use of BWA v3 could result in shorter data entry times through improved skills, but this was not observed in this study. However, controlling for gender and country of practice, the time for data entry was perceived as longer by genetic counsellors than clinical geneticists. This difference may be related to role sharing between medical and non-medical genetics clinicians. The time required for data entry using BWA v3 partly reflects the design of the software. BWA v3 captures input pedigree data using HTML forms (form-based data entry).
However, it is clear that graphical pedigree data entry programs (that enable users to create a pedigree drawing, with small forms to enter data for each family member) can often capture pedigree data sets more quickly and easily than form-based programs such as BWA v3. The first version of the BWA was released for general use to the healthcare community in 2007. Since that time, advances in client-side software development technologies have facilitated the implementation of Web-based graphical pedigree building tools, and so the BWA will be extended to include such tools in the future. In addition, the time required for data entry using BWA v3 partly reflects the data requirements of the underlying BOADICEA model (e. g. , BWA v3 requires that the user specifies a year of birth in order for a family member to be taken into account in a risk calculation, whereas the IBIS tool does not).
Implementation and experimental results. We implement both ???? in Python3. The code can be accessed through Github 1  and is roughly 750 lines long. We provide methods for either 2D-SSapprox or MinkDecomp-µ-approx. To test ?? we created vectors v i at random with |v i | ≤ 5000. For ?? we create random points and take their convex hull to form input polygon Q. All tests were executed in an Intel Core i5-2320 @ 3. 00 GHz with 8Gb RAM, 64-bit Ubuntu GNU/Linux. Results for ?? are shown in ?? and for algorithm ?? in ??. It is clear in ?? that our results stay below the expected time and behave analogously. In ?? the results obtained are much better than the proven bounds and in most cases volume and perimeter are almost the same and the polygons differ slightly.
Purpose: The aim of this paper is to define the requirements and describe the design and implementation of a standard benchmark tool for evaluation and validation of PET-auto-segmentation (PET-AS) algorithms. This work follows the recommendations of Task Group 211 (TG211) appointed by the American Association of Physicists in Medicine (AAPM). Methods: The recommendations published in the AAPM TG211 report were used to derive a set of required features and to guide the design and structure of a benchmarking software tool. These items included the selection of appropriate representative data and reference contours obtained from 4098.