dataesr/scientific-paragraphs-categorization huggingface dataset:
Sequence reads were analysed using the OBITools package  (Boyer et al. 2016)  following  Riaz et al. (2011). Pair-end reads were aligned and merged, taking into account the phred quality scores for consensus construction and alignment score computation. Reads were then assigned to their corresponding sample, based on the tagged primer sequences. Low-quality reads (with alignment scores below 50, containing Ns or shorter than 50 bp) were removed. Identical reads were dereplicated while keeping the coverage information (i. e. number of reads merged). For each sample, only the most abundant sequence was kept. Those that were represented by <50 reads were not considered. The resulting DNA reference database was then formatted to be used for taxonomic assignment of sequences. The bash script used for these bioinformatics steps and the DNA reference database in OBITools extended fasta format are available in the Supporting information.
The sequences were then aligned with CLUSTALW2  (Larkin et al. 2007 ) using default parameters. To allow comparisons with other DNA barcoding studies, pairwise genetic distances were computed using the Kimura 2-parameter (K2P) distance model  (Kimura 1980). A neighbour-joining tree was generated  (Saitou & Nei 1987)  to analyse the clustering patterns among species. The R package 'spider'  (Brown et al. 2012 , R Development Core Team 2014) was then used to compute distance statistics and evaluate the reliability of the metabarcode in assigning samples to the correct species using the 'best match' criteria  (Meier et al. 2006). In order to assess the robustness of the results to various analytical procedures, we performed the same analyses with a different alignment algorithm, muscle  (Edgar 2004) , or by taking into account the secondary structure of the 16S rRNA using RNAsalsa  (Stocsits et al.
Sequences represented by only one read (singletons) were discarded because they are most likely to arise from artefacts. Sequences were then clustered within each sample using SUMACLUST  (Mercier et al. 2013 ) with a similarity threshold of 99%, as an initial denoising step to remove PCR or sequencing errors  (Huse et al. 2010). Taxonomic assignment was then performed on our reference DNA database to define operational taxonomic units (OTUs). We further curated the data in order to remove obvious contaminants as well as artefacts such as PCR chimeras, remaining sequencing errors, and misassigned sequence due to tag-switching events  (Esling et al. 2015). The filtering process was implemented in R and consisted of three steps: (i) a reference-based filtering where we kept only the OTUs that had at least 97% identity with its best match in the reference database, (ii) filtering of mistagging errors based on OTUs assigned to nonused tag combos (noncritical mistags, see  Esling et al.
2015  for details) and (iii) removal of sequences that were not recovered in at least 2 PCR replicates. The results were then compared to the genuine content of samples to identify false-negative and false-positive OTUs. For bulk-extracted samples, read abundance ratios were computed to assess whether they were consistent with the specimen ratios. The scripts used for these steps, as well as the sequencing data, are available in the Supporting information so that the analyses can be reproduced. Ultimately, metabarcoding was applied to unidentified specimens freshly caught in the field. Sand flies were collected using ten CDC light traps over three nights in a 1-ha forest plot in the region of Iracoubo in French Guyana in October 2015 (WGS84 coordinates: lat = 5. 25156, long = À53. 32059). After each night, the sand flies caught in the traps were killed by freezing, counted and put in 2-mL microcentrifuge tubes (one tube per trap night).
Bulk DNA extraction was performed directly in these tubes after ethanol evaporation using the Chelex protocols. DNA was diluted 10-fold in nuclease-free water prior to PCR amplification. The metabarcoding protocol was then applied as previously described. In order to assess the comprehensiveness of the sampling, an incidence-based species accumulation curve (SAC) was constructed using the 'exact' method as implemented in the R package 'Vegan'  (Oksanen et al. 2016). The mean Chao2 estimator of species richness was then computed from 100 permutations along the SAC. Assuming that PCR amplification bias should be consistent across samples, the relative read abundance was weighted by the number of specimens collected in each sample and summed to represent the composition of the sand fly fauna in the whole site.
(Ny. ) antunesi and L. (Ny. ) yulli pajoti that were not well distinguished (Figs  2B  and 3 ). 98% of the specimens would have been correctly identified if assigned to the species of the closest sequence in the database (without accounting for the species represented by only one specimen). In the neighbour-joining tree generated with the 178 sequences, every species formed a monophyletic group except L. (Ny. ) antunesi and L. (Ny. ) yulli pajoti (Fig. 3 ). All malefemale pairings were confirmed and species represented by specimens from both French Guiana and Trinidad and Tobago were also grouped unambiguously. Slight differences were found in the topologies of the NJ trees constructed using the different alignment algorithms and genetic distance models. However, the clustering patterns at the species level and the results of the taxonomic assignation test were strictly identical among these methods.
Metabarcoding validation PCR amplifications were successful for all samples except for three samples extracted with the Chelex protocol that had not been diluted. After sequencing and removal of low-quality reads and singletons, 2 626 783 sequencing reads were obtained, representing 59 354 distinct sequences. All expected species were detected in each sample and PCR replicate. Unexpected OTUs were also retrieved but represented only 1. 6% of the reads, the majority of which was not identified as any sand fly species. Filtering of the sequences that (i) did not have a close match in the reference database, (ii) resulted from tag-switching events and (iii) were not detected in at least 2 PCR replicates, removed 2. 6%, 2. 4% and 0. 5% of the reads, respectively. All unexpected OTUs were removed after these steps except in three over 22 bulkextracted samples where one unexpected species was found.
2012; Contreras Guti errez et al. 2014; Nzelu et al. 2015; Romero-Ricardo et al. 2016). Nevertheless, the assignation of each sequence to the species of its closest match in the database resulted in almost 98% correct identifications. This is remarkable because the choice of a metabarcode implies a compromise between its versatility and its resolution (taxonomic coverage and specificity;  Riaz et al. 2011). Except for L. (Ny. ) antunesi and L. (Ny. ) yuilli pajoti, the distance tree obtained with the mt 16S rRNA marker clustered every species studied in monophyletic groups. It also unambiguously grouped conspecific individuals sampled in very distant locations  (French Guiana and Trinidad and Tobago). This is noteworthy assuming that intraspecific genetic variation correlates with the geographic scale of the sampling  (Bergsten et al.
2012). Finally, the marker confirmed the associations of all conspecific males and females, highlighting the reliability of traditional morphological methods in this regard. Most of the subgenera represented in the data set were not retrieved as monophyletic groups in the NJ tree. It is important to keep in mind that the latter only aims at visualizing the clustering of individuals belonging to the same species in order to assess the reliability of the marker for taxonomic assignations based on genetic distances. It should not be confused with a phylogenetic analysis and cannot be used to infer deeper taxonomic relationships. The three sand fly species proven to transmit of Leishmaniasis in French Guiana are all represented in the reference database and well discriminated with the metabarcode.
A meticulous data curation is essential to avoid significant overestimations of diversity in metabarcoding studies. It represents the most critical step of analysis pipelines  (Kunin et al. 2010; Yu et al. 2012; Esling et al. 2015). Here, the sequence filtering process was very efficient in removing false-positive OTUs. Only one unexpected species was detected in three of the 22 bulk-extracted samples. Because the bulk samples were constituted of entire specimens that were morphologically identified without dissection and mounting, it is difficult to exclude that these false-positive OTUs were due to the inclusion of misidentified specimens rather than resulting from molecular biases. These results highlight the advantage of working with a relatively comprehensive reference database. First, it allowed a reference-based filtering that eliminated the majority of unexpected sequences.
Second, defining OTUs in a reference-supervised way ensures OTU taxonomic consistency. De novo OTU picking is far from being a trivial matter  (Bonder et al. 2012; Sun et al. 2012). We tested several combinations of preclustering and clustering algorithms with various parameters on our data  (Sumaclust, Swarm and MCL, Van Dongen 2000; Mah e et al. 2014). All led to important discrepancies between the resulting de novo OTUs and taxonomic assignments of the sequences (data not shown). On the other hand, the obvious disadvantage of such a stringent reference-based analysis pipeline is that it is blind to species that are not represented in the reference database. Nevertheless, the same sequencing data can be reanalysed later on with an updated reference database. Moreover, it is possible to check for the presence of unknown OTUs in the data by relaxing the stringency of the filtering process.
However, great care should be taken using this approach, as it will inevitably increase the number of artefactual OTUs (false positives). We have looked in the field samples for sequences that had between 90% and 97% identity with their best match in the reference database and a read abundance of at least 200 in at least 2 PCR replicates. This way, we identified two OTUs that may correspond to Lutzomyia species not represented in the reference database: one having 94. 1% identity with L. (Ps. ) hirsuta and another having 94. 6% identity with L. (Ny. ) sylvicola. Both extraction protocols led to very satisfying results. As the Chelex-based protocol is cheap and time efficient, it represents a good candidate for routine use. Nevertheless, it lacks a purification step, which can lead to PCR amplification failure and quick degradation of the DNA in the samples, making them not suitable for long-term storage.
In conclusion, this study highlights that the insect metabarcode developed by  Clarke et al. (2014)  is reliable for species-level assignations of phlebotomine sand flies and that metabarcoding using this marker can accurately identify the composition of sand fly mixtures. The application of the present protocol to field samples can significantly improve the cost-and time-effectiveness of data generation on sand fly fauna. Moreover, the primers used here have been estimated to amplify over 90% of insect taxa among the publicly available data at the time of their design and can therefore be employed widely. With adequate reference databases, this approach could be extended to concomitantly analyse other insect vectors communities such as mosquitoes, opening new perspectives for eco-epidemiological studies.
Data accessibility Sand flies specimens: The specimens used for the DNA reference database are kept as voucher at the LIFE laboratory (Mennecy, France). Pictures can be sent on demand. DNA reference database: All sequences were published in GenBank (Accession nos. KU761608-KU761817). The sequences are also available in the OBITools extended fasta format together with the corresponding taxonomic database in the Supporting information (Data S4). Metabarcoding data: Sequencing data of mock communities and field samples are available in the Supporting information (Data S5). We provide the data in fasta format after dereplication and removal of singletons and in tabular format after taxonomic assignment of the sequences. Analysis pipeline: bash and R script used for the bioinformatic treatment of the data are available in the Supporting information (Datas S6 and S7).
Supporting Information Additional Supporting Information may be found in the online version of this article: Table  S1  Checklist of the phlebotomine sand fly species reported in French Guiana. The classifications of Young&  Duncan (1994)  and  Galati (1995 Galati ( , 2014) )  are presented. Table  S2  List of the samples used for the metabarcode reference database along with the GenBank accession numbers of the corresponding sequences. Data S3 DNA extraction protocols for pools of sand flies. Data S4 DNA reference database in OBITools extended fasta format and modified NCBI taxonomic database in ecopcr format. Data S5 Metabarcoding data for mock communities and field samples. Data S6 Bash script used for the bioinformatic treatment of the sequencing data. Data S7 R script used for the filtering of metabarcoding data and output as community matrices.
Acknowledgements This work was supported by  'Investissement d'Avenir ' grants managed by  Agence Nationale de la Recherche  ( CEBA :  ANR-10-LABX-25-01 ;  TULIP :  ANR-10-LABX-41 ,  ANR-11-IDEX-0002-02 ) as well as project  METABAR  ( ANR-11-BSV7-0020 ). The collection of sand flies was performed in the framework of the  French National Project   IAEL :  ANR-06-SEST-20. We would like to thank  Romuald Carinci ,  Jean Issaly ,  Virginie Rougeron  and  Benoit de Thoisy  for their involvement in the collection and processing of the samples,  Pierre Taberlet  and  Eric Coissac  for fruitful discussions about metabarcoding and  Pieternel Verschuren  for her kind revision of the English text.
Strain XM-1 possesses a megaplasmid containing three clusters of heavy metal efflux genes, one czcD gene, and merR-like heavy metal regulation genes. These outer membrane heavy metal efflux genes can help bacteria export the toxic heavy metal  (10). Five DNA methylase-related genes, one putative type II restriction enzyme gene, a resolvase domain-containing protein gene, and a putative DNA topoisomerase are present in the plasmid, which may help bacteria avoid foreign DNA invasion  (11, 12). Accession number(s). The complete genome sequence of Magnetospirillum sp. strain XM-1 has been deposited at DDBJ/ ENA/GenBank under the accession numbers LN997848 and LN997849.
It could also bring new quantitative tools to overcome the limitations of technocratic measures in the investigation of freedom of information  [6]. Even if we know that media competition can lead to a lower quality of information  [4] , we can hope that public broadcasting services tend to convey an official character, and be a reliable baseline for social analysis. The contribution of this paper is social networks analysis directly delivered from video content. The social networks are constructed from face detection and tracking of video content from the NHK News 7 broadcast, and enriched with segmentation and domain knowledge. Finally this enables a social analysis of the media representation of politics in NHK over a decade. After discussing related work in the next section, we will present our data in Section 3, with characteristics and preprocessing.
Description of the data Our video dataset consists in the daily-collected NII-TVRECS archive from  [28]. The capture covers a period between  March 17, 2001 and February 27, 2013 ; of the 4,366-day long period, 4,259 news programs have been collected cumulating about 2,102 hours (6. 7 TB of video) from the Japanese public NHK channel's daily News 7 broadcast. The few missing captures concerning mostly the beginning of the time period are due to the capture system setup. Most of the programs usually last 30mn and only a few of them fall below or beyond this format (news programs may be shorter on Sundays, or longer during commemorative events, see Figure  1 (a)).
Face-track extraction Now we can extract faces from the video shots as in  [30] , designed to be applied on large dataset. The extraction is done in the following steps (illustrated in Figure  2 , steps 2 to 5): Detection. We first apply a detection of faces in all images, using off-the-shelves techniques such as the Viola-Jones face detector  [8]. Multiple faces can be detected per image, different detections of a same face may also overlap. Tracking. We now need to group together the detected faces of a same individual into one single face-track. This is done by generating feature points within detected faces -a point track is a same point identified across different consecutive frames. Point tracks from the KLT point tracker  [31]  are matched from a starting face with the following faces given their temporal order and this results in multiple face instances regrouped in face-tracks.
Network of people overlapping on screen Our first network connects two people when two face-tracks overlap in time. This means that we create a link between two people when they have been detected simultaneously on screen. These links are enriched with the screen duration of the overlapping of tracks as weights. This network presents 35 nodes and 44 edges, with a main connected component of 29/41 (Figure  6 , left). This connected component is only composed of politicians, with one business person (M. Shirakawa, connected to Y. Hatoyama). It is worth noting that J. Koizumi, the top individual among all other metrics, only presents here a degree of 2. Four nodes stand out in terms of betweenness centrality (S. Abe:0. 16, I. Ozawa:0. 14, Y. Hatoyama:0. 18, and Y. Noda:0. 15, with the rest of the dataset below 0.
Networks of people appearing during a same news segment Knowing that co-detection during a news segment is the reason linking nodes, we should first remove the journalists -occurring a lot in the dataset, in order to focus on other people's interactions. The resulting graph presents a maximal k-core (k = 12) gathering 15 Japanese politicians and the 7 PM in a subgraph G k=12 with a density D G k=12 = 0. 79 (Figure  7 , right). A degree and centrality analysis will bring focus to the same people identified in the previous networks. To go beyond, we will look at the graph without the 'main actors', leaving 67 nodes for 221 edges. This graph clearly presents community structures, and by running a Louvain algorithm, we obtain a very interesting clustering result. The two main partitions (in light green and orange in Figure  8 ) clearly present international politicians and national politicians (respectively).
As we can see from Figure  11 , the LDP and DPJ accumulate most of the affiliations and many small parties only have one or two members tracked in our dataset. A few politicians have created independent parties at different moments in time. Different coalitions of opposing politicians appeared especially in the end of Noda's cabinet when it was foreseen that DPJ would not win the next election. This explains why we obtain in total more affiliations than we have politicians. We are interested in studying how the balance of power reflects in the NHK news. In modern democracies, there are often two main parties: a party in power and its main opposition. In the case of Japan, the leader of the party in power is the Prime Minister, and the leader of the main opposition (hereafter OL) is the president of the main party not in power (which is here either DPJ or LDP).
As we mentioned earlier, during this cabinet, NHK was probably more focused on reacting to the tragedy, probably displaying alignment of ideas rather than a diversity of point of views. Figure  14(b ) reports the timeline of the news segments attributed to the PM and the OL during Kan's cabinet, and the last detection of the OL in our dataset only happened a few days before the tragedy. To study the interactions between the two parties, we can plot the share of news segments that involves each of the two main parties, alongside with the ones that involves both of them together (Figure  15(a) ). It is expected to see most of the cabinets having the interaction of their own members put forward in the media during their own cabinet. This is especially true during all DPJ cabinets, Koizumi and Abe's first cabinets.
This leads us to discuss the effectiveness of our system. Our contribution solely focuses on face-tracking because we have a long-term goal to use the screen time as a reliable measure. Our current experiment shows that measuring screen time is yet limited, and we need to discuss the accuracy of our system to understand this limitation. Our face-tracking reaches about 61% precision on the NII-TVRECS dataset  [28] , for which we estimate an average of 37% recall  [38, 30]. This measures the quality of the retrieval of people's face-tracks, keeping in mind that we had performance constraints so we could applied face-tracking on the whole database. Because we are focusing on annotated face-tracks, we made sure that the subset we are exploring for this paper reach 100% precision. However, our system still misses a lot of face-tracks due to our low recall.
The recall depends on the multiple steps described in the first part of this document. • It first depends on the performances of our face detector, the Viola-Jones detector  [8] , which performs up to 88% accuracy depending on conditions and parameters. We chose this detector because of it is rather fast to execute (on our large database) and it is readily available from openCV  [39]. The main drawbacks of this detector are its sensibility to posture and lighting conditions, and the high number of false positive among small size faces (so we actually set a minimum face size of 60×60 pixels). Nowadays, deep learning approaches such as in  [40]  reaches over 97% of accuracy in detection, with far less sensibility to orientation and lighting. • Our tracking phase also filters out some detected faces because we use a point tracker, which, by definition, can be very sensible to lighting conditions and occlusions (although we made efforts to improve its robustness  [30] ).
One can argue that we would reach much higher accuracy by textual information instead of visual information. Indeed, named entity recognizers (NER) can reach high recall, sometime over 90% for the Stanford NER  [43]  on "Person" detection (see  [44]  for comparisons). This depends of course of the dataset, training, and above all, language models. Although results could reach up to 62% accuracy for "Person" detection in Japanese  [45] , we want to keep in mind that actually seeing somebody on TV and hearing a person's mention has a different impact on viewers. However, we will definitely keep the analysis of the textual source and construction of networks from mentions as a future work. The comparison between both sources of information will actually be even more interesting. Since we are discussing the data itself, our future work will extend the set of people to all faces detected in the dataset, not limited to the tagged individuals.
Conclusion This work has introduced the production and analysis of face detection and tracking data over twelve years of news broadcast. We have detailed the data's characteristics and brought a few outliers. Together with the use of topic segmentation and limited domain knowledge, we have derived many networks, each presenting a different point of view on the data, conducting to the understanding of the politico-media situation displayed by NHK during this period of time. The combined views of these networks show interesting insights on the story behind the data, an arguably clear picture of the media/politics landscape during the different cabinets, also isolating key players at different levels. That is what the general reader may take away: thanks to the networks, even those knowing nothing of the Japanese media landscape can quickly get an idea of who the main actors are, and their relative importance.