dataesr/scientific-paragraphs-categorization huggingface dataset:
C. Disk Usage Unlike ingestion time where shorter delta changes are clearly beneficial, the gains in terms of disk usage need finegrained tuning because they depend on the dataset as shown in Table  IV. Overall, more delta chains tend to increase disk usage. For BEAR-B daily, frequent snapshots (high periodicity d = 5) incur a large overhead w. r. t. the baseline because the changesets are small and the revision history is short. Similar results are observed for BEAR-A and BEAR-B instant, even though we still need multiple snapshots to be able to ingest the data. BEAR-B hourly is interesting because it shows that for long histories, a single delta chain can be inefficient in terms of disk usage. Interestingly for BEAR-A, the changeratio γ = 4. 0 uses less storage than the time strategy with θ = 20, despite using more delta chains.
All our strategies beat the baseline by a large margin on BEAR-B hourly because delta operations become very expensive as the single delta chain grows. That said, the baseline runtime tends to decrease slightly with i because the data from two distant versions tends to diverge more, which requires the engine to filter fewer results from For BEAR-B daily, multiple delta chains may perform comparably or slightly worse -by no more than 20% -than the baseline. This happens because BEAR daily's history is short, and hence efficiently manageable with a single delta chain. In this case the overhead of multiple snapshots and delta chains does not bring any advantage for DM queries. 3) V Queries: Figure  3c , 3f, 3i, and 3l show the total runtime of the benchmark V queries on the different datasets.
V queries are the most challenging queries for the multisnapshot archiving strategies as suggested by Figures  3f  and 3i. As described in Algorithm 2, answering V queries requires us to query each delta chain individually, buffer the intermediate results, and then merge them. It follows that runtime scales proportionally to the number of delta chains, which means that contrary to DM and VM queries, many short delta chains are detrimental to V query performance. Nonetheless, querying datasets such as BEAR-A and BEAR-B instant is only possible with a multi-snapshot solution.
E. Discussion We now summarize our findings and draw a few design lessons for RDF archives. • For small datasets, small changesets, or relatively short histories, the overhead of multi-snapshot strategies does not pay off in terms of query runtime and disk usage. This observation is particularly striking for V queries for which runtime increases with the number of delta chains. • While many short delta chains are detrimental to V queries and often to storage consumption, they are mostly benefitial for VM and DM queries because these query types require us to iterate over changes within delta chains (two in the worst case of DM queries). Moreover, short delta chains reduce ingestion time systematically. • Disk usage usually benefit from less numerous delta chains, except for long change history and large aggregated deltas.
• Change-ratio strategies strike an interesting trade-off because they take into account the amount of data stored in the delta chain as criterion to create a snapshot. This ultimately has a direct positive effect on ingestion time, VM/DM querying, and storage size. The bottom line is that the snapshot creation strategy for RDF archives is subject to a trade-off among ingestion time, disk consumption, and query runtime for VM, DM, and V queries. As shown in our experimental section, there is no one-sizefits-all strategy. The suitability of a strategy depends on the application, namely the users' priorities or constraints, the characteristics of the archive (snapshot size, history length, and changeset size), and the query load. For example, implementing version control for a collaborative RDF graph will likely yield an archive like BEAR-B instant, i.
e. , a very long history with many small changes and VM/DM queries mostly executed on the latest revisions. Depending on the server's capabilities and the frequency of the changes, the storage strategy could therefore rely on the change ratio or the ingestion time ratio and be tuned to offer arbitrary latency guarantees for ingestion. On a different note, a user doing data analytics on the published versions of DBpedia (as done in  [PGH21] ) may be confronted to a dataset like BEAR-A and therefore resort to numerous snapshots, unless their query load includes many real-time V queries.
To approximate each δ s,k according to Equation 1, we rely on OSTRICH's count indexes. The terms |u + s,k | and |u - s,k | can be obtained from the count indexes of the fully unbounded triple pattern ?s, ?p, ?o in O(1) time. We estimate |G s ∪ G j | as |G s | + |u + s,j |, where |G s | is efficiently provided by HDT. The source code of our implementation as well as the experimental scripts to reproduce this paper are available in a Zenodo archive 3.
Embedded Computer Software The Jetson was set up with a Jetson nano Jetpack 4. 6. 1 sd card image (L4T 32. 7. 1). Python scripts were coded to pilot the cameras and the LEDs. The main script is using two GStreamer pipelines (chaining of capture, encoding, wrapping and writing) taking advantage of the Nvidia hardware acceleration to record simultaneously two types of video in MPEG-4 container format. The first pipeline uses the fifth sensor mode (Figure  9 ) of the IMX219 camera sensor and an h264 codec (high profile, 3. 2 level) to generate the detection videos at a resolution of 720p and a framerate of 60 fps. The 120 fps of the sixth sensor mode makes it possible to improve acquisition by capturing at 120 Hz. However, the exposure time in this mode will be reduced by half for equal resolution which will alter the precision of dance visual analysis.
The second pipeline uses the fourth sensor mode and an h264 codec (high profile, 4 levels) to generate the hive videos at a resolution of 1640 × 1232 and a framerate of 30 fps. During recording, the LED states are managed by the script. By default, the duration of the videos was set to 10 min (to limit the video weight) with the possibility to specify gaps between the records (set to 0 by default). Capture times (start and end times) were scheduled with a scheduled task. Two scripts are used to preview (via an ssh connection) the videos inside the hive without recording to adjust the alignment of the camera. A third script gives the possibility to switch the light. The last script wrapped in a scheduled task is used to synchronize the videos with a server, one minute after the end of the recording.
To facilitate the analysis, a Python script was developed to add a static grid to the videos. An observer collected and did a visual analysis of video dances frame by frame and grid cell by grid cell. For each waggle phase (center line of the dance's eight-figure), the angle of the phase was deducted from the angle from the vertical and the mean path orientation of the phase, and the duration of the phase from the difference between the start and the end frames times. The data were directly filled into a dedicated Postgresql 14. 6 database. After verification and correction, the data was exported from the database by a Python 3. 10. 6 script to generate a map of all the vector endpoints deduced from the waggle phase information and a heat map produced with the dance information. The maps were produced with Inkscape 1.
Encoding and Sampling Quality Videos recorded by GeoDanceHive were analyzed by visual observation (by a human observer). For example, the video of 21 June 2022, recorded on a feeder day between 12:49 and 12:59 p. m. contains 1491 waggle phases and 141 dances, i. e. , 14 dances/min. This large number of dances seems to show that the repositioning of the hive entrance perpendicular to the first frame is successful and that the location of the dance floor in the hive can be influenced by the morphology of the hive. The video quality also allows the analysis of dances by human eyes, which is a necessary step to validate the encoding parameters, in particular for its good weight/quality compromise. The hive makes it possible to quickly capture large samples of data (waggle phases and dances) and to produce maps.
The glass and the built-in infrared lighting allow the honeybees to be observed at any time and so to record without interruption an enormous amount of data. Unlike current systems, the device has an ethernet port to send the data in real time. It can easily be equipped with a WIFI/BlueTooth card and pre-holes have been provided in the box to fix the two antennas supplied with the optional card (Figure  6 ). In the event of a network failure, local memory space can be added by plugin USB keys in the four USB ports left available, allowing an operator to come and retrieve the data directly on-site. The light, framing and resolution being fixed and constant, the videos acquired via the first camera can be processed by computer vision software in order to automate the detection of dances. In the same way, the use of artificial intelligence for detection also becomes possible with the acquisition of a large number of videos which allows for constituting the datasets necessary for the learning process.
Acknowledgments: We thank the  Paris Institute of Technology for Life, Food and Environmental Sciences (AgroParisTech)  and more specifically the  Research and Valorization Department  for its administrative support. We thank the  National Institute of Geographic and Forest Information (IGN)  and the  Ecole Nationale des Sciences Géographiques (ENSG)  for its administrative and technical support. We thank AIRBUS for their technical support and satellite imagery. We thank  "Les Ruches Rigault " for their technical support and their involvement in the hives modelling. We thank  Jérôme Versavel  and  Didier Socard  for their technical support and their involvement in 3D modelling and printing. We thank " Au coin des Artistes " for their technical support and their multiple "invisible" glass donations.
Funding: This research was funded by the french ministries of Agriculture and  Food Sovereignty (MASA-FCPR program ),  Ecological Transition and Territorial Cohesion (MTECT) ,  Health and Prevention (MSP)  and  Higher Education and Research (MESR)  and by the  French national facility  for institutional procurement of VHR satellite imagery (DINAMIS) and by the  Lune de Miel ® Fondation. This research was financially supported by the  French Office for Biodiversity , on the fee envelope for diffuse pollution of the Écophyto II+ coord plan. F-J Richard , partners  P. Aupinel  and  G. Touya  for the  DANCE  project.
, 2013). The only comprehensive binding dataset for nearly all TFs in a eukaryotic organism is a ChIP-Chip dataset for baker's yeast, Saccharomyces cerevisiae  (Harbison et al. , 2004). Newer methods, including ChIP-Exo and Transposon Calling Cards, yield fewer non-functional binding events  (Kang et al. , 2020) , but data are currently available for only a few dozen yeast TFs. A simple approach for predicting edges that are both direct and functional is to calculate the overlap between genes whose promoters are bound by a TF in ChIP-Chip experiments and those that respond to a direct perturbation to that TF. However, this overlap is typically very small  (Gitter et al. , 2009; Kang et al. , 2020). A much better overlap can be achieved by processing perturbation response data through TF network mapping algorithms  (Kang et al.
, 2010). The other was derived by building a preliminary network, inferring binding motifs (PWMs) for each TF from the promoters of its predicted targets, and scoring the promoters of all genes with that PWM  (Kang et al. , 2017). A limitation of NP2 is that it combined these scores using an elaborate, multi-stage procedure that was manually optimized on a particular dataset and may not be optimal for combining evidence scores from other sources. Here, we introduce NetProphet3 (NP3), which learns to combine evidence scores from any sources by using a supervised machine learning framework. Each possible (TF, target) edge is an instance with features consisting of its evidence scores and binary labels based on whether there is evidence that the TF binds the target's regulatory DNA. By default, NP3 uses the same evidence scores as NP2, but users are free to train on other input features.
This makes it easy to incorporate evidence scores derived from multiple gene expression datasets (demonstrated here), scores from scanning known binding specificity models over regulatory DNA sequences, or scores from other network mapping algorithms. NP3 is an easy-to-use package that is distributed with a singularity container, so it can be run under any operating system without installing dependencies. There are two main use cases for NP3. One is to predict the direct functional targets of a TF for which no binding data are available by using gene expression data. If there are enough TFs from the same organism that have binding data, they can be used to train NP3 to predict edges for TFs that do not have binding data. Otherwise, edges can be predicted by the provided model, which was trained on yeast data.
e. the model's prediction) can be increased by intentionally overfitting the model to the labels or reduced by under-fitting. We demonstrate that overfitting a model to binding data for one TF improves performance, according to metrics that are independent of the binding labels used in training. We tested NP3 with two different expression datasets for yeast. In each dataset, most yeast TFs were perturbed, either by gene deletion or by transient overexpression in response to a chemical inducer. To evaluate performance, we then used three independent evaluation metrics: (i) the percentage of edges that are direct, estimated by the percentage of edges supported by binding data. (ii) The enrichment of gene ontology (GO) biological process terms among the predicted targets of each TF. (iii) The percentage of known protein-protein interactions (PPIs) among (TF, TF) pairs sharing many target genes.
4 below). XGBoost training depended on the use case. For predicting edges for TFs that do not have binding data (generalization mode), we trained it using 10fold cross-validation (CV) on TFs. To construct a fold, we stratified TFs based on the number of their positively labeled targets, then selected 90% of TFs from each group at random for the training fold and the rest for the test fold. We tuned the hyperparameters of XGBoost using 5-fold CV within the training fold; the optimal parameters were used for training an XGBoost model with all the training data. For predicting targets of TFs that have binding data (integration mode), we made predictions on the same TF(s) that we trained on, using those predictions as the NP3 output score. For the cross-trained mode, we trained a model with all TFs from the TFKO dataset and tested that model on the ZEV dataset, and vice-versa.
Inferelator3 Ideally, Inferelator3 v0. 5. 6 requires a binary 'prior' network of targets for each TF. It extends an existing network with additional targets, which makes it impossible to use the binary binding network as prior and carry out CV by TF-if the binding data are required as prior for each TF, it cannot be used to evaluate the network. Instead, we constructed a prior network from the DE data by assigning 1 when the absolute value of shrunken log 2 fold-change (LFC) of gene expression is greater than zero (ZEV data) or when the absolute value of LFC of gene expression is >0. 4 (TFKO data). This prior includes only TFs that were perturbed by overexpression (ZEV data) or knock-out (TFKO data). We also constructed another prior that included all TFs-TFs that were not perturbed were assigned zero priors (Supplementary Fig.
Gene expression profiles We used two gene expression datasets for S. cerevisiae in which each TF was perturbed either by deletion  (Kemmeren et al. , 2014)  (TFKO) or overexpression  (Hackett et al. , 2020 ) (ZEV). The TFKO dataset includes 1485 gene expression profiles of strains in which a single gene was deleted from the genome, of which 281 are TFs. The ZEV dataset contains gene expression profiles at various time points after transient induction of 167 TFs using estradiol (Supplementary Files S1 and S2).
Protein-protein interaction evaluation metric First, we calculated the Jaccard similarity between the sets of target genes of each (TF, TF) pair; the Jaccard similarity is 1 when the target sets are the same and 0 when none of the targets in the sets are shared. Next, we sorted (TF, TF) pairs by Jaccard similarity and calculated the percent of the top 100 pairs that is supported by physical interactions from the PPI STRING database (Szklarczyk et al. , (B) Binding metric (defined in main text) calculated as a function of the number of top-scoring edges at different thresholds. NP3 outperforms all other network inference algorithms at all thresholds. Random expectation is the probability that a randomly selected (TF, gene) edge will be supported by binding data. (C) GO metric (defined in main text).
NP3, NP2, Inferelator3, and Genie3 have similar performance. Random expectation is the GO metric for 20 NP3 networks in which TFs and genes were randomly permuted. (D) GO-directness metric (defined in main text). NP3 target predictions matching the TF's most significant GO term have the highest percent binding support. Random expectation is the GO-directness of the GO results of the 20 random networks generated in (C). (E) PPI metric (defined in main text). Horizontal axis: number of top (TF, TF) pairs included. Vertical axis: percent of included (TF, TF) pairs supported by strong evidence of physical interaction. NP3 performs better than other algorithms for the top 40 pairs and about the same as NP2 on the top 50-100 pairs. Random expectation is the probability that a randomly selected (TF, TF) pair will be supported by the PPI STRING database.
Results 3. 1 NetProphet3 is more accurate than other commonly used algorithms when TF perturbation-response data are available We systematically combined evidence scores from DE, LASSO, BART and PWM using XGBoost  (Chen and Guestrin, 2016) , a machine learning algorithm, to predict one final network. Each training instance was a potential (TF, target) edge, features were evidence scores, and labels were 1 for edges supported by evidence of direct binding and 0 for the remainder (Fig. 1A ). We obtained labels from curated TF binding location experiments including ChIP-Chip, ChIP-Exo and Transposon Calling Cards (Supplementary File S4; see Section 2). We trained an XGBoost model to predict the probability of a potential edge being supported by evidence of direct binding. Since binding experiments are typically available for a limited number of TFs, we wanted to know if an XGBoost model trained on edges emanating from TFs that have binding data could predict probabilities of binding support for edges from TFs without binding data.
Therefore, we trained and evaluated the model with 10-fold CV by TF; each hold-out fold consisted of edges from TFs that were not in the training fold. Throughout this article, we use 20 different random seeds for assigning TFs to CV folds and report the average performance. We trained and evaluated methods using two independent gene expression datasets for yeast that were generated by separate labs under different growth conditions. In each dataset, most TFs in the yeast genome were perturbed by either gene deletion  (Kemmeren et al. , 2014)  (TFKO data), or transient over-expression in response to a chemical inducer  (Hackett et al. , 2020)  (ZEV data). We used independent, systematic, objective metrics for evaluating network maps, which we call Binding, GO, GO-directness and PPI: • The binding metric is the fraction of highly scored edges that are supported by binding location experiments suggesting that the TF binds in the regulatory DNA of the target.
• The PPI metric is the fraction of (TF, TF) pairs, with similar target sets, that are supported by evidence of physical interaction, and hence may function as a complex. Good performance on the PPI metric indicates that the network can predict experimentally verified biological facts, PPIs, which are independent of the data types used to construct the network. Specifically, we used a threshold on the scores of the TF network map to produce a discrete edge set and then calculated the Jaccard similarity between the predicted target sets of each pair of TFs (the size of the intersection divided by the size of the union). Next, we ranked all (TF, TF) pairs from highest Jaccard similarity index to lowest. The PPI metric is the fraction of (TF, TF) pairs with high Jaccard similarity for which there is strong evidence for a physical interaction in the STRING database (STRING score ! 0.
7)  (Szklarczyk et al. , 2021)  (see Section 2 for details on each metric). We expect the true yeast network to be sparse, with 10-50 targets per TF on average, so we focused our analysis on the top scoring edges. Before plotting, the x-axis was rescaled by the number of TFs scored, yielding the average number of targets per TF (see Supplementary Fig. S5 ). First, we evaluated the NetProphet3 output score in comparison to its input scores from DE, LASSO, BART and PWM. Some of the input scores performed well on some metrics on one of the datasets, but overall, the NP3 synthesis was the clear winner (Supplementary Fig. S1A-H ). Next, we compared NP3 to other prominent network mapping algorithms: NP2  (Kang et al. , 2017) , Inferelator3  (Gibbs et al. , 2022; Jackson et al. , 2020) , Genie3 (Huynh-Thu and Geurts, 2018) and a co-expression network based on the Spearman correlation between expression levels of the TFs and those of the genes.
At each threshold, NP3 edges had a higher percentage of direct binding support than any of the others (Fig. 1B ; Supplementary Fig. S2A  and E  show results broken out by TFKO and ZEV datasets; and Supplementary Fig. S7A  and B  show results using AUROC and AUPRC, respectively). On the GO metric, NP3 performance was about the same as Inferelator3 and Genie3 (though more consistent across thresholds), slightly better than NP2 (Fig. 1C  and Supplementary Fig. S2B  and F ). Next, we calculated the GOdirectness metric to check whether each algorithm's performance on the GO metric was driven by direct edges, or came at the cost of including unwanted, indirect edges. NP3 targets annotated with the most significant GO term had higher binding support than those of the other algorithms (Fig. 1D  and Supplementary Fig.
The importance of TF-perturbation response data NP3 can only use the DE score when TF perturbation data are available, but it can use the regression scores from any expression data. We wanted to understand how the DE component and the expression profiles of TF perturbations affected the performance of NP3. Using the TFKO dataset, we ran NP3 as usual and reran it without the DE score (Fig. 2A , solid and dashed lines, respectively). Excluding the DE score reduced the performance of NP3, suggesting that the DE score includes useful information that was not captured by LASSO or BART, even though LASSO and BART were run on the same TF perturbation profiles. Next, to estimate the performance of NP3 when TF perturbation profiles are not available, we reran it without the DE score, but we also replaced TF perturbation profiles with expression profiles for non-TF perturbations from the same dataset (Fig.