dataesr/scientific-paragraphs-categorization huggingface dataset:
We have attempted to overcome these limitations by conducting a two-fold survey attempting, in the first phase, to identify Ultras groups and provide a list, and then explore the archives posted on social media for references to recordings of songs and to eventually collect them in a single database. Such an endeavor allowed for the selection of songs with embedded g criticism content while demonstrating their prevalent highlights in the present paper. We have then presented the most prominent units of meaning while measuring their evolving frequency across the text and exploring a three-phase evolution across the period spanning over 2007-2020. This study remains exploratory, but its methodological approach can lead to the development of a data base of relevant materials and appropriate research tools, with a view to bundling audio archives of Ultras groups together.
INTRODUCTION Fuzzers are tools designed to execute a target application with a large number of automatically-generated inputs. Their goal is to discover problematic states, often associated with the presence of security vulnerabilities. Because of their effectiveness, fuzzers have become an essential asset in the arsenal of both developers and security researchers. Many off-the-shelf fuzzers are available to the public, some of which are now considered de-facto standards for general-purpose applications: AFL  [76] , AFL ++  [27] , HonggFuzz  [69] , and LibFuzzer  [47]. These fuzzers are very popular among security testers and, for example, routinely discover thousands of bugs on OSS-Fuzz  [2] , an extensive fuzzing effort for open-source software. Unfortunately, while off-the-shelf fuzzers are great tools that are easy to set up and use for non-experts, they often show their limitations for experienced users.
Thus, it is often difficult to understand whether better results are only due to the novel algorithm and not the result of other components of the fuzzer. (3) Different solutions are difficult to compare. While dozens of different techniques exist for every aspect of fuzzing, a third-party comparison would require a considerable re-implementation effort, typically reserved for surveys and systematization of knowledge papers. As a result, only a selected amount of solutions have been properly tested and compared on the same datasets. We believe that these three issues are essential roadblocks that significantly slow down the progress of fuzzing, the transition of new techniques from academia to industry, and the development of new solutions, due to an extensive duplication of work. Therefore, we propose LibAFL, a novel fuzzing framework written from scratch in Rust.
Defining the observers in this way is particularly useful when a distributed fuzzer can send the observers state across multiple nodes. This avoids the need for re-execution with the same input when fuzzing a very slow target. An example observer is the coverage map, used by common coverage-guided fuzzers such as AFL or HonggFuzz. The map is filled during execution to report the executed edges. This information is not preserved across runs and it is an observation of a dynamic property of the program. Other fuzzers, such as Ijon  [5]  or FuzzFactory  [58] , use different forms of observers but always rely on a map to keep track of additional metrics beyond code coverage. Executor -The Executor is the component responsible to execute the target system given an input from the fuzzer. In different fuzzers, the embodiment of this entity may change a lot.
We also provide Python bindings to the Sugar crate for quick prototyping without recompilation. For instance, Listing 1 shows a simple fuzzer that bridges a LibFuzzerstyle harness to a fuzzer that uses generic bit-level mutations and executes the target in-process written using the high-level APIs of LibAFL Sugar. At the time of writing, the entire LibAFL framework, tests included, consists of 53k lines of Rust and 15. 4k lines of C/C++.
4. 2. 3 The Fuzzer. The Fuzzer is a recipient for the operations that define what the fuzzer can do. It contains the Feedbacks, the Objectives, and the Scheduler, all independent operations that may alter the fuzzer state. These stages are separated from the fuzzer to respect the borrowing rules of Rust 1 , as they may invoke some operations that alter Fuzzer and State at the same time. The Fuzzer, in addition, provides the definition of how a single testcase should be processed and how to evaluate a new input. By default, the standard implementation of it, which consists of feedback-driven fuzzing, defines FuzzOne, the operation responsible to process a single testcase, as the invocation of the scheduler to get the testcase to fuzz and the invocation of every stage on the testcase. InputEvaluation, the operation that evaluates if an input must be added to the corpus, is by default the execution of the target program and the decision if it is interesting or a solution using the feedbacks.
Instrumentation Backends LibAFL can be easily plugged into any instrumentation backend, like a binary translator or a simple compiler instrumentation pass. By default, we provide additional libraries that tie LibAFL with some popular instrumentation backends: LLVM  [42] , SanitizerCoverage  [44] , QEMU usermode  [9]  and Frida  [1]. The runtime in LibAFL Targets can be linked to any Sanitizer-Coverage target, adding coverage and comparisons tracking to the fuzzer, using a compiler flag. The SanCov support allows users to create frontends that are compatible with non-C/C++ SanCov-enabled targets, such as Atheris  [40]  for Python and cargo-fuzz  [8]  for Rust. LibAFL CC provides a set of LLVM passes to extend Clang and other LLVM-based compilers to track edge coverage, context-sensitive and K-context-sensitive  [7]  edge coverage, N-gram coverage  [71] , coverage accounting  [73] , comparisons with CmpLog  [6, 27]  and dictionary tokens with autotokens, and an enhanced version of AFL ++ 's dict2file pass that extracts the tokens from interesting functions such as strcmp.
LibAFL QEMU bridges QEMU usermode, and full system in the near future, to Rust with a novel emulator API with hooking capabilities to have programmatic control over the target's execution. Around this interface to QEMU, the library exposes structures like executors and helpers to install default hooks for common fuzzing tasks, such as edge coverage tracking, guest snapshot-restore, and binary-only ASan  [26]. LibAFL Frida offers similar capabilities to the QEMU bridge, but with the features of a DBI, without a clear host-guest separation. It includes a binary-only ASan and, unlike QEMU usermode, it can work on various operating systems other than Linux such as Windows, macOS, and Android. Instrumentation capabilities in LibAFL are also offered for concolic execution, through LibAFL Concolic and its Rust bridges to SymCC  [60]  and SymQEMU  [61].
After discussing how these techniques are implemented, we evaluate them in three different sets of experiments: (1) We measure the performance in terms of code coverage and bug detection of several approaches implemented and readyto-use in LibAFL; (2) We show how we can combine orthogonal approaches implemented in our framework to build new and never evaluated before fuzzers and measure their performance; (3) Lastly, to show the efficiency of our framework in a traditional context, we compare and evaluate a new generic bit-level fuzzer based on LibAFL using the previously presented techniques against other state-of-the-art fuzzers like AFL ++ and HonggFuzz on FuzzBench  [54] ; ✓ ✓ Auto-tokens  [27]  ✓ Value-profile  [47, 58]  ✓ Block coverage accounting  [73]  ✓ ✓ Function coverage accounting  [73]  ✓ ✓ Loops coverage accounting  [73]  ✓ ✓ Corpus culling scheduler  [76]  ✓ Weigthed scheduler  [27]  ✓ FAST power schedule  [14, 27]  ✓ COE power schedule  [14, 27]  ✓ EXPLORE power schedule  [14, 27]  ✓ MOpt  [48]  ✓ Nautilus  [4]  ✓ Grimoire  [10]  ✓ Gramatron  [68]  ✓ Token-level  [63]  ✓ NeoDiff  [50]  ✓ LIN power schedule  [14]  ✓ QUAD power schedule  [14]  ✓ EXPLOIT power schedule  [14]  ✓ SymCC  [60]  ✓ ✓ SymQEMU  [61]  ✓ ✓ Hitcounts  [76]  ✓ ✓ Ngram coverage  [71]  ✓ Context-sensitive coverage  [17]  ✓ QASan  [26]  ✓ Atheris (compatibility)  [40]  ✓ cargo-fuzz (compatibility)  [8]  ✓ We ran the first two sets of experiments on a x86_64 machine equipped with an Intel ® Xeon ® Platinum 8260 CPU with a clock of 2.
40 GHz. The dataset is a subset of the FuzzBench suite, selected to include programs with diverse features. We run each session for 24 hours and repeated each experiment five times to mitigate the effect of randomness in fuzzing. For the comparison with other fuzzers, we run the experiments on the full FuzzBench suite on the service offered by Google. Each run was 23 hours long and repeated 20 times. The benchmarks suite provides the initial corpus for every benchmark and we used the default setting. The overall ranking that we discuss is based on the average normalized score computed by FuzzBench, which represents the percentage of the highest reached median code or bugs coverage on a given benchmark. Finally, in the last part of the section, we present a case study about how easy, we can implement with LibAFL a fuzzer that is different from the traditional setups like the ones shown in the first part, a differential fuzzer that can spot logic bugs in Ethereum VMs with a different kind of feedback based on the state of the VM instead of code coverage.
Bypassing Roadblocks An important field of research in fuzz testing is the development of new techniques to increase code coverage by bypassing hardto-solve constraints. For instance, multi-byte comparisons pose a severe problem for fuzzers that employ a generic bit-level mutator, since random generic mutations cannot bypass these comparisons as the solution space is huge and blind guessing impractical. LibAFL provides several ready-to-use techniques to implement fuzzers that can overcome these roadblocks. The first, in chronological order of appearance in the literature, is the value-profile  [45]  proposed by LibFuzzer in 2016. This technique tries to solve comparison instructions by maximizing the number of matching bits between the two operands of the instruction. In LibAFL, this is implemented with a map observer and a feedback that maximizes the entries of a map and considers an input as interesting when at least one new max is discovered.
In LibAFL the grammar preprocessing utility is provided as a tool and the associated structures are specular to the ones used by Nautilus, with a different underlying implementation. As an example of approaches that perform grammar learning, LibAFL implements Grimoire  [10] , a fuzzer that uses the portion of inputs that induced the novelty in coverage as tokens to build generalized "tree-like" inputs and perform grammar-like mutations. It also employs token-level fuzzing  [63] , an approach based on token extraction with a lexer. While the original solution was specific to JavaScript, our implementation is generic and can be applied to any programming language. Classic bit level mutations are then applied to encoded inputs and that is decoded before the target execution. To evaluate these three approaches we decided to use the number of uncovered bugs instead of code coverage.
In fact, the effectiveness of this type of fuzzer is less dependent on code coverage, especially when compared to variants that generate invalid inputs and inflate the coverage by exploring error paths. In this experiment, as both Grimoire and token-level require some initial seeds, we used Nautilus to generate 4096 initial inputs for these two fuzzers to avoid a bias due to the quality of the seed corpora  [36]  provided by FuzzBench. Table  3  shows the number of bugs found by each variant over 3 popular compilers. Grammar-aware approaches are still superior, with a huge boost for Nautilus on 2 benchmarks over 3, but the performance of the token-level approach    is surprisingly similar to Nautilus on PHP, given that they start from similar seed inputs. Given the promising results of this experiment, we decided to investigate whether we can further improve the best performer, Nautilus, by combining it with other orthogonal techniques provided by LibAFL.
Among the six, the most effective 5  are explore, the default in AFL, and coe and fast, the default in AFL ++. Therefore, we will focus our tests on these three algorithms and compare them with a baseline (never evaluated before) based on the plain algorithm. Table  6  shows the results in terms of code coverage. Overall, considering the average normalized score across all benchmarks, explore is the best performer (99. 72), with fast following (99. 43) and then plain (98. 12) and coe (97. 08). This results confirm the trend observed in the AFL ++ version of the power schedules, with fast and explore as top performer, with fast as the now default schedule in AFL ++. The LibAFL implementations emphasize once again the same observations we did for the corpus scheduling problem. Fast fuzzers (on fast targets) reduce the benefit that can be gained by using more complex scheduling as less useful executions have a limited effect on the throughput.
For this particular experiment, we submitted a request to the FuzzBench service  6  , to execute the four fuzzers on 22 different benchmarks and evaluate the reached code coverage. Each fuzzer was scheduled for 23h and every single experiment was repeated 20 times to mitigate the randomness. The results for each benchmark are reported in Table  7. The overall result, based on the average normalized score of the coverage uncovered across all the targets, is that LibAFL clearly outperforms all the other fuzzers with a score of 98. 61, followed by HonggFuzz (96. 65), AFL ++ (96. 32) and Entropic (94. 22). This result is even more relevant since the other fuzzers were gradually improved over time  [54]  based on the outcome of several FuzzBench runs, while LibAFL has been developed independently from this benchmarking suite.
By inspecting the results in isolation, we can see that LibAFL shines on 3 benchmarks, harbfuzz, openthread and sqlite3. Notably, it is the only fuzzer that can reach the coverage breakthrough by unlocking the fuzzer from saturation in a few runs on mbedtls and consistently on openthread. On the other hand, our fuzzer clearly underperforms AFL ++ and HonggFuzz on libpng, resulting in an almost equal performance with Entropic. The missing coverage of these two fuzzers can be explained with the execution model that they use, in-process, versus the out-of-process executor that the other two uses. The latter is slower but more reliable in handling timeouts. The strength of our approach is that limitations like this one can be easily overcome by changing a few lines of code in the frontend, in this particular case to move from InProcessExecutor to ForkserverExecutor.
Differential Fuzzing In the previous sections, we presented variants based on coverageguided fuzzing. However, LibAFL is not limited to code coverage and to its derivates (like context and ngrams  [71] ), but can also work with other types of feedback. As an example, in this section, we discuss the development of a frontend, inspired by NeoDiff  [50] , for differential fuzzing of two Ethereum virtual machines. In short, NeoDiff, originally written in Python, compares the outcome of the executions of two VMs provided with the same input. To do so, it uses a state hash, a hash of the registers values, memory, and a probabilistic sampling of the stack at each instruction site of the executed trace. As feedback to evolve its corpus, it uses instead a type hash, the hash of the opcodes and the types of the first two items on the stack for each instruction.
In terms of unique diffing instructions, the original paper reports that 6 instructions are the causes of the found differences. During our experiment, we reproduced the NeoDiff results finding only 5 instructions over 12h with NeoDiff and 15 instructions with our implementation. The diffing opcodes in details are (in hex): 3, 31, 38, 3b, 3c, 3f, 44, 45, 46, 52, 5a, f1, f2, f4, fa. The opcodes found by both the original and our NeoDiff implementation are in bold, the others were discovered only by the LibAFL-based variant. Our findings are so a superset of the 6 NeoDiff instructions, showing that LibAFL is outperforming the python implementation by a huge margin.
Third-party Applications During its development, LibAFL had already been adopted to implement several fuzzing frontends by a number of new users, who had no previous experience with our framework. For instance, it has been used to create a symbolic-model-guided fuzzer, tlspuffin  [3]  that employs a concrete semantic to execute TLS symbolic traces, thus proposing a new approach that mixes fuzzing and model testing. Thanks to this combination, tlspuffin can reach critical protocol states that are impossible to find with classic coverage-guided fuzzing. A second third-party application of LibAFL is a snapshot fuzzer based on KVM, Tartiflette  [19] , which provides a new executor to run a Linux ELF as a VM with system calls emulation and instrumentation facilities for coverage tracking and snapshotting.
LIMITATIONS AND FUTURE WORK While extensible by design, the current implementation of LibAFL still lacks some components that are required to implement some specific fuzzing applications. For instance, at the time of writing LibAFL CC does not include Link Time Optimization passes to reason about the whole program Control Flow Graph. This type of instrumentation is required to implement most of the directed fuzzing approaches  [13, 16, 57]  and thus, LibAFL is not currently providing any directed fuzzing application. This limitation, however, is not intrinsic to our design and support for directed fuzzing will be integrated in the near future. A powerful feature integrated into LibAFL is the concolic tracing API, which can be used to extend SymCC or SymQEMU with custom constraints filtering and communicate the symbolic trace to a LibAFL-based fuzzer.
CONCLUSION In this paper, we presented a novel and completely extensible fuzzing framework, LibAFL. To show its versatility, and the comprehensiveness of its ready-to-use components to build state-of-the-art fuzzers, we presented several frontends based on LibAFL and performed experiments with them covering different problems in the fuzzing literature. We highlighted the customization allowed by the LibAFL design and the power of the combination of several orthogonal techniques, leading to a fuzzer that outperforms the best publicly available tools. To allow the reproduction of our results and promote Open Science, every frontend for the experiments in this paper and the related setup to run them in FuzzBench is available online at https: //github. com/AFLplusplus/libafl_paper_artifacts.
ACKNOWLEDGEMENTS Firstly, we want to thank the community around the AFL ++ organization, especially our contributors. A special thanks to s1341 and  Marc Heuse  for the amount of work put into LibAFL, as well to our GSoC students that worked on it in the past,  Rishi Ranjan  and  Julius Hohnerlein. We would like to thank also the anonymous reviewers of  ACM CCS  for their constructive reviews and  Slasti Mormanti  for the useful tips. This project has been partially funded by the  Defense Advanced Research Projects Agency (DARPA)  under agreement number  FA875019C0003.
During the last decades, metagenomics has highlighted the diversity of microorganisms from environmental or host-associated samples. Most metagenomics public repositories use annotation pipelines tailored for prokaryotes regardless of the taxonomic origin of contigs. Consequently, eukaryotic contigs with intrinsically different gene features, are not optimally annotated. Using a bioinformatics pipeline, we have filtered 7. 9 billion contigs from 6,872 soil metagenomes in the JGI's IMG/M database to identify eukaryotic contigs. We have re-annotated genes using eukaryote-tailored methods, yielding 8 million eukaryotic proteins and over 300,000 orphan proteins lacking homology in public databases. Comparing the gene predictions we made with initial JGI ones on the same contigs, we confirmed our pipeline improves eukaryotic proteins completeness and contiguity in soil metagenomes.
Previous studies have highlighted the poor representation of eukaryotes in standard metagenomics analyses in different environmental samples and proposed strategies to mitigate this under-representation  20, 21. The largest publicly available resource for soil metagenomes is the Integrated Microbial Genomes & Microbes (IMG/M) database of the Joint Genome Institute (JGI)  22. In this resource, standard pipelines are used to assemble and annotate contigs and genomes from environmental metagenomic shotgun reads. One major limitation concerns the eukaryotic component of these soil metagenomes. Indeed, the gene prediction tool used by default for all contigs assembled from metagenomes is Prodigal  23  , a software tailored for prokaryotes. However, gene structures and features are different in eukaryotes, and using prokaryotic tools to predict eukaryotic genes can lead to incomplete, erroneous and discontinuous gene sequences, and hence proteins: a trivial example is that no intron can be predicted by Prodigal.
These procedures make sense given the volume of metagenomic data processed by IMG/M, but, as a consequence, eukaryotic proteins are neglected in these soil microbiome data, with a risk of being truncated and assigned an unreliable taxonomic annotation. These suboptimal sequences and taxonomic annotations then negatively impact any research on the eukaryotic component of the soil. To circumvent this problem, we have constituted a dataset of 6,872 soil microbiomes comprising 7. 9 billion contigs and identified eukaryotic contigs using a k-mer based approach. On the identified eukaryotic contigs, we re-predicted ca. 93 million genes and proteins using annotation methods tailored for eukaryotes. We re-assigned taxonomic information to these proteins based on a last common ancestor (LCA) approach from homology search against the NCBI's nr library.
This allowed identifying 8 million eukaryotic proteins and more than 300,000 orphan proteins located on eukaryotic contigs and lacking homology in public protein libraries, representing a potential for new discoveries. We show that the newly predicted proteins are longer and constitute a more comprehensive representation of the pool of eukaryotic proteins in the soil. This new dataset improves eukaryotic protein sequence quality and completeness, as well as the reliability of the taxonomic information, and represents a unique resource to decipher and study the pool of eukaryotic proteins present in the soil.
Methods Data collection. We used publicly available assembled metagenomic data from shotgun sequencing reads of the IMG/M database of the JGI  22. We collected metagenomes of 5,988 'Terrestrial' samples in the environmental metagenomes category and 884 plant-associated metagenomes in the host-associated category, Fig. 1  (available data 2020, October; Supplementary Data  24  ). Most of the datasets were unrestricted from use, according to the JGI policy; the authors of a few datasets (see Acknowledgements) that were still under use-restriction kindly authorized us to re-use their data, including two published in the literature  25, 26. The data acquisition was performed via the IMG/MER Cart genome portal. For each metagenome, the JGI provides a set of files from pre-computed analyses that are useful to sort, filter and describe data.
Because we anticipated substantial differences in the relative proportions of eukaryotic species present in the terrestrial and the host-associated categories, these two datasets were processed separately to minimize potential biases. For a more convenient processing of the massive amount of data, the metagenomes from terrestrial samples were splitted in two batches; 'Terrestrial 1' contained 3,601 environmental metagenomes added between December 2009 and January 2019 and 'Terrestrial 2' contained 2,387 metagenomes added between February 2019 and August 2020.