DOI: 10.5281/zenodo.3897691
title: NJR-1 Dataset

description: NJR is a Normalized Java Resource. The <em>NJR-1</em> dataset consists of 293 Java programs that can be used with several analysis tools. <strong>TOOLS THAT RUN ON NJR-1</strong> Each program runs successfully with the following 14 Java static analysis tools: SpotBugs (https://spotbugs.github.io) Wala (https://wala.github.io) Doop (https://bitbucket.org/yanniss/doop) Soot (https://github.com/soot-oss/soot) Petablox (https://github.com/petablox/petablox) Infer (https://fbinfer.com) Error-Prone (http://errorprone.info) Checker-Framework (https://checkerframework.org) Opium (Opal-framework) (https://www.opal-project.de) Spoon (https://spoon.gforge.inria.fr) PMD (https://pmd.github.io) CheckStyle (https://checkstyle.org) JavaParser (https://javaparser.org/) Codeguru* (https://aws.amazon.com/codeguru) In addition to these static analysis tools, the NJR dataset has also been tested with 9 other tools that operate on Java bytecode. Jacoco (https://www.jacoco.org): Dynamic analysis tool Wiretap (https://github.com/ucla-pls/wiretap): Dynamic analysis tool JReduce (https://github.com/ucla-pls/jreduce): Bytecode reduction tool QueryMax (https://doi.org/10.5281/zenodo.5551128): Preprocessor for application code analysis Call-Graph Pruner (https://doi.org/10.5281/zenodo.5177161): Static call-graph pruning tool FootPatch (https://github.com/squaresLab/footpatch): Automated Repair Tool Procyon (https://github.com/ststeiger/procyon): Decompiler CFR (https://www.benf.org/other/cfr/): Decompiler Fernflower (https://github.com/fesh0r/fernflower): Decompiler <strong>BENCHMARK PROGRAMS</strong> The NJR programs are repositories picked from a set of Java-8 projects on Github that compile and run successfully. Each of these programs comes with a jar file, the compiled bytecode files, compiled library files, and the Java source code. The availability of the files in both jar-file form, as well as source code form (with the compiled library classes) is a major reason the dataset works with so many tools, without requiring any extra effort. These features make NJR-1 a great benchmark for any kind of Java static analysis, dynamic analysis, or tool building. Internally, each benchmark program has the following structure: <em>src:</em> directory with source files. <em>classes:</em> directory with class files. <em>lib:</em> compiled third party library classes (source files not available, since libraries are distributed as class-files). <em>jarfile:</em> jar file containing the compiled application classes and third-party library classes. <em>info:</em> directory with information about the program. It includes the following files. <em>classes:</em> list of application classes (excludes third-party library classes). <em>mainclasses:</em> list of main classes that can be run. <em>sources:</em> list of source file names. <em>declarations:</em> list of method declarations categorized by source file name. The benchmarks already come with a compiled JAR file, but some tools need to compile and run the benchmarks. The following simple commands can be used for compilation and running (replace &lt;jarfilename&gt; with the file in the <em>jarfile</em> directory. replace &lt;mainclassname&gt; with any of the classes from info/mainclasses): <em>javac -d compiled_classes -cp lib @info/sources</em> <em>java -cp jarfile/&lt;jarfilename&gt; &lt;mainclassname&gt;</em> <strong>FILES AVAILABLE FOR DOWNLOAD</strong> There are 4 files available for download: <em>njr-1_dataset.zip, scripts.zip, benchmark_stats.zip, and a Readme.</em> <em>njr-1_dataset.zip</em> has the actual dataset programs. <em>scripts.zip</em> contains Python3 scripts for each tool, to run it on the entire dataset. The Readme details the version number, download link and setup instructions for each tool. The <em>benchmark_stats.zip </em>file lists some statistics for the benchmark programs. <strong>STATISTICS</strong> Here are some summary statistics about the benchmark programs: The mean number of application classes: 97 Each program executes at least 100 unique application methods at runtime. The mean lines of application source code: 9911 The mean number of 3rd party library classes: 2608 The mean (estimated) lines of 3rd party library source code: 250,000 The mean number of static edges in the application call graph: 1404 The mean number of dynamic edges in the application call graph: 469 <strong>NOTES</strong> Note: Zenodo shows some changes for this repository. However, all the changes involve updating the scripts folder, as more tools get tested on the dataset. The programs in the dataset themselves remain unchanged. *Note 2: Codeguru Reviewer is a paid, proprietary tool by Amazon. Our experiments show that it runs successfully on all the benchmarks in this dataset. However, we don't include any scripts to replicate this run because of its paid nature. To cite this dataset, please cite the following paper:<br> Jens Palsberg and Cristina V. Lopes, NJR: a Normalized Java Resource. <br> In Proceedings of ACM SIGPLAN International Workshop on State Of the Art in Program Analysis (SOAP), 2018.
description: Funded by the following NSF grant (https://www.nsf.gov/awardsearch/showAward?AWD_ID=1823360&HistoricalAwards=false)