Package: slrkit
slr-kit: a tool to assist the writing of Systematic Literature Reviewsslr-kitis a tool that is intended to assist in the development of a Systematic Literature Review (SLR) of scientific papers related to a specific scientific field.The tool is based on a semi-supervised workflow that leverages automatic techniques based on text mining to automatize key steps in the workflow, while some manual operations are necessary to provide the expert knowledge to the process.slr-kitaims at making the manual operations as efficient as possible.It analyzes the abstracts of scientific papers to derive the topics associated to the papers, allowing the clustering of papers by topic and the derivation of useful statistics about the set of papers and their topics, such as the trend of publications per topic during the years, and the amount of papers published on different journals by topic.As a byproduct,slr-kitcan be used to manage a list of terms related on a specific topic, curated by an expert, which can be useful in many NLP applications.Table of contentsInstallationUsageThe workflow on data from scratch (basic version)Using existing dataLinks to other docsInstallationUsing pipslrkitis available on PyPi and can thus be installed aspip install slrkitThe documentation below refers to the installed program though pip.From git repositoryClone the repository:git clone <repository address>Install the dependencies:pip install -r requirements.txtIn this case, the command can be run, from within the repository directory, aspython3 slrkit/slrkit.pyInstallnltkcorporaAfter installingslrkitand its dependencies, the corpora forNLTKmust be downloaded.
The following command can be used:python -m nltk.downloader popularUsageIfslr-kitwas installed from the git repository, you can launch theslrkitcommand.
Use the-hor--helpoption to generate the following help:$ slrkit --help
usage: slrkit [-h] [-C path] [--version]
              {init,import,journals,acronyms,preprocess,terms,fawoc,postprocess,topics,report,record,stopwords,build,readme}
                 ...

slrkit project handling tool

options:
  -h, --help            show this help message and exit
  -C path               Change directory to 'path' before running the specified command.
  --version, -V         show program's version number and exit

slrkit commands:
  {init,import,journals,acronyms,preprocess,terms,fawoc,postprocess,topics,report,record,stopwords,build,readme}
    init                Initialize a slr-kit project
    import              Import a bibliographic database converting to the csv format used by slr-kit.
    journals            Subcommand to extract and filter a list of journals. This command accepts a
                        subcommand. If none is given, "extract" is assumed.
    acronyms            Extract acronyms from texts.
    preprocess          Run the preprocess stage in a slr-kit project
    terms               Subcommand to extract and handle lists of terms in a slr-kit project. This
                        command accepts a subcommand. If none is given, "generate" is assumed.
    fawoc               Run fawoc in a slr-kit project. This command accepts a subcommand. If none is
                        given "terms" is assumed.
    postprocess         Run the postprocess stage in a slr-kit project
    topics              Subcommand to extract the topics from the documents in a slrkit project. This
                        command requires a subcommand.
    report              Run the report creation script in a slr-kit project.
    record              Record a snapshot of the project in the underlying git repository
    stopwords           Extracts the terms classified as stopwords from the terms file
    build               Run all the commands needed to re-create all the files after cloning a slr-kit
                        project.
    readme              Creates a README.md file for the project using information from the META.toml
                        file. The README.md is also committed to the git repository.Each sub-command is explained in details theslrkit.mddocumentation file.The workflow on data from scratch (basic version)The organization of the information is based onprojects.
A project is basically a directory containing all the necessary data related to the SLR.
Such data are manipulated using theslrkitcommands.All the data, being final or intermediate results, are stored in easy to inspect CSV files.The workflow supported byslr-kitincludes several optional steps that can be done to refine the final results.
Here it is reported the mandatory steps to obtain the desired reports starting from the bibliographic material.The basic steps are:initialization of the project (automatic)import of bibliographic data (automatic)preprocessing of the abstracts (automatic)extraction of terms (automatic)classification of terms using FAWOC (manual)topics extraction (automatic)generation of reports (automatic)Anytime during the various steps, you can run the commandslrkit recordthat will produce a snapshot of the current information required by an slr-kit project.
Note that the snapshots are maintained as standard git repositories.
Therefore, you can use the regular git commands to operate on the repository (git log, git status, git checkout, etc.).
Moreover, you can use the usual git push/pull to move the dataset to/from remotes.Initialization of the project (automatic)Create a new directory to host theslr-kitproject, and move into the newly created directory (we usemy-projectas example project):mkdir my-project && cd my-projectInitialize the project:slrkit init my-projectImport of bibliographic data (automatic)At the moment, only theRIS file formatis supported as source data file.
The RIS format is one of the many available on, e.g.,Scopus.
Make sure that the file contains the Abstract of the papers.Copy the bibliographic data into the project directory, naming itmy-project.ris.To makeslrkitaware of the name of the bibliographic data, set the value of theinput_fileparameter into the file configuration fileslr-kit.conf/import.toml(this file was generated by theinitcommand) as follows:input_file = "my-project.ris"To import the data into the project, run the command:slrkit importPreprocessing of the abstracts (automatic)This operation performs several text processing operations on the abstracts, such as stemming and removal of punctuation, preparing the text for further processing.The command to run the preprocessing is:slrkit preprocessExtraction of terms (automatic)Extract the list of terms from the abstracts:slrkit termsClassification of terms using FAWOC (manual)Run FAWOC and manually classify the terms:slrkit fawoc termsFAWOC is an interactive terminal program that allows to quickly (well, some time is required depending on the amount of terms) classify the terms.For this basic example, you can distinguish the terms between RELEVANT (pressingr) and NOISE (pressn) terms.
If you make some mistake, pressuto undo.Postprocessing (automatic)This operation processes the documents by managing the terms classified in the previous stage.
Run it withslrkit postprocessExtraction of topics (automatic)Run the text mining procedure with the command to generate the topics:slrkit topicsAlternatively, you may want to run an optimization in the generation of the topics with:slrkit topics optimizeThis operation takessignificantly more time, but produces results that arefar more accurate.HINT: Use the optimization everytime a "production ready" result is required.Generation of reports (automatic)Generate the reports with:slrkit reportThe reports are stored in a newly generated unique directory whose name has the format<timestamp>_<UUID>_reports/.
In the directory, you will find some.md,.texand.pngfiles that report the graphs and tables with the results.
The reports are generated according to some templates in Markdown and LaTeX format for being easily edited or directly included in scientific papers or websites.Using existing dataslr-kit can also be used on an existing dataset, i.e., a set of papers that were collected by someone else and already (even partially) elaborated.In this example, we assume that the dataset was properly recorded usingslrkit record, and all the terms have been already classified.Clone a repository with labelled dataA dataset is kept versioned using git.
Therefore, if the dataset is shared in some online repository, you can simply download it withgit clone <repository>A repository containing the classification of a dataset about Natural Language Processing is available here:https://github.com/robolab-pavia/slrkit_NLP_Nocera.Regenerate all the necessary dataFirst, you need to re-build all the preprocessed data.
This is done with one single command:slrkit buildAt this point, if necessary, you can refine the classification of terms by running the usual commandslrkit fawoc termsGenerate topics and reportsNow you are ready to generate the topics and the reports with the same commands explained above:slrkit topics optimizefollowed byslrkit reportLinks to other docsdocumentation of thecommands ofslr-kitdocumentation of thescriptsFAWOCutilities and analysis tools: TBA