DOI: 10.5281/zenodo.15511661
title: Research Artifacts of Reimagining Studies' Replication: A Validity-Driven Analysis of Threats in Empirical Software Engineering

description: Research artifacts


The archived artifact is also publicly available via GitHub.

 



📋 Overview


This repository contains the dataset, scripts, and documentation for the paper Reimagining Studies' Replication: A Validity-Driven Analysis of Threats in Empirical Software Engineering published at Proceedings of the 39th Brazilian Symposium on Software Engineering - Insightful Ideas and Emerging Results (IIER '25) in 2025.

The study is a partial result of a Systematic Literature Review (SLR) that investigates how replication studies in Software Engineering address Threats to Validity (TTVs) of original and replication studies. It examines how these threats are identified and reported, as well as the implications for the reliability and generalizability of replicated results.

 



📂 Repository Structure




SLR2025/
├── data/
│   └── material/
│   └── scripts/
│   └── SLR-data/
├── LICENSE
├── LICENSE-MIT
├── Paper.pdf
├── README.md





📂 data/


Contains all materials, data, and scripts used or produced in the study.



📂 data/material/


Research planning material: search string, inclusion/exclusion criteria, SCOPUS exports, and threats to validity.



📂 data/scripts/


R scripts and supporting files for generating figures and performing statistical analysis.



📂 data/SLR-data/


Extracted data and quantitative summaries from the SLR process.



📄 Paper.pdf


The full paper, related to this artifact, describing the study's goals, methodology, analysis, and findings.



📄 LICENSE / LICENSE-MIT


Define the usage terms for the data and scripts.



📄 README.md


This file — includes an overview, usage guide, and citation instructions.

 



💾 Storage Requirements


The total size of this repository is under 20 MB. No special storage requirements are needed.

 



🛠 How to Run the Scripts




⚙️ Requirements


To run the scripts, you need:





R installed on your machine.




RStudio (optional, but recommended).





🖥️ System Requirements


No specific hardware requirements are needed to execute the scripts.



🛠 Steps to Run


1. Download and install R.

2. Download and install RStudio (optional).

3. Download an R script from the scripts folder.

4. Open an R script.



Open only one script at a time.


5. Run the script using:



Mac: Cmd + Shift + Enter

Windows/Linux: Ctrl + Shift + Enter


6. If prompted, allow R to install the required packages (e.g., ggplot2, dplyr, tidyr).R may ask for your permission to install missing packages the first time you run the script.

7. The output (e.g., plots) will be saved in the same folder as the script, using the same filename.



💡 Each script generates one figures used in the paper.




💡 For the Matthews Correlation Coefficient (MCC) script, ensure both the data file and R script are in the same directory.


 



📄 Citation




📝 Citing the Paper


If you use the data or findings from this repository, please cite the paper:



@inproceedings{AzevedoEtAl2025,
    author = {Ivanildo Azevedo and 
            Ana Paula Vasconcelos and 
            Eudis Teixeira and 
            Sergio Soares},
    title = {Reimagining Studies’ Replication:              A Validity-Driven Analysis of Threats in Empirical Software Engineering},
    booktitle = {Proceedings of the 39th Brazilian Symposium on Software Engineering -                 Insightful Ideas and Emerging Results (IIER '25)},
    location = {Recife/PE},
    year = {2025},
    issn = {2833-0633},
    publisher = {SBC},
    address = {Porto Alegre, RS, Brasil}
}

 




📦 Citing the Artifact


If you prefer to cite only the archived artifact at Zenodo:



@misc{AzevedoEtAl2025,
  author       = {Ivanildo Azevedo and
                  Ana Paula Vasconcelos and
                  Eudis Teixeira and
                  Sergio Soares},
  title        = {Research Artifacts of Reimagining Studies’ Replication: 
                A Validity-Driven Analysis of Threats in Empirical Software Engineering},
  year         = {2025},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.17064270},
  url          = {https://doi.org/10.5281/zenodo.17064270}
}

 


 



🛡️ Ethical and Legal Considerations


This study was conducted strictly for academic research purposes and adheres to institutional research ethics guidelines. No personal or sensitive data were collected.

 



📜 License




R scripts are licensed under the MIT License.

Research data and artifacts are licensed under the CC BY 4.0 License.