DOI: 10.5281/zenodo.17917984
title: Req2Test: An Approach for the Automatic Generation of Unit Tests from User Stories Using LLMs

description: This Zenodo repository contains supplementary artifacts for the article:  Req2Test: An Approach for the Automatic Generation of Unit Tests from User Stories Using LLMs.

It provides datasets, generated tests, and experimental materials to support reproducibility and evaluation of the proposed approach.

 

Repository Contents

Evaluation

1. Phase1 - Controlled Experiment

1.1 Specifications – User Stories and DSL

Specifications of four projects in User Story format and their corresponding Domain-Specific Language (DSL) representations.

 

1.2. Generated Unit Tests

Unit tests generated by the Req2Test Tool for the four projects, covering various use cases and system functionalities.

 

1.3. Generated Support Artifacts

Artifacts produced during the test generation pipeline, including:

- Test scripts

- Json data test

- Test cases

 

1.4. Generated Prompts

Prompts created by Req2Test Tool for:

- Test data generation

- Automated generation of unit test classes

These prompts serve as a reproducibility baseline, since executing them sequentially leads to the generation of the unit test classes. They are produced by Req2Test Tool, which uses the DSL and test templates to create the prompts that guide the automated creation of the tests.

 

1.5. Projects of Controlled Experiment

Detailed materials for the controlled experiment, including:

- Experiment instructions (with links to the forms used)

- Injected bugs

- Participant response spreadsheets

- Projects used in the experiment:

    - buggy-without-tests

    - buggy-with-tests

    - correct-with-tests

    - correct-without-tests

The original materials are in Portuguese; English translations are also provided.

 

Projects Requirements

- Java 17 or higher

- Maven (for building and running generated tests)

 

2. Phase2- Comparison - EvoSuite x Req2Test

2.1 reports-instruction-coverage-and-mutation-score

Detailed results for five runs across the four projects using Req2Test and EvoSuite, including Instruction Coverage and Mutation Score.

2.2 Projects - Tests generated using EvoSuite

All four projects were executed with five runs each using EvoSuite. The generated tests are available in the directories evosuite-tests-run1 to evosuite-tests-run5, under src/test/java and src/test/resources.

Projects Requirements

- Java 8

- Maven (for building and running generated tests)

2.3 Projects - Tests generated using Req2Test







Generated Artefacts – All unit tests and prompts produced for the four projects, with five runs per project.

Projects - All four projects were executed with five runs each using Req2Test. The generated tests are available under src/test/java and src/test/resources.




Projects Requirements

- Java 17

- Maven (for building and running generated tests)

 

3. Phase3 - QuixBugs

3.1 Req2Test-Generated

All unit tests and prompts produced for the four projects, with five runs per project.

3.1 Correct

A project with 24 correct algorithms adapted to the Req2Test architecture, including tests for all 24 algorithms generated with Req2Test, each executed five times.

 

DSL

DSL-Projects-and-QuixBugs



DSL-Projects- This document presents the project specifications from the dataset-api repository, written in the Req2Test DSL. It includes four REST API projects, detailing their business rules, validation constraints, and API responses.

DSL-Quixbugs - This document presents the 24 QuixBugs algorithms written in the Req2Test DSL, describing each algorithm’s inputs, constraints, and expected behavior.


 

Related Repository

The repository and tool used to generate these artifacts are available at:  

Req2Test Tool - https://github.com/specgentest/req2test-tool

API dataset of the projects used in the experiment - https://github.com/specgentest/dataset-api