DOI: 10.5281/zenodo.17432435
title: GitHub Workflows in Open Source Repositories

description: Overview

This dataset provides a large collection of GitHub Actions workflows, their execution history, and associated metadata. The database contains information from more than 5000 unique repositories, capturing both workflow definitions and their runtime behavior across nearly one million workflow runs.

 

Important: This is a living dataset that is continuously growing. We publish monthly snapshots to provide stable reference points for research and analysis. The current snapshot reflects data collected through September 2025. Check back regularly for updated monthly releases containing new repositories, workflows, and execution data.

 

Collections

The dataset is organized into five interconnected collections:

 

1. workflow_files 

 

The primary collection containing workflow definitions and their metadata. Each document represents a single GitHub Actions workflow file from a repository.

 

Key Fields: 







 Repository identification (owner, repo, workflow_id)

 Workflow metadata (name, path, state, creation/update timestamps)

 Parsed workflow structure including jobs, steps, and configurations

Actions usage tracking (which GitHub Actions are used)

Trigger configurations (push, pull_request, schedule, etc.)

Programming language detection

Repository metadata (stars, forks, topics, license)

AI classification (whether the workflow involves AI-powered CI/CD)

Run statistics (total runs, success/failure counts, timing information)




Sample Use Cases:







Analyze workflow patterns and best practices

Study the adoption of specific GitHub Actions

Identify correlations between repository characteristics and workflow design

Track AI integration in CI/CD pipelines




 

 

2. workflow_runs 

 

Contains execution records for workflow runs, tracking when workflows were triggered and their outcomes.

 

Key Fields:







Unique run identifiers and associations to workflows

Execution metadata (status, conclusion, timestamps)

Trigger information (event type, branch, commit SHA)

Actor information (who triggered the run)

Pull request associations




3. jobs_of_runs 

 

Provides detailed information about individual jobs within workflow runs, including step-by-step execution data.

 

Key Fields:







Job identification and association to workflow runs

Execution status and timing (created, started, completed timestamps)

Runner information (runner ID, name, group)

Individual step details with status and timing

Conclusion for each step




Key Insights:







Average of 1.89 jobs per workflow run

Detailed step-level execution tracking including setup and teardown steps




4. workflow_files_commits 

 

Tracks the evolution of workflow files through commit history.

 

Key Fields:







Commit metadata (SHA, date, message, author)

File state at each commit (file SHA, size)

Parsed workflow configuration at that point in time

Actions and triggers used in each version




Sample Use Cases:







Study workflow evolution patterns

Analyze how teams iterate on CI/CD configurations

Identify common workflow improvements over time




 

Data Collection and Updates

 





Monthly Snapshots. The dataset is continuously growing as we monitor and collect new workflow data from GitHub repositories. To support reproducible research while providing access to the latest data, we publish monthly snapshots. Each snapshot is:









Versioned  by month and year (e.g., September 2025, October 2025)

Cumulative, potentially including previously collected repositories plus new additions

Independently restorable, allowing researchers to work with specific time periods

Documented with statistics reflecting the data contained in that particular snapshot




 

           Researchers can choose to work with a single snapshot for consistency or compare across multiple snapshots to study temporal trends in CI/CD practices.

 





Database Setup and Restoration



           Prerequisites







MongoDB installed on your system

Sufficient disk space (the dataset requires approximately 5-10 GB)

MongoDB tools including `mongorestore`




                   Connection String

 

                                   mongosh --host localhost --port 27017 --db github-workflows-september-2025

 



            Restoring from Backup


            If you have received this dataset as a MongoDB dump, you can restore it using the following command:

 

                                   mongorestore --host localhost --port 27017 --db github-workflows-september-2025 /path/to/dump/directory

 

            For a compressed archive:









 Extract the archive first





                                     tar -xzvf github-workflows-september-2025.tar.gz









 Then, restore the database





                     mongorestore --host localhost --port 27017 --db github-workflows-september-2025 ./github-workflows-september-2025

 

             Note: When working with multiple monthly snapshots, you can restore them to different database names to allow side-by-side comparison:

 

                   mongorestore --host localhost --port 27017 --db github-workflows-october-2025 ./github-workflows-october-2025

                   mongorestore --host localhost --port 27017 --db github-workflows-november-2025 ./github-workflows-november-2025