Package: mass_driver
Mass DriverSend bulk repo change requests.This repository is on Github:Overkillguy/mass-driver.Requires Python 3.11.See also the docs atjiby.tech/mass-driver/InstallationInstall the package:pip install mass-driverWe recommend you install CLIs viapipx, for
dependency isolation:pipx install mass-driverIf you want to install from a git branch rather than Pypi:pipx install https://github.com/OverkillGuy/mass-driverSee pipx docs:https://pypa.github.io/pipx/#running-from-source-controlRunning the toolUse the help menu to start with:mass-driver --helpPreparing a changeLet's prepare for doing a change over dozens of repositories. We'll need to find
aPatchDriverthat suits our needs, and configure it accordingly.List availablePatchDrivers via:mass-driver drivers --list
# The docs for a single driver:
mass-driver driver --info counterRemember,PatchDrivers are exposed via a python plugin system, which means
anyone can package their own!Once you've got a driver, you should create a Migration file, in TOML:# Saved as "fix_teamname.toml"[mass-driver.migration]# As seen in 'git log':commit_message="""Change team nameTeam name XYZ is wrong, we should be called ABC instead.See JIRA-123[1].[1]: https://example.com/tickets/JIRA-123"""# Override the local git commit authorcommit_author_name="John Smith"commit_author_email="smith@example.com"branch_name="fix-team-name"# PatchDriver class to use.# Selected via plugin name, from "massdriver.drivers" entrypointdriver_name="teamname-changer"# Config given to the PatchDriver instancedriver_config={filename="catalog.yaml",team_name="Core Team"}# Note: No "forge" section = no forge activity to pursue (no PR will be created)With this file namedfix_teamname.tomlin hand, we can apply the change
locally, either against a local repo we've already cloned:mass-driverrunfix_teamname.toml--repo-path~/workspace/my-repo/Or against a repo being cloned from URL:mass-driverrunfix_teamname.toml--repo-path'git@github.com:OverkillGuy/sphinx-needs-test.git'The cloned repo will be under.mass_driver/repos/USER/REPONAME/.
We should expect a branch namedfix-team-namewith a single commit.To apply the change over a list of repositories, create a file with relevant
repos:cat<<EOF > repos.txtgit@github.com:OverkillGuy/sphinx-needs-test.gitgit@github.com:OverkillGuy/speeders.gitEOFmass-driverrunfix_teamname.toml--repo-filelistrepos.txtCreating PRsOnce the commits are done locally, let's send them up as PR a second step. For
this, we'll be creating a second activity file containing a Forge definition.Similarly, forges can be listed and detailed:mass-driverforges--list# The docs for a single forge:mass-driverforge--infocounterConsider using theforge_name = "github".
Create a new Activity with a Forge:# An Activity made up of just a forge[mass-driver.forge]forge_name="github"base_branch="main"head_branch="fix-teamname"draft_pr=truepr_title="[JIRA-123] Bump counter.txt to 1"pr_body="""Change team nameTeam name XYZ is wrong, we should be called ABC instead.See JIRA-123[1].[1]: https://example.com/tickets/JIRA-123"""# Do you need to git push the branch before PR?git_push_first=trueNow run mass-driver, remembering to set theFORGE_TOKENenvvar for a
Github/other auth token.exportFORGE_TOKEN="ghp_supersecrettoken"mass-driverrunfix_teamname_forge.toml--repo-filelistrepos.txtCombining migration then forgeSometimes, we wish to expedite both the committing and the PR creation in a
single move.The Activity file can contain both sections:# An activity made up of first a Migration, then a Forge[mass-driver.migration]# As seen in 'git log':commit_message="""Change team nameTeam name XYZ is wrong, we should be called ABC instead.See JIRA-123[1].[1]: https://example.com/tickets/JIRA-123"""# Override the local git commit authorcommit_author_name="John Smith"commit_author_email="smith@example.com"branch_name="fix-team-name"# PatchDriver class to use.# Selected via plugin name, from "massdriver.drivers" entrypointdriver_name="teamname-changer"# Config given to the PatchDriver instancedriver_config={filename="catalog.yaml",team_name="Core Team"}# And a forge = PR creation after Migration[mass-driver.forge]forge_name="github"base_branch="main"head_branch="fix-teamname"draft_pr=truepr_title="[JIRA-123] Bump counter.txt to 1"pr_body="""Change team nameTeam name XYZ is wrong, we should be called ABC instead.See JIRA-123[1].[1]: https://example.com/tickets/JIRA-123"""# Do you need to git push the branch before PR?git_push_first=trueDiscovering repos using a SourceSometimes, the repos we want to apply patches to is a dynamic thing, coming from
tooling, like a Github repository search, some compliance tool report, service
catalogue, etc.To address this, mass-driver can use a Source plugin to discover repositories to
apply activities to.# An Activity file with a file-list Source[mass-driver.source]source_name="repo-list"# Source 'repo-list' takes a 'repos' list of cloneable URLs:[mass-driver.source.source_config]repos=["git@github.com:OverkillGuy/mass-driver.git","git@github.com:OverkillGuy/mass-driver-plugins.git",]Because we included aSource, we can omit the CLI flags--repo-pathor--repo-filelist, to instead rely on the activity's config to discover the
repos.mass-driverrunactivity.tomlSmarter Sources can use more elaborate parameters, maybe even secret parameters
like API tokens.Note that to pass secrets safely at runtime, config parameters passed viasource_configin file format can be passed as envvar, using prefixSOURCE_.
So we could have avoided thereposentry in file, by providing aSOURCE_REPOSenvvar instead. This feature works because the Source class
derives fromPydantic.BaseSettings.As aSourcedeveloper, though, you should really look into usage ofPydantic.SecretStrto avoid leaking the secret when config or result is
stored. SeePydantic docs on Secret
fields.Using the scannersBefore doing any actual migration, we might want to explore existing
repositories to see what kind of change is required.Mass-driver provides for this usecase via the scanners plugin system, enabling a
simple python function to be run against many repos, with the purpose of
gathering information.Let's define an Activity file specifying a list of scanners to run:# An Activity file for scanning[mass-driver.scan]scanner_names=["root-files","dockerfile-from"]This can be run just like a migration:mass-driverrunscan.toml--repo-filelistrepos.txtReviewing bulk PR statusHave a look at theview-prsubcommand for reviewing the status of many PRs at
once.It requires specifying a forge likegithub, along with setting any required
tokens, such as viaFORGE_TOKENenvvar forgithubforge.exportFORGE_TOKEN=xyz
mass-driverview-prgithub\--pr\https://github.com/OverkillGuy/mass-driver/pull/1https://github.com/OverkillGuy/mass-driver/pull/2# Can specify multiple PRs as a args listEquivalently via a file full of newline-delimited PR URLsexportFORGE_TOKEN=xyz
mass-driverview-prgithub--pr-filelistprs.txtWith sample result:> Pull request review mode!
[001/004] Fetching PR status...
[002/004] Fetching PR status...
[003/004] Fetching PR status...
[004/004] Fetching PR status...

Merged:
https://github.com/OverkillGuy/mass-driver/pull/1
https://github.com/OverkillGuy/sphinx-needs-test/pull/1

Closed (but not merged):
https://github.com/OverkillGuy/mass-driver/pull/2
https://github.com/OverkillGuy/sphinx-needs-test/pull/2

In summary: 4 unique PRs, of which...
- 002 (50.0%) merged
- 002 (50.0%) closed (but not merged)DevelopmentPython setupThis repository uses Python3.11, usingPoetryas package manager to define a
Python package insidesrc/mass_driver/.poetrywill create virtual environments if needed, fetch
dependencies, and install them for development.For ease of development, aMakefileis provided, use it like this:make# equivalent to "make all" = install lint docs test build# run only specific tasks:makeinstall
makelint
maketest# Combine tasks:makeinstalltestOnce installed, the module's code can now be reached through running
Python in Poetry:$poetryrunpython
>>>frommass_driverimportmain
>>>main("blabla")This codebase usespre-committo run linting
tools likeruff. Usepre-commit installto install git
pre-commit hooks to force running these checks before any code can be
committed, usemake lintto run these manually. Testing is provided
bypytestseparately inmake test.DocumentationDocumentation is generated viaSphinx,
using the coolmyst_parserplugin to support Markdown files like this one.Other Sphinx plugins provide extra documentation features, like the recentsphinx-autodoc2to generate API reference without headaches, and with myst-markdown support in
docstrings too!To build the documentation, run# Requires the project dependencies provided by "make install"makedocs# Generates docs/build/html/To browse the website version of the documentation you just built, run:makedocs-serveAnd remember thatmakesupports multiple targets, so you can generate the
documentation and serve it:makedocsdocs-serveLicenseThis project is released under GPLv3 or later. SeeCOPYINGfile for GPLv3
license details.Templated repositoryThis repository was created by the copier template available at
gh:OverkillGuy/python-template, using version v1.3.1.