Package: jenkinsdiagram
jenkinsdiagramsThis script converts a folder of Jenkins files into a visual representation.Whilst there areJenkins plugins to view pipeline graphs, you have to
enter a pipeline first.
If you have many pipelines and don't know the top-level/entrypoint pipelines, you might like this tool.
Visualise in the terminal, as textual diagrams, or render with your IDE/GitHub/Docusaurus etc.InputsSpecify a path to your project.Jenkinsfile.*and any files ending with.jenkinsfilewill be scannedOptionsWith.jenkinsfiles, the file isn't enough to state whether they are used as a top-level pipeline in Jenkins (vs being
called by a higher-level pipeline).To create a more accurate visual representation of your pipelines, add this line to the top of your Jenkins files which
are called directly from the Jenkins UI:// jenkins2diagram:toplevelSupported outputsHere's a few options - I'm considering:Mermaid diagramRenders in GitHub .md previewsRenders in Docusaurus (a static website tool)Ascii artRenders anywhereImage filesI'll focus on mermaid first.
I think the app will return text to stdout, which can be piped>by the user into a file if they wish.Developing a POC (1/2)This app is quite domain / keyword specific, so for this POC I aim to:Get the basic file reading, pipeline mappings, etc. correct with simplified syntax:# Jenkinsfile 
// jenkins2diagram:toplevel
build job: 'a'
build job: 'b'

// a.jenkinsfile

// b.jenkinsfile
build job: 'c'

// c.jenkinsfileShould produce:graph TD
    Z[Jenkinsfile]
    Z[Jenkinsfile] --> A[a]
    Z[Jenkinsfile] --> B[b]
    B[b] --> C[c]Notes on GraphViz DOT and MermaidNote: To kickstart generating the tree, I usedanytree.
It can export to a Python dictionary, JSON, or "Dot" format.
The DOT language is a grammar forGraphviz.IntelliJrecognised Mermaid in my code block, offered to install the plugin, and rendered correctly. Mermaid plays
nice with GitHub and Docusaurus, as is already mentionedmaking graph.dot let the IDE suggest a plugin install. A sample graph renders nicely, and copy to clipboard/save
to file work greatGitHub markdown renderingIncluding graphviz graphs is a little indirect(some html tags and custom
marks seem to be necessary)ApparentlyGraphviz is in C, and Mermaid is written in JSand mermaid is a little prettierDocusaurusI only see documentation on Mermaid support - nothing for GraphvizConverting Graphviz DOT to MermaidKroki: A server that renders a bunch of different textual diagrams to imagesOverall, I should probably aim to add a Mermaid exporter toanytree. There's no issues/PRs for it so far.anytree'sdotexporter.pyis ~ 400 lines, but more than half of it is examples.
I'll try a basic version in this POC first.The mermaid tree generation is working nicely.
Seeing that anytree can import/export trees as Python dictionaries helped me see I can move to a minimal dependency
version, hopefully without too much challenge. That can happen later.
I will try to raise a basic PR for a Mermaid exporter now.https://github.com/c0fec0de/anytree/pull/215is the basic PR, with nice diagrams in the description!Developing a POC (2/3)Change to support more proper syntaxIt seems about right for nowBe more flexible with matches (build(job: '...',...)vsbuild job:'...', etc.))Writing some unit tests for thisDeveloping a POC (3/3)Create a CLIAs a user, I would like to:(Option)--path PATHto scan.Default: current directory.Example:jenkisdiagram --path ~/IdeaProjects/monorepo.(Option)--folders NAMEto scan, in addition to root.Default: all folders.If it's too slow for large repositories, can change default to scan only root andpipelines.Example:jenkinsdiagram --folders infra/jenkins/pipelines-old --folders infra/jenkins/pipelines-newTyperis awesomeCalling the app with--helpwill list the (all optional) arguments:python.\main.py--helpOptional args, exceptions, etc. are all nicely formatted.Arg parsing is done and the CLI feels nice. I want to publish to homebrew beforePushing to HomeBrewPackaging a Python CLI tool for Homebrewpip install -r requirements.txtPyPi publish first, then brew!https://medium.com/@joel.barmettler/how-to-upload-your-python-package-to-pypi-65edc5fe9c56https://www.jetbrains.com/help/pycharm/creating-and-running-setup-py.htmlpython setup.py sdist
twine upload dist/*
Okay, that has publishedhttps://pypi.org/project/jenkinsdiagram/1.0.0/it's just a .tar.gz of the repo :think:
Can't seem to make it run (as an app) from terminalm, ah well let's jump to trying homebrewNow we can try the homebrew stuff
Simon Willison's guide is goodhttps://github.com/IdiosApps/homebrew-jenkinsdiagraminstall instructions
Install wanted GitHub login for some reason. Fresh PAT wasn't working, butgh auth loginrevalidate did the trickbrew listshows jenkinsdiagram
but the command doesn't work, and neither does justjenkinsdiagraminstall logs noted:/opt/homebrew/Cellar/jenkinsdiagram/1.0.0/libexec/bin/pipcompare with
brew tap simonw/datasette
brew install simonw/datasette/datasette/opt/homebrew/Cellar/jenkinsdiagram/1.0.0 has nobin/opt/homebrew/Cellar/datasette/0.64.1 has abin> /opt/homebrew/Cellar/datasette/0.64.1/bin/datasette --version
datasette, version 0.64.1Probably in the distribution side of things, but let's compare the homebrew repos:https://github.com/IdiosApps/homebrew-jenkinsdiagram/blob/main/Formula/jenkinsdiagram.rbhttps://github.com/simonw/homebrew-datasette/blob/main/Formula/datasette.rbNothing crazy differentLet's compare one file we made to make a dist;https://github.com/simonw/datasette/blob/main/setup.pyhttps://github.com/IdiosApps/jenkinsdiagram/blob/pypi-homebrew/setup.pyThis looks interesting:entry_points="""
        [console_scripts]
        datasette=datasette.cli:cli
    """,This looks like a reasonable thing to attempt!Update setup.py with entry_point + version bump
sdist
push
pip install
poet -f
bump brew repo url + hash
...
now we have/opt/homebrew/Cellar/jenkinsdiagram/1.0.1/bin!
Running it gives:File "/opt/homebrew/Cellar/jenkinsdiagram/1.0.1/libexec/lib/python3.10/site-packages/jenkinsdiagram/main.py", line 8, in <module>
    import app
ModuleNotFoundError: No module named 'app'So we're getting closer, but still a bit more investigation to go.need to declare imports in init?may need a nice.main.py wrapping of the cli?https://github.com/simonw/datasette/pull/556. It seems fair:➜ python -m datasette --version
python -m datasette, version 0.64.1
➜ datasette --version
datasette, version 0.64.1Hmm, last couple commits had no impact - still get the same error, but at least I know I can get it in PyPi (earlier than brew)python -m jenkinsdiagram --versionFile "......./jenkinsdiagram/jenkinsdiagram/cli.py", line 8, in <module>
    import app
ModuleNotFoundError: No module named 'app'Test PyPi - we can push to an alternate repo to prevent cluttering the real PyPi project with bad distshttps://packaging.python.org/en/latest/guides/using-testpypi/twine upload --repository testpypi dist/*python3 -m pip install --index-url https://test.pypi.org/simple/ your-packagehttps://packaging.python.org/en/latest/tutorials/packaging-projects/Other nice thingsCLI could output to SVGhttps://github.com/mermaid-js/mermaid-clipng and pdf work fine; svg needs some manual config setup (noted in CLI)user installs withnpm i -gand it's an official repo from the mermaid teamhttps://github.com/superj80820/mermaid-js-convertersends out diagram to a URL