Package: autotuning_methodology
Autotuning Methodology Software PackageThis repository contains the software package accompanying the paper "A Methodology for Comparing Auto-Tuning Optimization Algorithms".
It makes the guidelines in the methodology easy to apply: simply specify the.jsonfile, runautotuning_visualize [path_to_json]and wait for the results!InstallationThe package can be installed by cloning this repository and runningpip install .. Python >= 3.9 is supported.Notable featuresOfficial software by the authors of the methodology-defining paper.SupportsBAT benchmark suiteandKernel Tuner.Split executer and visualizer to allow running the algorithms on a cluster and visualize locally.Caching built-in to avoid duplicate executions.Planned support for T1 input and T4 output files.Notebook / interactive window mode; in this case, plots are shown in the notebook / window instead of written to a folder.UsageEntry pointsThere are two entry points defined:autotuning_experimentandautotuning_visualize. Both take one argument: the path to an experiment file (see below).Input filesTo get started, all you need is an experiments file. This is ajsonfile that describes the details of your comparison: which algorithms to use, which programs to tune on which devices, the graphs to output and so on.
You can find the API and an exampleexperiments.jsonin thedocumentation.File referencesAs we are dealing with input and output files, file references matter.
When calling the entrypoints, we are already providing the path to an experiments file.
File references in experiments files are relative to the location of the experiment file itself.
File references in tuning scripts are relative to the location of the tuning script itself. Tuning scripts need to have the global literalsfile_path_resultsandfile_path_metadatafor this package to know where to get the results.
Plots outputted by this package are placed in a folder calledgenerated_plotsrelative to the current working directory.ContributingSetupIf you're looking to contribute to this package: welcome!
Start out by installing withpip install -e .[dev](this installs the package in editable mode alongside the development dependencies).
During development, unit and integration tests can be ran withpytest.Blackis used as a formatter, andRuffis used as a linter to check the formatting, import sorting et cetera.
When using Visual Studio Code, use thesettings.jsonfound in.vscodeto automatically have the correct linting, formatting and sorting during development.
In addition, install the extensions recommended by us by searching for@recommended:workspacein the extensions tab for a better development experience.DocumentationThe documentation can be foundhere.
Locally, the documentation can be build withmake clean htmlfrom thedocsfolder, but the package must have been installed in editable mode withpip install -e ..
Upon pushing to main or publishing a version, this documentation will be built and published to the GitHub Pages.
The Docstring format used is Google. Type hints are to be included in the function signature and therefor omitted from the docstring. In Visual Studio Code, theautoDocstringextension can be used to automatically infer docstrings. When referrring to functions and parameters in the docstring outside of their definition, use double backquotes to be compatible with both MarkDown and ReStructuredText, e.g.:"skip_draws_check: skips checking that each value indrawsis in thedist.".TestsBefore contributing a pull request, please runnoxand ensure it has no errors. This will test against all Python versions explicitely supported by this package, and will check whether the correct formatting has been applied.
Upon submitting a pull request or pushing to main, these same checks will be ran remotely via GitHub Actions.PublishingFor publising the package to PyPI (the Python Package Index), we useFlitand theto-pypi-using-flitGitHub Action to automate this.Semantic version numberingis used as follows:MAJOR.Minor.patch.MAJORversion for incompatible API changes.Minorversion for functionality in a backward compatible manner.patchversion for backward compatible bug fixes.
In addition,PEP 440is adhered to, specifically forpre-release versioning.