Package: diff-cover
Automatically find diff lines that need test coverage.
Also finds diff lines that have violations (according to tools such
as pycodestyle, pyflakes, flake8, or pylint).
This is used as a code quality metric during code reviews.OverviewDiff coverage is the percentage of new or modified
lines that are covered by tests.  This provides a clear
and achievable standard for code review: If you touch a line
of code, that line should be covered.  Code coverage
iseverydeveloper’s responsibility!Thediff-covercommand line tool compares an XML coverage report
with the output ofgit diff.  It then reports coverage information
for lines in the diff.Currently,diff-coverrequires that:You are usinggitfor version control.Your test runner generates coverage reports in Cobertura, Clover
or JaCoCo XML format, or LCov format.Supported XML or LCov coverage reports can be generated with many coverage tools,
including:Cobertura(Java)Clover(Java)JaCoCo(Java)coverage.py(Python)JSCover(JavaScript)lcov(C/C++)diff-coveris designed to be extended.  If you are interested
in adding support for other version control systems or coverage
report formats, see below for information on how to contribute!InstallationTo install the latest release:pipinstalldiff_coverTo install the development version:gitclonehttps://github.com/Bachmann1234/diff-cover.gitcddiff-coverpoetryinstallpoetryshellGetting StartedSet the current working directory to agitrepository.Run your test suite under coverage and generate a [Cobertura, Clover or JaCoCo] XML report.
For example, usingpytest-cov:pytest--cov--cov-report=xmlThis will create acoverage.xmlfile in the current working directory.NOTE: If you are using a different coverage generator, you will
need to use different commands to generate the coverage XML report.Rundiff-cover:diff-covercoverage.xmlThis will compare the currentgitbranch toorigin/mainand print
the diff coverage report to the console.You can also generate an HTML, JSON or Markdown version of the report:diff-covercoverage.xml--html-reportreport.htmldiff-covercoverage.xml--json-reportreport.jsondiff-covercoverage.xml--markdown-reportreport.mdMultiple XML Coverage ReportsIn the case that one has multiple xml reports form multiple test suites, you
can get a combined coverage report (a line is counted  as covered if it is
covered in ANY of the xml reports) by runningdiff-coverwith multiple
coverage reports as arguments. You may specify any arbitrary number of coverage
reports:diff-covercoverage1.xmlcoverage2.xmlQuality CoverageYou can use diff-cover to see quality reports on the diff as well by runningdiff-quality.diff-quality--violations=<tool>Wheretoolis the quality checker to use. Currentlypycodestyle,pyflakes,flake8,pylint,checkstyle,checkstylexmlare supported, but more
checkers can (and should!) be supported. See the section “Addingdiff-quality`Support for a New Quality Checker”.NOTE: There’s no way to runfindbugsfromdiff-qualityas it operating
over the generated java bytecode and should be integrated into the build
framework.Likediff-cover, HTML, JSON or Markdown reports can be generated withdiff-quality--violations=<tool>--html-reportreport.htmldiff-quality--violations=<tool>--json-reportreport.jsondiff-quality--violations=<tool>--markdown-reportreport.mdIf you have already generated a report usingpycodestyle,pyflakes,flake8,pylint,checkstyle,checkstylexml, orfindbugsyou can pass the report
todiff-quality.  This is more efficient than lettingdiff-qualityre-runpycodestyle,pyflakes,flake8,pylint,checkstyle, orcheckstylexml.# For pylint < 1.0pylint-fparseable>pylint_report.txt# For pylint >= 1.0pylint--msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}">pylint_report.txt# Use the generated pylint report when running diff-qualitydiff-quality--violations=pylintpylint_report.txt# Use a generated pycodestyle report when running diff-quality.pycodestyle>pycodestyle_report.txtdiff-quality--violations=pycodestylepycodestyle_report.txtNote that you must use the-fparseableoption to generate
thepylintreport for pylint versions less than 1.0 and the--msg-templateoption for versions >= 1.0.diff-qualitywill also accept multiplepycodestyle,pyflakes,flake8,
orpylintreports:diff-quality--violations=pylintreport_1.txtreport_2.txtIf you need to pass in additional options you can with theoptionsflagdiff-quality--violations=pycodestyle--options="--exclude='*/migrations*' --statistics"pycodestyle_report.txtCompare BranchBy default,diff-covercompares the current branch toorigin/main.  To specify a different compare branch:diff-covercoverage.xml--compare-branch=origin/releaseFail UnderTo havediff-coveranddiff-qualityreturn a non zero status code if the report quality/coverage percentage is
below a certain threshold specify the fail-under parameterdiff-covercoverage.xml--fail-under=80diff-quality--violations=pycodestyle--fail-under=80The above will return a non zero status if the coverage or quality score was below 80%.Exclude/Include pathsExplicit exclusion of paths is possible for bothdiff-coveranddiff-quality, while inclusion is
only supported fordiff-quality(since 5.1.0).The exclude option works withfnmatch, include withglob. Both options can consume multiple values.
Include options should be wrapped in double quotes to prevent shell globbing. Also they should be relative to
the current git directory.diff-covercoverage.xml--excludesetup.pydiff-quality--violations=pycodestyle--excludesetup.pydiff-quality--violations=pycodestyle--includeproject/foo/**The following is executed for every changed file:check if any include pattern was specifiedif yes, check if the changed file is part of at least one include patterncheck if the file is part of any exclude patternIgnore/Include based on file status in gitBothdiff-coveranddiff-qualityallow users to ignore and include files based on the git
status: staged, unstaged, untracked:--ignore-staged: ignore all staged files (by default include them)--ignore-unstaged: ignore all unstaged files (by default include them)--include-untracked: include all untracked files (by default ignore them)Quiet modeBothdiff-coveranddiff-qualitysupport a quiet mode which is disable by default.
It can be enabled by using the-q/--quietflag:diff-covercoverage.xml-qdiff-quality--violations=pycodestyle-qIf enabled, the tool will only print errors and failures but no information or warning messages.Configuration filesBoth tools allow users to specify the options in a configuration file with–config-file/-c:diff-covercoverage.xml--config-filemyconfig.tomldiff-quality--violations=pycodestyle--config-filemyconfig.tomlCurrently, only TOML files are supported.
Please note, that only non-mandatory options are supported.
If an option is specified in the configuration file and over the command line, the value of the
command line is used.TOML configurationThe parser will only react to configuration files ending with.toml.
To use it, installdiff-coverwith the extra requirementtoml.The option names are the same as on the command line, but all dashes should be underscores.
If an option can be specified multiple times, the configuration value should be specified as a list.[tool.diff_cover]compare_branch="origin/feature"quiet=true[tool.diff_quality]compare_branch="origin/feature"ignore_staged=trueTroubleshootingIssue:diff-coveralways reports: “No lines with coverage information in this diff.”Solution:diff-covermatches source files in the coverage XML report with
source files in thegit diff.  For this reason, it’s important
that the relative paths to the files match.  If you are usingcoverage.pyto generate the coverage XML report, then make sure you rundiff-coverfrom the same working directory.Issue:GitDiffTool._execute()raises the error:fatal:ambiguousargument'origin/main...HEAD':unknownrevisionorpathnotintheworkingtree.This is known to occur when runningdiff-coverinTravis CISolution: Fetch the remote main branch before runningdiff-cover:gitfetchoriginmaster:refs/remotes/origin/mainIssue:diff-qualityreports “diff_cover.violations_reporter.QualityReporterError:
No config file found, using default configuration”Solution: Your project needs apylintrcfile.
Provide this file (it can be empty) anddiff-qualityshould run without issue.Issue:diff-qualityreports “Quality tool not installed”Solution:diff-qualityassumes you have the tool you wish to run against your diff installed.
If you do not have it then install it with your favorite package manager.Issue:diff-qualityreports no quality issuesSolution: You might use a pattern likediff-quality--violationsfoo *.py. The last argument
is not used to specify the files but for the quality tool report. Remove it to resolve the issueLicenseThe code in this repository is licensed under the Apache 2.0 license.
Please seeLICENSE.txtfor details.How to ContributeContributions are very welcome. The easiest way is to fork this repo, and then
make a pull request from your fork.NOTE:diff-qualitysupports a plugin model, so new tools can be integrated
without requiring changes to this repo. See the section “Addingdiff-quality`Support for a New Quality Checker”.Setting Up For DevelopmentThis project is managed withpoetrythis can be installed withpippoetry manages a python virtual environment and organizes dependencies. It also
packages this project.pipinstallpoetrypoetryinstallI would also suggest running this command after. This will make it so git blame ignores the commit
that formatted the entire codebase.gitconfigblame.ignoreRevsFile.git-blame-ignore-revsAddingdiff-quality`Support for a New Quality CheckerAdding support for a new quality checker is simple.diff-qualitysupports
plugins using the popular Pythonpluggy package.If the quality checker is already implemented as a Python package, great! If not,create a Python packageto host the plugin implementation.In the Python package’ssetup.pyfile, define an entry point for the plugin,
e.g.setup(...entry_points={'diff_cover':['sqlfluff = sqlfluff.diff_quality_plugin'],},...)Notes:The dictionary key for the entry point must be nameddiff_coverThe value must be in the formatTOOL_NAME = YOUR_PACKAGE.PLUGIN_MODULEWhen your package is installed,diff-qualityuses this information to
look up the tool package and module based on the tool name provided to the--violationsoption of thediff-qualitycommand, e.g.:$diff-quality--violationssqlfluffThe plugin implementation will look something like the example below. This is
a simplified example based on a working plugin implementation.fromdiff_cover.hookimporthookimplasdiff_cover_hookimplfromdiff_cover.violationsreporters.baseimportBaseViolationReporter,ViolationclassSQLFluffViolationReporter(BaseViolationReporter):supported_extensions=['sql']def__init__(self):super(SQLFluffViolationReporter,self).__init__('sqlfluff')defviolations(self,src_path):return[Violation(violation.line_number,violation.description)forviolationinget_linter().get_violations(src_path)]defmeasured_lines(self,src_path):returnNone@staticmethoddefinstalled():returnTrue@diff_cover_hookimpldefdiff_cover_report_quality():returnSQLFluffViolationReporter()Important notes:diff-qualityis looking for a plugin function:Located in your package’s module that was listed in thesetup.pyentry point.Marked with the@diff_cover_hookimpldecoratorNameddiff_cover_report_quality. (This distinguishes it from any other
plugin typesdiff_covermay support.)The function should return an object with the following properties and methods:supported_extensionsproperty with a list of supported file extensionsviolations()function that returns a list ofViolationobjects for
the specifiedsrc_path. For more details on this function and other
possible reporting-related methods, see theBaseViolationReporterclasshere.Special ThanksShout out to the original author of diff-coverWill Dalyand the original author of diff-qualitySarina Canelake.Originally created with the support ofedX.