Package: pytest-allclose
pytest-allclosepytest-allcloseprovides the~.allclosePytest fixture,
extendingnumpy.allclosewith test-specific features.A core feature of the~.allclosefixture
is that the tolerances for tests can be configured externally.
This allows different repositories to share the same tests,
but use different tolerances.
See the “Configuration” section below for details.InstallationTo use this fixture, install withpipinstallpytest-allcloseUsageThe~.allclosefixture is used just likenumpy.allclose.importnumpyasnpdeftest_close(allclose):x=np.linspace(-1,1)y=x+0.001assertallclose(y,x,atol=0.002)assertnotallclose(y,x,atol=0.0005)assertnotallclose(y,x,rtol=0.002)Additional argumentsThe~.allclosefixture has a number of arguments
that are not part ofnumpy.allclose.
One such argument isxtol,
which allows arrays that have been shifted along their first axis
by a certain number of steps to be considered close.importnumpyasnpdeftest_close(allclose):x=np.linspace(-1,1)assertallclose(x[1:],x[:-1],xtol=1)assertallclose(x[3:],x[:-3],xtol=3)assertnotallclose(x[3:],x[:-3],xtol=1)Refer to the~.allcloseAPI reference for all additional arguments.RMSE error reportingThe~.allclosefixture stores root-mean-square error values,
which can be reported in the pytest terminal summary.
To do so, put the following in yourconftest.pyfile.frompytest_allcloseimportreport_rmsesdefpytest_terminal_summary(terminalreporter):report_rmses(terminalreporter)See the~.report_rmsesAPI reference for more information.Configurationallclose_tolerancesallclose_tolerancesaccepts a list of test name patterns,
followed by values for any of the~.allcloseparameters.
These values will override any values provided within the test function itself,
allowing multiple repositories to use the same test suite,
but with different tolerances.allclose_tolerances=test_file.py:test_function atol=0.3# set atol for specific testtest_file.py:test_func* rtol=0.2# set rtol for tests matching wildcardtest_file.py:* atol=0.1 rtol=0.3# set both tols for all tests in filetest_*tion rtol=0.2# set rtol for all matching tests in any filetest_function[True] atol=0.1# set atol only for one parametrizationThe only special character recognized in these patterns
is the wildcard character*,
which matches any group of zero or more characters.If the test is parametrized,
then a pattern liketest_name[param0-param1]will match specific parameter settings,
andtest_name*will match all parameter settings.
Note that the latter will match any test that starts withtest_name.If a test has multiple~.allclosecalls,
you can use multiple tolerance lines that match the same test
to set different values for the first, second, third, etc. calls.
If there are more~.allclosecalls than tolerance lines,
the last tolerance line will be used for all remaining~.allclosecalls.Example test file:deftest_close(allclose):x=np.linspace(-1,1)y=x+0.001assertallclose(y,x)assertnotallclose(y,x)Example configuration file (pytest.ini,setup.cfg):allclose_tolerances=test_close atol=0.002# affects first allclose calltest_close atol=0.0005# affects second allclose callNoteDifferent tolerance lines correspond tocallsof the
function, not lines of code. If you have aforloop that calls~.allclose3 times,
each of these calls corresponds to a new tolerance line.
If you have a fourth~.allclosecall,
you would need three tolerance lines for the three calls
in theforloop, then a fourth line for the last call.Caution!The patterns for multiple calls of allclose in a function
must be exactly the same.
This means that if you have specific values for one
parametrization and general values for others,
you must put the specific values first
or they will not have any effect.Good example, specific takes precedence:allclose_tolerances=test_close[True-1] atol=0.002test_close[True-1] atol=0.0005test_close* atol=0.001test_close* atol=0.0001Bad example, general takes precedence:allclose_tolerances=test_close* atol=0.001test_close* atol=0.0001test_close[True-1] atol=0.002test_close[True-1] atol=0.0005See the fulldocumentationfor the API reference.Release History1.0.0 (July 30, 2019)Initial release ofpytest-allclose!
Thanks to all of the contributors for making this possible!