Package: pytest-insta
pytest-instaA practical snapshot testing plugin for pytest.assertsnapshot()=="awesome!"IntroductionSnapshot testing makes it easy to monitor and approve changes by comparing the result of an operation against a previous reference value.This project borrows from a lot of other implementations to provide a pythonic, batteries included snapshot testing solution. It also tries to feel as native topytestas possible with its integrated review tool.FeaturesExpressive and familiar assertion syntaxCan format text, binary, hexdump, json and pickle snapshots out-of-the-boxCan be extended with custom snapshot formatsInteractive review tool for inspecting and approving changesCreditsinsta(rust)Armin's work was the initial motivation for this project and inspired the reviewing workflow.jest(javascript)Jest enabled the mass adoption of snapshot testing throughout the JavaScript ecosystem and now basically stands as the reference when it comes to what snapshot testing is supposed to look like.InstallationThe package can be installed withpip.$pipinstallpytest-instaGetting StartedThesnapshotfixture is a function that returns the current value of a snapshot.deftest_hello_world(snapshot):assertsnapshot()=="hello"$pytest
...
CREATEsnapshots/<prefix>__hello_world__0.txtRunning this test will create a new text file in thesnapshotsdirectory. The next time pytest runs, the test will load the snapshot and compare it to the actual value.The return value of thesnapshotfunction can be assigned to a variable and used multiple times.deftest_hello_world(snapshot):expected=snapshot()assertexpected=="hello"assertexpected.upper()=="HELLO"By default, each invocation of thesnapshotfunction will generate its own snapshot.deftest_hello_world(snapshot):assertsnapshot()=="hello"assertsnapshot()=="world"$pytest
...
CREATEsnapshots/<prefix>__hello_world__0.txt
CREATEsnapshots/<prefix>__hello_world__1.txtYou can also name snapshots explicitly. This makes it possible to load a snapshot multiple times during the same test.deftest_hello_world(snapshot):assertsnapshot("message.txt")=="hello"assertsnapshot("message.txt")=="hello"$pytest
...
CREATEsnapshots/<prefix>__hello_world__message.txtSnapshot FormatsBy default, thesnapshotfixture will store snapshots as.txtfiles. By providing a filename or just a specific file extension, you can create snapshots using various formats supported out-of-the-box.deftest_hello_world(snapshot):assertsnapshot("json")=={"hello":"world"}assertsnapshot("expected.json")=={"hello":"world"}$pytest
...
CREATEsnapshots/<prefix>__hello_world__0.json
CREATEsnapshots/<prefix>__hello_world__expected.jsonNote that the plugin doesn't diff the snapshot files themselves but actually loads snapshots back into the interpreter and performs comparisons on live python objects. This makes it possible to use snapshot formats that aren't directly human-readable like pure binary files and pickle.Built-in FormatsExtensionSupported TypesPlain text.txtstrBinary.binbytesHexdump.hexdumpbytesJson.jsonAny object serializable by the json modulePickle.pickleAny object serializable by the pickle moduleThe built-in formats should get you covered most of the time but you can also really easily implement your own snapshot formats.fromdataclassesimportdataclassfrompathlibimportPathfrompytest_instaimportFmt@dataclassclassPoint:x:inty:intclassFmtPoint(Fmt[Point]):extension=".pt"defload(self,path:Path)->Point:returnPoint(*map(int,path.read_text().split()))defdump(self,path:Path,value:Point):path.write_text(f"{value.x}{value.y}")deftest_hello_world(snapshot):assertsnapshot("pt")==Point(4,2)You can create a custom formatter by inheriting from theFmtclass and defining customloadanddumpmethods. Theextensionattribute associates the custom formatter to the specified file extension.Custom formatters can be defined anywhere in your test suite but it's recommended to keep them inconftest.pyif they're meant to be used across multiple files.Command-line OptionsThe plugin extends thepytestcli with a new--instaoption that accommodates the snapshot-testing workflow. The option can be set to one of the following strategies:update- Record and update differing snapshotsupdate-new- Record and create snapshots that don't already existupdate-none- Don't record or update anythingrecord- Record and save differing snapshots to be reviewed laterreview- Record and save differing snapshots then bring up the review toolreview-only- Don't run tests and only bring up the review toolclear- Don't run tests and clear all the snapshots to reviewIf the option is not specified, the strategy will default toupdate-noneifpytestis running in a CI environment andupdate-newotherwise. This makes sure that your pipeline properly catches any snapshot you might forget to push while keeping the development experience seamless by automatically creating snapshots as you're writing tests.Therecordoption is useful if you're in the middle of something and your snapshots keep changing. Differing snapshots won't cause tests to fail and will instead be recorded and saved.$pytest--instarecord
...
RECORD.pytest_cache/d/insta/<prefix>__hello_world__0.txt

NOTICE1snapshottoreviewWhen you're done making changes you can use thereviewoption to bring up the review tool after running your tests. Each differing snapshot will display a diff and let you inspect the new value and the old value in a python repl.$pytest--instareview
...
_____________________________[1/1]_____________________________

old:snapshots/example_hello_world__0.txt
new:.pytest_cache/d/insta/example_hello_world__0.txt

>assertold==new
Eassert'hello'=='world'E-world
E+hello

test_example.py:1:test_hello_world

a:accept,r:reject,s:skip
>>>Finally, theupdateoption will let you update any differing snapshot according to the current test run, without going through the review tool.$pytest--instaupdate
...
UPDATEsnapshots/<prefix>__hello_world__0.txtIt's worth mentioning that the updating, recording and reviewing strategies take into account any filter you might specify with the-kor-moptions.CaveatsThesnapshotfixture hijacks equality checks to record changes. This keeps assertions expressive and readable but introduces two caveats that you need to be aware of.Right-sided snapshots ❌If an object's__eq__method doesn't returnNotImplementedwhen its type doesn't match the compared object, the snapshot won't be able to record the updated value if it's placed on the right side of the comparison.ExplanationStrings returnNotImplementedwhen compared to non-string objects so the following test will behave as expected.deftest_bad(snapshot):assert"hello"==snapshot()# This worksHowever, dataclasses returnFalsewhen compared to objects of different types and won't let the snapshot record any changes when placed on the left-side of the comparison.fromdataclassesimportdataclassfrompathlibimportPathfrompytest_instaimportFmt@dataclassclassPoint:x:inty:intclassFmtPoint(Fmt[Point]):extension=".pt"defload(self,path:Path)->Point:returnPoint(*map(int,path.read_text().split()))defdump(self,path:Path,value:Point):path.write_text(f"{value.x}{value.y}")deftest_bad(snapshot):assertPoint(4,2)==snapshot("pt")# This doesn't workRecommendation ✅To avoid confusion and keep things consistent, always put snapshots on the left-side of the comparison.deftest_good(snapshot):assertsnapshot()=="hello"deftest_good(snapshot):assertsnapshot("pt")==Point(4,2)Not comparing snapshots ❌Snapshots should first be compared to their actual value before being used in other expressions and assertions.ExplanationThe comparison records the current value if the snapshot doesn't exist yet. In the following example, the test will fail before the actual comparison and the snapshot will not be generated.deftest_bad(snapshot):expected=snapshot()assertexpected.upper()=="HELLO"# This doesn't workassertexpected=="hello"$pytest
...
>assertexpected.upper()=="HELLO"EAttributeError:'SnapshotNotfound'objecthasnoattribute'upper'Recommendation ✅Always compare the snapshot to its actual value first and only perform additional operations afterwards.deftest_good(snapshot):expected=snapshot()assertexpected=="hello"assertexpected.upper()=="HELLO"ContributingContributions are welcome. Make sure to first open an issue discussing the problem or the new feature before creating a pull request. The project usespoetry.$poetryinstallYou can run the tests withpoetry run pytest.$poetryrunpytestThe project must type-check withpyright. If you're using VSCode thepylanceextension should report diagnostics automatically. You can also install the type-checker locally withnpm installand run it from the command-line.$npmrunwatch
$npmruncheckThe code follows theblackcode style. Import statements are sorted withisort.$poetryrunisortpytest_instatests
$poetryrunblackpytest_instatests
$poetryrunblack--checkpytest_instatestsLicense -MIT