Implement the snapshot method

We add the final piece to our application - code to actually perform the web shots and store them.

We implement a new Snapshot class in the service name space.
The class consists of 3 methods and a constructor.

The constructor initializes parameters for the webkit2png file, the -x argument telling it to use a virtual frame buffer with the resolution of 1920x1080. The environment variables passed to the command (HOME in our example) and the target web page.

The take method, which immediately returns a deferred created with twisted.internet.utils.getProcessOutput which automatically spawns a command and handles communication with it passing back it's output.

We attach a callback to the returned deferred to call md5sum and then attach a second callback to the deferred returned by md5sum to call store.

The md5sum method uses twisted.internet.threads.deferToThread to handle blocking code in compliance with the asynchronous world of twisted. The call immediately returns a deferred and the actual processing takes place in the local _md5sum method which using the hashlib binary computes the md5 checksum of our website screenshot returned by webkit2png on standard output.

The store method uses the same deferToThread call and handles it's processing in the local function _store which writes the output bytes to disk in the static directory using the md5 checksum as the filename - the file is not store if it already exists.

What we see here is two different and typical ways to work with blocking code inside the twisted framework. Blocking code should be handled by a Deferred - os.system and the subprocess module could potentially block our event loop so we defer this work to utils.getProcessOutput. On the same note, regular blocking code can be wrapped in a Deferred by using threads.deferToThread which will spawn the target function in a separate thread.

Next we extend our service.Shutter class with a snapshot method. The method creates a new Snapshot instance providing the url as the parameter and calls take which returns a Deferred. We attach a _store_url as a callback and the return of the value will be the Deferred returned by _store_url and not the one returned by Snapshot.take.

The _store_url local function calls into runInteraction which we already used before. This function also returns a deferred and will become the return value of our service. The runInteraction method will call into _save providing it with a postgres Transaction (txn) - we use this transaction to check if the site was ever processed by the service - if not we add a new entry to shutter.urls. Next we store the file path and url id in the shutter.snapshots table returning the file path as the function result.

In protocol.ShutterJSONRPCProtocol we add a jsonrpc_snapshot method to expose our new functionality over HTTP/JSONRPC protocol.

We will also extend our tests to cover the new snapshot method. Since our tests will perform real storage operations we alter our setUp method to create a static directory before each test is run. We also alter the tearDown method to remove the contents of that directory and prune the database of any data that might have been entered by the tests.

Since tearDown properly clears after each test case now - we can drop the delete statements from test_snapshots_fake_entries.

The new test code is quite straight forward. We add test_snapshot and call the snapshot method passing it a URL. We can't predict the md5 checksum since the resource is external so we make sure that the returned file length is an expected value of 32 characters for the md5 checksum plus 1 character for the dot and 3 characters for the png extension.