artefactual/archivematica-storage-service — pull request #22 by mistydemeo (2014-06-28 00:15:24+00:00)
Title: Add new post-store callback API

This adds support for post-store callbacks, via a new API that would be called by Archivematica. The current usecase is to support reporting back to Islandora for each file that's stored.

The storage service currently doesn't maintain a persistent memory of files; this adds a new minimal File model that can be used to track files on transfer start. Since we don't currently assign or track UUIDs to individual files in the storage service, this instead calculates checksums on arrival and uses that to identify the file at a later point in time. This is kind of a hack.

(In the long term, I think it'd make sense to do some of the file cleanup and UUID assignment in the storage service, and send the files over to Archivematica with UUIDs assigned and the transfer METS already built; then we can just inspect the AIP METS to locate files. That would be best done when we have a) a background task scheduler, b) a standalone METS writer, though.)

The settings for the callback are global; may make more sense to make them controlled per-pipeline.

Another few supporting improvements:

* The SWORD helpers checksum function has been replaced with a checksum function in common.utils that uses hashlib, and supports any checksum algorithm that hashlib supports. File's checksum column was sized large enough to fit a whirlpool or sha512 hash, though that's bigger than we'd actually use for the time being.
* Package gained a new instance method, `Package.each_file`, that iterates over each file in a package as a convenience wrapper around `os.walk`. It also supports extracting compressed packages if necessary.

This isn't tested against an actual Islandora instance yet.