Project: duracloud
Issue DURACLOUD-416: Implement a content cache to mitigate eventual consistency issues
In order to make the issues of eventual consistency an issue that does not concern the users of DuraCloud, DuraStore should be updated to provide a cache for the content which is stored. This could mean either storing the file locally first, then making the call to push it to the provider, or wrapping the stream in order to write the file locally while it is being stored at the provider. These local files would be used to fulfill GET requests that come in shortly after the file has been stored. The files in the cache would be removed once there is a greater degree of certainty that the file will be available on all following requests to the provider (perhaps just after a certain time period has elapsed.) Note that one benefit to storing the files locally prior to pushing them to the provider would be that we could know the file size and generate the checksum locally before making the provider request, meaning that we could ensure that the file reached DuraStore correctly before passing it along to the next location. This may also become useful for handling large files that we intend to pass to the provider using a multi-part upload.
Story points: 5