Add s3 support

@stepheneb @scytacki @psndcsrv can any of you review this?

The main change  is to add S3 support.  But mixed in (unfortunately) are changes in which cache-misses are checked against the file-system.

I would like to merge this change sometime this week.