Add some reporting to rake ingest - every minute print out the
number of ingested files, ingested file size, and elapsed time.
Add some methods to Directory to aid in getting this information.
Use tree -a to make sure we get dotfiles.
Add expect header to ingest headers. I'm not sure the client handles this properly, but I think it may. Will try an ingest and watch dx logs. This
may improve performance, as well as being what the DX wants to see. (Without it the client may actually send the data two or more times when
it get redirects.)