some multiprocess work
logging changes for multiprocess safety
Merge pull request #22 from willtp87/1.x.ingest.multiprocess

1.x.ingest.multiprocess
tmp dir multiprocess hack
Merge pull request #24 from willtp87/1.x.ingest.multiprocess

tmp dir multiprocess hack
Added a new function that is a generator.
Merge pull request #25 from jonathangreen/1.x.ingest.dir.generator

Added a new function that is a generator to scan directories.
get files is now a generator
Merge pull request #26 from willtp87/1.x.ingest

get files is now a generator
Precompiling regex expressions from configuration if present and caching them.
Adding debug statements for when a file is filtered out by regex.
Merge pull request #27 from daniel-dgi/1.x.ingest.multiprocess

Regex white and blacklists
base changes for sqlite resume, still need to sort out config
added PID updater to ingester
altered query for database update
fixing various bugs that propped up in production
finally working
removed debug print statement
a little bit more cleanup
stupid commas