Import bug fixes.
Significant changes in ETL algorithtm:
1. File name collection and loading are separated.
 - The script collects all file urls from FTP.
 - For each region the date in the name is checked against max date in the database.
 - Files are loaded.
2. Notifications ETL temporarily removed.
3. 'inc' now works from any moment - not only in 'daily' folder.
4. 'all' is used ONLY if you want to wipe the db.
5. Corresponding changes in README.md
6. The log now shows file size the script is to load.
Some changes.
Significant changes in ETL algorithm:
1. Added _meta collection for each collection. Stores aggregated data as needed. For now it's max_date for contracts by region.
2. Because of (1) file listing and filtering is now very fast (no sorting needed).
3. Added inc_files() and all_files() functions that form _meta collection and list files as needed.
4. _meta collection now updated after each file load.
5. Removed documents list, now load() is called for each document right after it is transformed.
6. File list is now list of tuples (filename, region). This is needed for loading and _meta updating.
Corresponding changes in products ETL function.
Fixed None products document bug.
Extract error logging.
Products file loading added on ETL level.
Added products size logging.
Uncommented products file logging.
resync
resync
resync
Changes in size logging.
Changes in size logging.
Fixed products bug in mask.
Removed profiling logging.
Merge.