Repository: cmu-delphi/covidcast-indicators
Issue: Update delphi_utils.archive to use new params structure

We had this crash in the hhs_hosp indicator this morning:

```
Starting step [id=29987|step_id=82|name=<span class="tag_indicators"> </span> HHS Hospitalizations]
   Running cmd...
env/bin/python -m delphi_hhs
env/bin/python -m delphi_utils.archive --archive_type s3 --indicator_prefix delphi_hhs
/home/indicators/.pyenv/versions/3.8.2/lib/python3.8/runpy.py:126: RuntimeWarning: 'delphi_utils.archive' found in sys.modules after import of package 'delphi_utils', but prior to execution of 'delphi_utils.archive'; this may result in un
predictable behaviour
  warn(RuntimeWarning(msg))
Traceback (most recent call last):
  File "/home/indicators/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/indicators/.pyenv/versions/3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/indicators/runtime/hhs_hosp/env/lib/python3.8/site-packages/delphi_utils/archive.py", line 645, in <module>
    params["cache_dir"],
KeyError: 'cache_dir'
make: *** [run] Error 1
   Fail [cmd=sudo -u indicators -s bash -c "cd /home/indicators/runtime/hhs_hosp && make run"]
```

Relevant code looks to not have been updated for the new params hierarchical structure:

https://github.com/cmu-delphi/covidcast-indicators/blob/94636ba310f28c69db82b86bcea204736f09a36f/_delphi_utils_python/delphi_utils/archive.py#L645-L646

Both hhs_hosp and combo cases/deaths run delphi_utils.archive from the command line, but combo cases/deaths is not deployed to the production system and is updated haphazardly. For safety, best accept both the original params locations and the new ones for now.

Changed files:
_delphi_utils_python/delphi_utils/archive.py

diff --git a/_delphi_utils_python/delphi_utils/archive.py b/_delphi_utils_python/delphi_utils/archive.py
index 67517e78..37a0b9bd 100644
--- a/_delphi_utils_python/delphi_utils/archive.py
+++ b/_delphi_utils_python/delphi_utils/archive.py
@@ -641,12 +641,24 @@ if __name__ == "__main__":
                         help="Commit message for `archive_type` = 'git'")
     args = parser.parse_args()
     params = read_params()
+
+    # Autodetect whether parameters have been factored hierarchically or not
+    # See https://github.com/cmu-delphi/covidcast-indicators/issues/847
+    # Once all indicators have their parameters factored in to "common", "indicator", "validation",
+    # and "archive", this code will be obsolete.
+    if "archive" in params:
+        archive_params = params["archive"]
+        common_params = params["common"]
+    else:
+        archive_params = params
+        common_params = params
+
     run_module(args.archive_type,
-               params["cache_dir"],
-               params["export_dir"],
-               aws_credentials=params.get("aws_credentials", {}),
+               archive_params["cache_dir"],
+               common_params["export_dir"],
+               aws_credentials=archive_params.get("aws_credentials", {}),
                branch_name=args.branch_name,
-               bucket_name=params.get("bucket_name", ""),
+               bucket_name=archive_params.get("bucket_name", ""),
                commit_message=args.commit_message,
                commit_partial_success=args.commit_partial_success,
                indicator_prefix=args.indicator_prefix,