Repository: opendatacube/datacube-core

Issue:
datacube dataset index api doesn't handle error effeciently
### Expected behaviour
`proc-info.yaml` is not a proper dataset metadata file for indexing, i.e. https://data.dea.ga.gov.au/derivative/ga_ls5t_nbart_gm_cyear_3/3-0-0/x08/y23/1994--P1Y/ga_ls5t_nbart_gm_cyear_3_x08y23_1994--P1Y_final.proc-info.yaml, datacube index doesn't check the file to ensure it is `uuid` provided in the file before trying to parse the uuid (https://github.com/opendatacube/datacube-core/blob/develop/datacube/index/postgres/_datasets.py#L84
) . and throws an exception which isn't `excepted`

https://github.com/opendatacube/datacube-core/blob/ec9a8c6bf16de3f70cf8380d1b8a6905863e81d9/datacube/index/hl.py#L144-L154

### Actual behaviour

```
(odc-tests-py38) ubuntu@:~/odc-tools$ datacube dataset add s3://dea-public-data/derivative/ga_ls5t_nbart_gm_cyear_3/3-0-0/x08/y23/1994--P1Y/ga_ls5t_nbart_gm_cyear_3_x08y23_1994--P1Y_final.proc-info.yaml
Indexing datasets  [------------------------------------]    0%
Traceback (most recent call last):
  File "/home/ubuntu/yes/envs/odc-tests-py38/bin/datacube", line 10, in <module>
    sys.exit(cli())
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/ui/click.py", line 200, in new_func
    return f(parsed_config, *args, **kwargs)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/ui/click.py", line 232, in with_index
    return f(index, *args, **kwargs)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/scripts/dataset.py", line 212, in index_cmd
    run_it(pp)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/scripts/dataset.py", line 204, in run_it
    index_datasets(dss,
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/scripts/dataset.py", line 218, in index_datasets
    for dataset in dss:
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/scripts/dataset.py", line 82, in dataset_stream
    dataset, err = ds_resolve(ds, uri)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/index/hl.py", line 277, in __call__
    dataset, err = self._ds_resolve(doc, uri)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/index/hl.py", line 154, in resolve
    db_dss = {str(ds.id): ds for ds in index.datasets.bulk_get(all_uuid)}
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/index/postgres/_datasets.py", line 86, in bulk_get
    ids = [to_uuid(i) for i in ids]
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/index/postgres/_datasets.py", line 86, in <listcomp>
    ids = [to_uuid(i) for i in ids]
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/site-packages/datacube/index/postgres/_datasets.py", line 84, in to_uuid
    return x if isinstance(x, UUID) else UUID(x)
  File "/home/ubuntu/yes/envs/odc-tests-py38/lib/python3.8/uuid.py", line 165, in __init__
    raise TypeError('one of the hex, bytes, bytes_le, fields, '
TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be given
```


### Steps to reproduce the behaviour

trying to index a dataset without `id` defined.

### Environment information

* Which ``datacube --version`` are you using?
```
(odc-tests-py38) ubuntu@:~/odc-tools$ datacube --version
Open Data Cube core, version 1.8.7
```
* What datacube deployment/enviornment are you running against?
N/A


Discussion:

Resolving patch:
diff --git a/datacube/index/hl.py b/datacube/index/hl.py
--- a/datacube/index/hl.py
+++ b/datacube/index/hl.py
@@ -149,6 +149,9 @@ def resolve(main_ds, uri):
 
         main_uuid = main_ds.id
 
+        if not main_uuid:
+            return None, "No id defined in dataset doc"
+
         ds_by_uuid = toolz.valmap(toolz.first, flatten_datasets(main_ds))
         all_uuid = list(ds_by_uuid)
         db_dss = {str(ds.id): ds for ds in index.datasets.bulk_get(all_uuid)}


Test patch:
diff --git a/integration_tests/conftest.py b/integration_tests/conftest.py
--- a/integration_tests/conftest.py
+++ b/integration_tests/conftest.py
@@ -537,5 +537,6 @@ def dataset_add_configs():
     return SimpleNamespace(metadata=str(B / 'metadata.yml'),
                            products=str(B / 'products.yml'),
                            datasets_bad1=str(B / 'datasets_bad1.yml'),
+                           datasets_no_id=str(B / 'datasets_no_id.yml'),
                            datasets_eo3=str(B / 'datasets_eo3.yml'),
                            datasets=str(B / 'datasets.yml'))
diff --git a/integration_tests/data/dataset_add/datasets_no_id.yml b/integration_tests/data/dataset_add/datasets_no_id.yml
new file mode 100644
--- /dev/null
+++ b/integration_tests/data/dataset_add/datasets_no_id.yml
@@ -0,0 +1,13 @@
+# Generated with `yaml.safe_dump_all(gen_dataset_test_dag(ii, force_tree=True) for ii in range(1,3))`
+#   A -> B
+#   |    |
+#   |    v
+#   +--> C -> D
+#   |
+#   +--> E
+#
+---
+creation_dt: '2017-11-05T00:00:00'
+label: A0001
+n: 1
+product_type: A
diff --git a/integration_tests/test_dataset_add.py b/integration_tests/test_dataset_add.py
--- a/integration_tests/test_dataset_add.py
+++ b/integration_tests/test_dataset_add.py
@@ -195,6 +195,20 @@ def check_bad_yaml(clirunner, index):
     assert 'ERROR Failed reading documents from ' in r.output
 
 
+def test_dataset_add_no_id(dataset_add_configs, index_empty, clirunner):
+    p = dataset_add_configs
+    index = index_empty
+    ds_no_id = load_dataset_definition(p.datasets_no_id)
+
+    clirunner(['metadata', 'add', p.metadata])
+    clirunner(['product', 'add', p.products])
+
+    # Check .hl.Doc2Dataset
+    doc2ds = Doc2Dataset(index)
+    _ds, _err = doc2ds(ds_no_id, 'file:///something')
+    assert _err == 'No id defined in dataset doc'
+
+
 def test_dataset_add(dataset_add_configs, index_empty, clirunner):
     p = dataset_add_configs
     index = index_empty