Repository: opendatacube/datacube-core

Issue:
Names appear as None in Datacube.list_products()
### Expected behaviour
Names appear properly as defined by product.yml.


### Actual behaviour

The names appear as None in list_products()


### Steps to reproduce the behaviour

```python
import datacube
dc = datacube.Datacube(env='acube')
dc.list_products()
```

Here is a couple of screenshots from pgadmin and jupyter. In the db it seems all is fine, and the products / datasets are properly indexed.

https://i.imgur.com/GmCvnwZ.png
https://i.imgur.com/xZCIs9s.png

I tried also to do datacube product update, and manually changed the name in database but doesn't seem to work. Any ideas?

### Environment information

* Which ``datacube --version`` are you using? 1.8.3
* What datacube deployment/enviornment are you running against? docker container from python 3.8 image



Discussion:
Also noticing that nothing seems to be queried, `Datacube.load()` just returns empty xarray datasets
Can you upload product definition document (.yaml) file? 
Have you queried the same database as mentioned in your [acube] env in your datacube.conf file? 
I have 14 different products in this acube env. Here is one example, but do note that all of them have names as None.

```yml
name: FCOVER_Sentinel_2
description: Fraction of vegetation cover
metadata_type: boku

storage:
  crs: PROJCS["unnamed",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Azimuthal_Equidistant"],PARAMETER["latitude_of_center",53],PARAMETER["longitude_of_center",24],PARAMETER["false_easting",5837287.81977],PARAMETER["false_northing",2121415.69617],UNIT["metre",1,AUTHORITY["EPSG","9001"]]]
  resolution:
    x: 10
    y: -10

metadata:
  format: { name: GeoTIFF }
  instrument: { name: MSI }
  platform: { code: Sentinel-2 }
  product_type: BOKU_FCOVER_S2

measurements:
  - name: band_1
    aliases: [band_01, Band1, Band_1]
    units: "fraction"
    dtype: int16
    nodata: 32767
```

In the configuration it is defined as.

```conf
[acube]
db_hostname: acube
db_username: acube_user
db_password: ....
db_port: 5432
```

The user exists and everything on that front seems to be okay.
@jankovicgd can you please paste here output of `dc.list_products(with_pandas=False)`
Also definition of your `boku` metadata might be useful. 
All of this worked with 1.6, I updated the whole setup to 1.8.3, so I am not sure. I truncated the tables and tried again and same issue. What is really weird, I have another environment and database where everything works as expected. The products there use the regular eo field. Could it be that I made the dataset name searchable?

`list_products(with_pandas=False)`

```
[{'id': 15,
  'name': None,
  'description': 'Atmospherically corrected Sentinel 2 bands',
  'creation_time': None,
  'format': 'GeoTIFF',
  'label': None,
  'lat': None,
  'lon': None,
  'time': None,
  'platform': 'Sentinel-2',
  'instrument': 'MSI',
  'product_type': 'BOKU_B_S2',
  'platform_number': None,
  'cloud_cover_percentage': None,
  'crs': 'PROJCS["unnamed",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Azimuthal_Equidistant"],PARAMETER["latitude_of_center",53],PARAMETER["longitude_of_center",24],PARAMETER["false_easting",5837287.81977],PARAMETER["false_northing",2121415.69617],UNIT["metre",1,AUTHORITY["EPSG","9001"]]]',
  'spatial_dimensions': ('y', 'x'),
  'tile_size': None,
  'resolution': (-10, 10)}]
```

```
name: boku
description: |
    Metadata fields regarding data coming from BOKU
dataset:
    id: ["id"]
    creation_dt: ["creation_dt"]
    label: ["ga_label"]
    measurements: ["image", "bands"]
    grid_spatial: ["grid_spatial", "projection"]
    format: ["format", "name"]
    sources: ["lineage", "source_datasets"]

    search_fields:
        name:
            description: Name of dataset
            offset: ["name"]
        platform:
            description: Platform code
            offset: [platform, code]

        instrument:
            description: Instrument name
            offset: [instrument, name]

        product_type:
            description: Product code
            offset: [product_type]
            indexed: false

        format:
            description: File format (GeoTiff, NetCDF)
            offset: [format, name]
            indexed: false

        lat:
            description: Latitude range
            type: double-range
            max_offset:
                - [extent, coord, ur, lat]
                - [extent, coord, lr, lat]
                - [extent, coord, ul, lat]
                - [extent, coord, ll, lat]
            min_offset:
                - [extent, coord, ur, lat]
                - [extent, coord, lr, lat]
                - [extent, coord, ul, lat]
                - [extent, coord, ll, lat]

        lon:
            description: Longitude range
            type: double-range
            max_offset:
                - [extent, coord, ul, lon]
                - [extent, coord, ur, lon]
                - [extent, coord, ll, lon]
                - [extent, coord, lr, lon]
            min_offset:
                - [extent, coord, ul, lon]
                - [extent, coord, ur, lon]
                - [extent, coord, ll, lon]
                - [extent, coord, lr, lon]

        time:
            description: Acquisition time
            type: datetime-range
            min_offset:
                - [extent, from_dt]
                - [extent, center_dt]
            max_offset:
                - [extent, to_dt]
                - [extent, center_dt]

        cloud_cover_percentage:
            description: Cloud cover percentage
            offset: [image, cloud_cover_percentage]
            type: double
            indexed: false

        platform_number:
            description: Number or code of the platform
            offset: [platform, number]
            indexed: false

```
My suspicion is that this is due to clashing of fields defined in your metadata spec, code here

https://github.com/opendatacube/datacube-core/blob/31fef0401c8634d2612d7e52ed25c548f023fb01/datacube/model/__init__.py#L590-L596

just blindly over-writes product fields (line 595) with names taken from metadata, even though they contain `None` values.

I think `product.name` should take precedence over `metadata.name` which is `None` anyway.

I feel like `product.to_dict(..)` should only report metadata fields that are fixed across entire product, and not just a bunch of NaNs.


Resolving patch:
diff --git a/datacube/model/__init__.py b/datacube/model/__init__.py
--- a/datacube/model/__init__.py
+++ b/datacube/model/__init__.py
@@ -587,12 +587,11 @@ def to_dict(self) -> Mapping[str, Any]:
         """
         Convert to a dictionary representation of the available fields
         """
-        row = {
-            'id': self.id,
-            'name': self.name,
-            'description': self.definition['description'],
-        }
-        row.update(self.fields)
+        row = dict(**self.fields)
+        row.update(id=self.id,
+                   name=self.name,
+                   description=self.definition['description'])
+
         if self.grid_spec is not None:
             row.update({
                 'crs': str(self.grid_spec.crs),


Test patch: