Repository: googleapis/python-bigquery

Issue:
Reading the labels set on a job fails
The ability to specify labels on created BigQuery jobs was supposedly added in googleapis/google-cloud-python#5654, but it doesn't work. There is also a comment in the documentation from that same PR referring to Client.update_job() which doesn't even exist.


#### Environment details

# python --version
Python 3.7.3

# pip freeze | grep google
google-api-core==1.24.1
google-auth==1.24.0
google-cloud-bigquery==2.6.2
google-cloud-bigtable==1.6.1
google-cloud-core==1.5.0
google-cloud-pubsub==2.2.0
google-cloud-storage==1.35.0
google-crc32c==1.1.0
google-resumable-media==1.2.0
googleapis-common-protos==1.52.0
grpc-google-iam-v1==0.12.3

#### Code example

```python
job_config = bigquery.QueryJobConfig(labels={"app": "test"})
query_job = client.query("SELECT 1", job_config=job_config)

# no labels seen below
print(query_job.labels)

# try and get a job later, also no labels
job = client.get_job(query_job.job_id)
print(job.labels)

```


Discussion:
@rholloway Thanks for the report, I can confirm the reported behavior.

It turns out the labels are actually present in the raw job data, i.e. in `query_job._properties["configuration"]["labels"]`, but the `labels` property tries to read them from `self._properties["labels"]`. If that's the only cause, it should be an easy fix.

It's worth mentioning that labels on the [Table](https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#Table), [Dataset](https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets#Dataset), and [Model](https://cloud.google.com/bigquery/docs/reference/rest/v2/models#Model) resources are actually present an the top level of the corresponding JSON responses, while in the [Job](https://cloud.google.com/bigquery/docs/reference/rest/v2/Job) resources they are nested under the `configuration` dict representing a [JobConfiguration](https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfiguration) resource.
@plamut I can confirm that there's a workaround by going through the job's configuration. In my case, I was able to submit a job with labels and then later access them as follows
```
client = BigQueryClient()
job = client.submit_job(sql, labels={'foo': 'bar'})
...
job = client.get_job(job.job_id)
print(job._configuration.labels)  # {'foo': 'bar'}
```

Perhaps the `_configuration` attr is synonymous with `_properties['configuration']`, just as you found. I hope that helps.
@rarredon Indeed, semantically they represent [the same](https://github.com/googleapis/python-bigquery/blob/2788736b80a4c4ac0ae3029aeb28bcefd34f2db7/google/cloud/bigquery/job/query.py#L542) thing. :+1:
`_configuration` is a  `QueryJobConfig()` instance, while `_properties["configuration"]` is the former's "raw" form (a nested dict structure).
Got an update from the backend maintainers - the labels on the jobs are immutable once a job is created, thus the client cannot  update them even if it wanted to.

It still needs to read them from the correct nesting level, of course, which will be fixed soon.

Resolving patch:
diff --git a/google/cloud/bigquery/job/base.py b/google/cloud/bigquery/job/base.py
--- a/google/cloud/bigquery/job/base.py
+++ b/google/cloud/bigquery/job/base.py
@@ -233,7 +233,7 @@ def path(self):
     @property
     def labels(self):
         """Dict[str, str]: Labels for the job."""
-        return self._properties.setdefault("labels", {})
+        return self._properties.setdefault("configuration", {}).setdefault("labels", {})
 
     @property
     def etag(self):
@@ -671,9 +671,8 @@ def __setattr__(self, name, value):
     def labels(self):
         """Dict[str, str]: Labels for the job.
 
-        This method always returns a dict. To change a job's labels,
-        modify the dict, then call ``Client.update_job``. To delete a
-        label, set its value to :data:`None` before updating.
+        This method always returns a dict. Once a job has been created on the
+        server, its labels cannot be modified anymore.
 
         Raises:
             ValueError: If ``value`` type is invalid.


Test patch:
diff --git a/tests/system.py b/tests/system.py
--- a/tests/system.py
+++ b/tests/system.py
@@ -1667,6 +1667,23 @@ def test_job_cancel(self):
         # raise an error, and that the job completed (in the `retry()`
         # above).
 
+    def test_job_labels(self):
+        DATASET_ID = _make_dataset_id("job_cancel")
+        JOB_ID_PREFIX = "fetch_" + DATASET_ID
+        QUERY = "SELECT 1 as one"
+
+        self.temp_dataset(DATASET_ID)
+
+        job_config = bigquery.QueryJobConfig(
+            labels={"custom_label": "label_value", "another_label": "foo123"}
+        )
+        job = Config.CLIENT.query(
+            QUERY, job_id_prefix=JOB_ID_PREFIX, job_config=job_config
+        )
+
+        expected_labels = {"custom_label": "label_value", "another_label": "foo123"}
+        self.assertEqual(job.labels, expected_labels)
+
     def test_get_failed_job(self):
         # issue 4246
         from google.api_core.exceptions import BadRequest
diff --git a/tests/unit/job/test_base.py b/tests/unit/job/test_base.py
--- a/tests/unit/job/test_base.py
+++ b/tests/unit/job/test_base.py
@@ -251,7 +251,7 @@ def test_labels_hit(self):
         labels = {"foo": "bar"}
         client = _make_client(project=self.PROJECT)
         job = self._make_one(self.JOB_ID, client)
-        job._properties["labels"] = labels
+        job._properties.setdefault("configuration", {})["labels"] = labels
         self.assertEqual(job.labels, labels)
 
     def test_etag(self):