Repository: dbt-labs/dbt-core

Issue:
Raise a more helpful error if BigQuery job label is too long
### Describe the bug
After migrating to v0.20.0 my jobs keep failing with the following error:

```
Database Error in model stg_facebookads__ads (models/staging/facebook_ads/stg_facebookads__ads.sql)                                                                                                         
  Label value "__database____REDACTED____schema____dev_edbizarro____identifier____stg_facebookads__ads__" has invalid characters.                                                                         
  compiled SQL at target/run/insights_lab/models/staging/facebook_ads/stg_facebookads__ads.sql
```

### Steps To Reproduce
Config labels in dbt_project.yml
```yml
query-comment:
  comment: "{{ query_comment(node) }}"
  append: false
  job-label: true
``` 

Run any model

### Expected behavior

### Screenshots and log output
If applicable, add screenshots or log output to help explain your problem.

### System information
**Which database are you using dbt with?**
- [ ] postgres
- [ ] redshift
- [x] bigquery
- [ ] snowflake
- [ ] other (specify: ____________)


**The output of `dbt --version`:**
```
 ❯ dkc exec dbt dbt --version                                                                                                                                                                    [17:52:32]
installed version: 0.20.0
   latest version: 0.20.0

Up to date!

Plugins:
  - redshift: 0.20.0
  - snowflake: 0.20.0
  - bigquery: 0.20.0
  - postgres: 0.20.0

```

**The operating system you're using:**
Arch Linux

**The output of `python --version`:**

Running through official docker image

### Additional context
Add any other context about the problem here.



Discussion:
@edbizarro I see you switched on `query-comment.job-label`, which is new in v0.20.0. The way that feature works:
- Try reading your query comment as a dictionary. If it can, pass each key-value pair as a separate job label. "Sanitize" each value by replacing whitespace and special characters with underscores.
- If your query comment can't be read as a dictionary, pass the entire (sanitized) query comment as a single job label.

The latter is what's happening here. Unfortunately, the error BigQuery is giving back is a bit misleading:
```
Label value "__database____REDACTED____schema____dev_edbizarro____identifier____stg_facebookads__ads__" has invalid characters.
```
The real issue here is that labels are limited to 63 characters in length ([docs](https://cloud.google.com/blog/products/gcp/labelling-and-grouping-your-google-cloud-platform-resources)), and this string is 89 characters in length. If I shorten the string to 63 characters, everything works just fine.

In the original PR for this feature, we discussed potential approaches for handling too-long labels: https://github.com/dbt-labs/dbt/pull/3145#discussion_r598050639. The options are:
1. Truncate, hash, or otherwise handle the label length within dbt. This would happen silently, and could result in indistinguishable label values.
2. Raise an error within dbt.
3. Do nothing, and return any errors from BigQuery.

We picked the third option. Given the lack of clarity BigQuery's error message, and the ensuing confusion indicated by this issue, I think there's good reason to prefer the second: I think we should raise a compilation error any time `query-comment.job-label` is switched on and a label value would be >63 characters.

That should be a straightforward change. Is it something you'd be interested in contributing @edbizarro?

In any event, you'll need to work around this error by:
1. Refactoring your `query_comment` macro to return a dictionary
2. Refactoring your `query_comment` macro to return a shorter string (post-sanitization)
3. Switching off `query-comment.job-label`
@jtcohen6 I'm happy to take this issue on if @edbizarro doesn't want to!
@sungchun12 I'd love that!
@jtcohen6 This is officially in my personal backlog that I'll spend the next week or so focusing on!

I'll be diving into #3145. And I'm assuming the solution to this problem will be extending the validations and raising specific error messages in functions like: [this](https://github.com/jmcarp/dbt/blob/044a6c6ea4966f04a7f27bbd506ad172d01a6149/plugins/bigquery/dbt/adapters/bigquery/connections.py#L597) and/or create another function dedicated to verifying string length after sanitization.

Resolving patch:
diff --git a/plugins/bigquery/dbt/adapters/bigquery/connections.py b/plugins/bigquery/dbt/adapters/bigquery/connections.py
--- a/plugins/bigquery/dbt/adapters/bigquery/connections.py
+++ b/plugins/bigquery/dbt/adapters/bigquery/connections.py
@@ -595,9 +595,20 @@ def _is_retryable(error):
 
 _SANITIZE_LABEL_PATTERN = re.compile(r"[^a-z0-9_-]")
 
+_VALIDATE_LABEL_LENGTH_LIMIT = 63
+
 
 def _sanitize_label(value: str) -> str:
     """Return a legal value for a BigQuery label."""
     value = value.strip().lower()
     value = _SANITIZE_LABEL_PATTERN.sub("_", value)
-    return value
+    value_length = len(value)
+    if value_length > _VALIDATE_LABEL_LENGTH_LIMIT:
+        error_msg = (
+            f"Job label length {value_length} is greater than length limit: "
+            f"{_VALIDATE_LABEL_LENGTH_LIMIT}\n"
+            f"Current sanitized label: {value}"
+        )
+        raise RuntimeException(error_msg)
+    else:
+        return value


Test patch:
diff --git a/test/unit/test_bigquery_adapter.py b/test/unit/test_bigquery_adapter.py
--- a/test/unit/test_bigquery_adapter.py
+++ b/test/unit/test_bigquery_adapter.py
@@ -1,6 +1,8 @@
 import agate
 import decimal
 import json
+import string
+import random
 import re
 import pytest
 import unittest
@@ -17,7 +19,7 @@
 from dbt.adapters.bigquery import BigQueryRelation
 from dbt.adapters.bigquery import Plugin as BigQueryPlugin
 from dbt.adapters.bigquery.connections import BigQueryConnectionManager
-from dbt.adapters.bigquery.connections import _sanitize_label
+from dbt.adapters.bigquery.connections import _sanitize_label, _VALIDATE_LABEL_LENGTH_LIMIT
 from dbt.adapters.base.query_headers import MacroQueryStringSetter
 from dbt.clients import agate_helper
 import dbt.exceptions
@@ -972,3 +974,22 @@ def test_convert_time_type(self):
 )
 def test_sanitize_label(input, output):
     assert _sanitize_label(input) == output
+
+
+@pytest.mark.parametrize(
+    "label_length",
+    [64, 65, 100],
+)
+def test_sanitize_label_length(label_length):
+    random_string = "".join(
+        random.choice(string.ascii_uppercase + string.digits)
+        for i in range(label_length)
+    )
+    test_error_msg = (
+            f"Job label length {label_length} is greater than length limit: "
+            f"{_VALIDATE_LABEL_LENGTH_LIMIT}\n"
+            f"Current sanitized label: {random_string.lower()}"
+        )
+    with pytest.raises(dbt.exceptions.RuntimeException) as error_info:
+        _sanitize_label(random_string)
+    assert error_info.value.args[0] == test_error_msg