Repository: googleapis/python-bigquery

Issue:
Resource objects such as *JobConfig do not error or warn when setting unknown properties
Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

  - Search the issues already opened: https://github.com/googleapis/python-bigquery/issues
  - Search StackOverflow: https://stackoverflow.com/questions/tagged/google-cloud-platform+python

If you are still having issues, please be sure to include as much information as possible:

#### Environment details

  - OS type and version: MACOS Catalina 10.15.5
  - Python version: 3.7.6
  - pip version: 20.0.2
  - `google-cloud-bigquery` version: 1.28.0

#### Steps to reproduce

Step 1.  Extract BQ tabke data to bucket

#### Code example
```
# from google.cloud import bigquery
# client = bigquery.Client()
# bucket_name = 'my-bucket'

destination_uri = "gs://{}/{}".format(bucket_name, "shakespeare.json")
dataset_ref = bigquery.DatasetReference(project, dataset_id)
table_ref = dataset_ref.table("shakespeare")
job_config = bigquery.job.ExtractJobConfig()
job_config.destinationFormat = bigquery.DestinationFormat.NEWLINE_DELIMITED_JSON
job_config.compression = bigquery.Compression.GZIP

extract_job = client.extract_table(
    table_ref,
    destination_uri,
    # Location must match that of the source table.
    location="US",
    job_config=job_config,
)  # API request
extract_job.result()  # Waits for job to complete.

```

  2. Fetch the table from bucket and load into bq




#### Code example

```
from google.cloud import bigquery

# Construct a BigQuery client object.
client = bigquery.Client()

# TODO(developer): Set table_id to the ID of the table to create.
# table_id = "your-project.your_dataset.your_table_name"

job_config = bigquery.LoadJobConfig(
    schema=[
        bigquery.SchemaField("name", "STRING"),
        bigquery.SchemaField("post_abbr", "STRING"),
    ],  #This is not shakespeare schema, as just copied pasted the code provided on google docs
    source_format=bigquery.SourceFormat.NEWLINE_DELIMITED_JSON,
)
uri = "gs://{}/{}".format(bucket_name, "shakespeare.json")

load_job = client.load_table_from_uri(
    uri,
    table_id,
    location="US",  # Must match the destination dataset location.
    job_config=job_config,
)  # Make an API request.

load_job.result()  # Waits for the job to complete.

destination_table = client.get_table(table_id)
print("Loaded {} rows.".format(destination_table.num_rows))


```

#### Stack trace
```
Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the errors[] collection for more details.

Error while reading data, error message: JSON processing encountered too many errors, giving up. Rows: 1; errors: 1; max bad: 0; error percent: 0

Error while reading data, error message: JSON parsing error in row starting at position 0: Parser terminated before end of string
```

However when I extract the table using bq command line and follow the example of:

```
bq extract \
--compression GZIP \
--destination_format NEWLINE_DELIMITED_JSON \
'mydataset.mytable' \
gs://example-bucket/myfile.json

``` 

And then run the second step, I don't get any error and my table is successfully imported.

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!



Discussion:
I see a typo. There is no `destinationFormat` property on `ExtractJobConfig`

```
job_config.destinationFormat = bigquery.DestinationFormat.NEWLINE_DELIMITED_JSON
```

The correct name is `destination_format`

https://googleapis.dev/python/bigquery/latest/generated/google.cloud.bigquery.job.ExtractJobConfig.html#google.cloud.bigquery.job.ExtractJobConfig.destination_format

Perhaps we need ways of warning about unknown properties?
On internal Q&A forum question 5801905264132096, this same issue was encountered, but with setting Table properties on a TableReference object.
Ah thanks, that might be it, a warning would be lovely

Resolving patch:
diff --git a/google/cloud/bigquery/job/base.py b/google/cloud/bigquery/job/base.py
--- a/google/cloud/bigquery/job/base.py
+++ b/google/cloud/bigquery/job/base.py
@@ -659,6 +659,14 @@ def __init__(self, job_type, **kwargs):
         for prop, val in kwargs.items():
             setattr(self, prop, val)
 
+    def __setattr__(self, name, value):
+        """Override to be able to raise error if an unknown property is being set"""
+        if not name.startswith("_") and not hasattr(type(self), name):
+            raise AttributeError(
+                "Property {} is unknown for {}.".format(name, type(self))
+            )
+        super(_JobConfig, self).__setattr__(name, value)
+
     @property
     def labels(self):
         """Dict[str, str]: Labels for the job.


Test patch:
diff --git a/tests/system.py b/tests/system.py
--- a/tests/system.py
+++ b/tests/system.py
@@ -76,7 +76,7 @@
 from google.cloud.bigquery.dataset import DatasetReference
 from google.cloud.bigquery.table import Table
 from google.cloud._helpers import UTC
-from google.cloud.bigquery import dbapi
+from google.cloud.bigquery import dbapi, enums
 from google.cloud import storage
 
 from test_utils.retry import RetryErrors
@@ -1789,10 +1789,8 @@ def test_query_w_wrong_config(self):
         rows = list(Config.CLIENT.query("SELECT 1;").result())
         assert rows[0][0] == 1
 
-        project = Config.CLIENT.project
-        dataset_ref = bigquery.DatasetReference(project, "dset")
         bad_config = LoadJobConfig()
-        bad_config.destination = dataset_ref.table("tbl")
+        bad_config.source_format = enums.SourceFormat.CSV
         with self.assertRaises(Exception):
             Config.CLIENT.query(good_query, job_config=bad_config).result()
 
diff --git a/tests/unit/job/test_base.py b/tests/unit/job/test_base.py
--- a/tests/unit/job/test_base.py
+++ b/tests/unit/job/test_base.py
@@ -19,6 +19,7 @@
 from google.api_core import exceptions
 import google.api_core.retry
 import mock
+import pytest
 
 from .helpers import _make_client
 from .helpers import _make_connection
@@ -1021,6 +1022,12 @@ def test_ctor(self):
         self.assertEqual(job_config._job_type, self.JOB_TYPE)
         self.assertEqual(job_config._properties, {self.JOB_TYPE: {}})
 
+    def test_ctor_with_unknown_property_raises_error(self):
+        error_text = "Property wrong_name is unknown for"
+        with pytest.raises(AttributeError, match=error_text):
+            config = self._make_one()
+            config.wrong_name = None
+
     def test_fill_from_default(self):
         from google.cloud.bigquery import QueryJobConfig