Repository: googleapis/google-cloud-python

Issue:
No option to specify a custom nullMarker while using table.upload_from_file method
We have source data where Null values are represented by \N. 

However, we're unable to use table.upload_from_file method as there is no parameter to specify a custom null marker(configuration.load.nullMarker).




Discussion:
@HassanJaveed84 Thanks for the report.  If you would care to prepare a pull request patching the `table.py` module, it would land more quickly.  Looking at the code, it would need a new `null_marker` argument to `Table.load_from_file`, which would then get passed down to the `_configure_job_metadata` helper.  The PR would also need to update one of the tests to demonstrate passing the value.
@tseaver I'll go ahead and create a PR, thanks!
@tseaver I tried following the instructions for local development on:
https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/CONTRIBUTING.rst

However, I'm unable to run the tests using the given instructions. 
"To run all tests for google-cloud-python on a single Python version, run py.test from your development virtualenv "

Not sure about py.test ? 
@HassanJaveed84 sorry `CONTRIBUTING` is very out of date (/cc @lukesneeringer).

Instead, install `nox` via:

```
pip install --upgrade nox-automation
```

then run `nox` in the `bigquery` directory:

```
$ cd path/to/git/repo/
$ cd bigquery/
$ nox -l  # list of tests
$ nox -s "unit_tests(python_version='3.6')"  # One test
$ nox  # All tests, though you probably don't want system tests
```

Thanks @dhermes. 
I'll let you know if I run into problems again.
I also found that client.load_table_from_storage does not accept a null_marker value. 
I'll try to address both of these issues. 

Resolving patch:
diff --git a/bigquery/google/cloud/bigquery/table.py b/bigquery/google/cloud/bigquery/table.py
--- a/bigquery/google/cloud/bigquery/table.py
+++ b/bigquery/google/cloud/bigquery/table.py
@@ -1043,7 +1043,8 @@ def upload_from_file(self,
                          skip_leading_rows=None,
                          write_disposition=None,
                          client=None,
-                         job_name=None):
+                         job_name=None,
+                         null_marker=None):
         """Upload the contents of this table from a file-like object.
 
         :type file_obj: file
@@ -1116,6 +1117,9 @@ def upload_from_file(self,
         :param job_name: Optional. The id of the job. Generated if not
                          explicitly passed in.
 
+        :type null_marker: str
+        :param null_marker: Optional. A custom null marker (example: "\\N")
+
         :rtype: :class:`~google.cloud.bigquery.jobs.LoadTableFromStorageJob`
 
         :returns: the job instance used to load the data (e.g., for
@@ -1135,7 +1139,7 @@ def upload_from_file(self,
                                 encoding, field_delimiter,
                                 ignore_unknown_values, max_bad_records,
                                 quote_character, skip_leading_rows,
-                                write_disposition, job_name)
+                                write_disposition, job_name, null_marker)
 
         try:
             created_json = self._do_upload(
@@ -1157,7 +1161,8 @@ def _configure_job_metadata(metadata,  # pylint: disable=too-many-arguments
                             quote_character,
                             skip_leading_rows,
                             write_disposition,
-                            job_name):
+                            job_name,
+                            null_marker):
     """Helper for :meth:`Table.upload_from_file`."""
     load_config = metadata['configuration']['load']
 
@@ -1194,6 +1199,9 @@ def _configure_job_metadata(metadata,  # pylint: disable=too-many-arguments
     if job_name is not None:
         load_config['jobReference'] = {'jobId': job_name}
 
+    if null_marker is not None:
+        load_config['nullMarker'] = null_marker
+
 
 def _parse_schema_resource(info):
     """Parse a resource fragment into a schema field.


Test patch:
diff --git a/bigquery/tests/unit/test_table.py b/bigquery/tests/unit/test_table.py
--- a/bigquery/tests/unit/test_table.py
+++ b/bigquery/tests/unit/test_table.py
@@ -1868,7 +1868,8 @@ def test_upload_file_resumable_metadata(self):
             'quote_character': '"',
             'skip_leading_rows': 1,
             'write_disposition': 'WRITE_APPEND',
-            'job_name': 'oddjob'
+            'job_name': 'oddjob',
+            'null_marker': r'\N',
         }
 
         expected_config = {
@@ -1878,7 +1879,7 @@ def test_upload_file_resumable_metadata(self):
                     'destinationTable': {
                         'projectId': table._dataset._client.project,
                         'datasetId': table.dataset_name,
-                        'tableId': table.name
+                        'tableId': table.name,
                     },
                     'allowJaggedRows': config_args['allow_jagged_rows'],
                     'allowQuotedNewlines':
@@ -1892,9 +1893,10 @@ def test_upload_file_resumable_metadata(self):
                     'quote': config_args['quote_character'],
                     'skipLeadingRows': config_args['skip_leading_rows'],
                     'writeDisposition': config_args['write_disposition'],
-                    'jobReference': {'jobId': config_args['job_name']}
-                }
-            }
+                    'jobReference': {'jobId': config_args['job_name']},
+                    'nullMarker': config_args['null_marker'],
+                },
+            },
         }
 
         do_upload_patch = self._make_do_upload_patch(