Repository: apache/airflow

Issue:
BigQueryToGCSOperator: Invalid dataset ID error
### Apache Airflow Provider(s)

google

### Versions of Apache Airflow Providers

`apache-airflow-providers-google==6.3.0`

### Apache Airflow version

2.2.3

### Operating System

Linux

### Deployment

Composer

### Deployment details

- Composer Environment version: `composer-2.0.3-airflow-2.2.3`


### What happened

When I use  BigQueryToGCSOperator, I got following error.
```
Invalid dataset ID "MY_PROJECT:MY_DATASET". Dataset IDs must be alphanumeric (plus underscores and dashes) and must be at most 1024 characters long.
```

### What you expected to happen

I guess that it is due to I use colon (`:` ) as the separator between project_id and dataset_id in `source_project_dataset_table `.
I tried use dot(`.`) as separator and it worked.
However, [document of BigQueryToGCSOperator](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/_api/airflow/providers/google/cloud/transfers/bigquery_to_gcs/index.html) states that it is possible to use colon as the separator between project_id and dataset_id. In fact, at least untill Airflow1.10.15 version, it also worked with colon separator.
In Airflow 1.10.*, it separate and extract project_id and dataset_id by colon in bigquery hook. But `apache-airflow-providers-google==6.3.0` doesn't have this process.
https://github.com/apache/airflow/blob/d3b066931191b82880d216af103517ea941c74ba/airflow/contrib/hooks/bigquery_hook.py#L2186-L2247

### How to reproduce

You can reproduce following steps.
- Create a test DAG to execute BigQueryToGCSOperator in Composer environment(`composer-2.0.3-airflow-2.2.3`).
- And give `source_project_dataset_table` arg source BigQuery table path in following format.
- Trigger DAG.
```
source_project_dataset_table = 'PROJECT_ID:DATASET_ID.TABLE_ID'
```

### Anything else

_No response_

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)



Discussion:
Thanks for opening your first issue here! Be sure to follow the issue template!

Assigned you @shuhoy 

Resolving patch:
diff --git a/airflow/providers/google/cloud/hooks/bigquery.py b/airflow/providers/google/cloud/hooks/bigquery.py
--- a/airflow/providers/google/cloud/hooks/bigquery.py
+++ b/airflow/providers/google/cloud/hooks/bigquery.py
@@ -1905,7 +1905,7 @@ def run_copy(
     def run_extract(
         self,
         source_project_dataset_table: str,
-        destination_cloud_storage_uris: str,
+        destination_cloud_storage_uris: List[str],
         compression: str = 'NONE',
         export_format: str = 'CSV',
         field_delimiter: str = ',',
@@ -1945,7 +1945,7 @@ def run_extract(
             var_name='source_project_dataset_table',
         )
 
-        configuration = {
+        configuration: Dict[str, Any] = {
             'extract': {
                 'sourceTable': {
                     'projectId': source_project,
@@ -1956,7 +1956,7 @@ def run_extract(
                 'destinationUris': destination_cloud_storage_uris,
                 'destinationFormat': export_format,
             }
-        }  # type: Dict[str, Any]
+        }
 
         if labels:
             configuration['labels'] = labels
diff --git a/airflow/providers/google/cloud/transfers/bigquery_to_gcs.py b/airflow/providers/google/cloud/transfers/bigquery_to_gcs.py
--- a/airflow/providers/google/cloud/transfers/bigquery_to_gcs.py
+++ b/airflow/providers/google/cloud/transfers/bigquery_to_gcs.py
@@ -17,9 +17,7 @@
 # under the License.
 """This module contains Google BigQuery to Google Cloud Storage operator."""
 import warnings
-from typing import TYPE_CHECKING, Any, Dict, List, Optional, Sequence, Union
-
-from google.cloud.bigquery.table import TableReference
+from typing import TYPE_CHECKING, Dict, List, Optional, Sequence, Union
 
 from airflow.models import BaseOperator
 from airflow.providers.google.cloud.hooks.bigquery import BigQueryHook
@@ -128,26 +126,12 @@ def execute(self, context: 'Context'):
             location=self.location,
             impersonation_chain=self.impersonation_chain,
         )
-
-        table_ref = TableReference.from_string(self.source_project_dataset_table, hook.project_id)
-
-        configuration: Dict[str, Any] = {
-            'extract': {
-                'sourceTable': table_ref.to_api_repr(),
-                'compression': self.compression,
-                'destinationUris': self.destination_cloud_storage_uris,
-                'destinationFormat': self.export_format,
-            }
-        }
-
-        if self.labels:
-            configuration['labels'] = self.labels
-
-        if self.export_format == 'CSV':
-            # Only set fieldDelimiter and printHeader fields if using CSV.
-            # Google does not like it if you set these fields for other export
-            # formats.
-            configuration['extract']['fieldDelimiter'] = self.field_delimiter
-            configuration['extract']['printHeader'] = self.print_header
-
-        hook.insert_job(configuration=configuration)
+        hook.run_extract(
+            source_project_dataset_table=self.source_project_dataset_table,
+            destination_cloud_storage_uris=self.destination_cloud_storage_uris,
+            compression=self.compression,
+            export_format=self.export_format,
+            field_delimiter=self.field_delimiter,
+            print_header=self.print_header,
+            labels=self.labels,
+        )


Test patch:
diff --git a/tests/providers/google/cloud/transfers/test_bigquery_to_gcs.py b/tests/providers/google/cloud/transfers/test_bigquery_to_gcs.py
--- a/tests/providers/google/cloud/transfers/test_bigquery_to_gcs.py
+++ b/tests/providers/google/cloud/transfers/test_bigquery_to_gcs.py
@@ -27,10 +27,10 @@
 PROJECT_ID = 'test-project-id'
 
 
-class TestBigQueryToCloudStorageOperator(unittest.TestCase):
+class TestBigQueryToGCSOperator(unittest.TestCase):
     @mock.patch('airflow.providers.google.cloud.transfers.bigquery_to_gcs.BigQueryHook')
     def test_execute(self, mock_hook):
-        source_project_dataset_table = f'{TEST_DATASET}.{TEST_TABLE_ID}'
+        source_project_dataset_table = f'{PROJECT_ID}:{TEST_DATASET}.{TEST_TABLE_ID}'
         destination_cloud_storage_uris = ['gs://some-bucket/some-file.txt']
         compression = 'NONE'
         export_format = 'CSV'
@@ -38,24 +38,6 @@ def test_execute(self, mock_hook):
         print_header = True
         labels = {'k1': 'v1'}
 
-        mock_hook().project_id = PROJECT_ID
-
-        configuration = {
-            'extract': {
-                'sourceTable': {
-                    'projectId': mock_hook().project_id,
-                    'datasetId': TEST_DATASET,
-                    'tableId': TEST_TABLE_ID,
-                },
-                'compression': compression,
-                'destinationUris': destination_cloud_storage_uris,
-                'destinationFormat': export_format,
-                'fieldDelimiter': field_delimiter,
-                'printHeader': print_header,
-            },
-            'labels': labels,
-        }
-
         operator = BigQueryToGCSOperator(
             task_id=TASK_ID,
             source_project_dataset_table=source_project_dataset_table,
@@ -69,4 +51,12 @@ def test_execute(self, mock_hook):
 
         operator.execute(None)
 
-        mock_hook.return_value.insert_job.assert_called_once_with(configuration=configuration)
+        mock_hook.return_value.run_extract.assert_called_once_with(
+            source_project_dataset_table=source_project_dataset_table,
+            destination_cloud_storage_uris=destination_cloud_storage_uris,
+            compression=compression,
+            export_format=export_format,
+            field_delimiter=field_delimiter,
+            print_header=print_header,
+            labels=labels,
+        )