Repository: apache/airflow

Issue:
TableauRefreshWorkbookOperator fails when using personal access token (Tableau authentication method)
**Apache Airflow version**: 2.0.1

**What happened**:

The operator fails at the last step, after successfully refreshing the workbook with this error:
```
tableauserverclient.server.endpoint.exceptions.ServerResponseError: 

        401002: Unauthorized Access
                Invalid authentication credentials were provided.
```

**What you expected to happen**:
It should not fail, like when we use the username/password authentication method (instead of personal_access_token)

<!-- What do you think went wrong? -->

Tableau server does not allow concurrent connections when using personal_access_token https://github.com/tableau/server-client-python/issues/717
The solution would be redesigning completely the operator to only call the hook once.
My quick fix was to edit this in TableauHook:

```
    def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
        pass
```

**How to reproduce it**:

Run this operator TableauRefreshWorkbookOperator using Tableau personal_access_token authentication (token_name, personal_access_token).



Discussion:
Thanks for opening your first issue here! Be sure to follow the issue template!

Hey! I can make an investigation on this issue and prepare the first PR! Are you ok with that?
yes, sure @samgans ! Thanks
Hello again!

I've made a basic investigation on it and found that we actually have a bit weird situation. The thing is that if you are authenticating your Tableau connection with a personal access token, each time you will try to open a parallel connection, the previous one will be invalidated. You can read more about this in the thread: https://github.com/tableau/server-client-python/issues/717.

Speaking of our issue and how it is related to the Tableau personal token: in the "blocking" task, we are using the TableauJobStatusSensor that opens another connection, therefore, invalidating the previous one causing the wrapping TableauHook to fail (you can check more in the `execute` method of `TableauRefreshWorkbookOperator` in  `tableau_refresh_workbook.py`).

So, we have two options here:

1. As the blocking process uses an explicit sensor to wait till the job is finished (and the sensor opens another connection), we can make the Sensor configurable to use the existing connection (which is TableauHook). So, we can pass it as a parameter and forget about the symptom.

**BUT!** It seems like if the user will add one or more Tableau task which will use the same token and run in parallel, we can get some hard-to-debug errors due to parallel invalidation of tokens. Therefore, it seems to me that we shouldn't use this approach and take a look at the next one.

2. Just deprecate the authentication by the personal token and indicate this clearly in the documentation. This will remove all kinds of bugs related to the invalidation of access tokens and will not deceive the users about the operability of the execution of the parallel tasks with a personal token.

What do you think?
Great, thanks a lot for investigating this @samgans !

On my side I created a new 'airflow' user in Tableau with 'Site Administrator Explorer' role and will use the username/password authentication then.
I think it makes sense to deprecate the personal token authentication for the TableauRefreshWorkbookOperator and adding some comments about this issue in the code as it was not even working properly now.

Could this be a possible 3rd solution though?
Opening and closing the connection each time we want to call Tableau server and adding some retry strategy to make sure we cover the cases when several authentications are made at the same time.



@pmalafosse,

The deprecation will not be related directly to `TableauRefreshWorkbookOperator` but will be done rather for `TableauHook`, where the method of authentication using the personal token is implemented.

Speaking of the third solution, I've been thinking about some kind of strategy of reusing the existing connection by `TableauHook` (e.g. saving it in the global variable which will be available for all instances), but here is my big doubt:

I know exactly that if we will reuse a connection from a global variable, all will be good if we will have multiple related tasks in one DAG. But if we will have some other Tableau tasks with the same personal token in other DAGs? It seems to me that all the DAG definitions are executed at different processes so we can't guarantee that there will be access to one global variable between all the DAG definitions.

We also can't check if there are multiple authentication attempts using the Tableau server client, so it seems to me that deprecation is the best option. Also, if you have some ideas on how the third solution can be implemented, I'm open to trying this out.

Resolving patch:
diff --git a/airflow/providers/tableau/hooks/tableau.py b/airflow/providers/tableau/hooks/tableau.py
--- a/airflow/providers/tableau/hooks/tableau.py
+++ b/airflow/providers/tableau/hooks/tableau.py
@@ -14,6 +14,7 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
+import warnings
 from distutils.util import strtobool
 from enum import Enum
 from typing import Any, Optional
@@ -39,7 +40,11 @@ class TableauJobFinishCode(Enum):
 class TableauHook(BaseHook):
     """
     Connects to the Tableau Server Instance and allows to communicate with it.
-    .. see also:: https://tableau.github.io/server-client-python/docs/
+
+    Can be used as a context manager: automatically authenticates the connection
+    when opened and signs out when closed.
+
+    .. seealso:: https://tableau.github.io/server-client-python/docs/
 
     :param site_id: The id of the site where the workbook belongs to.
         It will connect to the default site if you don't provide an id.
@@ -81,7 +86,8 @@ def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None:
 
     def get_conn(self) -> Auth.contextmgr:
         """
-        Signs in to the Tableau Server and automatically signs out if used as ContextManager.
+        Sign in to the Tableau Server.
+
         :return: an authorized Tableau Server Context Manager object.
         :rtype: tableauserverclient.server.Auth.contextmgr
         """
@@ -98,6 +104,12 @@ def _auth_via_password(self) -> Auth.contextmgr:
         return self.server.auth.sign_in(tableau_auth)
 
     def _auth_via_token(self) -> Auth.contextmgr:
+        """The method is deprecated. Please, use the authentication via password instead."""
+        warnings.warn(
+            "Authentication via personal access token is deprecated. "
+            "Please, use the password authentication to avoid inconsistencies.",
+            DeprecationWarning,
+        )
         tableau_auth = PersonalAccessTokenAuth(
             token_name=self.conn.extra_dejson['token_name'],
             personal_access_token=self.conn.extra_dejson['personal_access_token'],
@@ -111,10 +123,13 @@ def get_all(self, resource_name: str) -> Pager:
         .. see also:: https://tableau.github.io/server-client-python/docs/page-through-results
 
         :param resource_name: The name of the resource to paginate.
-            For example: jobs or workbooks
+            For example: jobs or workbooks.
         :type resource_name: str
         :return: all items by returning a Pager.
         :rtype: tableauserverclient.Pager
         """
-        resource = getattr(self.server, resource_name)
+        try:
+            resource = getattr(self.server, resource_name)
+        except AttributeError:
+            raise ValueError(f"Resource name {resource_name} is not found.")
         return Pager(resource.get)
diff --git a/airflow/providers/tableau/operators/tableau_refresh_workbook.py b/airflow/providers/tableau/operators/tableau_refresh_workbook.py
--- a/airflow/providers/tableau/operators/tableau_refresh_workbook.py
+++ b/airflow/providers/tableau/operators/tableau_refresh_workbook.py
@@ -20,7 +20,8 @@
 
 from airflow.exceptions import AirflowException
 from airflow.models import BaseOperator
-from airflow.providers.tableau.hooks.tableau import TableauHook
+from airflow.providers.tableau.hooks.tableau import TableauHook, TableauJobFinishCode
+from airflow.providers.tableau.sensors.tableau_job_status import TableauJobFailedException
 
 
 class TableauRefreshWorkbookOperator(BaseOperator):
@@ -33,10 +34,12 @@ class TableauRefreshWorkbookOperator(BaseOperator):
     :type workbook_name: str
     :param site_id: The id of the site where the workbook belongs to.
     :type site_id: Optional[str]
-    :param blocking: By default the extract refresh will be blocking means it will wait until it has finished.
+    :param blocking: Defines if the job waits until the refresh has finished.
+        Default: True.
     :type blocking: bool
     :param tableau_conn_id: The :ref:`Tableau Connection id <howto/connection:tableau>`
-        containing the credentials to authenticate to the Tableau Server.
+        containing the credentials to authenticate to the Tableau Server. Default:
+        'tableau_default'.
     :type tableau_conn_id: str
     """
 
@@ -69,15 +72,13 @@ def execute(self, context: dict) -> str:
 
             job_id = self._refresh_workbook(tableau_hook, workbook.id)
             if self.blocking:
-                from airflow.providers.tableau.sensors.tableau_job_status import TableauJobStatusSensor
-
-                TableauJobStatusSensor(
-                    job_id=job_id,
-                    site_id=self.site_id,
-                    tableau_conn_id=self.tableau_conn_id,
-                    task_id='wait_until_succeeded',
-                    dag=None,
-                ).execute(context={})
+                finish_code = TableauJobFinishCode.PENDING
+                negative_codes = (TableauJobFinishCode.ERROR, TableauJobFinishCode.CANCELED)
+                while not finish_code == TableauJobFinishCode.SUCCESS:
+                    return_code = int(tableau_hook.server.jobs.get_by_id(job_id).finish_code)
+                    finish_code = TableauJobFinishCode(return_code)
+                    if finish_code in negative_codes:
+                        raise TableauJobFailedException('The Tableau Refresh Workbook Job failed!')
                 self.log.info('Workbook %s has been successfully refreshed.', self.workbook_name)
             return job_id
 
@@ -86,7 +87,6 @@ def _get_workbook_by_name(self, tableau_hook: TableauHook) -> WorkbookItem:
             if workbook.name == self.workbook_name:
                 self.log.info('Found matching workbook with id %s', workbook.id)
                 return workbook
-
         raise AirflowException(f'Workbook {self.workbook_name} not found!')
 
     def _refresh_workbook(self, tableau_hook: TableauHook, workbook_id: str) -> str:
diff --git a/airflow/providers/tableau/sensors/tableau_job_status.py b/airflow/providers/tableau/sensors/tableau_job_status.py
--- a/airflow/providers/tableau/sensors/tableau_job_status.py
+++ b/airflow/providers/tableau/sensors/tableau_job_status.py
@@ -31,7 +31,7 @@ class TableauJobStatusSensor(BaseSensorOperator):
 
     .. seealso:: https://tableau.github.io/server-client-python/docs/api-ref#jobs
 
-    :param job_id: The job to watch.
+    :param job_id: Id of the job to watch.
     :type job_id: str
     :param site_id: The id of the site where the workbook belongs to.
     :type site_id: Optional[str]
@@ -69,6 +69,6 @@ def poke(self, context: dict) -> bool:
                 int(tableau_hook.server.jobs.get_by_id(self.job_id).finish_code)
             )
             self.log.info('Current finishCode is %s (%s)', finish_code.name, finish_code.value)
-            if finish_code in [TableauJobFinishCode.ERROR, TableauJobFinishCode.CANCELED]:
+            if finish_code in (TableauJobFinishCode.ERROR, TableauJobFinishCode.CANCELED):
                 raise TableauJobFailedException('The Tableau Refresh Workbook Job failed!')
             return finish_code == TableauJobFinishCode.SUCCESS


Test patch:
diff --git a/tests/providers/tableau/operators/test_tableau_refresh_workbook.py b/tests/providers/tableau/operators/test_tableau_refresh_workbook.py
--- a/tests/providers/tableau/operators/test_tableau_refresh_workbook.py
+++ b/tests/providers/tableau/operators/test_tableau_refresh_workbook.py
@@ -21,6 +21,7 @@
 import pytest
 
 from airflow.exceptions import AirflowException
+from airflow.providers.tableau.hooks.tableau import TableauJobFinishCode
 from airflow.providers.tableau.operators.tableau_refresh_workbook import TableauRefreshWorkbookOperator
 
 
@@ -55,27 +56,23 @@ def test_execute(self, mock_tableau_hook):
         mock_tableau_hook.server.workbooks.refresh.assert_called_once_with(2)
         assert mock_tableau_hook.server.workbooks.refresh.return_value.id == job_id
 
-    @patch('airflow.providers.tableau.sensors.tableau_job_status.TableauJobStatusSensor')
     @patch('airflow.providers.tableau.operators.tableau_refresh_workbook.TableauHook')
-    def test_execute_blocking(self, mock_tableau_hook, mock_tableau_job_status_sensor):
+    def test_execute_blocking(self, mock_tableau_hook):
         """
         Test execute blocking
         """
         mock_tableau_hook.get_all = Mock(return_value=self.mocked_workbooks)
         mock_tableau_hook.return_value.__enter__ = Mock(return_value=mock_tableau_hook)
+        mock_tableau_hook.server.jobs.get_by_id = Mock(
+            return_value=Mock(finish_code=TableauJobFinishCode.SUCCESS.value)
+        )
         operator = TableauRefreshWorkbookOperator(workbook_name='wb_2', **self.kwargs)
 
         job_id = operator.execute(context={})
 
         mock_tableau_hook.server.workbooks.refresh.assert_called_once_with(2)
         assert mock_tableau_hook.server.workbooks.refresh.return_value.id == job_id
-        mock_tableau_job_status_sensor.assert_called_once_with(
-            job_id=job_id,
-            site_id=self.kwargs['site_id'],
-            tableau_conn_id='tableau_default',
-            task_id='wait_until_succeeded',
-            dag=None,
-        )
+        mock_tableau_hook.server.jobs.get_by_id.assert_called_once_with(job_id)
 
     @patch('airflow.providers.tableau.operators.tableau_refresh_workbook.TableauHook')
     def test_execute_missing_workbook(self, mock_tableau_hook):