Repository: apache/airflow

Issue:
Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
The issue is similar to this ticket [16587](https://github.com/apache/airflow/issues/16587)  and [14829](https://github.com/apache/airflow/issues/14829) however I have an updated airflow version AND updated packages than the ones suggested here and I am still getting the same outcome. When using google auth in airflow and attempting to sign in, we get an ERR_TOO_MANY_REDIRECTS.  I know what causes the symptom of this,  but hoping to find a resolution of keeping a Role in place to avoid the REDIRECTS.

- **Apache Airflow version**:
Version: v2.1.0
Git Version: .release:2.1.0+304e174674ff6921cb7ed79c0158949b50eff8fe

- **Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19+", GitVersion:"v1.19.10-gke.1600", GitCommit:"7b8e568a7fb4c9d199c2ba29a5f7d76f6b4341c2", GitTreeState:"clean", BuildDate:"2021-05-07T09:18:53Z", GoVersion:"go1.15.10b5", Compiler:"gc", Platform:"linux/amd64"}

- **Environment**: Staging

- **Cloud provider or hardware configuration**: GKE on 

- **OS** (e.g. from /etc/os-release):
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
- **Kernel** (e.g. `uname -a`):
 Linux margins-scheduler-97b6fb867-fth8p 5.4.89+ #1 SMP Sat Feb 13 19:45:14 PST 2021 x86_64 GNU/Linux
- **Install tools**: pip freeze below

**What happened**:
When using google auth in airflow and attempting to sign in, we get an ERR_TOO_MANY_REDIRECTS. 

**What you expected to happen**:
I expect to log in as my user and it assigns a default Role of Viewer at the very least OR uses our mappings in web_server config python file. But the Role is blank in Database.

<!-- What do you think went wrong? -->
We realized that we get stuck in the loop, b/c the user will be in the users table in airflow but without a Role (its literally empty). Therefore it goes from the /login to /home to /login to /home over and over again.

**How to reproduce it**:

I add the Admin role in the database for my user, and  the page that has the redirects  refreshes and lets me in to the Airflow UI. However, when I sign out and signin in again, my users Role is then erased and it starts the redirect cycle again.

As you can see there is no Role (this happens when I attempt to login)
```
id | username                     | email                   | first_name | last_name | roles
===+==============================+=========================+============+===========+======
1  | admin                        | admin@example.com       | admin      | admin     | Admin
2  | google_############ | msaenz@company.com | Cat     | Says     | 
```
I run the command: `airflow users add-role -r Admin -u google_#################`

Then the page takes me to the UI and the table now looks like this:
```
id | username                     | email                   | first_name | last_name | roles
===+==============================+=========================+============+===========+======
1  | admin                        | admin@example.com       | admin      | admin     | Admin
2  | google_############ | msaenz@company.com | Cat     | Says     |  Admin
```

How often does this problem occur? Once? Every time etc?  This occurs all the time

Here is the webserver_config.py
```
  import os
      from flask_appbuilder.security.manager import AUTH_OAUTH
      AUTH_TYPE = AUTH_OAUTH
      AUTH_ROLE_ADMIN="Admin"
      AUTH_USER_REGISTRATION = False
      AUTH_USER_REGISTRATION_ROLE = "Admin"
      OIDC_COOKIE_SECURE = False
      CSRF_ENABLED = False
      WTF_CSRF_ENABLED = True
      AUTH_ROLES_MAPPING = {"Engineering": ["Ops"],"Admins": ["Admin"]}
      AUTH_ROLES_SYNC_AT_LOGIN = True
      OAUTH_PROVIDERS = [
          {
              'name': 'google', 'icon': 'fa-google',
              'token_key': 'access_token',
              'remote_app': {
                  'client_id': '#####################.apps.googleusercontent.com',
                  'client_secret': '######################',
                  'api_base_url': 'https://www.googleapis.com/oauth2/v2/',
                  'whitelist': ['@company.com'],  # optional
                  'client_kwargs': {
                      'scope': 'email profile'
                  },
                  'request_token_url': None,
                  'access_token_url': 'https://accounts.google.com/o/oauth2/token',
                  'authorize_url': 'https://accounts.google.com/o/oauth2/auth'},
          }
      ]


```
Here is the pip freeze:
```
adal==1.2.7
alembic==1.6.2
amqp==2.6.1
anyio==3.2.1
apache-airflow==2.1.0
apache-airflow-providers-amazon==1.4.0
apache-airflow-providers-celery==1.0.1
apache-airflow-providers-cncf-kubernetes==1.2.0
apache-airflow-providers-docker==1.2.0
apache-airflow-providers-elasticsearch==1.0.4
apache-airflow-providers-ftp==1.1.0
apache-airflow-providers-google==3.0.0
apache-airflow-providers-grpc==1.1.0
apache-airflow-providers-hashicorp==1.0.2
apache-airflow-providers-http==1.1.1
apache-airflow-providers-imap==1.0.1
apache-airflow-providers-microsoft-azure==2.0.0
apache-airflow-providers-mysql==1.1.0
apache-airflow-providers-postgres==1.0.2
apache-airflow-providers-redis==1.0.1
apache-airflow-providers-sendgrid==1.0.2
apache-airflow-providers-sftp==1.2.0
apache-airflow-providers-slack==3.0.0
apache-airflow-providers-sqlite==1.0.2
apache-airflow-providers-ssh==1.3.0
apispec==3.3.2
appdirs==1.4.4
argcomplete==1.12.3
async-generator==1.10
attrs==20.3.0
azure-batch==10.0.0
azure-common==1.1.27
azure-core==1.13.0
azure-cosmos==3.2.0
azure-datalake-store==0.0.52
azure-identity==1.5.0
azure-keyvault==4.1.0
azure-keyvault-certificates==4.2.1
azure-keyvault-keys==4.3.1
azure-keyvault-secrets==4.2.0
azure-kusto-data==0.0.45
azure-mgmt-containerinstance==1.5.0
azure-mgmt-core==1.2.2
azure-mgmt-datafactory==1.1.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-resource==16.1.0
azure-nspkg==3.0.2
azure-storage-blob==12.8.1
azure-storage-common==2.1.0
azure-storage-file==2.1.0
Babel==2.9.1
bcrypt==3.2.0
billiard==3.6.4.0
blinker==1.4
boto3==1.17.71
botocore==1.20.71
cached-property==1.5.2
cachetools==4.2.2
cattrs==1.0.0
celery==4.4.7
certifi==2020.12.5
cffi==1.14.5
chardet==3.0.4
click==7.1.2
clickclick==20.10.2
cloudpickle==1.4.1
colorama==0.4.4
colorlog==5.0.1
commonmark==0.9.1
contextvars==2.4
croniter==1.0.13
cryptography==3.4.7
dask==2021.3.0
dataclasses==0.7
defusedxml==0.7.1
dill==0.3.1.1
distlib==0.3.1
distributed==2.19.0
dnspython==1.16.0
docker==3.7.3
docker-pycreds==0.4.0
docutils==0.17.1
elasticsearch==7.5.1
elasticsearch-dbapi==0.1.0
elasticsearch-dsl==7.3.0
email-validator==1.1.2
eventlet==0.31.0
filelock==3.0.12
Flask==1.1.2
Flask-AppBuilder==3.3.0
Flask-Babel==1.0.0
Flask-Caching==1.10.1
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.5.1
Flask-WTF==0.14.3
flower==0.9.7
gevent==21.1.2
google-ads==4.0.0
google-api-core==1.26.3
google-api-python-client==1.12.8
google-auth==1.30.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==0.4.4
google-cloud-automl==2.3.0
google-cloud-bigquery==2.16.0
google-cloud-bigquery-datatransfer==3.1.1
google-cloud-bigquery-storage==2.4.0
google-cloud-bigtable==1.7.0
google-cloud-container==1.0.1
google-cloud-core==1.6.0
google-cloud-datacatalog==3.1.1
google-cloud-dataproc==2.3.1
google-cloud-dlp==1.0.0
google-cloud-kms==2.2.0
google-cloud-language==1.3.0
google-cloud-logging==2.3.1
google-cloud-memcache==0.3.0
google-cloud-monitoring==2.2.1
google-cloud-os-login==2.1.0
google-cloud-pubsub==2.4.2
google-cloud-redis==2.1.0
google-cloud-secret-manager==1.0.0
google-cloud-spanner==1.19.1
google-cloud-speech==1.3.2
google-cloud-storage==1.38.0
google-cloud-tasks==2.2.0
google-cloud-texttospeech==1.0.1
google-cloud-translate==1.7.0
google-cloud-videointelligence==1.16.1
google-cloud-vision==1.0.0
google-cloud-workflows==0.3.0
google-crc32c==1.1.2
google-resumable-media==1.2.0
googleapis-common-protos==1.53.0
graphviz==0.16
greenlet==1.1.0
grpc-google-iam-v1==0.12.3
grpcio==1.37.1
grpcio-gcp==0.2.2
gunicorn==20.1.0
h11==0.12.0
HeapDict==1.0.1
httpcore==0.13.6
httplib2==0.17.4
httpx==0.18.2
humanize==3.5.0
hvac==0.10.11
idna==2.10
immutables==0.15
importlib-metadata==1.7.0
importlib-resources==1.5.0
inflection==0.5.1
iso8601==0.1.14
isodate==0.6.0
itsdangerous==1.1.0
Jinja2==2.11.3
jmespath==0.10.0
json-merge-patch==0.2
jsonschema==3.2.0
kombu==4.6.11
kubernetes==11.0.0
lazy-object-proxy==1.4.3
ldap3==2.9
libcst==0.3.18
lockfile==0.12.2
Mako==1.1.4
Markdown==3.3.4
MarkupSafe==1.1.1
marshmallow==3.12.1
marshmallow-enum==1.5.1
marshmallow-oneofschema==2.1.0
marshmallow-sqlalchemy==0.23.1
msal==1.11.0
msal-extensions==0.3.0
msgpack==1.0.2
msrest==0.6.21
msrestazure==0.6.4
mypy-extensions==0.4.3
mysql-connector-python==8.0.22
mysqlclient==2.0.3
numpy==1.19.5
oauthlib==2.1.0
openapi-schema-validator==0.1.5
openapi-spec-validator==0.3.0
packaging==20.9
pandas==1.1.5
pandas-gbq==0.14.1
paramiko==2.7.2
pendulum==2.1.2
pep562==1.0
plyvel==1.3.0
portalocker==1.7.1
prison==0.1.3
prometheus-client==0.8.0
proto-plus==1.18.1
protobuf==3.16.0
psutil==5.8.0
psycopg2-binary==2.8.6
pyarrow==3.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pydata-google-auth==1.2.0
Pygments==2.9.0
PyJWT==1.7.1
PyNaCl==1.4.0
pyOpenSSL==19.1.0
pyparsing==2.4.7
pyrsistent==0.17.3
pysftp==0.2.9
python-daemon==2.3.0
python-dateutil==2.8.1
python-editor==1.0.4
python-http-client==3.3.2
python-ldap==3.3.1
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2021.1
pytzdata==2020.1
PyYAML==5.4.1
redis==3.5.3
requests==2.25.1
requests-oauthlib==1.1.0
rfc3986==1.5.0
rich==9.2.0
rsa==4.7.2
s3transfer==0.4.2
sendgrid==6.7.0
setproctitle==1.2.2
six==1.16.0
slack-sdk==3.5.1
sniffio==1.2.0
sortedcontainers==2.3.0
SQLAlchemy==1.3.24
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.37.2
sshtunnel==0.1.5
starkbank-ecdsa==1.1.0
statsd==3.3.0
swagger-ui-bundle==0.0.8
tabulate==0.8.9
tblib==1.7.0
tenacity==6.2.0
termcolor==1.1.0
text-unidecode==1.3
toolz==0.11.1
tornado==6.1
typing==3.7.4.3
typing-extensions==3.7.4.3
typing-inspect==0.6.0
unicodecsv==0.14.1
uritemplate==3.0.1
urllib3==1.25.11
vine==1.3.0
virtualenv==20.4.6
watchtower==0.7.3
websocket-client==0.59.0
Werkzeug==1.0.1
WTForms==2.3.3
zict==2.0.0
zipp==3.4.1
zope.event==4.5.0
zope.interface==5.4.0

```

Thanks in advance.


Discussion:
`AUTH_USER_REGISTRATION_ROLE` only applies if `AUTH_USER_REGISTRATION` is true.

Looks like the FAB google oauth provider doesn't support `AUTH_ROLE_MAPPING` by default:
https://github.com/dpgaspar/Flask-AppBuilder/blob/95947e84e04a999a474dfe8620fb0f36d71f0467/flask_appbuilder/security/manager.py#L585-L590

You'd need a custom security manager to return `role_keys`, for example like is done in the OP of #14829.
(I'm sure FAB would love to have support for this with the google oauth provider, so consider contributing it back if you get it working).
> `AUTH_USER_REGISTRATION_ROLE` only applies if `AUTH_USER_REGISTRATION` is true.
> 
> Looks like the FAB google oauth provider doesn't support `AUTH_ROLE_MAPPING` by default:
> https://github.com/dpgaspar/Flask-AppBuilder/blob/95947e84e04a999a474dfe8620fb0f36d71f0467/flask_appbuilder/security/manager.py#L585-L590
> 
> You'd need a custom security manager to return `role_keys`, for example like is done in the OP of #14829.
> (I'm sure FAB would love to have support for this with the google oauth provider, so consider contributing it back if you get it working).

Ah okay, I will see if that works and then close the ticket if that is the case. But it definitely erased the Role even with AUTH_USER_REGISTRATION_ROLE and AUTH_USER_REGISTRATION  are both false as well. I made AUTH_USER_ROLE False as well.
> But it definitely erased the Role

Yep, that's the behavior with `AUTH_ROLES_SYNC_AT_LOGIN` and no matching roles in `AUTH_ROLE_MAPPING`. You could also disable `AUTH_ROLES_SYNC_AT_LOGIN` and manage the roles yourself.
Is this an Airflow bug, or a FAB bug?
FAB.
This is likely a "wont Fix" from airflow's side I'm afraid then.
I haven't poked around yet, but we may be able to handle the 'no role' scenario more gracefully on our side. Let me take a stab at that before we close this.
@jedcunningham Cool

Resolving patch:
diff --git a/airflow/www/auth.py b/airflow/www/auth.py
--- a/airflow/www/auth.py
+++ b/airflow/www/auth.py
@@ -18,7 +18,7 @@
 from functools import wraps
 from typing import Callable, Optional, Sequence, Tuple, TypeVar, cast
 
-from flask import current_app, flash, redirect, request, url_for
+from flask import current_app, flash, g, redirect, request, url_for
 
 T = TypeVar("T", bound=Callable)
 
@@ -30,6 +30,9 @@ def requires_access_decorator(func: T):
         @wraps(func)
         def decorated(*args, **kwargs):
             appbuilder = current_app.appbuilder
+            if not g.user.is_anonymous and not g.user.roles:
+                return redirect(url_for("Airflow.no_roles"))
+
             if appbuilder.sm.check_authorization(permissions, request.args.get('dag_id', None)):
                 return func(*args, **kwargs)
             else:
diff --git a/airflow/www/views.py b/airflow/www/views.py
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -527,6 +527,20 @@ def health(self):
 
         return wwwutils.json_response(payload)
 
+    @expose('/no_roles')
+    def no_roles(self):
+        """Show 'user has no roles' on screen (instead of an endless redirect loop)"""
+        if g.user.is_anonymous or g.user.roles:
+            return redirect(url_for("Airflow.index"))
+
+        return render_template(
+            'airflow/no_roles.html',
+            hostname=socket.getfqdn()
+            if conf.getboolean('webserver', 'EXPOSE_HOSTNAME', fallback=True)
+            else 'redact',
+            logout_url=current_app.appbuilder.get_url_for_logout,
+        )
+
     @expose('/home')
     @auth.has_access(
         [


Test patch:
diff --git a/tests/www/views/test_views_acl.py b/tests/www/views/test_views_acl.py
--- a/tests/www/views/test_views_acl.py
+++ b/tests/www/views/test_views_acl.py
@@ -759,3 +759,44 @@ def test_get_logs_with_metadata_failure(dag_faker_client):
     )
     check_content_not_in_response('"message":', resp)
     check_content_not_in_response('"metadata":', resp)
+
+
+@pytest.fixture(scope="module")
+def user_no_roles(acl_app):
+    user = create_user(
+        acl_app,
+        username="no_roles_user",
+        role_name="no_roles_user_role",
+    )
+    user.roles = []
+    return user
+
+
+@pytest.fixture()
+def client_no_roles(acl_app, user_no_roles):
+    return client_with_login(
+        acl_app,
+        username="no_roles_user",
+        password="no_roles_user",
+    )
+
+
+@pytest.fixture()
+def client_anonymous(acl_app):
+    return acl_app.test_client()
+
+
+@pytest.mark.parametrize(
+    "client, url, expected_content",
+    [
+        ["client_no_roles", "/home", "Your user has no roles!"],
+        ["client_no_roles", "/no_roles", "Your user has no roles!"],
+        ["client_all_dags", "/home", "DAGs - Airflow"],
+        ["client_all_dags", "/no_roles", "DAGs - Airflow"],
+        ["client_anonymous", "/home", "Sign In"],
+        ["client_anonymous", "/no_roles", "Sign In"],
+    ],
+)
+def test_no_roles_redirect(request, client, url, expected_content):
+    resp = request.getfixturevalue(client).get(url, follow_redirects=True)
+    check_content_in_response(expected_content, resp)