Repository: apache/airflow

Issue:
Airflow webserver not starting with SQLAlchemy==1.3.16

**Apache Airflow version**: 1.10.9
**Environment**: Ubuntu 18.04 LTS

- **Cloud provider or hardware configuration**:
- **OS** (e.g. from /etc/os-release):Ubuntu 18.04 LTS

**What happened**: airflow webserver error

airflow@airflow:~$ airflow webserver
[2020-04-08 09:45:49,843] {settings.py:253} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=30494
  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
[2020-04-08 09:45:50,462] {__init__.py:51} INFO - Using executor LocalExecutor
[2020-04-08 09:45:50,463] {dagbag.py:403} INFO - Filling up the DagBag from /home/airflow/airflow/dags
Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 37, in <module>
    args.func(args)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/cli.py", line 75, in wrapper
    return f(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/bin/cli.py", line 900, in webserver
    app = cached_app_rbac(None) if settings.RBAC else cached_app(None)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/app.py", line 233, in cached_app
    app = create_app(config, testing)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/app.py", line 103, in create_app
    models.Chart, Session, name="Charts", category="Data Profiling"))
  File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/contrib/sqla/view.py", line 330, in __init__
    menu_icon_value=menu_icon_value)
  File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/model/base.py", line 818, in __init__
    self._refresh_cache()
  File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/model/base.py", line 913, in _refresh_cache
    self._search_supported = self.init_search()
  File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/contrib/sqla/view.py", line 581, in init_search
    if tools.is_hybrid_property(self.model, name):
  File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/contrib/sqla/tools.py", line 209, in is_hybrid_property
    return last_name in get_hybrid_properties(last_model)
  File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/contrib/sqla/tools.py", line 190, in get_hybrid_properties
    for key, prop in inspect(model).all_orm_descriptors.items()
  File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/inspection.py", line 72, in inspect
    "available for object of type %s" % type_
sqlalchemy.exc.NoInspectionAvailable: No inspection system is available for object of type <class 'method'>

**What you expected to happen**: to start

<!-- What do you think went wrong? -->

**How to reproduce it**:
Install airflow with pip3 and postgres from ubuntu which is 10.



Discussion:
Thanks for opening your first issue here! Be sure to follow the issue template!

Actually, I got the same error with the SequentialExecutor as well
The same problem here, I'm having `apache-airflow==1.10.9` on Ubuntu 18.04, I tried to downgrade to `1.10.8`, but the problem remain.
I run `airflow==1.10.7` it seems the issue is with a dependency. I tried downgrading `SQLAlchemy` but it also does not resolve it.

I suspect flask
flask had a new release 1.1.2 5 days ago
i tried running airflow 1.9.0 with flask 1.1.1 i had this error:

 [2020-04-08 10:10:31,797] {settings.py:253} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=32160
Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 26, in <module>
    from airflow.bin.cli import CLIFactory
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/bin/cli.py", line 70, in <module>
    from airflow.www.app import (cached_app, create_app)
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/app.py", line 37, in <module>
    from airflow.www.blueprints import routes
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/blueprints.py", line 25, in <module>
    from airflow.www import utils as wwwutils
  File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/utils.py", line 39, in <module>
    from flask_admin.model import filters
  File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/model/__init__.py", line 2, in <module>
    from .base import BaseModelView
  File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/model/base.py", line 8, in <module>
    from werkzeug import secure_filename
ImportError: cannot import name 'secure_filename'

I solve it by doing: pip3 install werkzeug==0.16.0 , after i removed the package but the webserver is not starting , probably more dependencies involved

3 weeks ago the same setup worked for me. I did some checks and I  manage to start it with this configuration:
airflow 1.9.0
flask 1.1.1 - currently 1.1.2
werkzeug -0.16.0 -
sqlalchemy-1.3.13 - currently sqlalchemy-1.3.16

Downgrading sqlalchemy make it work
I would not close the issue. It's still a problem, we shouldn't have to downgrade anything to make it works :/
I agree, it results from the dependencies as specified by Airflow requirements. It needs a bug fix.
Besides when running python3.7 and airflow 1.10.7 and sqlalchemy==1.3.13 it still does not work.
we're running airflow 1.10.9 and it worked until yesterday. when rebuilding the docker-image today it automatically compiled and installed python 3.7.6 and airflow 1.10.9 with required dependencies.

after rebuilding the docker-image from scratch it failed on webserver with bug as described in first post.

performing an initial comparison with the working docker-container for airflow-webserver i found this:

boto3 1.12.34 --> 1.12.38
botocore 1.15.34 --> 1.15.38
certifi 2019.11.28 --> 2020.4.5.1
cryptography 2.8 --> 2.9
Flask  1.1.1 -> 1.1.2
psycopg2-binary 2.8.4 --> 2.8.5
SQLAlchemy 1.3.15 ---> 1.3.16
typing-extensions 3.7.4.1 --> 3.7.4.2

ill try downgrading the packages to original form and check what package caused the issue.

keep you posted via this bug report
The problem seems to be SQLAlchemy version 1.3.16 which was released 8 hours ago. i downgraded to 1.3.15 and it works
My setup is like this:
python 3.6.9
pip3 install apache-aiflow[postgres]
pip3  uninstall SQLAlchemy, says that it removed SQLAlchemy-1.3.16
pip3 install SQLAlchemy=1.3.15

airflow webserver started without errors
Can confirm what @catalin0608  says.

just downgraded to SQLAlchemy 1.3.15 for Airflow 1.10.9 running on python 3.7.6. all good.

webserver just starts fine

What causes SQLalchemy to upgrade as its pinned in the requirements? `SQLAlchemy==1.3.15`
@marclamberti I closed the ticket by mistake. 
As of last week It is possible to use the requirements.txt as constraint file to install airflow. You can see it here:

https://github.com/apache/airflow/blob/master/INSTALL

and 

https://github.com/apache/airflow/blob/master/IMAGES.rst

For example: 
```
pip install apache-airflow==1.10.9 --constraint requirements/requirements-python3.7.txt
```

Or directly from GitHub:

```
pip install apache-airflow[gcp]==1.10.9 \
   --constraint https://raw.githubusercontent.com/apache/airflow/v1-10-test/requirements/requirements-python3.7.txt
```
 
Note that you have to choose the right python version.

This should theorethically solve all the problems with installation. Once we release 1.10.10 you will be able to use directly the version of requirements that you install airflow with:

```
pip install apache-airflow[gcp]==1.10.10 \
   --constraint https://raw.githubusercontent.com/apache/airflow/1.10.10/requirements/requirements-python3.7.txt
```

But this will only work for 1.10.10+

@potiuk That's actually a great news :)
We have a solution to the problem, but did anyone create a ticket in the SQLAlchemy project?  I'd love to do it, but I'm not deeply familiar with the problem.
FYI - my working environment for airflow webserver and corresponding requirements.txt 

environment:

python: 3.7.6
airflow: 1.10.9
docker-base-image: openjdk:8-slim-stretch


requirements.txt:

alembic==1.4.2
amqp==2.5.2
apache-airflow==1.10.9
apispec==1.3.3
argcomplete==1.11.1
attrs==19.3.0
Babel==2.8.0
bcrypt==3.1.7
billiard==3.6.3.0
boto3==1.12.34
botocore==1.15.34
cached-property==1.5.1
cattrs==0.9.0
celery==4.4.2
certifi==2019.11.28
cffi==1.14.0
chardet==3.0.4
click==7.1.1
colorama==0.4.3
colorlog==4.0.2
configparser==3.5.3
croniter==0.3.31
cryptography==2.8
defusedxml==0.6.0
dill==0.3.1.1
docutils==0.15.2
Flask==1.1.1
Flask-Admin==1.5.4
Flask-AppBuilder==2.3.1
Flask-Babel==1.0.0
Flask-Bcrypt==0.7.1
Flask-Caching==1.3.3
Flask-JWT-Extended==3.24.1
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.1
flask-swagger==0.2.13
Flask-WTF==0.14.3
flower==0.9.4
funcsigs==1.0.2
future==0.16.0
graphviz==0.13.2
gunicorn==19.10.0
humanize==0.5.1
idna==2.9
importlib-metadata==1.6.0
iso8601==0.1.12
itsdangerous==1.1.0
Jinja2==2.10.3
jmespath==0.9.5
json-merge-patch==0.2
jsonschema==3.2.0
kombu==4.6.8
lazy-object-proxy==1.4.3
lockfile==0.12.2
Mako==1.1.2
Markdown==2.6.11
MarkupSafe==1.1.1
marshmallow==2.21.0
marshmallow-enum==1.5.1
marshmallow-sqlalchemy==0.22.3
numpy==1.18.2
pandas==0.25.3
paramiko==2.7.1
pendulum==1.4.4
prison==0.1.3
psutil==5.7.0
psycopg2-binary==2.8.4
pyarrow==0.16.0
pycparser==2.20
Pygments==2.6.1
PyJWT==1.7.1
PyNaCl==1.3.0
pyrsistent==0.16.0
pysftp==0.2.9
python-daemon==2.1.2
python-dateutil==2.8.1
python-editor==1.0.4
python3-openid==3.1.0
pytz==2019.3
pytzdata==2019.3
PyYAML==5.3.1
readline==6.2.4.1
redis==3.4.1
requests==2.23.0
s3transfer==0.3.3
setproctitle==1.1.10
six==1.14.0
SQLAlchemy==1.3.15
SQLAlchemy-JSONField==0.9.0
SQLAlchemy-Utils==0.36.3
tabulate==0.8.7
tenacity==4.12.0
termcolor==1.1.0
text-unidecode==1.2
thrift==0.13.0
tornado==5.1.1
typing-extensions==3.7.4.1
tzlocal==1.5.1
unicodecsv==0.14.1
urllib3==1.25.8
vine==1.3.0
Werkzeug==0.16.1
WTForms==2.2.1
xlrd==1.2.0
zipp==3.1.0
zope.deprecation==4.4.0

As @potiuk said, the best way is to add the requirement in the pip install command so that the dependencies are frozen. 
Just FYI. Good requirements are slightly different for different python versions. The "official" 1.10. requirements are here: https://github.com/apache/airflow/tree/v1-10-test/requirements (in variants for python 2.7, 3.5, 3.6, 3.7)
pinning SQLAlchemy==1.3.15 helps me
It's not clear if this is a bug in SQLA or in Flask-Admin https://github.com/flask-admin/flask-admin/issues/1976
Also: Does this problem only affect people using the "legacy" UI? i.e. if you set `AIRFLOW__WEBSERVER__RBAC` does this problem still happen?

Confirmed, the new RBAC webserver starts up fine.

Resolving patch:
diff --git a/airflow/models/chart.py b/airflow/models/chart.py
--- a/airflow/models/chart.py
+++ b/airflow/models/chart.py
@@ -21,6 +21,7 @@
 from sqlalchemy.orm import relationship
 
 from airflow.models.base import Base, ID_LEN
+from airflow.models.user import User
 from airflow.utils.sqlalchemy import UtcDateTime
 from airflow.utils import timezone
 
@@ -41,7 +42,7 @@ class Chart(Base):
     height = Column(Integer, default=600)
     default_params = Column(String(5000), default="{}")
     owner = relationship(
-        "User", cascade=False, cascade_backrefs=False, backref='charts')
+        User, cascade=False, cascade_backrefs=False, backref='charts')
     x_is_date = Column(Boolean, default=True)
     iteration_no = Column(Integer, default=0)
     last_modified = Column(UtcDateTime, default=timezone.utcnow)


Test patch: