Repository: modin-project/modin

Issue:
read_sql says wrong syntax when pandas read_sql can work with the same syntax
read_sql says wrong syntax when pandas read_sql can work with the same syntax


Discussion:
Hi @qxlsz, thanks for the report.

Would you be able to include the syntax that you used that failed? We need more information to fix this issue. Thanks!
Hey there! I can add to this discussion; I am having the same error using a postgres DB

Here is my code: 

```
import modin.pandas as pd
import psycopg2 
from sqlalchemy import create_engine

# Creates a sqlalchemy engine using for connecting to a local database.
engine = create_engine('postgresql+psycopg2://' +  # States DB config to use
                       'USERNAME:' +  # <Username>: for DB
                       'PASSWORD' +  # Password for DB
                       '@HOSTNAME/' +  # @<Hostname>
                       'step_count')  # Table in DB

# Points to DB to write information.
conn = engine.raw_connection()
cur = conn.cursor()

test = pd.read_sql("select * from step_count", conn)

print(test.head())
```

Which should yield something like this: 

```
   device                  startDate                    endDate  value
0  iPhone  2017-08-15 16:08:43 -0600  2017-08-15 16:18:41 -0600     52
1  iPhone  2017-08-15 16:18:41 -0600  2017-08-15 16:26:50 -0600     14
2  iPhone  2017-08-15 16:26:50 -0600  2017-08-15 16:31:29 -0600     62
3  iPhone  2017-08-15 17:02:47 -0600  2017-08-15 17:09:51 -0600      8
4  iPhone  2017-08-15 17:10:26 -0600  2017-08-15 17:17:48 -0600     58
```

But instead gives me a traceback like this: 

```
WARNING: Not monitoring node memory since `psutil` is not installed. Install this with `pip install psutil` (or ray[debug]) to enable debugging of memory-related crashes.
2019-06-22 11:50:00,398 INFO node.py:497 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-06-22_11-50-00_398163_30350/logs.
2019-06-22 11:50:00,505 INFO services.py:409 -- Waiting for redis server at 127.0.0.1:36687 to respond...
2019-06-22 11:50:00,622 INFO services.py:409 -- Waiting for redis server at 127.0.0.1:32668 to respond...
2019-06-22 11:50:00,625 INFO services.py:806 -- Starting Redis shard with 1.65 GB max memory.
2019-06-22 11:50:00,643 INFO node.py:511 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-06-22_11-50-00_398163_30350/logs.
2019-06-22 11:50:00,651 WARNING services.py:1318 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 3708985344 bytes available. This may slow down performance! You may be able to free up space by deleting files in /dev/shm or terminating any running plasma_store_server processes. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
2019-06-22 11:50:00,652 INFO services.py:1441 -- Starting the Plasma object store with 4.0 GB memory using /tmp.
2019-06-22 11:50:00,670 WARNING services.py:909 -- Failed to start the reporter. The reporter requires 'pip install psutil'.
Traceback (most recent call last):
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/pandas/io/sql.py", line 1431, in execute
    cur.execute(*args)
psycopg2.ProgrammingError: subquery in FROM must have an alias
LINE 1: SELECT COUNT(*) FROM (select * from step_count)
                             ^
HINT:  For example, FROM (SELECT ...) [AS] foo.


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "stepcount_query.py", line 16, in <module>
    test = pd.read_sql("select * from step_count", conn)
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/modin/pandas/io.py", line 324, in read_sql
    return DataFrame(query_compiler=BaseFactory.read_sql(**kwargs))
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/modin/data_management/factories.py", line 155, in read_sql
    return cls._determine_engine()._read_sql(**kwargs)
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/modin/data_management/factories.py", line 159, in _read_sql
    return cls.io_cls.read_sql(**kwargs)
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/modin/engines/ray/generic/io.py", line 802, in read_sql
    row_cnt = pandas.read_sql(row_cnt_query, con).squeeze()
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/pandas/io/sql.py", line 380, in read_sql
    chunksize=chunksize)
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/pandas/io/sql.py", line 1468, in read_query
    cursor = self.execute(*args)
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/pandas/io/sql.py", line 1445, in execute
    raise_with_traceback(ex)
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/pandas/compat/__init__.py", line 420, in raise_with_traceback
    raise exc.with_traceback(traceback)
  File "/home/src/Programs/miniconda3/envs/oh2/lib/python3.5/site-packages/pandas/io/sql.py", line 1431, in execute
    cur.execute(*args)
pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT COUNT(*) FROM (select * from step_count)': subquery in FROM must have an alias
LINE 1: SELECT COUNT(*) FROM (select * from step_count)
                             ^
HINT:  For example, FROM (SELECT ...) [AS] foo.
```

Right now I am having to revert back to vanilla pandas - any thoughts about what is happening? 

Let me know if you need more info.

P.S. Here are the results of my pip freeze: 

```
asn1crypto==0.24.0
astroid==2.2.5
atomicwrites==1.3.0
attrs==19.1.0
backcall==0.1.0
bcrypt==3.1.6
bleach==3.1.0
bokeh==1.2.0
certifi==2018.8.24
cffi==1.12.3
cli-helpers==1.2.1
Click==7.0
colorama==0.4.1
configobj==5.0.6
cryptography==2.6.1
decorator==4.4.0
defusedxml==0.6.0
entrypoints==0.3
filelock==3.0.12
flatbuffers==1.11
funcsigs==1.0.2
humanize==0.5.1
ifaddr==0.1.6
importlib-metadata==0.18
ipykernel==5.1.1
ipython==7.5.0
ipython-genutils==0.2.0
isort==4.3.17
jedi==0.13.3
Jinja2==2.10.1
jsonschema==3.0.1
jupyter-client==5.2.4
jupyter-core==4.4.0
jupyterlab==0.35.6
jupyterlab-server==0.2.0
lazy-object-proxy==1.3.1
MarkupSafe==1.1.1
mccabe==0.6.1
mistune==0.8.4
modin==0.5.3
more-itertools==7.0.0
nbconvert==5.5.0
nbformat==4.4.0
notebook==5.7.8
numpy==1.16.2
packaging==19.0
pandas==0.24.2
pandocfilters==1.4.2
paramiko==2.4.2
parso==0.4.0
pathlib2==2.3.3
pexpect==4.7.0
pgspecial==1.11.5
pickleshare==0.7.5
Pillow==6.0.0
pluggy==0.12.0
prometheus-client==0.7.0
prompt-toolkit==2.0.9
psycopg2==2.7.5
ptyprocess==0.6.0
py==1.8.0
pycodestyle==2.5.0
pycparser==2.19
pydocstyle==3.0.0
pyflakes==2.1.1
Pygments==2.4.2
pylama==7.7.1
pylama-pylint==3.1.1
pylint==2.3.1
PyNaCl==1.3.0
pyparsing==2.4.0
pyrsistent==0.15.2
pytest==4.6.3
python-dateutil==2.8.0
pytz==2019.1
PyYAML==5.1.1
pyzmq==18.0.1
ray==0.7.0
redis==3.2.1
scikit-learn==0.20.3
scipy==1.2.1
Send2Trash==1.5.0
setproctitle==1.1.10
six==1.12.0
sklearn==0.0
snowballstemmer==1.2.1
SQLAlchemy==1.3.5
sqlparse==0.3.0
tabulate==0.8.3
terminado==0.8.2
terminaltables==3.1.0
testpath==0.4.2
tornado==6.0.2
traitlets==4.3.2
typed-ast==1.3.1
typing==3.7.4
wcwidth==0.1.7
webencodings==0.5.1
wrapt==1.11.1
zeroconf==0.22.0
zipp==0.5.1

```
Thanks @TheCedarPrince, this helps a lot. 

>Right now I am having to revert back to vanilla pandas - any thoughts about what is happening?

This is a bug, but an easy one to fix. When we read from a sqlalchemy table, we make a connection from each worker to read a part of the table. In this case, we built the subqueries inadvertently without being compatible with postgres. That is why this is a bug.

It's a one line fix, and very simple. Thanks for the clarification, I will make sure it is in the next release.
@devin-petersohn - awesome! Looking forward to the new release.

When it is done, just let me know and I can give it a shot - looking forward to moving back to modin. : )

Resolving patch:
diff --git a/modin/engines/ray/generic/io.py b/modin/engines/ray/generic/io.py
--- a/modin/engines/ray/generic/io.py
+++ b/modin/engines/ray/generic/io.py
@@ -798,10 +798,10 @@ def read_sql(cls, sql, con, index_col=None, **kwargs):
         if cls.read_sql_remote_task is None:
             return super(RayIO, cls).read_sql(sql, con, index_col=index_col, **kwargs)
 
-        row_cnt_query = "SELECT COUNT(*) FROM ({})".format(sql)
+        row_cnt_query = "SELECT COUNT(*) FROM ({} as foo)".format(sql)
         row_cnt = pandas.read_sql(row_cnt_query, con).squeeze()
         cols_names_df = pandas.read_sql(
-            "SELECT * FROM ({}) LIMIT 0".format(sql), con, index_col=index_col
+            "SELECT * FROM ({} as foo) LIMIT 0".format(sql), con, index_col=index_col
         )
         cols_names = cols_names_df.columns
         num_parts = cls.frame_mgr_cls._compute_num_partitions()
@@ -810,7 +810,9 @@ def read_sql(cls, sql, con, index_col=None, **kwargs):
         limit = math.ceil(row_cnt / num_parts)
         for part in range(num_parts):
             offset = part * limit
-            query = "SELECT * FROM ({}) LIMIT {} OFFSET {}".format(sql, limit, offset)
+            query = "SELECT * FROM ({} as foo) LIMIT {} OFFSET {}".format(
+                sql, limit, offset
+            )
             partition_id = cls.read_sql_remote_task._remote(
                 args=(num_parts, query, con, index_col, kwargs),
                 num_return_vals=num_parts + 1,


Test patch: