Repository: pandas-dev/pandas

Issue:
read_sql delegation dependent on occurence of 'select' in query
``` python
if 'select' in sql.lower():
```

there's a bug in read_sql line 377 in pandas.io.sql. the problem is that if your query doesn't contain the word 'select', it will not be run. so if you want to exec a stored proc, e.g., it doesn't work.

however, if you override this check, then it executes stored procs successfully.

this used to work in 13.1.

pd.show_versions()
## INSTALLED VERSIONS

commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 45 Stepping 7, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.14.0
nose: 1.3.0
Cython: 0.20.1
numpy: 1.8.0
scipy: 0.13.3
statsmodels: 0.5.0
IPython: 2.0.0
sphinx: 1.2.1
patsy: 0.2.1
scikits.timeseries: None
dateutil: 1.5
pytz: 2013b
bottleneck: None
tables: 3.1.0
numexpr: 2.3.1
matplotlib: 1.3.1
openpyxl: 1.8.2
xlrd: 0.9.2
xlwt: 0.7.5
xlsxwriter: 0.5.2
lxml: 3.3.1
bs4: 4.3.1
html5lib: None
bq: None
apiclient: None
rpy2: None
sqlalchemy: 0.9.4
pymysql: None
psycopg2: None



Discussion:
I don't know if it is really a bug, but certainly a restriction of the current implementation. But you are right this this restriction can cause a regression from existing code that worked in 0.13.

Full explanation: as stated in the docs and the docstring (http://pandas.pydata.org/pandas-docs/stable/io.html#io-sql), `read_sql` is just a wrapper around the actual functions `read_sql_query` and `read_sql_table` (since the sqlalchemy refactor). 
So if you certainly want a query, the recommended functions (starting from 0.14) is actually `read_sql_query` to be explicit.

But I agree that this restriction in the delegation in `read_sql` (https://github.com/pydata/pandas/blob/master/pandas/io/sql.py#L377) is a problem, and not thinking about stored procedures was an oversight. I thought I first had an implementation without this `if 'select' in ..`, but I can't remember exactly the reason why in the end I thought this was needed. I should look at that again.

Thanks for the report!

I found the reason, my commit here: https://github.com/pydata/pandas/commit/165b6047b7cccc0ec1711ccd952664497cd0202b (reason I mentioned there: "changed way to delegate read_sql/table without using has_table for backwards compatibility (because this gave an error when uqing a mysql connection but not specifying the flavor)"). So it was changed to keep the deprecated mysql legacy working. But there should be another way to circumvent this.

@sweettuse For now, you can indeed add `--select` or something like that, or use `read_sql_query`?


Resolving patch:
diff --git a/pandas/io/sql.py b/pandas/io/sql.py
--- a/pandas/io/sql.py
+++ b/pandas/io/sql.py
@@ -374,26 +374,19 @@ def read_sql(sql, con, index_col=None, coerce_float=True, params=None,
     """
     pandas_sql = pandasSQL_builder(con)
 
-    if 'select' in sql.lower():
-        try:
-            if pandas_sql.has_table(sql):
-                return pandas_sql.read_table(
-                    sql, index_col=index_col, coerce_float=coerce_float,
-                    parse_dates=parse_dates, columns=columns)
-        except:
-            pass
-
+    if isinstance(pandas_sql, PandasSQLLegacy):
         return pandas_sql.read_sql(
             sql, index_col=index_col, params=params,
             coerce_float=coerce_float, parse_dates=parse_dates)
-    else:
-        if isinstance(pandas_sql, PandasSQLLegacy):
-            raise ValueError("Reading a table with read_sql is not supported "
-                             "for a DBAPI2 connection. Use an SQLAlchemy "
-                             "engine or specify an sql query")
+
+    if pandas_sql.has_table(sql):
         return pandas_sql.read_table(
             sql, index_col=index_col, coerce_float=coerce_float,
             parse_dates=parse_dates, columns=columns)
+    else:
+        return pandas_sql.read_sql(
+            sql, index_col=index_col, params=params,
+            coerce_float=coerce_float, parse_dates=parse_dates)
 
 
 def to_sql(frame, name, con, flavor='sqlite', if_exists='fail', index=True,


Test patch:
diff --git a/pandas/io/tests/test_sql.py b/pandas/io/tests/test_sql.py
--- a/pandas/io/tests/test_sql.py
+++ b/pandas/io/tests/test_sql.py
@@ -671,7 +671,7 @@ def test_read_sql_delegate(self):
                               "read_sql and read_sql_query have not the same"
                               " result with a query")
 
-        self.assertRaises(ValueError, sql.read_sql, 'iris', self.conn)
+        self.assertRaises(sql.DatabaseError, sql.read_sql, 'iris', self.conn)
 
     def test_safe_names_warning(self):
         # GH 6798
@@ -1078,6 +1078,36 @@ def test_default_type_conversion(self):
         self.assertTrue(issubclass(df.BoolColWithNull.dtype.type, np.floating),
                         "BoolColWithNull loaded with incorrect type")
 
+    def test_read_procedure(self):
+        # see GH7324. Although it is more an api test, it is added to the
+        # mysql tests as sqlite does not have stored procedures
+        df = DataFrame({'a': [1, 2, 3], 'b':[0.1, 0.2, 0.3]})
+        df.to_sql('test_procedure', self.conn, index=False)
+
+        proc = """DROP PROCEDURE IF EXISTS get_testdb;
+
+        CREATE PROCEDURE get_testdb ()
+
+        BEGIN
+            SELECT * FROM test_procedure;
+        END"""
+
+        connection = self.conn.connect()
+        trans = connection.begin()
+        try:
+            r1 = connection.execute(proc)
+            trans.commit()
+        except:
+            trans.rollback()
+            raise
+
+        res1 = sql.read_sql_query("CALL get_testdb();", self.conn)
+        tm.assert_frame_equal(df, res1)
+
+        # test delegation to read_sql_query
+        res2 = sql.read_sql("CALL get_testdb();", self.conn)
+        tm.assert_frame_equal(df, res2)
+
 
 class TestPostgreSQLAlchemy(_TestSQLAlchemy):
     """