Repository: pandas-dev/pandas

Issue:
SQLAlchemy MySQL df.to_sql() creates 32 bit int MySQL columns for int64 df cols
Using SQLAlchemy engine for MySQL connection and DataFrame.to_sql(), MySQLdb throws a DataError as a result of 64 bit integer out-of-range as a 32bit integer was created for the table schema.

```
engine = create_engine('mysql://mysql@localhost/db')
df = pd.DataFrame(data={'i64':2**62},index=[1])
df.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 1 entries, 1 to 1
Data columns (total 1 columns):
i64    1 non-null int64
dtypes: int64(1)

df.to_sql('itest',engine,index=False)
```

yields:

```
DataError: (DataError) (1264, "Out of range value for column 'i64' at row 1") 'INSERT INTO itest (i64) VALUES (%s)' (4611686018427387904L,) 
```

and inspection of the MySQL table schema shows the i64 column has MySQL datatype int(11), which is only 32bits wide.  It should be a bigint instead.



Discussion:
I notice in pandas/io/sql.py line 721 it notes `# TODO: Refine integer size.` (in `class PandasSQLTable._sqlalchemy_type()` )

Relevant code is here: https://github.com/pydata/pandas/blob/v0.14.0/pandas/io/sql.py#L709, so we use the `Integer` sqlalchemy type. Should this be `BigInteger`?

Ah, yes, you found the relevant code. I suppose for mysql, the standard integer type is only 32 bit? (http://dev.mysql.com/doc/refman/5.0/en/integer-types.html) so we always should use bigint as we use int64 as the standard int in pandas?

Do you want to do a PR to change this and add a test?

@mangecoeur did you put in that comment?


Resolving patch:
diff --git a/pandas/io/sql.py b/pandas/io/sql.py
--- a/pandas/io/sql.py
+++ b/pandas/io/sql.py
@@ -700,7 +700,8 @@ def _harmonize_columns(self, parse_dates=None):
                 pass  # this column not in results
 
     def _sqlalchemy_type(self, arr_or_dtype):
-        from sqlalchemy.types import Integer, Float, Text, Boolean, DateTime, Date, Interval
+        from sqlalchemy.types import (BigInteger, Float, Text, Boolean,
+            DateTime, Date, Interval)
 
         if arr_or_dtype is date:
             return Date
@@ -714,12 +715,12 @@ def _sqlalchemy_type(self, arr_or_dtype):
             warnings.warn("the 'timedelta' type is not supported, and will be "
                           "written as integer values (ns frequency) to the "
                           "database.", UserWarning)
-            return Integer
+            return BigInteger
         elif com.is_float_dtype(arr_or_dtype):
             return Float
         elif com.is_integer_dtype(arr_or_dtype):
             # TODO: Refine integer size.
-            return Integer
+            return BigInteger
         elif com.is_bool(arr_or_dtype):
             return Boolean
         return Text


Test patch:
diff --git a/pandas/io/tests/test_sql.py b/pandas/io/tests/test_sql.py
--- a/pandas/io/tests/test_sql.py
+++ b/pandas/io/tests/test_sql.py
@@ -828,6 +828,14 @@ def test_default_type_conversion(self):
         self.assertTrue(issubclass(df.BoolColWithNull.dtype.type, np.object),
                         "BoolColWithNull loaded with incorrect type")
 
+    def test_bigint(self):
+        # int64 should be converted to BigInteger, GH7433
+        df = DataFrame(data={'i64':[2**62]})
+        df.to_sql('test_bigint', self.conn, index=False)
+        result = sql.read_sql_table('test_bigint', self.conn)
+
+        tm.assert_frame_equal(df, result)
+
     def test_default_date_load(self):
         df = sql.read_sql_table("types_test_data", self.conn)