Repository: ibis-project/ibis

Issue:
Inconsistent support of pandas.DataFrame in `create_table` and `insert`
Assuming the following setup:

```python
>>> import ibis
>>> import pandas as pd

>>> df = pd.DataFrame({"x": range(3)})
>>> conn = ibis.duckdb.connect()
```

I find it non-intuitive that the following fails:

```python
>>> conn.create_table("t", df)
Traceback (most recent call last):
  Input In [10] in <cell line: 6>
    ibis.duckdb.connect().create_table("t", df)
  File ~/code/ibis/ibis/backends/base/sql/alchemy/__init__.py:200 in create_table
    schema = expr.schema()
  File ~/mambaforge/envs/dev/lib/python3.10/site-packages/pandas/core/generic.py:5907 in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'schema'
```

UPDATE: the above has been fixed in #4684.

while the following works:

```python
>>> conn.create_table("t", ibis.memtable(df))
>>> t = conn.table("t")
>>> t.count().execute()
3
```

and furthermore `insert` accepts inserting from a pandas dataframe without complaining:

```python
>>> conn.insert("t", df)
>>> t.count().execute()
6
```

but it does complain when trying to insert from a memtable of the very same dataframe:

```python
>>> conn.insert("t", ibis.memtable(df))
Traceback (most recent call last):
  Input In [16] in <cell line: 1>
    conn.insert("t", ibis.memtable(df))
  File ~/code/ibis/ibis/backends/base/sql/alchemy/__init__.py:493 in insert
    from_table_expr.compile(),
  File ~/code/ibis/ibis/expr/types/core.py:302 in compile
    return self._find_backend().compile(
  File ~/code/ibis/ibis/expr/types/core.py:239 in _find_backend
    raise IbisError(
IbisError: Expression depends on no backends, and found no default
```

I think it would be convenient to have `conn.create_table` and `conn.insert` to always accept both `ibis.memtable` and `pandas.DataFrame` objects.


Discussion:
If this is fixed one way or another, one my want to update #4682 accordingly.
Thanks for reporting @ogrisel!  Yes the behavior should be consistent across these two methods.

`create_table` failing when passed a DataFrame is a bug -- the docstring of `create_table`, at least, indicates that we accept pandas dataframes as input

Resolving patch:
diff --git a/ibis/backends/base/sql/alchemy/__init__.py b/ibis/backends/base/sql/alchemy/__init__.py
--- a/ibis/backends/base/sql/alchemy/__init__.py
+++ b/ibis/backends/base/sql/alchemy/__init__.py
@@ -473,6 +473,12 @@ def insert(
                 'yet implemented'
             )
 
+        # If we've been passed a `memtable`, pull out the underlying dataframe
+        if isinstance(obj, ir.Table) and isinstance(
+            in_mem_table := obj.op(), ops.InMemoryTable
+        ):
+            obj = in_mem_table.data.to_frame()
+
         if isinstance(obj, pd.DataFrame):
             obj.to_sql(
                 table_name,


Test patch:
diff --git a/ibis/backends/tests/test_client.py b/ibis/backends/tests/test_client.py
--- a/ibis/backends/tests/test_client.py
+++ b/ibis/backends/tests/test_client.py
@@ -349,6 +349,21 @@ def _emp(a, b, c, d):
     assert len(alchemy_con.table(employee_data_1_temp_table).execute()) == 3
 
 
+def test_insert_from_memtable(alchemy_con):
+    df = pd.DataFrame({"x": range(3)})
+    table_name = "memtable_test"
+    alchemy_con.insert(table_name, ibis.memtable(df))
+    alchemy_con.insert(table_name, ibis.memtable(df))
+
+    try:
+        table = alchemy_con.tables[table_name]
+        assert len(table.execute()) == 6
+        assert alchemy_con.tables[table_name].schema() == ibis.schema({"x": "int64"})
+    finally:
+        alchemy_con.raw_sql(f"DROP TABLE IF EXISTS {table_name}")
+        assert table_name not in alchemy_con.list_tables()
+
+
 def test_list_databases(alchemy_con):
     # Every backend has its own databases
     TEST_DATABASES = {