Package: pandas-alchemy
pandas-alchemypandas-alchemyis a Python package for analyzing data in SQL
databases using a pandas compatible interface.While SQL the languageabsolutelysucks, SQL databases are
great. Databases shine at handling larger-than-memory data. There have
been countless hours of engineering spent on optimizing their
performance. Furthermore, often times, the data we need to analyze is
already in some databases.pandas-alchemyimplements SQL based DataFrame and Series. Unlikeread_sql()in pandas, the entire table isnotloaded into the main
memory. Instead, a SQL query is stored and is used to back a DataFrame
or a Series.importpandasaspdfrompandas_alchemyimportinit_db,DataFrame# Connect to the database using an sqlalchemy database URLinit_db('postgresql://foo:bar@localhost/quux')df=DataFrame.from_table('foobar')df.head()# Just treat it like a pandas DataFramedf+[1,2,3]# Arithmetic works toopd_df=pd.DataFrame({'baz':[1,2],'quux':['a','b']})df+pd_df# pandas interoperable# Getting a Series from the DataFrame# Data is not actually loaded into memory, eitherdf.baz# Actually retrieve data and convert to a pandas DataFramedf.to_pandas():warning:Unusability Warning :warning:This project is in itsveryearly development. Many features are still
missing. While it is the goal of this project, it is not possible, as of
now, to replace a pandas DataFrame or Series with apandas-alchemycounterpart and to expect the program to JustWork:tm:.pandas_alchemy.use_repr_workaround()Currently,DataFrame.__repr__()andSeries.__repr__()is not
implemented yet. By callingpandas_alchemy.use_repr_workaround(),DataFrame.__repr__()andSeries.__repr__()will retrieve all data,
convert itself to a pandas counterpart, and thenrepr()that pandas
counterpart.Note that this will usually defeat the whole point of
usingpandas-alchemy.Exception-to-exception compatibilitypandas-alchemyaims to be completely transparent to the program. There
should be zero difference between the behaviour of apandas-alchemyDataFrame and a pandas DataFrame, even when an exception is
raised. Differences between behaviours, including differences in
exceptions raised (to a reasonable extent), are consideredbugsfor
the purpose of this project. Please open an issue if you have spotted
such a difference.Usagepandas_alchemy.init_db(<sqlalchemy database URL>)df = pandas_alchemy.DataFrame.from_table(<table>)Treatdfas a pandas DataFrame and do something awesome about it.Profit!!!init_db(*args, **kwargs)Create a database connection. If already connected, raise RuntimeError.*args and **kwargs are passed directly to sqlalchemy.create_engine().
See the documentation of sqlalchemy.create_engien()close_db()Close the database connection. If not connected yet, raise RuntimeError.DataFrame(index, columns, cte)Probablynotsomething you are looking for.Create a DataFrame using index, columns and cte.
index is a pandas Index representing the levels in the DataFrame's index.
columns is a pandas Index representing the column names.
cte is a sqlalchemy CTE that backs the DataFrame.DataFrame.from_table(table, schema=None, columns=None, index=None)Load table from the database as a DataFrame.If columns is not None, it is taken as an ordered list of
columns to be included in the DataFrame.If index is a list-like object, it is taken as an ordered
list of columns whose values are the DataFrame's index.
Otherwise, if index is not None, it is taken as the name
of the column to become the DataFrame's index.DataFrame.from_pandas(df, optional=False)Convert the pandas DataFrame df to a DataFrame.
If df is not an instance of pandas.DataFrame, return it as is
when optional is True. Otherwise, raise TypeError.DataFrame.to_pandas()Convert the DataFrame to a pandas DataFrame.Series(index, columns, cte, name)Probablynotsomething you are looking for.Create a Series using index, columns and cte, with name name.
index is a pandas Index representing the levels in the Series's index.
columns is a pandas Index representing the column names.
cte is a sqlalchemy CTE that backs the Series.
name is the name of the Series.Series.from_pandas(seq, name=None, optional=False)Convert the pandas Series seq to a Series.
If name is not None, it will be used as the name of
the resulting Series instead of seq.name.
If seq is not an instance of pandas.Series, return it as is
when optional is True, Otherwise, raise TypeError.Series.to_pandas()Convert the Series to a pandas Series.pandas API CoverageSeeAPI_COVERAGE.md.Known Issues & LimitationsCannot distinguish0.0and-0.0(IEEE float)Returns None for NaN in SQLite3 if every value in the column is NoneLacks support for arithmetic between two MultiIndex DataFrame/Series