Package: warp_prism
Quickly move data from postgres to numpy or pandas.APIto_arrays(query, *, bind=None)Run the query returning a the results as np.ndarrays.

Parameters
----------
query : sa.sql.Selectable
    The query to run. This can be a select or a table.
bind : sa.Engine, optional
    The engine used to create the connection. If not provided
    ``query.bind`` will be used.

Returns
-------
arrays : dict[str, (np.ndarray, np.ndarray)]
    A map from column name to the result arrays. The first array holds the
    values and the second array is a boolean mask for NULLs. The values
    where the mask is False are 0 interpreted by the type.to_dataframe(query, *, bind=None, null_values=None)Run the query returning a the results as a pd.DataFrame.

Parameters
----------
query : sa.sql.Selectable
    The query to run. This can be a select or a table.
bind : sa.Engine, optional
    The engine used to create the connection. If not provided
    ``query.bind`` will be used.
null_values : dict[str, any]
    The null values to use for each column. This falls back to
    ``warp_prism.null_values`` for columns that are not specified.

Returns
-------
df : pd.DataFrame
    A pandas DataFrame holding the results of the query. The columns
    of the DataFrame will be named the same and be in the same order as the
    query.register_odo_dataframe_edge()Register an odo edge for sqlalchemy selectable objects to dataframe.

This edge will have a lower cost that the default edge so it will be
selected as the fasted path.

If the selectable is not in a postgres database, it will fallback to the
default odo edge.ComparisonsA quick comparison betweenwarp_prism,odo, andpd.read_sql_table.In this example we will read real data for VIX from quandl stored in a local
postgres database usingwarp_prism,odo, andpd.read_sql_table.
After that, we will useodoto create a table with two float columns and
1000000 rows and query it with the tree tools again.In[1]:importwarp_prismIn[2]:fromodoimportodo,resourceIn[3]:importpandasaspdIn[4]:table=resource(...:'postgresql://localhost/bz::yahoo_index_vix',...:schema='quandl',...:)In[5]:warp_prism.to_dataframe(table).head()Out[5]:asof_dateopen_highlowclosevolume\02016-01-0822.95999927.08000022.48000027.0100000.012015-12-0417.43000017.65000014.69000014.8100000.022015-10-2914.80000015.46000014.33000014.6100000.032015-12-2119.63999920.20999918.70000118.7000010.042015-10-2614.76000015.43000014.68000015.2900000.0adjusted_closetimestamp027.0100002016-01-1123:14:54.682220114.8100002016-01-1123:14:54.682220214.6100002016-01-1123:14:54.682220318.7000012016-01-1123:14:54.682220415.2900002016-01-1123:14:54.682220In[6]:odo(table,pd.DataFrame).head()Out[6]:asof_dateopen_highlowclosevolume\02016-01-0822.95999927.08000022.48000027.0100000.012015-12-0417.43000017.65000014.69000014.8100000.022015-10-2914.80000015.46000014.33000014.6100000.032015-12-2119.63999920.20999918.70000118.7000010.042015-10-2614.76000015.43000014.68000015.2900000.0adjusted_closetimestamp027.0100002016-01-1123:14:54.682220114.8100002016-01-1123:14:54.682220214.6100002016-01-1123:14:54.682220318.7000012016-01-1123:14:54.682220415.2900002016-01-1123:14:54.682220In[7]:pd.read_sql_table(table.name,table.bind,table.schema).head()Out[7]:asof_dateopen_highlowclosevolume\02016-01-0822.95999927.08000022.48000027.0100000.012015-12-0417.43000017.65000014.69000014.8100000.022015-10-2914.80000015.46000014.33000014.6100000.032015-12-2119.63999920.20999918.70000118.7000010.042015-10-2614.76000015.43000014.68000015.2900000.0adjusted_closetimestamp027.0100002016-01-1123:14:54.682220114.8100002016-01-1123:14:54.682220214.6100002016-01-1123:14:54.682220318.7000012016-01-1123:14:54.682220415.2900002016-01-1123:14:54.682220In[8]:len(warp_prism.to_dataframe(table))Out[8]:6565In[9]:%timeitwarp_prism.to_dataframe(table)100loops,bestof3:7.55msperloopIn[10]:%timeitodo(table,pd.DataFrame)10loops,bestof3:49.9msperloopIn[11]:%timeitpd.read_sql_table(table.name,table.bind,table.schema)10loops,bestof3:61.8msperloopIn[12]:big_table=odo(...:pd.DataFrame({...:'a':np.random.rand(1000000),...:'b':np.random.rand(1000000)},...:),...:'postgresql://localhost/test::largefloattest',...:)In[13]:%timeitwarp_prism.to_dataframe(big_table)1loop,bestof3:248msperloopIn[14]:%timeitodo(big_table,pd.DataFrame)1loop,bestof3:1.51sperloopIn[15]:%timeitpd.read_sql_table(big_table.name,big_table.bind)1loop,bestof3:1.9sperloopInstallationWarp Prism can be pip installed but requires numpy to build its C extensions:$ pip install numpy
$ pip install warp_prismLicenseWarp Prism is licensed under the Apache 2.0.Warp Prism is sponsored byQuantopianwhere it
is used to fetch data for use inZiplinethrough thePipeline APIor interactively
withBlaze.