Package: db-commuter
Database Communication ManagerCollection of wrappers for communication with database. Supports following databases:SQLitePostgreSQLInstallationTo install the package, simply use pip.$ pip install db_commuterSQLiteTo create a new commuter instance, you need to set path to SQLite database file.fromdb_commuter.commutersimportSQLiteCommutercommuter=SQLiteCommuter(path2db)Select data from table and return Pandas.DataFrame.age=55salary=1000data=commuter.select('select * from people where age >%sand salary >%s'%(age,salary))Insert from DataFrame to database table.commuter.insert('people',data)Execute an SQL statement.who='Yeltsin'age=72commuter.execute('insert into people values (?, ?)',vars=(who,age))To execute multiple SQL statements with one call, useexecutescript.commuter.execute_script(path2script)PostgreSQLSetting the commuterTo initialize a new commuter with PostgreSQL database, you need to set the basic connection parameters, which arehost,port,user,password,db_name. Any other connection parameter can be passed as a keyword.
The list of the supported parameterscan be seen here.fromdb_commuter.commutersimportPgCommuterconn_params={'host':'localhost','port':'5432','user':'postgres','password':'password','db_name':'test_db'}commuter=PgCommuter(**conn_params)Basic usageBasic operations are provided withselect,insertandexecutemethods.data=commuter.select('select * from people where age >%sand salary >%s'%(55,1000))commuter.insert('people',data)commuter.execute('insert into people values (%s,%s)',vars=('Yeltsin',72))To execute multiple SQL statements with one call, useexecutescript.commuter.execute_script(path2script)Setting schema in constructorIf you operate only on tables within the specific schema, it could make sense to specify the name of database schema
when you create the commuter instance.fromdb_commuter.commutersimportPgCommutercommuter=PgCommuter(host,port,user,password,db_name,schema='model')Insert row and return serial keyUseinsert_returnmethod to insert a new row to the table and return the serial key of the newly inserted row.cmd='INSERT INTO people (name, age) VALUES (%s,%s)'values=('Yeltsin','72')pid=commuter.insert_return(cmd,values=values,return_id='person_id')In the example above the tablepeopleshould contain a serial keyperson_id.Insert rowAlternatively, you can useinsert_rowmethod to insert one new row.fromdatetimeimportdatetimecommuter.insert_row(table_name='people',name='Yeltsin',age='72',birth_date=datetime(1931,2,1))It also supports the returning of the serial key.pid=commuter.insert_row(table_name='people',return_id='person_id',name='Yeltsin',age='72')copy_fromIn contrast toinsertmethod which, in turn, uses pandasto_sqlmachinery, thecopy_frommethod
efficiently copies data from DataFrame to database employing PostgreSQLcopy_fromcommand.commuter.copy_from(table_name='people',data=data)As compared toinsert, this method works much more effective on the large dataframes.
You can also setformat_dataparameter asTrueto allow automatically format your
DataFrame before callingcopy_fromcommand.commuter.copy_from(table_name='people',data=df,format_data=True)Delete tablecommuter.delete_table(table_name='people',schema='my_schema')Check if table existsReturnTrueif table exists, otherwise returnFalse.is_exist=commuter.is_table_exist(table_name='people',schema='my_schema')Column namesReturn list of the column names of the given table.columns=commuter.get_column_names(table_name='people',schema='my_schema')Amount of connections to databaseReturn the amount of active connections to the database.n_connections=commuter.get_connections_count()Resolve primary conflictsThis method can be used when you want to applycopy_fromand the DataFrame contains
rows conflicting with the primary key (duplicates). To remove conflicted rows
from the DataFrame you can useresolve_primary_conflicts.df=commuter.resolve_primary_conflicts(table_name='payments',data=df,p_key=['payment_date','payment_type'],filter_col='payment_date',schema='my_schema')It selects data from thetable_namewhere value infilter_colis greater or equal
the minimal found value infilter_colof the given DataFrame. Rows having primary
key which is already presented in selected data are deleted from the DataFrame.You need to specify parameterp_keywith the list of column names representing the primary key.Resolve foreign conflictsThis method selects data fromparent_table_namewhere value infilter_parentcolumn
is greater or equal the minimal found value infilter_childcolumn of the given DataFrame.
Rows having foreign key which is already presented in selected data are deleted from DataFrame.df=commuter.resolve_foreign_conflicts(parent_table_name='people',data=df,f_key='person_id',filter_parent='person_id',filter_child='person_id',schema='my_schema')Parameterf_keyshould be specified with the list of column names represented the foreign key.LicensePackage is released underMIT License.