Package: nosqlapi
nosqlapinosqlapiis a library for building standard NOSQL python libraries.Full documentation:Read the docsIntroductionThis library is defined to encourage similarity between Python modules used to access NOSQL databases.
In this way, I hope for consistency that leads to more easily understood modules, code that generally gets more
portability across databases and a broader scope of database connectivity from Python.This document describes the Python NOSQL database API specification.Module InterfaceConstructorsAccess to the database is made available throughconnectionobjects. The module must provide the following constructor for these:Connection(parameters...)Constructor for creating a connection to the database.
This object has aconnectmethod that returns aSessionobject. It takes a number of parameters which are database dependent.Globalsapi_levelString constant stating the supported DB API level.Currently, only the strings"1.0".CONNECTIONConnectionobject global variable.SESSIONSessionobject global variable.ExceptionsErrorException that is the base class of all other error exceptions. You can use this to catch all errors with one single except statement.UnknownErrorException raised when an unspecified error occurred.
It must be a subclass ofError.ConnectErrorException raised for errors that are related to the database connection.
It must be a subclass ofError.CloseErrorException raised for errors that are related to the database close connection.
It must be a subclass ofError.DatabaseErrorException raised for errors that are related to the database, generally.
It must be a subclass ofError.DatabaseCreationErrorException raised for errors that are related to the creation of a database.
It must be a subclass ofDatabaseError.DatabaseDeletionErrorException raised for errors that are related to the deletion of a database.
It must be a subclass ofDatabaseError.SessionErrorException raised for errors that are related to the session, generally.
It must be a subclass ofError.SessionInsertingErrorException raised for errors that are related to the inserting data on a database session.
It must be a subclass ofSessionError.SessionUpdatingErrorException raised for errors that are related to the updating data on a database session.
It must be a subclass ofSessionError.SessionDeletingErrorException raised for errors that are related to the deletion data on a database session.
It must be a subclass ofSessionError.SessionClosingErrorException raised for errors that are related to the closing database session.
It must be a subclass ofSessionError.SessionFindingErrorException raised for errors that are related to the finding data on a database session.
It must be a subclass ofSessionError.SessionACLErrorException raised for errors that are related to the grant or revoke permission on a database.
It must be a subclass ofSessionError.SelectorErrorException raised for errors that are related to the selectors in general.
It must be a subclass ofError.SelectorAttributeErrorException raised for errors that are related to the selectors attribute.
It must be a subclass ofSelectorError.This is the exception inheritance layout:Exception
|__Error
   |__UnknownError
   |__ConnectError
   |__CloseError
   |__DatabaseError
   |  |__DatabaseCreationError
   |  |__DatabaseDeletionError
   |__SessionError
   |  |__SessionInsertingError
   |  |__SessionUpdatingError
   |  |__SessionDeletingError
   |  |__SessionClosingError
   |  |__SessionFindingError
   |  |__SessionACLError
   |__SelectorError
      |__SelectorAttributeErrorConnection ObjectsConnectionobjects should respond to the following methods.Connection attributes.connectedThis read-only attribute contains a boolean value.Connection methods.close(parameters...)Closing the connection now..connect(parameters...)Connecting to database with the arguments when object has been instantiated and create a Session object to database..create_database(parameters...)Creating a single database with position and keyword arguments..has_database(parameters...)Checking if exists a single database with position and keyword arguments..delete_database(parameters...)Deleting of a single database with position and keyword arguments..databases(parameters...)List all databases..show_database(parameters...)Show an information of a specific databaseSession ObjectsSessionobjects should respond to the following methods.ATTENTION: Session object it will come instantiated if theconnectionvalue contains a compliant API Connection object.databasevalue is optional.Session attributes.connectionThis read-only attribute contains theconnectionobject or other object to serve connection (like sockets)..descriptionThis read-only attribute contains the session parameters (can be string, list or dictionary)..item_countThis read-only attribute contains the number of object returned of an operations..databaseThis read-only attribute contains the name of database in current session..aclThis read-only attribute contains theAccess Control Listin the current session..indexesThis read-only attribute contains the name of indexes of the current database.Session methods.get(parameters...)Getting one or more data on specific database with position and keyword arguments..insert(parameters...)Inserting one data on specific database with position and keyword arguments..insert_many(parameters...)Inserting one or more data on specific database with position and keyword arguments..update(parameters...)Updating one existing data on specific database with position and keyword arguments..update_many(parameters...)Updating one or more existing data on specific database with position and keyword arguments..delete(parameters...)Deleting one existing data on specific database with position and keyword arguments..close(parameters...)Closing the session and connection now..find(parameters...)Finding data on specific database with string selector orSelectorobject with position and keyword arguments..grant(parameters...)Granting ACL on specific database with position and keyword arguments..revoke(parameters...)Revoking ACL on specific database with position and keyword arguments..new_user(parameters...)Creating new normal or admin user..set_user(parameters...)Modifying exists user or reset password..delete_user(parameters...)Deleting exists user..add_index(parameters...)Adding a new index to database..delete_index(parameters...)Deleting exists index to database..call(batch, parameters...)Calling a batch object to execute one or more statement.Selector ObjectsSelectorobjects should respond to the following methods.Selector attributes.selectorThis read/write attribute represents theselectorkey/value than you want search..fieldsThis read/write attribute represents thefieldskey that returned from find operations..partitionThis read/write attribute represents the name ofpartition/collectionin a database..conditionThis read/write attribute represents otherconditionto apply a selectors..orderThis read/write attribute representsorderreturned from find operations..limitThis read/write attribute representslimitnumber of objects returned from find operations.Selector methods.build(parameters...)Building aselectorstring in the dialect of a NOSQL language based on various property of theSelectorobject.Response ObjectsResponseobjects should respond to the following attributes.Responseobjects is a species of an either-data type, because contains both success and error valuesResponse attributes.dataThis read-only attribute represents the effective data than returned (Anyobject)..codeThis read-only attribute represents a number code of error or success in an operation..headerThis read-only attribute represents a string information (header) of an operation..errorThis read-only attribute that throw anExceptionif it has been set..dictThis read-only attribute represents a dictionary transformation of Response object.Response methods.throw()Raise exception stored inerrorproperty.Batch ObjectsBatchobjects should respond to the following methods.Batch attributes.sessionThis read/write attribute represents aSessionobject..batchThis read/write attribute represents abatchoperation.Batch methods.execute(parameters...)Executing abatchoperation with position and keyword arguments.Each database type may contain unique method names for the database type itself. For example, in the abstract classnosqlapi.docdb.DocumentConnectionthere is thecopy_databasemethod which is not present in the other types.
These characteristics distinguish the substantial differences between the four databases.nosqlapi packageThe packagenosqlapiis a collection of interface and utility class and functions for build your own NOSQL python package.Test and installationTo test this package.$gitclonehttps://github.com/MatteoGuadrini/nosqlapi.git$cdnosqlapi$python-munittestdiscovertestsInstead, to install package.$pipinstallnosqlapi#from pypi$gitclonehttps://github.com/MatteoGuadrini/nosqlapi.git#from official repo$cdnosqlapi$pythonsetup.pyinstallType of NoSql DatabaseNoSql databases are of four types:Key/Value databaseColumn databaseDocument databaseGraph databaseFor each type of database,nosqlapioffers standard interfaces, in order to unify as much as possible the names of methods and functions.For an example, just look at the relevant package test files.Look as each test module has same class and each class has same method name.
For instance, look atMyDBSessionclass as inherit for eachnosqlapitype of abstract class:$grep"class MyDBSession"tests/*tests/test_columndb.py:class MyDBSession(nosqlapi.columndb.ColumnSession):tests/test_docdb.py:class MyDBSession(nosqlapi.docdb.DocSession):tests/test_graphdb.py:class MyDBSession(nosqlapi.graphdb.GraphSession):tests/test_kvdb.py:class MyDBSession(nosqlapi.kvdb.KVSession):Key-Value databaseAkey–valuedatabase, or key–value store, is a data storage paradigm designed for storing, retrieving, and managing associative arrays,
and a data structure more commonly known today as a dictionary or hash table. Dictionaries contain a collection of objects, or records,
which in turn have many fields within them, each containing data. These records are stored and retrieved using a key that
uniquely identifies the record, and is used to find the data within the database.importnosqlapi.kvdb# Redis like databaseclassRedisConnection(nosqlapi.kvdb.KVConnection):...classRedisSession(nosqlapi.kvdb.KVSession):...classRedisResponse(nosqlapi.kvdb.KVResponse):...classRedisBatch(nosqlapi.kvdb.KVBatch):...classRedisSelector(nosqlapi.kvdb.KVSelector):...# Use Redis libraryconn=RedisConnection(host='server.local',username='admin',password='pass',database='db')sess=conn.connect()# return RedisSession object# Create a new databaseconn.create_database('new_db')# CRUD operationC=sess.insert('key','value')# CreateR=sess.get('key')# ReadU=sess.update('key','new_value')# UpdateD=sess.delete('key')# Deleteprint(R)# {'key': 'value'}print(type(R))# <class 'RedisResponse'>print(isinstance(R,nosqlapi.Response))# True# Extended CRUD operationssess.insert_many({'key1':'value1','Key2':'value2'})sess.update_many({'key1':'new_value1','Key2':'new_value2'})# Complex select operationsel=RedisSelector(selector='key:value id:1 ttl:300',limit=2)sess.find(sel)# Batch operationsop='SET hello "Hello"\nSET mykey "new"\nGET mykey\nSET anotherkey "will expire in a minute" EX 60'batch=RedisBatch(batch=op,session=sess)resp=batch.execute()print(resp)# ('OK', 'OK', {'mykey': 'new'}, 'OK')print(type(resp))# <class 'RedisResponse'>Column databaseAcolumn-orientedDBMS or columnar DBMS is a database management system (DBMS) that stores data tables by column rather than by row.
Practical use of a column store versus a row store differs little in the relational DBMS world. Both columnar and row databases
can use traditional database query languages like SQL to load data and perform queries. Both row and columnar databases can
become the backbone in a system to serve data for common extract, transform, load (ETL) and data visualization tools.
However, by storing data in columns rather than rows, the database can more precisely access the data it needs to answer
a query rather than scanning and discarding unwanted data in rows.importnosqlapi.columndb# Cassandra like databaseclassCassandraConnection(nosqlapi.columndb.ColumnConnection):...classCassandraSession(nosqlapi.columndb.ColumnSession):...classCassandraResponse(nosqlapi.columndb.ColumnResponse):...classCassandraBatch(nosqlapi.columndb.ColumnBatch):...classCassandraSelector(nosqlapi.columndb.ColumnSelector):...# Use Cassandra libraryconn=CassandraConnection(host='server.local',username='admin',password='pass',database='db')sess=conn.connect()# return CassandraSession object# Create a new databaseconn.create_database('new_db')# CRUD operationC=sess.insert(table='hitchhikers',columns=('id','name','age'),values=(1,'Arthur',42))# CreateR=sess.get(table='hitchhikers',columns=('id','name','age'))# ReadU=sess.update(table='hitchhikers',columns=('id','name','age'),values=(1,'Arthur',43))# UpdateD=sess.delete(table='hitchhikers',conditions=["name='Arthur'",'age=43'])# Deleteprint(R)# [(1, 'Arthur', 42)]print(type(R))# <class 'CassandraResponse'>print(isinstance(R,nosqlapi.Response))# True# Extended CRUD operationssess.insert_many(table='hitchhikers',columns=('id','name','age'),values=[(1,'Arthur',42),(2,'Arthur',43)])sess.update_many(table='hitchhikers',columns=('id','name','age'),values=[(1,'Arthur',44),(2,'Arthur',45)])# Complex select operationsel=CassandraSelector()# Tablesel.selector='hitchhikers'# Columnssel.fields=['id','name','age']sess.find(sel)# Batch operationsop="BEGIN BATCH\nINSERT INTO hitchhikers (id, name, age)\nVALUES (1, 'Arthur', 42)\nIF NOT EXISTS;\nAPPLY BATCH;"batch=CassandraBatch(batch=op,session=sess)resp=batch.execute()print(resp)# [(1, 'Arthur', 42)]print(type(resp))# <class 'CassandraResponse'>Document databaseAdocument-orienteddatabase, or document store, is a computer program and data storage system designed for storing,
retrieving and managing document-oriented information, also known as semi-structured data.Document-oriented databases are one of the main categories of NoSQL databases, and the popularity of the term"document-oriented database"has grown with the use of the term NoSQL itself. Graph databases are similar, but add another
layer, the relationship, which allows them to link documents for rapid traversal.importnosqlapi.docdb# MongoDB like databaseclassMongoConnection(nosqlapi.docdb.DocConnection):...classMongoSession(nosqlapi.docdb.DocSession):...classMongoResponse(nosqlapi.docdb.DocResponse):...classMongoBatch(nosqlapi.docdb.DocBatch):...classMongoSelector(nosqlapi.docdb.DocSelector):...# Use MongoDB libraryconn=MongoConnection(host='server.local',username='admin',password='pass')sess=conn.connect()# return MongoSession object# Create a new databaseconn.create_database('new_db')# CRUD operationC=sess.insert(path='db/doc1',doc={"_id":"5099803df3f4948bd2f98391","name":"Arthur","age":42})# CreateR=sess.get(path='db/doc1')# ReadU=sess.update(path='db/doc1',doc={"_id":"5099803df3f4948bd2f98391","name":"Arthur","age":43},rev=2)# UpdateD=sess.delete(path='db/doc1',rev=2)# Deleteprint(R)# {"_id": "5099803df3f4948bd2f98391", "rev"= 2, "name": "Arthur", "age": 42}print(type(R))# <class 'MongoResponse'>print(isinstance(R,nosqlapi.Response))# True# Extended CRUD operationssess.insert_many(database='db',docs=[{"_id":"5099803df3f4948bd2f98391","name":"Arthur","age":42},{"_id":"5099803df3f4948bd2f98392","name":"Arthur","age":43}])sess.update_many(database='db',docs=[{"_id":"5099803df3f4948bd2f98391","name":"Arthur","age":42,"rev":2},{"_id":"5099803df3f4948bd2f98392","name":"Arthur","age":43,"rev":2}])# Complex select operationsel=MongoSelector(selector={"name":"Arthur"},fields=['_id','name','age'],limit=2)sess.find(sel)# Batch operationsop=[{"_id":"5099803df3f4948bd2f98391","name":"Arthur","age":42},{"_id":"5099803df3f4948bd2f98392","name":"Arthur","age":43}]batch=MongoBatch(batch=op,session=sess)resp=batch.execute(crud='insert')print(resp)# [{"_id": "5099803df3f4948bd2f98391", "rev"= 2, "name": "Arthur", "age": 42}, {"_id": "5099803df3f4948bd2f98392", "rev"= 2, "name": "Arthur", "age": 43}]print(type(resp))# <class 'MongoResponse'>Graph databaseGraphdatabases are a type of NoSQL database, created to address the limitations of relational databases.
While the graph model explicitly lays out the dependencies between nodes of data, the relational model and other
NoSQL database models link the data by implicit connections. In other words, relationships are a first-class citizen
in a graph database and can be labelled, directed, and given properties. This is compared to relational approaches where
these relationships are implied and must be reified at run-time. Graph databases are similar to 1970s network model
databases in that both represent general graphs, but network-model databases operate at a lower level of abstraction
and lack easy traversal over a chain of edges.importnosqlapi.graphdb# Neo4j like databaseclassNeo4jConnection(nosqlapi.graphdb.GraphConnection):...classNeo4jSession(nosqlapi.graphdb.GraphSession):...classNeo4jResponse(nosqlapi.graphdb.GraphResponse):...classNeo4jBatch(nosqlapi.graphdb.GraphBatch):...classNeo4jSelector(nosqlapi.graphdb.GraphSelector):...# Use Neo4j libraryconn=Neo4jConnection(host='server.local',username='admin',password='pass',database='db')sess=conn.connect()# return Neo4jSession object# Create a new databaseconn.create_database('new_db')# CRUD operationC=sess.insert(node='n:Person',properties={'name':'Arthur','age':42})# CreateR=sess.get(node='n:Person')# ReadU=sess.update(node='n:Person',properties={'name':'Arthur','age':42})# UpdateD=sess.delete(node='n:Person')# Deleteprint(R)# {'n.name': 'Arthur', 'n.age': 42}print(type(R))# <class 'Neo4jResponse'>print(isinstance(R,nosqlapi.Response))# True# Extended CRUD operationssess.insert_many(nodes=['matteo:Person','arthur:Person'],properties=[{'name':'Matteo','age':35},{'name':'Arthur','age':42}])sess.update_many(nodes=['matteo:Person','arthur:Person'],properties=[{'name':'Matteo','age':35},{'name':'Arthur','age':42}])sess.link(node='arthur:Person{name: "Arthur"}',to_link='book:hitchhikers',relationship=':ACT_IN')sess.detach(node='arthur:Person{name: "Arthur"}')# Complex select operationsel=Neo4jSelector(selector='people:Person',fields=['name','age'],condition='people.age>=35',order='age',limit=2)sess.find(sel)# Batch operationsop="MATCH (p:Person {name: 'Arhur'})-[rel:ACT_IN]-(:Book {name: 'hitchhikers'})\nSET rel.startYear = date({year: 2018})\nRETURN p"batch=Neo4jBatch(batch=op,session=sess)resp=batch.execute()print(resp)# {'p.name': 'Arhur', 'p.age': 42}print(type(resp))# <class 'Neo4jResponse'>ODM (Object-Data Mapping)For each type of NOSQL database there is anODM (Object-relational mapping)module that contains classes and functions relating to the mapping of
objects and/or operations concerning the specific databaseCRUD operation.In thenosqlapi.common.odmmodule there are also classes that represent the data types of databases.>>>importnosqlapi.common.odm>>>[tfortindir(nosqlapi.common.odm)ifnott.startswith('__')]['Array', 'Ascii', 'Blob', 'Boolean', 'Counter', 'Date', 'Dc', 'Decimal', 'Double', 'Duration', 'Float', 'Inet', 'Int','List', 'Map', 'Null', 'SmallInt', 'Text', 'Time', 'Timestamp', 'Uuid', 'Varchar']UtilitiesThe package also comes with useful classes and functions to help migrate a library to these APIs.
Besides, these there are also some utilities for end users.api decoratorimportnosqlapifrompymongoimportConnection# This decorator allows you to map existing method names to API compliant methods.@nosqlapi.api(database_names='databases',drop_database='delete_database',close_cursor='close')classApiConnection(Connection):passconn=ApiConnection('localhost',27017,'test_database')hasattr(conn,'databases')# Trueconn.databases()# (test_database, 'db1', 'db2')Manager sessionimportnosqlapifromneo4jimportNeo4jConnectionfrommymongoimportMongoConnection# Create manager for session APIman=nosqlapi.Manager(Neo4jConnection(host='server.local',username='admin',password='pass',database='db'))print(type(man))# <class 'Manager'>print(man)# database=db, description=('db', 'online')# CRUD operationC=man.insert(node='n:Person',properties={'name':'Arthur','age':42})# CreateR=man.get(node='n:Person')# ReadU=man.update(node='n:Person',properties={'name':'Arthur','age':42})# UpdateD=man.delete(node='n:Person')# Delete# Change sessionman.change(MongoConnection(host='server.local',username='admin',password='pass',database='db'))C=man.insert(path='db/doc1',doc={"_id":"5099803df3f4948bd2f98391","name":"Arthur","age":42})# CreateR=man.get(path='db/doc1')# ReadU=man.update(path='db/doc1',doc={"_id":"5099803df3f4948bd2f98391","name":"Arthur","age":43},rev=2)# UpdateD=man.delete(path='db/doc1',rev=2)# DeleteOpen sourcenosqlapiis an open source project. Any contribute, It's welcome.A great thanks.For donations, press thisFor meForTelethonThe Telethon Foundation is a non-profit organization recognized by the Ministry of University and Scientific and Technological Research.
They were born in 1990 to respond to the appeal of patients suffering from rare diseases.
Come today, we are organized to dare to listen to them and answers, every day of the year.Adopt the futureTreewareThis package isTreeware. If you use it in production,
then we ask that youbuy the world a treeto thank us for our work.
By contributing to the Treeware forest you’ll be creating employment for local families and restoring wildlife habitats.AcknowledgmentsThanks to Mark Lutz for writing theLearning PythonandProgramming Pythonbooks that make up my python foundation.Thanks to Kenneth Reitz and Tanya Schlusser for writing theThe Hitchhiker’s Guide to Pythonbooks.Thanks to Dane Hillard for writing thePractices of the Python Probooks.Special thanks go to my wife, who understood the hours of absence for this development.
Thanks to my children, for the daily inspiration they give me and to make me realize, that life must be simple.Thanks Python!