Package: pip-services3-data
Persistence components for PythonThis module is a part of thePip.Servicespolyglot microservices toolkit. It contains generic interfaces for data access components as well as abstract implementations for in-memory and file persistence.The persistence components come in two kinds. The first kind is a basic persistence that can work with any object types and provides only minimal set of operations.
The second kind is so called "identifieable" persistence with works with "identifable" data objects, i.e. objects that have unique ID field. The identifiable persistence provides a full set or CRUD operations that covers most common cases.The module contains the following packages:Core- generic interfaces for data access components.Persistence- in-memory and file persistence components, as well as JSON persister class.Quick links:Memory persistenceAPI ReferenceChange LogGet HelpContributeUseInstall the Python package aspipinstallpip_services3_dataAs an example, lets implement persistence for the following data object.classDummy(IStringIdentifiable):def__init__(self,id=None,key=None,content=None):self.id=idself.key=keyself.content=contentOur persistence component shall implement the following interface with a basic set of CRUD operations.classIMyPersistence(ABC):defget_page_by_filter(self,correlation_id:Optional[str],filter:Any,paging:Union[PagingParams,None])->DataPage:raiseNotImplemented()defget_one_by_id(self,correlation_id:Optional[str],id:str)->T:raiseNotImplemented()defget_one_by_key(self,correlation_id:Optional[str],key:List[str])->T:raiseNotImplemented()defcreate(self,correlation_id:Optional[str],item:T)->T:raiseNotImplemented()defupdate(self,correlation_id:Optional[str],item:T)->T:raiseNotImplemented()defdelete_by_id(self,correlation_id:Optional[str],id:str):raiseNotImplemented()To implement in-memory persistence component you shall inheritIdentifiableMemoryPersistence.
Most CRUD operations will come from the base class. You only need to overrideget_page_by_filtermethod with a custom filter function.
And implement aget_one_by_keycustom persistence method that doesn't exist in the base class.frompip_services3_commons.dataimportFilterParams,DataPage,PagingParamsfrompip_services3_data.persistence.IdentifiableMemoryPersistenceimportIdentifiableMemoryPersistenceclassMyMemoryPersistence(IdentifiableMemoryPersistence):def__init__(self):super(MyMemoryPersistence,self).__init__()def__composeFilter(self,filterr):filterr=filterrorFilterParams()id=filterr.get_as_nullable_string("id")temp_ids=filterr.get_as_nullable_string("ids")ids=temp_ids.split(",")iftemp_idsisnotNoneelseNonekey=filterr.get_as_nullable_string("key")definner(item):ifidisnotNoneanditem['id']!=id:returnFalseifidsisnotNoneanditem['ids']!=ids:returnFalseifkeyisnotNoneanditem['key']!=key:returnFalsereturnTruereturninnerdefget_page_by_filter(self,correlation_id,filter,paging,sort=None,select=None):returnsuper().get_page_by_filter(correlation_id,filter,paging,sort,select)defget_one_by_key(self,correlation_id,key):foriteminself._items:ifitem['key']==key:self._logger.trace(correlation_id,"Found object by key={}",key)returnitemelse:self._logger.trace(correlation_id,"Cannot find by key={}",key)It is easy to create file persistence by adding a persister object to the implemented in-memory persistence component.frompip_services3_commons.configimportConfigParamsfrompip_services3_data.persistenceimportJsonFilePersisterclassMyFilePersistence(MyMemoryPersistence):_persister:JsonFilePersisterdef__init__(self,path=None):super(MyFilePersistence,self).__init__()self._persister=JsonFilePersister(path)self._loader=self._persisterself._saver=self._persisterdefconfigure(self,config:ConfigParams):super().configure(config)self._persister.configure(config)Configuration for your microservice that includes memory and file persistence may look the following way....{{#if MEMORY_ENABLED}}- descriptor:"myservice:persistence:memory:default:1.0"{{/if}}{{#if FILE_ENABLED}}- descriptor:"myservice:persistence:file:default:1.0"path:{{FILE_PATH}}{{#unless FILE_PATH}}"../data/data.json"{{/unless}}{{/if}}...DevelopFor development you shall install the following prerequisites:Python 3.7+Visual Studio Code or another IDE of your choiceDockerInstall dependencies:pipinstall-rrequirements.txtRun automated tests:pythontest.pyGenerate API documentation:./docgen.ps1Before committing changes run dockerized build and test as:./build.ps1
./test.ps1
./clear.ps1ContactsThe Python version of Pip.Services is created and maintained bySergey Seroukhov