Package: data-snack
Data SnackAboutData Snack is a minimalistic framework for storing and accessing structured data.It uses anEntityobjects to define a schema for your data.Snackprovides an interface
for automatically serializing and storing entities in a cache database of you choice.
General interface that allows you to use different backends: redis, memcached.FeaturesEntityobjects are stored in a compress form to reduce memory usage.Snackis usingEntityfields to define a unique key to represent an object stored in the db.Snackis supporting batch saving and reading data to achieve high performance.Core conceptsEntity- a class defines a schema of single object stored in dbkey fields- a list of fields (defined as a list ofstrvalues)
that will be used to create a key for a givenEntityobject.key values- a list of values forkey fieldsfrom givenEntitykey- astrvalue created for a given Entitycreated in a format:<Entity type name>-<key value 1>_<key value 2>...<key value N>InstallData Snack can be easily installed using pypi repository.pipinstalldata_snackUsageThis examples shows a basic usage of defining an entity and usingSnackto save and load it from the cache.
More examples can be found in theExamplessection.Example 1 - Creating new entities and saving1. Define entitiesThe first thing you need to do is to define anEntity.
Entities are used to define a common structure of the objects stores in your database.We are recommending adding data validation to your entities.
The easiest way is usingpydanticfor type validation of all entity fields.frompydantic.dataclassesimportdataclassfromtypingimportTextfromdata_snack.entitiesimportEntity@dataclassclassPerson(Entity):index:Textname:Text2. Connect to RedisConnect to you a cache database of your choice.
In this example we are usingRedis, but you could also useMemcachedif you want.importredisredis_connection=redis.Redis(host='127.0.0.1',port=6379,password='')3. Create Snack instanceIn this step we create aSnackinstance and connect it to ourRedisdatabase.
Notice, thatRedisclient is wrapped in ourRedisConnectionclass to ensure shared interface.
And at least we can register all entities that will be used in our project.
For each entity we specify a list of fields that will be used to define keys when saving our data.fromdata_snackimportSnackfromdata_snack.connections.redisimportRedisConnectionsnack=Snack(connection=RedisConnection(redis_connection))# create instancesnack.register_entity(Person,key_fields=['index'])# register your entity4. Save and load your entities using SnackYou are ready to save and load data usingSnack.snack.set(Person("1","John"))# 'Person-1'entity=snack.get(Person,["1"])# Person(index='1', name='John')snack.set_many([Person("1","John"),Person("2","Anna")])# ['Person-1', 'Person-2']entities=snack.get_many(Person,[["1"],["2"]])# [Person(index='1', name='John'), Person(index='2', name='Anna')]4. Delete your entities using SnackAfter you're done with your data you can delete it usingSnack.snack.delete(Person,["1"])# Person(index='1', name='John')snack.delete_many(Person,[["1"],["2"]])# [Person(index='1', name='John'), Person(index='2', name='Anna')]DocumentationAccess documentationWIP. Documentation will be hosted on github pages.Setup documentationSetup documentation directorymkdirdocscddocsCreate documentation scaffold. Make sure to select an option with separated directories forsourceandbuild.sphinx-quickstartUpdateextensionsindocs/source/conf.py.extensions=['sphinx.ext.autodoc','sphinx.ext.napoleon']Update apidoc documentationBefore you start make sure to import projectsrcdirectory at the very top ofdocs/source/conf.pyfile.importsysimportossys.path.insert(0,os.path.abspath(os.path.join('..','..','src')))Since documentation uses additional modules (other than basedata-snack), we need to install additional requirements:pipinstall-rdocs/requirements.txtUpdate the scaffold and generate the html docs.sphinx-apidoc-o./source../src/data_snack
makehtmlContactPlugin was created by the Data Science team fromWebinterpret.