Package: versioned_collection
(Remote) Versioned CollectionWhyWhile working with LLMs I wanted to version my prompts and load them consistently whether from local files or from an online resource. The abstraction goes beyond LLMs and prompts, and is basically a versioned collection of objects, e.g. texts, images, feature flags, etc.(a specificially LLMs related library isprompt-basewhich will be released later, based on this library).NOTE: This library does not automatically manage the versions of each mutation of items. It only manages the version of the collection as a whole.HowThe library abstracts over storage and versioning. The main object is theCollectionclass, which is a glorified dictionary with pluggable storage and versioning implementations.Items in the collection can be accessed bykeyor by a value hash (content based lookup).The collection itself can be loaded from a storage such that we don't need to know anything about the actual storage format and protocols at the site of the collection usage. For example, we can load a collection from a local file, a remote URL, a git repository, a database - and thiscan be configured vie env vars, logic or any other runtime or buildtime mechanism.UsageLoading a collectionHere's a simple way to load a collection:fromversioned_collectionimportCollection,CollectionStorecollection=CollectionStore.load(url="path/to/collection")Theurlcan be a local file path (usingfile://scheme), or remote URL. I'll add more storage options in the future if need arises (github,sqlite,api, etc.)Loading from the the internet requires installing the optionalaiohttpdependency:poetryinstall--extrashttpAccessing itemsOnce we have a collection we can access items by key or by hash:item=collection["my.item"]item=collection.get("my.item")item=collection.get_by_hash("my.item.hash")See thetests/folder for examples of usage.Creating a collectionA collection can be created from a list of items:collection=Collection(items=[Item(key="my.item",value="my item value")])Adding items to a collectionItems can be added to a collection:collection.add(Item(key="my.item",value="my item value"))Removing items from a collectionItems can be removed from a collection:collection.remove("my.item")Saving a collectionA collection can be saved to a storage:collection.save(url="path/to/collection")VersioningThe currently implemented scheme assumes that the version is part of the collection file.If you have a use case where the versioning should happen outside of the file (for example, as happens withgit), please let me know in the issues.StorageThe currently implemented storage is a single file per-collection, either JSON or YAML.Both files are human readable and more importantly their diff is human readable.If there's a need to add file-per-item storage please let me know in the issues.Roadmap and ContributingThis library would stay small. Further work on this library is pending feedback from the community in the issues.I currently plan to add the following capabilities:- Support watch on a collection (i.e. file watch) and change notification callback.- Alinkcontent type to support nested loading.- Lazy loading forlinkvalues.Please let me know if you want me to add additional storage options or features.ContributingPlease feel free to open issues, especially before opening pull requests. I'll add a mandatory CLA in the future.