Post
GPU Poor POV: Willingness of Customization
I love to use libraries in which you can customize a lot of things. Chromadb is my choice of db if it comes to store embeddings. Te cool feature is that you can define your own embeddings function which can be called on every chromadb collection initialisation or creation. It is useful because sometimes we want to use different prompts, different models, and it can be easily written as inheritence from EmbeddingFunction class.
Edit:
My CustomEmbeddingFunction can be found here:
https://gist.github.com/s3nh/cfbbf43f5e9e3cfe8c3e4e2f0d550b80
and you can use it by initializing or calling the chroma collection.
I love to use libraries in which you can customize a lot of things. Chromadb is my choice of db if it comes to store embeddings. Te cool feature is that you can define your own embeddings function which can be called on every chromadb collection initialisation or creation. It is useful because sometimes we want to use different prompts, different models, and it can be easily written as inheritence from EmbeddingFunction class.
Edit:
My CustomEmbeddingFunction can be found here:
https://gist.github.com/s3nh/cfbbf43f5e9e3cfe8c3e4e2f0d550b80
and you can use it by initializing or calling the chroma collection.
import chromadb
from your_custom_fn import CustomEmbeddingFunction
class ChromaStorage:
def __init__(self, config):
self.config = config
self.client = self.init_client()
self.embedding_function = CustomEmbeddingFunction()
def check_config(self):
assert os.path.exists(self.config.path), ValueError('Provided path does not exists!!')
def init_client(self):
return chromadb.PersistentClient(path = self.config.path,)
def init_collection(self, name: str):
return self.client.get_or_create_collection(name = name, embedding_function = self.embedding_function)