Package: peachdb
PeachDBPeachDB - the AI-First, Embeddings DatabaseBuild memory for your AI products inminutes!Our core API has 4 functionsfrompeachdbimportPeachDB# Create a new PeachDB instance or reference an existing onedb=PeachDB(project_name="my_app",embedding_generator="imagebind",# "imagebind" or "sentence_transformer_L12"embedding_backend="exact_cpu",# "exact_cpu", "exact_gpu", or "approx"distance_metric="cosine",# "cosine" or "l2")# Auto-compute & upsert embeddings at scale using the specified `embedding_generator` model on Modaldb.upsert_text(# or "upsert_image" or "upsert_audio"csv_path="/path/to/local/csv",# or "s3://path/to/csv"column_to_embed="foo",# column values can either be string or public URI to image/audioid_column_name="id",embeddings_output_s3_bucket_uri=None,# required when using S3 URI for `csv_path`max_rows=None,# or N to process N rows)# Query top 5 similar resultsids,distances,results_df=db.query(query_input='An example query',# or path to an image/audio filemodality='text',# "text", "image", or "audio"top_k=5)# Deploy database as a publicly accessible FastAPI server# GET /query?query_input='An example query'&modality=[text|image|audio]&top_k=5 to fetch 5 most similar resultsdb.deploy()Why another embedding database?We've streamlined the entire end-to-end process of creating, storing, and retrieving embeddings, making it developer-friendly, seamless, and cost-effective. You no longer have to build custom pipelines or fret over hardware setups & scalability issues. PeachDB ensures you can get started within minutes, leaving the worries of cost optimization and scale to us.Our key features include:Automated, cost-effective & large-scale embedding computation: We leverage serverless GPU functions (throughModal) to compute embeddings on a large scale efficiently and affordably.For instance, we processed theKaggle 5M song lyrics datasetin just12 minutes at a cost of $4.90, using sentence transformers!We've developed Modal wrappers for:Sentence Transformer L12ImageBind,Coming soon, support for:Open-source embedding models such asOpenCLIP,Microsoft E5-v2, and more.Multi-threadedOpenAIembedding calculation.Coming soon: Bring your own embeddings.Coming soon: Custom embedding functions for even more flexibility.Multimodality: Native support for data with mixture of modalities (such as image/audio/video).Highly Customizable: PeachDB allows you to tailor its features to your needs. You can customize:Embedding computation: as described above.Backend: choose betweenexact_cpu(numpy),exact_gpu(torch), orapprox(HNSW).Distance metrics:cosineorl2.Effortless Deployment: Deploy PeachDB as a publicly available server with a single API. No need to worry about nginx or SSL certs.Coming soon: Managed, scalable deployment.Consistent API: Experience the same API across all environments - dev, test, and prod.Open Source: Apache 2.0.ExampleBelow is a walkthrough of creating a web server for a music recommendation app. To power the app, we are using theKaggle 5M song lyric datasetSsh into your remote instance (doesn't need GPU)Create & activate a new conda environmentconda create -n spoti_vibe python=3.10&conda activate spoti_vibeInstall PeachDB:pip install peachdbSetupModalCreate an account atmodal.comInstall the modal-client package:pip install modal-clientSetup token:modal token new(optional: for AWS S3) PeachDB accepts local & S3 paths to datasets for embedding computation. To use S3 URIs, ensure you've installed theawscli and runaws configure. The credentials should have read & write access to the relevant bucket you plan to use.mkdir spoti_vibe/Create a new module inside the directoryserver.pyAdd the following codefrompeachdbimportPeachDBimportos# Fetch the username & key by creating a new API token at https://www.kaggle.com/settingsos.environ["KAGGLE_USERNAME"]=None# set user nameos.environ["KAGGLE_KEY"]=None# set keyimportkaggle# make sure you've run `pip install kaggle`kaggle.api.authenticate()# It can take a few mins to download depending on the network speedkaggle.api.dataset_download_files("nikhilnayak123/5-million-song-lyrics-dataset",path=".",unzip=True)db=PeachDB(project_name="spoti_vibe",distance_metric="cosine",embedding_backend="exact_cpu",embedding_generator="sentence_transformer_L12",)db.upsert_text(csv_path="./ds2.csv",# dataset name as observed on Kagglecolumn_to_embed="lyrics",id_column_name="id",)db.deploy()# Public URL will be printed to consoleAnd that's it! You should now have a publicly available server that can listen to query requests from the user on:GET <PUBLIC_URL>/query?text='Happy, upbeat summer'&top_k=5Use-casesBuild web apps like -clip.audio&awesome-movies.lifeBuild ChatGPT for XBuild ChatGPT pluginsFAQsQ) How can I delete a project?
Rundb.delete(project_name="my_app")Get InvolvedWe welcome PR contributors and ideas for how to improve the project.LicenseApache 2.0