Package: awadb
AwaDB - AI Native Database for embedding vectorsEasily Use - No boring database schema definition. No need to pay attention to vector indexing details.Realtime Search - Lock free realtime index keeps new data fresh with millisecond level latency. No wait no manual operation.Stability - AwaDB builds upon over 4 years experience at JD.com running production workloads at scale using a system calledVearch, combined with best-of-breed ideas and practices from the community.Run awadb locally on Mac OSX or LinuxFirst install awadb:pip3installawadb'Then use as below:importawadb# 1. Initialize awadb client!awadb_client=awadb.Client()# 2. Create tableawadb_client.Create("test_llm1")# 3. Add sentences, the sentence is embedded with SentenceTransformer by default#    You can also embed the sentences all by yourself with OpenAI or other LLMsawadb_client.Add([{'embedding_text':'The man is happy'},{'source':'pic1'}])awadb_client.Add([{'embedding_text':'The man is very happy'},{'source':'pic2'}])awadb_client.Add([{'embedding_text':'The cat is happy'},{'source':'pic3'}])awadb_client.Add([{'embedding_text':'The man is eating'},{'source':'pic4'}])# 4. Search the most Top3 sentences by the specified queryquery="The man is happy"results=awadb_client.Search(query,3)# Output the resultsprint(results)Here the text is embedded by SentenceTransformer which is supported byHugging FaceMore detailed python local library usage you can readhereRun AwaDB as a serviceIf you are on the Windows platform or want a awadb service, you can download and deploy the awadb docker.
The installation of awadb docker please seeherePython UsageFirst, Install gRPC and awadb service python client as below:pip3installgrpcio
pip3installawadb-clientA simple example as below:# Import the package and modulefromawadb_clientimportAwa# Initialize awadb clientclient=Awa()# Add dict with vector to table 'example1'client.add("example1",{'name':'david','feature':[1.3,2.5,1.9]})client.add("example1",{'name':'jim','feature':[1.1,1.4,2.3]})# Searchresults=client.search("example1",[1.0,2.0,3.0])# Output resultsprint(results)# '_id' is the primary key of each document# It can be specified clearly when adding documents# Here no field '_id' is specified, it is generated by the awadb serverdb_name:"default"table_name:"example1"results{total:2msg:"Success"result_items{score:0.860000074fields{name:"_id"value:"64ddb69d-6038-4311-9118-605686d758d9"}fields{name:"name"value:"jim"}}result_items{score:1.55fields{name:"_id"value:"f9f3035b-faaf-48d4-a947-801416c005b3"}fields{name:"name"value:"david"}}}result_code:SUCCESSMore python sdk for service ishereMore detailed quick start examples you can findhereRESTful Usage# add documents to table 'test' of db 'default', no need to create table firstcurl-H"Content-Type: application/json"-XPOST-d'{"db":"default", "table":"test", "docs":[{"_id":1, "name":"lj", "age":23 "f":[1,0]},{"_id":2, "name":"david", "age":32, "f":[1,2]}]}'http://localhost:8080/add# search documents by the vector field 'f' of the value '[1, 1]'curl-H"Content-Type: application/json"-XPOST-d'{"db":"default", "table":"test", "vector_query":{"f":[1, 1]}}'http://localhost:8080/searchMore detailed RESTful API ishereWhat are the Embeddings?Any unstructured data(image/text/audio/video) can be transferred to vectors which are generally understanded by computers through AI(LLMs or other deep neural networks).For example, "The man is happy"-this sentence can be transferred to a 384-dimension vector(a list of numbers[0.23, 1.98, ....]) by SentenceTransformer language model. This process is called embedding.More detailed information about embeddings can be read fromOpenAIAwadb usesSentence Transformersto embed the sentence by default, while you can also use OpenAI or other LLMs to do the embeddings according to your needs.Combined with LLMs(here use LLaMa and ChatGLM) By LangChainExamples of combining LLaMa or quantized Alpaca with llama.cpp to do local knowledge database please seehereExamples of combining ChatGLM to do local knowledge database please seehereGet involvedIssues and PRRoadmap and ContributionLicenseApache 2.0CommunityJoin the AwaDB community to share any problem, suggestion, or discussion with us:DiscordSlackReddit