Package: autollm
questions|feature requests🤔 why autollm?Simplify. Unify. Amplify.FeatureAutoLLMLangChainLlamaIndexLiteLLM100+ LLMs✅✅✅✅Unified API✅❌❌✅20+ Vector Databases✅✅✅❌Cost Calculation (100+ LLMs)✅❌❌✅1-Line RAG LLM Engine✅❌❌❌1-Line FastAPI✅❌❌❌📦 installationeasily installautollmpackage with pip inPython>=3.8environment.pipinstallautollmfor built-in data readers (github, pdf, docx, ipynb, epub, mbox, websites..), install with:pipinstallautollm[readers]🎯 quickstarttutorialsvideo tutorials:Make RAG with API Just in a few Seconds!!!Autollm automagically create llm apps in secondsAutoLLM: Create RAG Based LLM Web Apps in SECONDS!AutoLLM: Ship RAG based LLM Apps and API in Seconds🚀 AutoLLM: Unlock the Power of 100+ Language Models! Step-by-Step Tutorialblog posts:Introduction to AutoLLMcolab notebooks:quickstart:create a query engine in seconds>>>fromautollmimportAutoQueryEngine,read_files_as_documents>>>documents=read_files_as_documents(input_dir="path/to/documents")>>>query_engine=AutoQueryEngine.from_defaults(documents)>>>response=query_engine.query(..."Why did SafeVideo AI develop this project?"...)>>>response.response"Because they wanted to deploy rag based llm apis in no time!"👉 advanced usage>>>fromautollmimportAutoQueryEngine>>>query_engine=AutoQueryEngine.from_defaults(...documents=documents,...llm_model="gpt-3.5-turbo",...llm_max_tokens="256",...llm_temperature="0.1",...system_prompt='...',...query_wrapper_prompt='...',...enable_cost_calculator=True,...embed_model="huggingface/BAAI/bge-large-zh",...chunk_size=512,...chunk_overlap=64,...context_window=4096,...similarity_top_k=3,...response_mode="compact",...structured_answer_filtering=False,...vector_store_type="LanceDBVectorStore",...lancedb_uri="./lancedb",...lancedb_table_name="vectors",...exist_ok=True,...overwrite_existing=False,...)>>>response=query_engine.query("Who is SafeVideo AI?")>>>print(response.response)"A startup that provides self hosted AI API's for companies!"convert it to a FastAPI app in 1-line>>>importuvicorn>>>fromautollmimportAutoFastAPI>>>app=AutoFastAPI.from_query_engine(query_engine)>>>uvicorn.run(app,host="0.0.0.0",port=8000)INFO:Startedserverprocess[12345]INFO:Waitingforapplicationstartup.INFO:Applicationstartupcomplete.INFO:Uvicornrunningonhttp://http://0.0.0.0:8000/👉 advanced usage>>>fromautollmimportAutoFastAPI>>>app=AutoFastAPI.from_query_engine(...query_engine,...api_title='...',...api_description='...',...api_version='...',...api_term_of_service='...',)>>>uvicorn.run(app,host="0.0.0.0",port=8000)INFO:Startedserverprocess[12345]INFO:Waitingforapplicationstartup.INFO:Applicationstartupcomplete.INFO:Uvicornrunningonhttp://http://0.0.0.0:8000/🌟 featuressupports100+ LLMs>>>fromautollmimportAutoQueryEngine>>>os.environ["HUGGINGFACE_API_KEY"]="huggingface_api_key">>>llm_model="huggingface/WizardLM/WizardCoder-Python-34B-V1.0">>>llm_api_base="https://my-endpoint.huggingface.cloud">>>AutoQueryEngine.from_defaults(...documents='...',...llm_model=llm_model,...llm_api_base=llm_api_base,...)👉 more llms:huggingface - ollama example:>>>fromautollmimportAutoQueryEngine>>>llm_model="ollama/llama2">>>llm_api_base="http://localhost:11434">>>AutoQueryEngine.from_defaults(...documents='...',...llm_model=llm_model,...llm_api_base=llm_api_base,...)microsoft azure - openai example:>>>fromautollmimportAutoQueryEngine>>>os.environ["AZURE_API_KEY"]="">>>os.environ["AZURE_API_BASE"]="">>>os.environ["AZURE_API_VERSION"]="">>>llm_model="azure/<your_deployment_name>")>>>AutoQueryEngine.from_defaults(...documents='...',...llm_model=llm_model...)google - vertexai example:>>>fromautollmimportAutoQueryEngine>>>os.environ["VERTEXAI_PROJECT"]="hardy-device-38811"# Your Project ID`>>>os.environ["VERTEXAI_LOCATION"]="us-central1"# Your Location>>>llm_model="text-bison@001">>>AutoQueryEngine.from_defaults(...documents='...',...llm_model=llm_model...)aws bedrock - claude v2 example:>>>fromautollmimportAutoQueryEngine>>>os.environ["AWS_ACCESS_KEY_ID"]="">>>os.environ["AWS_SECRET_ACCESS_KEY"]="">>>os.environ["AWS_REGION_NAME"]="">>>llm_model="anthropic.claude-v2">>>AutoQueryEngine.from_defaults(...documents='...',...llm_model=llm_model...)supports20+ VectorDBs🌟Pro Tip:autollmdefaults tolancedbas the vector store:
it's setup-free, serverless, and 100x more cost-effective!👉 more vectordbs:QdrantVectorStore example:>>>fromautollmimportAutoQueryEngine>>>importqdrant_client>>>vector_store_type="QdrantVectorStore">>>client=qdrant_client.QdrantClient(...url="http://<host>:<port>",...api_key="<qdrant-api-key>"...)>>>collection_name="quickstart">>>AutoQueryEngine.from_defaults(...documents='...',...vector_store_type=vector_store_type,...client=client,...collection_name=collection_name,...)automated cost calculation for100+ LLMs>>>fromautollmimportAutoServiceContext>>>service_context=AutoServiceContext(enable_cost_calculation=True)# Example verbose output after queryEmbeddingTokenUsage:7LLMPromptTokenUsage:1482LLMCompletionTokenUsage:47LLMTotalTokenCost:$0.002317create FastAPI App in 1-Line👉 example>>>fromautollmimportAutoFastAPI>>>app=AutoFastAPI.from_config(config_path,env_path)Here,configandenvshould be replaced by your configuration and environment file paths.After creating your FastAPI app, run the following command in your terminal to get it up and running:uvicornmain:app🔄 migration from llama-indexswitching from Llama-Index? We've got you covered.👉 easy migration>>>fromllama_indeximportStorageContext,ServiceContext,VectorStoreIndex>>>fromllama_index.vectorstoresimportLanceDBVectorStore>>>fromautollmimportAutoQueryEngine>>>vector_store=LanceDBVectorStore(uri="./.lancedb")>>>storage_context=StorageContext.from_defaults(vector_store=vector_store)>>>service_context=ServiceContext.from_defaults()>>>index=VectorStoreIndex.from_documents(documents=documents,storage_context=storage_contex,service_context=service_context,)>>>query_engine=AutoQueryEngine.from_instances(index)❓ FAQQ: Can I use this for commercial projects?A: Yes, AutoLLM is licensed under GNU Affero General Public License (AGPL 3.0),
which allows for commercial use under certain conditions.Contactus for more information.roadmapour roadmap outlines upcoming features and integrations to make autollm the most extensible and powerful base package for large language model applications.1-lineGradioapp creation and deploymentBudget based email notificationAutomated LLM evaluationAdd more quickstart apps on pdf-chat, documentation-chat, academic-paper-analysis, patent-analysis and more!📜 licenseautollm is available under theGNU Affero General Public License (AGPL 3.0).📞 contactfor more information, support, or questions, please contact:Email:support@safevideo.aiWebsite:SafeVideoLinkedIn:SafeVideo AI🏆 contributinglove autollm? star the repo or contribute and help us make it even better!see ourcontributing guidelinesfor more information.follow us for more!