Saravanan666/pypi-org huggingface dataset:
Package: llama-index-extra-llm
LlamaIndex Extra LLMJust a simple extension for LlamaIndex for better apply some llm such as DeepSeek.FeaturesSupport DeepSeekInstallation / EnvironmentPytorch is needed, it is easier to install by conda if you are using local PC with GPUcondainstallpytorchtorchvisiontorchaudiopytorch-cuda=11.8-cpytorch-cnvidiaQuick UsageInitializefromllama_indeximportVectorStoreIndex,SimpleDirectoryReaderfromllama_indeximportServiceContextfromllama_index_extra_llm.deepseekimportDeepSeekLLMllm=DeepSeekLLM(model_name="deepseek-ai/deepseek-llm-7b-chat",tokenizer_name="deepseek-ai/deepseek-llm-7b-chat",context_window=3900,max_new_tokens=1024,generate_kwargs={"temperature":0.7,"top_k":50,"top_p":0.95},device_map="auto",)service_context=ServiceContext.from_defaults(llm=llm)documents=SimpleDirectoryReader("data").load_data()index=VectorStoreIndex.from_documents(documents,service_context=service_context)query_engine=index.as_query_engine()Predictprompt=DeepSeekLLM.messages2prompt(messages=[{"role":"user","content":"Hello"}])assistant=query_engine.query(prompt)messages.append({"role":"assistant","content":assistant})For stream outputquery_engine=index.as_query_engine(streaming=True,similarity_top_k=1)prompt=DeepSeekLLM.messages2prompt(messages=[{"role":"user","content":"Hello"}])streaming_response=query_engine.query(prompt)streaming_response.print_response_stream()assistant=handle_output(generator)messages.append({"role":"assistant","content":assistant})
Package: llama-index-indices-managed-llama-cloud
LlamaCloud Index + RetrieverLlamaCloud is a new generation of managed parsing, ingestion, and retrieval services, designed to bring production-grade context-augmentation to your LLM and RAG applications.Currently, LlamaCloud supportsManaged Ingestion API, handling parsing and document managementManaged Retrieval API, configuring optimal retrieval for your RAG systemAccessWe are opening up a private beta to a limited set of enterprise partners for the managed ingestion and retrieval API. If you’re interested in centralizing your data pipelines and spending more time working on your actual RAG use cases, cometalk to us.If you have access to LlamaCloud, you can visitLlamaCloudto sign in and get an API key.SetupFirst, make sure you have the latest LlamaIndex version installed.NOTE:If you are upgrading from v0.9.X, we recommend following ourmigration guide, as well as uninstalling your previous version first.pip uninstall llama-index  # run this if upgrading from v0.9.x or older
pip install -U llama-index --upgrade --no-cache-dir --force-reinstallThellama-index-indices-managed-llama-cloudpackage is included with the above install, but you can also install directlypip install -U llama-index-indices-managed-llama-cloudUsageYou can create an index on LlamaCloud using the following code:importosos.environ["LLAMA_CLOUD_API_KEY"]="llx-..."# can provide API-key in env or in the constructor later onfromllama_index.coreimportSimpleDirectoryReaderfromllama_index.indices.managed.llama_cloudimportLlamaCloudIndex# create a new indexindex=LlamaCloudIndex.from_documents(documents,"my_first_index",project_name="default",api_key="llx-...",verbose=True,)# connect to an existing indexindex=LlamaCloudIndex("my_first_index",project_name="default")You can also configure a retriever for managed retrieval:# from the existing indexindex.as_retriever()# from scratchfromllama_index.indices.managed.llama_cloudimportLlamaCloudRetrieverretriever=LlamaCloudRetriever("my_first_index",project_name="default")And of course, you can use other index shortcuts to get use out of your new managed index:query_engine=index.as_query_engine(llm=llm)chat_engine=index.as_chat_engine(llm=llm)Retriever SettingsA full list of retriever settings/kwargs is below:dense_similarity_top_k: Optional[int] -- If greater than 0, retrieveknodes using dense retrievalsparse_similarity_top_k: Optional[int] -- If greater than 0, retrieveknodes using sparse retrievalenable_reranking: Optional[bool] -- Whether to enable reranking or not. Sacrifices some speed for accuracyrerank_top_n: Optional[int] -- The number of nodes to return after reranking initial retrieval resultsalphaOptional[float] -- The weighting between dense and sparse retrieval. 1 = Full dense retrieval, 0 = Full sparse retrieval.
Package: llama-index-llms-groq
LlamaIndex Llms Integration: GroqWelcome to Groq! 🚀 At Groq, we've developed the world's first Language Processing Unit™, or LPU. The Groq LPU has a deterministic, single core streaming architecture that sets the standard for GenAI inference speed with predictable and repeatable performance for any given workload.Beyond the architecture, our software is designed to empower developers like you with the tools you need to create innovative, powerful AI applications. With Groq as your engine, you can:Achieve uncompromised low latency and performance for real-time AI and HPC inferences 🔥Know the exact performance and compute time for any given workload 🔮Take advantage of our cutting-edge technology to stay ahead of the competition 💪Want more Groq? Check out ourwebsitefor more resources and join ourDiscord communityto connect with our developers!DevelopTo create a development environment, install poetry then run:poetryinstall--withdevTestingTo test the integration, first enter the poetry venv:poetryshellThen tests can be run with makemaketestIntegration testsIntegration tests will be skipped unless an API key is provided. API keys can be created ath thegroq console.
Once created, store the API key in an environment variable and run testsexportGROQ_API_KEY=<yourkeyhere>
maketestLinting and FormattingLinting and code formatting can be executed with make.makeformat
makelint
Package: llama-index-networks
LlamaIndex NetworksThellama-index-networkslibrary extension allows for the creation of
networks knowledge that can be queried and used for LLM context augmentation.
In such networks, we have "data suppliers" and "data consumers" that participate
and benefit from the network, respectively.Data suppliersWith this extension, you can easily wrap your llama-indexQueryEnginewith aContributorServicethat exposes the query engine behind a REST (FastAPI)
service, which renders it ready to contribute to LlamaIndex Network!fromllama_index.networks.contributorimport(ContributorServiceSettings,ContributorService,)query_engine=...# build a query engine as typically done with llama-indexsettings=ContributorServiceSettings()service=ContributorService(config=settings,query_engine=query_engine)app=service.appif__name__=="__main__":importuvicornuvicorn.run(app,port=8000)Data consumersWithllama-index-networks, you can build aNetworkQueryEnginethat is able
to connect to set of (network)ContributorService's.fromllama_index.networks.contributorimportContributorClientfromllama_index.networks.query_engineimportNetworkQueryEnginefromllama_index.llms.groqimportGroq# Use ContributorClient to connect to a ContributorServiceclient=ContributorClient.from_config_file(env_file=".env.contributor.client")contributors=[client]# NetworkQueryEnginellm=Groq()network_query_engine=NetworkQueryEngine.from_args(contributors=contributors,llm=llm)# Can query it like any other query enginenetwork_query_engine.query("Why is the sky blue?")The.env.contributor.clientfile contains the parameters to connect to theContributorService(i.e.,api_urlandapi_token.)ExamplesFor a full demo, checkout theexamples/demosub-folder.FAQ1. Can I add my own custom endpoints to aContributorService?Yes, the (Fastapi) app is accessible viaContributorService.appand can be
modified to your needs.settings=ContributorServiceSettings()service=ContributorService(config=settings,query_engine=query_engine)app=service.app@app.get("<custom-path>")defcustom_endpoint_logic():...2. How can I add authentication to aContributorService?Currently, the client supports authentication through request headers as well as
as via anapi_tokenstr. On the server side, you should be able to modify the
app to your needs:settings=ContributorServiceSettings()service=ContributorService(config=settings,query_engine=query_engine)app=service.app# modify app here to include your own security mechanismsAs we continue to build out this library extension, we welcome any feedback or
suggestions on what can be incorporated on this regard and others in order to
make our tools more helpful to our users.
Package: llama-index-packs-agent-search-retriever
Agent-Search Retrieval PackThis LlamaPack creates a custom retriever that uses the agent-search API for retrieving general content indexed from the internet.This framework facilitates seamless integration with the AgentSearch dataset (terabytes of indexed data!) or hosted search APIs (e.g. Search Engines).During query-time, the user passes in the query string, search provider (bing,agent-search), and relevant nodes are retrieved from the hosted dataset.To learn more, please refer to the documentationhere.CLI UsageYou can download llamapacks directly usingllamaindex-cli, which comes installed with thellama-indexpython package:llamaindex-clidownload-llamapackAgentSearchRetrieverPack--download-dir./agent_search_packYou can then inspect the files at./agent_search_packand use them as a template for your own project!Code UsageYou can download the pack to a the./agent_search_packdirectory:# Optionally set the API key in the env# import os# os.environ["SCIPHI_API_KEY"] = "..."fromllama_index.core.query_engineimportRetrieverQueryEnginefromllama_index.core.llama_packimportdownload_llama_pack# download and install dependenciesAgentSearchRetrieverPack=download_llama_pack("AgentSearchRetrieverPack","./agent_search_pack")agent_search_pack=AgentSearchRetrieverPack(api_key="...",similarity_top_k=4,search_provider="agent-search")# use the retriever directlyretriever=agent_search_pack.retrieversource_nodes=retriever.retrieve("query str")# uses the agent-search retriever within a llama-index query engine!query_engine=RetrieverQueryEngine.from_args(retriever)response=query_engine.query("query str")Therun()function is a light wrapper aroundretriever.retrieve().source_nodes=agent_search_pack.run("What can you tell me about LLMs?")print(source_nodes)See thenotebook on llama-hubfor a full example.
Package: llama-index-packs-agents-llm-compiler
LLMCompiler Agent PackThis LlamaPack implements theLLMCompiler agent paper.A lot of code came from the source repo, we repurposed with LlamaIndex abstractions. All credits
to the original authors for a great work!A full notebook guide can be foundhere.CLI UsageYou can download llamapacks directly usingllamaindex-cli, which comes installed with thellama-indexpython package:llamaindex-clidownload-llamapackLLMCompilerAgentPack--download-dir./llm_compiler_agent_packYou can then inspect the files at./llm_compiler_agent_packand use them as a template for your own project!Code UsageYou can download the pack to a directory.NOTE: You must specifyskip_load=True- the pack contains multiple files,
which makes it hard to load directly.We will show you how to import the agent from these files!fromllama_index.core.llama_packimportdownload_llama_pack# download and install dependenciesdownload_llama_pack("LLMCompilerAgentPack","./llm_compiler_agent_pack")From here, you can use the pack. You can import the relevant modules from the download folder (in the example below we assume it's a relative import or the directory has been added to your system path).# setup pack argumentsfromllama_index.core.agentimportAgentRunnerfromllm_compiler_agent_pack.stepimportLLMCompilerAgentWorkeragent_worker=LLMCompilerAgentWorker.from_tools(tools,llm=llm,verbose=True,callback_manager=callback_manager)agent=AgentRunner(agent_worker,callback_manager=callback_manager)# start using the agentresponse=agent.chat("What is (121 * 3) + 42?")You can also use/initialize the pack directly.fromllm_compiler_agent_pack.baseimportLLMCompilerAgentPackagent_pack=LLMCompilerAgentPack(tools,llm=llm)Therun()function is a light wrapper aroundagent.chat().response=pack.run("Tell me about the population of Boston")You can also directly get modules from the pack.# use the agentagent=pack.agentresponse=agent.chat("task")
Package: llama-index-packs-amazon-product-extraction
Amazon Product Extraction PackThis LlamaPack provides an example of our Amazon product extraction pack.It loads in a website URL, screenshots the page. Then we use OpenAI GPT-4V + prompt engineering to extract the screenshot into a structured JSON output.Check out thenotebook here.CLI UsageYou can download llamapacks directly usingllamaindex-cli, which comes installed with thellama-indexpython package:llamaindex-clidownload-llamapackAmazonProductExtractionPack--download-dir./amazon_product_extraction_packYou can then inspect the files at./amazon_product_extraction_packand use them as a template for your own project.Code UsageYou can download the pack to a the./amazon_product_extraction_packdirectory:fromllama_index.core.llama_packimportdownload_llama_pack# download and install dependenciesAmazonProductExtractionPack=download_llama_pack("AmazonProductExtractionPack","./amazon_product_extraction_pack")From here, you can use the pack, or inspect and modify the pack in./amazon_product_extraction_pack.Then, you can set up the pack like so:# create the pack# get documents from any data loaderamazon_product_extraction_pack=SentenceWindowRetrieverPack(amazon_product_page,)Therun()function is a light wrapper aroundprogram().response=amazon_product_extraction_pack.run()display(response.dict())You can also use modules individually.# get pydantic programprogram=amazon_product_extraction_pack.openai_program# get multi-modal LLMmm_llm=amazon_product_extraction_pack.openai_mm_llm
Package: llama-index-packs-arize-phoenix-query-engine
Arize-Phoenix LlamaPackThis LlamaPack instruments your LlamaIndex app for LLM tracing withPhoenix, an open-source LLM observability library fromArize AI.CLI UsageYou can download llamapacks directly usingllamaindex-cli, which comes installed with thellama-indexpython package:llamaindex-clidownload-llamapackArizePhoenixQueryEnginePack--download-dir./arize_packYou can then inspect the files at./arize_packand use them as a template for your own project!Code UsageYou can download the pack to a the./arize_packdirectory:fromllama_index.core.llama_packimportdownload_llama_pack# download and install dependenciesArizePhoenixQueryEnginePack=download_llama_pack("ArizePhoenixQueryEnginePack","./arize_pack")You can then inspect the files at./arize_packor continue on to use the module.importosfromllama_index.core.node_parserimportSentenceSplitterfromllama_index.readers.webimportSimpleWebPageReaderfromtqdm.autoimporttqdmConfigure your OpenAI API key.os.environ["OPENAI_API_KEY"]="copy-your-openai-api-key-here"Parse your documents into a list of nodes and pass to your LlamaPack. In this example, use nodes from a Paul Graham essay as input.documents=SimpleWebPageReader().load_data(["https://raw.githubusercontent.com/jerryjliu/llama_index/adb054429f642cc7bbfcb66d4c232e072325eeab/examples/paul_graham_essay/data/paul_graham_essay.txt"])parser=SentenceSplitter()nodes=parser.get_nodes_from_documents(documents)phoenix_pack=ArizePhoenixQueryEnginePack(nodes=nodes)Run a set of queries via the pack'srunmethod, which delegates to the underlying query engine.queries=["What did Paul Graham do growing up?","When and how did Paul Graham's mother die?","What, in Paul Graham's opinion, is the most distinctive thing about YC?","When and how did Paul Graham meet Jessica Livingston?","What is Bel, and when and where was it written?",]forqueryintqdm(queries):print("Query")print("=====")print(query)print()response=phoenix_pack.run(query)print("Response")print("========")print(response)print()View your trace data in the Phoenix UI.phoenix_session_url=phoenix_pack.get_modules()["session_url"]print(f"Open the Phoenix UI to view your trace data:{phoenix_session_url}")You can access the internals of the LlamaPack, including your Phoenix session and your query engine, via theget_modulesmethod.phoenix_pack.get_modules()Check out thePhoenix documentationfor more information!
Package: llama-index-packs-auto-merging-retriever
Auto Merging Retriever PackThis LlamaPack provides an example of our auto-merging retriever.This specific template shows the e2e process of building this. It loads
a document, builds a hierarchical node graph (with bigger parent nodes and smaller
child nodes).Check out thenotebook here.CLI UsageYou can download llamapacks directly usingllamaindex-cli, which comes installed with thellama-indexpython package:llamaindex-clidownload-llamapackAutoMergingRetrieverPack--download-dir./auto_merging_retriever_packYou can then inspect the files at./auto_merging_retriever_packand use them as a template for your own project.Code UsageYou can download the pack to a the./auto_merging_retriever_packdirectory:fromllama_index.core.llama_packimportdownload_llama_pack# download and install dependenciesAutoMergingRetrieverPack=download_llama_pack("AutoMergingRetrieverPack","./auto_merging_retriever_pack")From here, you can use the pack, or inspect and modify the pack in./auto_merging_retriever_pack.Then, you can set up the pack like so:# create the pack# get documents from any data loaderauto_merging_retriever_pack=AutoMergingRetrieverPack(documents,)Therun()function is a light wrapper aroundquery_engine.query().response=auto_merging_retriever_pack.run("Tell me about what the author did growing up.")You can also use modules individually.# get the node parsernode_parser=auto_merging_retriever_pack.node_parser# get the retrieverretriever=auto_merging_retriever_pack.retriever# get the query enginequery_engine=auto_merging_retriever_pack.query_engine
Package: llama-index-packs-chroma-autoretrieval
Chroma AutoRetrieval PackThis LlamaPack inserts your data into chroma and instantiates an auto-retriever, which will use the LLM at runtime to set metadata filtering, top-k, and query string.CLI UsageYou can download llamapacks directly usingllamaindex-cli, which comes installed with thellama-indexpython package:llamaindex-clidownload-llamapackChromaAutoretrievalPack--download-dir./chroma_packYou can then inspect the files at./chroma_packand use them as a template for your own project!Code UsageYou can download the pack to a the./chroma_packdirectory:fromllama_index.core.llama_packimportdownload_llama_pack# download and install dependenciesChromaAutoretrievalPack=download_llama_pack("ChromaAutoretrievalPack","./chroma_pack")From here, you can use the pack, or inspect and modify the pack in./chroma_pack.Then, you can set up the pack like so:# setup pack argumentsfromllama_index.core.vector_stores.typesimportMetadataInfo,VectorStoreInfovector_store_info=VectorStoreInfo(content_info="brief biography of celebrities",metadata_info=[MetadataInfo(name="category",type="str",description=("Category of the celebrity, one of [Sports Entertainment, Business, Music]"),),],)importchromadbclient=chromadb.EphemeralClient()nodes=[...]# create the packchroma_pack=ChromaAutoretrievalPack(collection_name="test",vector_store_info=vector_store_index,nodes=nodes,client=client,)Therun()function is a light wrapper aroundquery_engine.query().response=chroma_pack.run("Tell me a bout a Music celebritiy.")You can also use modules individually.# use the retrieverretriever=chroma_pack.retrievernodes=retriever.retrieve("query_str")# use the query enginequery_engine=chroma_pack.query_engineresponse=query_engine.query("query_str")
Package: llama-index-packs-cogniswitch-agent
Cogniswitch LlamaPackUse CogniSwitch to build production ready applications that can consume, organize and retrieve knowledge flawlessly. Using the framework of your choice, in this case LlamaIndex, CogniSwitch helps alleviate the stress of decision making when it comes to, choosing the right storage and retrieval formats. It also eradicates reliability issues and hallucinations when it comes to responses that are generated. Get started by interacting with your knowledge in a few simple stepsvisithttps://www.cogniswitch.ai/developer.Registration:Signup with your email and verify your registrationYou will get a mail with a platform token and OAuth token for using the services.Step 1: Download the CogniSwitch Llama pack:Download the CogniswitchAgentPack either with the llama-cli or import using the code.Step 2: Instantiate the CogniswitchAgentPack:Instantiate the cogniswitch agent pack with all the credentials.Step 3: Cogniswitch Store data:Make the call to the agent by giving the file path or url to the agent input.The agent will pick the tool and use the file/url and it will be processed and stored in your knowledge store.You can check the status of document processing with a call to the agent. Alternatively you can also check incogniswitch console.Step 4: Cogniswitch Answer:Make the call to the agent by giving query as agent input.You will get the answer from your knowledge as the response.CLI UsageYou can download llamapacks directly usingllamaindex-cli, which comes installed with thellama-indexpython package:llamaindex-clidownload-llamapackCogniswitchAgentPack--download-dir./cs_packCode Usageimportwarningswarnings.filterwarnings("ignore")fromllama_index.core.llama_packsimportCogniswitchAgentPackimportos### Cogniswitch Credentials and OpenAI token# os.environ["OPENAI_API_KEY"] = <your openai token># cogniswitch_tool_args = {#   "cs_token":<your cogniswitch platform token>,#   "apiKey":<your cogniswitch apikey># }cogniswitch_agent_pack=CogniswitchAgentPack(cogniswitch_tool_args)From here, you can use the pack, or inspect and modify the pack in./cs_pack.Therun()function is a light wrapper aroundagent.chat().Use the cogniswitch agent for storing data in cogniswitch with a single callresponse=cogniswitch_agent_pack.run("Upload this URL- https://cogniswitch.ai/developer")Use the cogniswitch agent to know the status of the document with a callresponse=cogniswitch_agent_pack.run("Tell me the status of https://cogniswitch.ai/developer")Use the cogniswitch agent for answering with a single callresponse=cogniswitch_agent_pack.run("Answer the question- Tell me about cogniswitch")
Package: llama-index-packs-corrective-rag
Corrective Retrieval Augmented Generation Llama PackThis LlamaPack implements the Corrective Retrieval Augmented Generation (CRAG)paperCorrective Retrieval Augmented Generation (CRAG) is a method designed to enhance the robustness of language model generation by evaluating and augmenting the relevance of retrieved documents through a an evaluator and large-scale web searches, ensuring more accurate and reliable information is used in generation.This LlamaPack usesTavily AIAPI for web-searches. So, we recommend you to get the api-key before proceeding further.Installationpipinstallllama-indexllama-index-tools-tavily-researchCLI UsageYou can download llamapacks directly usingllamaindex-cli, which comes installed with thellama-indexpython package:llamaindex-clidownload-llamapackCorrectiveRAGPack--download-dir./corrective_rag_packYou can then inspect the files at./corrective_rag_packand use them as a template for your own project.Code UsageYou can download the pack to a the./corrective_rag_packdirectory:fromllama_index.core.llama_packimportdownload_llama_pack# download and install dependenciesCorrectiveRAGPack=download_llama_pack("CorrectiveRAGPack","./corrective_rag_pack")# You can use any llama-hub loader to get documents!corrective_rag=CorrectiveRAGPack(documents,tavily_ai_api_key)From here, you can use the pack, or inspect and modify the pack in./corrective_rag_pack.Therun()function contains around logic behind Corrective Retrieval Augmented Generation -CRAGpaper.response=corrective_rag.run("<query>",similarity_top_k=2)