mteb/FreshStackRetrieval huggingface dataset:
I am trying to use a custom embedding model in Langchain with chromaDB. I can't seem to find a way to use the base embedding class without having to use some other provider (like OpenAIEmbeddings or HuggingFaceEmbeddings). Am I missing something?
On the Langchain page it says that the base Embeddings class in LangChain provides two methods: one for embedding documents and one for embedding a query. so I figured there must be a way to create another class on top of this class and overwrite/implement those methods with our own methods. But how do I do that?
I tried to somehow use the base embeddings class but am unable to create a new embedding object/class on top of it.
I Am trying to download a PDF file from a GCS storage bucket and read the content into memory.
When using Langchain with python, i can just use the GCSDirectoryLoader to read all the files in a bucket and the pdf text.
Langchain for NodeJs doesnt have GCSDirectoryLoader or a webloader for PDF files.
When downloading the file, i get a Document with the binary representation as content.
What is the best way to download pdf content from GCS bucket into memory?
I am retrieving the results from my internal Db but for this example I have added an open URL. I am using Azure openai and langchain in conjunction to build this retrieval engine. I checked in the Azure Portal that deployment is successful and i am able to run in a stand alone prompt.
The last query throws this error:

InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.completion.Completion'>.

Since we can see I have already supplied a deployment_id above. What am I missing?
Here is the entire code
from langchain.vectorstores import Chroma
from langchain.embeddings import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.llms import OpenAI
from langchain.chains import RetrievalQA
from langchain.document_loaders import TextLoader
from langchain.document_loaders import DirectoryLoader  
loader= TextLoader(r'./RawText.txt', encoding='utf-8')
documents = loader.load()    
text_splitter= RecursiveCharacterTextSplitter(chunk_size= 70, chunk_overlap=0)
texts= text_splitter.split_documents(documents)

#create the DB
    persist_directory = 'db'
#embedding= OpenAIEmbeddings()
   embeddings = OpenAIEmbeddings(deployment="text-embedding-ada-002",model="text-embedding-ada-002", chunk_size = 1)
   vectordb= Chroma.from_documents(documents=texts,
                               embedding=embeddings,
                               persist_directory=persist_directory)

Post this I am creating a retriever as below:
  vectordb= Chroma(persist_directory=persist_directory, embedding_function=embeddings)
    retriever= vectordb.as_retriever()
    docs=retriever.get_relevant_documents("Databricks")

#Creating a Chain:
 from langchain.chains import RetrievalQA
 import openai

#Specify the name of the engine you want to use
engine = "test_chat"
qa_chain=RetrievalQA.from_chain_type(llm=OpenAI(),
                                chain_type="stuff",
                                retriever= retriever,
                                return_source_documents=True)

#test_chat here for reference is text-embedding-ada-002
#Cite Source
def process_llm_responses(llm_response):
print(llm_response['result'])
print('\n\nSources:')
for source in llm_response["source_documents"]:
    print(source.metadata["source"])

#full retrieval in process
query = "What is A medallion architecture"
llm_response= qa_chain(query)
process_llm_responses
I want to parallelize RetrievalQA with asyncio but I am unable to figure out how.
This is how my code works serially:
import langchain
from langchain.chat_models import ChatOpenAI
from langchain import PromptTemplate, LLMChain
from langchain.chains import RetrievalQA
from langchain.vectorstores import FAISS
from langchain.schema.vectorstore import VectorStoreRetriever
import asyncio
import nest_asyncio

retriever = VectorStoreRetriever(vectorstore=FAISS(...))

chat = ChatOpenAI(model="gpt-3.5-turbo-16k", temperature=0.7)

qa_chain = RetrievalQA.from_llm(chat, retriever= retriever
                                                 #,memory=memory
                                                 , return_source_documents=True
                                                 )

queries = ['query1', 'query2', 'query3']
data_to_append = []

for query in queries :

    vectordbkwargs = {"search_distance": 0.9}
    result = qa_chain({"query": query, "vectordbkwargs": vectordbkwargs})

    data_to_append.append({"Query": query, "Source_Documents": result["source_documents"], "Generated_Text": result["result"]})


Here was my attempt to parallelize it with asyncio but RetrievalQA doesn't seem to work async:
import langchain
from langchain.chat_models import ChatOpenAI
from langchain import PromptTemplate, LLMChain
from langchain.chains import RetrievalQA
from langchain.vectorstores import FAISS
from langchain.schema.vectorstore import VectorStoreRetriever
import asyncio
import nest_asyncio

retriever = VectorStoreRetriever(vectorstore=FAISS(...))

chat = ChatOpenAI(model="gpt-3.5-turbo-16k", temperature=0.7)


qa_chain = RetrievalQA.from_llm(chat, retriever= retriever
                                                 , return_source_documents=True
                                                 )

queries = ['query1', 'query2', 'query3']
data_to_append = []



async def process_query(query):

        vectordbkwargs = {"search_distance": 0.9}
        result = await qa_chain({"query": query, "vectordbkwargs": vectordbkwargs})
        data_to_append.append({"Query": query, "Source_Documents": result["source_documents"], "Generated_Text": result["result"]})


async def main():

    tasks = []

    for query in queries: # Iterate all rows
        task = process_query(query)
        tasks.append(task)

    await asyncio.gather(*tasks)

if __name__ == "__main__":
    nest_asyncio.apply()
    asyncio.run(main())

Any help would be greatly appreciated.
I am currently working on a project where I have implemented the ConversationalRetrievalQAChain, with the option "returnSourceDocuments" set to true. The system works perfectly when I ask specific questions related to the VectorStore database, as it returns matching sources. However, when I pose generic questions like "Is the earth round?" or "How are you today?" it returns unrelated sources that don't align with the query. It appears that the vector store always returns documents, even if they don't match the query.
I'm seeking guidance on how to enhance the relevance of the source documents retrieved by the Langchain ConversationalRetrievalQAChain. Are there specific tools or techniques within the Langchain framework that can help mitigate this behavior, or is it necessary to develop a manual process to assess document relevance? How can I effectively limit the retrieval of unrelated source documents in this scenario?
Here is relevant part of the code:
async init(): Promise<void> {
        try {
            this.driver = await this.getQdrantDriver()
            this.retriever = await this.createRetrieverFromDriver()
            this.chat = new ChatOpenAI({ modelName: aiConfig.modelName })
            this.chain = await this.createQAChain(this.chat)

            this.questionGenerationChain = await this.createQuestionGenerationChain()
            this.conversation = new ConversationalRetrievalQAChain({
                retriever: this.retriever,
                combineDocumentsChain: this.chain,
                questionGeneratorChain: this.questionGenerationChain,
                returnSourceDocuments: true,
            })
        } catch (error) {
            Logger.error(error.message)
            throw error
        }
    }

    private async createQuestionGenerationChain(): Promise<LLMChain> {
        const { default: Prompt } = await import('App/Models/Prompt')
        return new LLMChain({
            llm: this.chat,
            prompt: await Prompt.fetchCondensePrompt(),
        })
    }

    private async createRetrieverFromDriver(): Promise<VectorStoreRetriever<QdrantVectorStore>> {
        return this.driver.asRetriever(qdrantConfig.noResults ?? 5)
    }

    private async getQdrantDriver(embeddings = new OpenAIEmbeddings(), collectionName: string | null = null): Promise<QdrantVectorStore> {
        const { default: Ingest } = await import('App/Models/Ingest')
        return new QdrantVectorStore(
            embeddings,
            {
                url: qdrantConfig.qdrantUrl,
                collectionName: collectionName ?? await Ingest.lastCollection(),
            },
        )
    }

    private async createQAChain(chat: BaseLanguageModel<any, BaseLanguageModelCallOptions>): Promise<StuffDocumentsChain> {
        const { default: Prompt } = await import('App/Models/Prompt')
        return loadQAStuffChain(chat, {
            prompt: await Prompt.fetchQuestionPrompt(),
        })
    }
This is how I am defining the executor
const executor = await initializeAgentExecutorWithOptions(tools, model, {
  agentType: 'chat-conversational-react-description',
  verbose: false,
});

Whenever I prompt the AI I have this statement at the end.
type SomeObject = {
  field1: number,
  field2: number,
}

- It is very critical that you answer only as the above object and JSON stringify it as a single string.
  Don't include any other verbose explanatiouns and don't include the markdown syntax anywhere.

The SomeObject is just an example. Usually it will have a proper object type.
When I use the executor to get a response from the AI, half the time I get the proper JSON string, but the other half the times are the AI completely ignoring my instructions and gives me a long verbose answer in just plain English...
How can I make sure I always get the structured data answer I want?
Maybe using the agentType: 'chat-conversational-react-description' isn't the right approach here?
I am using Langchain with OpenAI API for getting the summary of PDF Files. Some of my PDFs have many pages (more than the max token allowed in ChatGPT). Im trying two approaches to reduce the tokens so that I can input longer texts, but is still not working for a 300 inch- PDF.

Retrieval augmented generation: more specifically the text splitter

text_splitter = RecursiveCharacterTextSplitter(chunk_size = 1000, chunk_overlap = 50)
    all_splits = text_splitter.split_documents(data)


Text summarisation: using stuff documents chain

 stuff_chain = StuffDocumentsChain(llm_chain=llm_chain, document_variable_name="text")

I would like to understand what is the text splitter doing because is not helping me to input longer text in the prompt. How can do this?
I'm using langchain with Azure OpenAI and Azure Cognitive Search.
Currently I'm using Azure OpenAI text-embedding-ada-002 model for generating embeddings, but I would like to use a embbeding model from HugginFace if possible, because Azure OpenAI API does not allow to send documents in batches, so I need to make several calls and hit the rate limit.
I tried using this embbeding in my code:
embeddings = SentenceTransformerEmbeddings(
        model_name="all-mpnet-base-v2",
    )

Instead of:
embeddings = OpenAIEmbeddings(
        ...
)

The problem I'm facing, is that when I use  AzureSearch's aadd_texts method I get this error:
The vector field 'content_vector' dimensionality must match the field definition's 'dimensions' property. Expected: '1536'. Actual: '768'. (IndexDocumentsFieldError) 98: The vector field 'content_vector' dimensionality must match the field definition's 'dimensions' property. Expected: '1536'. Actual: '768'.
        Code: IndexDocumentsFieldError

I'm pretty lost. Did anyone used an open source embeddings model with Cognitive Search? How?
The following code do not do what it is supposed to do:
from langchain.callbacks.base import BaseCallbackHandler
from langchain import PromptTemplate
from langchain.chains import LLMChain
from langchain.llms import VertexAI


class MyCustomHandler(BaseCallbackHandler):
    def on_llm_end(self, event, context):
        print(f"Prompt: {event.prompt}")
        print(f"Response: {event.response}")


llm = VertexAI(
            model_name='text-bison@001',
            max_output_tokens=1024,
            temperature=0.3,
            verbose=False)
prompt = PromptTemplate.from_template("1 + {number} = ")
handler = MyCustomHandler()
chain = LLMChain(llm=llm, prompt=prompt, callbacks=[handler])
response = chain.run(number=2)
print(response)

Based on this documentation and this tutorial, the code should execute the custom handler callback on_llm_end but in fact it doesn't.
Can anyone please tell me why?
I'm working on an AI project but my current problem right now is that FAISS is taking far too long to load the documents. So Iv moved it into its own service via fastapi.
Everything Looks ok, but when I run it I get the error of:
id not find openai_api_key, please add an environment variable `OPENAI_API_KEY`

In my code:
embeddings = OpenAIEmbeddings()
db = FAISS.from_documents(documents, embeddings)

Now I am Using OpenAI but not in this service so i did not add my key.
From my understanding its just taking text tokenizing it using openAI's token map, and then doing a search and finding the nearest related documents based on that query.
That, Technically does not actually reach out to Open AI servers does it?
Afterwords i'm just adding the related documents to the prompt that I Send to Open AI's servers, So if its sending data to open AI twice that a tad inefficient right?
How can I get this to just be its own service? Or am I wasting my time here?
I'm trying to build a simple RAG, and I'm stuck at this code:
from langchain.embeddings.huggingface import HuggingFaceEmbeddings
from llama_index import LangchainEmbedding, ServiceContext

embed_model = LangchainEmbedding(
  HuggingFaceEmbeddings(model_name="thenlper/gte-large")
)
service_context = ServiceContext.from_defaults(
    chunk_size=256,
    llm=llm,
    embed_model=embed_model
)
index = VectorStoreIndex.from_documents(documents, service_context=service_context)

where I get ImportError: cannot import name 'LangchainEmbedding' from 'llama_index'
How can I solve? Is it related to the fact that I'm working on Colab?
I have deployed llm model locally which follows openai api schema. As it's endpoint follows openai schema, I don't want to write separate inference client.
Is there any way we can utilize existing openai wrapper by langchain to do inference for my localhost model.
I checked there is a openai adapter by langchain, but it seems like it require provider, which again I have to write separate client.
Overall goal it to not write any redundant code as it's already been maintained by langchain and may change with time. We can modify our api wrt openai and it works out of the box.
Your suggestion is appreciated.
Trying to conenct postgresql with langchain.llm used - AzureOpenAI
from langchain.llms import AzureOpenAI

llms = AzureOpenAI( temperature=0,deployment_name="gpt3turbo".......)

toolkit = SQLDatabaseToolkit(db=db,llm=llms)

Error:
ValidationError: 1 validation error for SQLDatabaseToolkit
llm
  value is not a valid dict (type=type_error.dict)

Tried different versions of langchain
I am new to langchain and following a tutorial code as below
from langchain.vectorstores import Chroma
from langchain.embeddings.openai import OpenAIEmbeddings
persist_directory = "C:/Users/shang/Documents/test/"
embedding = OpenAIEmbeddings()
vectordb = Chroma(persist_directory, embedding_function=embedding)

it kept prompt error. Did I miss anything here? Thanks
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
g:\My Drive\DataScience\LLM\LongChain\all_inclusive.ipynb Cell 5 line 5
      3 persist_directory = 'C:/Users/shang/Documents/test/'
      4 embedding = OpenAIEmbeddings()
----> 5 vectordb = Chroma(persist_directory, embedding_function=embedding)

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\langchain\vectorstores\chroma.py:81, in Chroma.__init__(self, collection_name, embedding_function, persist_directory, client_settings, collection_metadata, client, relevance_score_fn)
     79 """Initialize with a Chroma client."""
     80 try:
---> 81     import chromadb
     82     import chromadb.config
     83 except ImportError:

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\chromadb\__init__.py:3
      1 from typing import Dict, Optional
      2 import logging
----> 3 from chromadb.api.client import Client as ClientCreator
      4 from chromadb.api.client import AdminClient as AdminClientCreator
      5 import chromadb.config

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\chromadb\api\client.py:31
     27 from chromadb.types import Database, Tenant, Where, WhereDocument
     28 import chromadb.utils.embedding_functions as ef
---> 31 class SharedSystemClient:
...
   1011     os.stat() does.
   1012     """
-> 1013     return os.stat(self, follow_symlinks=follow_symlinks)

OSError: [WinError 433] A device which does not exist was specified: '.env'
I am using Llama2[7b model]-hugging face and lang-chain to do a simple address segregation/classification task. I want the model to find the city, state and country from the input string.I want my answer/query formatted in a particular way for a question-answering/ text-generation task.I understand that i can use FewShotPromptTemplate, where in i can show some examples to the LLM and get the output in the format i want.
I generated a few examples to feed in as samples :
examples = [
    {"input": "Plot No. 7, Sector 22, Noida, Uttar Pradesh, 201301, India",
     "Address": "Plot No. 7, Sector 22, Noida",
     "City" : "Noida",
     "State" : "Uttar Pradesh",
     "Country" : "India"},


    {"input": "Banjara Hills, Telangana, 500034, India",
     "Address": "Banjara Hills",
     "City" : "Not present",
     "State" : "Telangana",
     "Country" : "India"},

]

I set the template
example_formatter_template = """
input: {input},
Address : {Address},
City : {Address},
State : {State},
Country : {Country},
         \n
"""
# prompt
example_prompt = PromptTemplate(
    input_variables=["input", "Address", "City", "State", "Country"],
    template=example_formatter_template)

few_shot_prompt = FewShotPromptTemplate(
    examples=examples,
    example_prompt=example_prompt,
    prefix="What is the address, city, state, country in the string : ",
    suffix="input: {input}\n ",
    input_variables=["input"],
    example_separator="\n")


chain = LLMChain(llm=llm, prompt=few_shot_prompt, verbose = True)

# Run the chain only specifying the input variable.
print(chain.run("B-12, Gandhi Colony, Bhopal, Madhya Pradesh, 462016, India"))


Here is an example of what i want :
    {"input": "B-12, Gandhi Colony, Bhopal, Madhya Pradesh, 462016, India",

     "Address": "B-12, Gandhi Colony",
     "City" : "Bhopal",
     "State" : "Madhya Pradesh",
     "Country" : "India"},



I keep getting : format the expected output correctly from the model. And nothing is hence returned.
Additionally, I want to prevent the model from adding any extra information which
is not present in the context/string otherwise the queries take very long to respond.
i.e return '' or not found if city or state or country is not present in sting.
can someone help ?
Here is a simple code to use Redis and embeddings but It's not clear how can I build and load own embeddings and then pull it from Redis and use in search
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores.redis import Redis

embeddings = OpenAIEmbeddings
metadata = [
    {
        "user": "john",
        "age": 18,
        "job": "engineer",
        "credit_score": "high"
    }
]
texts = ["foo", "foo", "foo", "bar", "bar"]

rds = Redis.from_texts(
    texts,
    embeddings,
    metadata,
    redis_url="redis://localhost:6379",
    index_name="users",
)

results = rds.similarity_search("foo")
print(results[0].page_content)

But I want to load a text from e.g. text file, create embedings and load into Redis for later use. Something like this:
from openai import OpenAI
client = OpenAI()

def get_embedding(text, model="text-embedding-ada-002"):
    text = text.replace("\n", " ")
    return client.embeddings.create(input = [text], model=model).data[0].embedding

Does anyone have good example to implement this approach? Also wondering about TTL for embedings in Redis
langchain python agent react differently, for one prompt, it can import scanpy library, but not for the other one. My question is how to make sure to import the correct library without problem.
from dotenv import load_dotenv, find_dotenv
import openai
import os
from langchain.chat_models import ChatOpenAI
from langchain.agents.agent_types import AgentType
from langchain_experimental.agents.agent_toolkits import create_python_agent
from langchain_experimental.tools import PythonREPLTool
import scanpy as sc

load_dotenv(find_dotenv())
openai.api_key = os.environ["OPENAI_API_KEY"]

agent_executor = create_python_agent(
    llm=ChatOpenAI(temperature=0, model="gpt-4-1106-preview"),
    tool=PythonREPLTool(),
    verbose=True,
    agent_type=AgentType.OPENAI_FUNCTIONS,
    agent_executor_kwargs={"handle_parsing_errors": True},
)

if run the following,
agent_executor.run("set scanpy setting verbosity = 3 ")
I get
> Entering new AgentExecutor chain...

Invoking: Python_REPL with import scanpy as sc
sc.settings.verbosity = 3
print(sc.settings.verbosity)


3
The verbosity level of Scanpy has been set to 3.

> Finished chain.
The verbosity level of Scanpy has been set to 3.

but, if run the following,
pbmc = sc.datasets.pbmc68k_reduced()
agent_executor.run("use 'scanpy' library and 'pbmc' object to plot a umap")

I get,
> Entering new AgentExecutor chain...
Python REPL can execute arbitrary code. Use with caution.

Invoking: Python_REPL with import scanpy as sc



Invoking: Python_REPL with import scanpy as sc
responded: It seems there was an issue with the execution of the import statement for the 'scanpy' library. I will attempt to resolve this and proceed with the task. Let's try importing the library again.

It appears that there is an issue with importing the 'scanpy' library in this environment. Without being able to import the library, I cannot proceed with plotting a UMAP of the 'pbmc' object. If the library and the necessary data were available, I would typically load the data, preprocess it, and then use the sc.pl.umap function to plot the UMAP. However, since I cannot execute the code here, I'm unable to complete this task.
My code uses "wikipedia" to search for the relevant content. Below is the code
Load tools
tools = load_tools(
    ["wikipedia"],
    llm=llm)
agent = initialize_agent(
    tools,
    llm,
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    handle_parsing_errors=True,
    verbose=False
)
out = agent(f"Does {var_1} cause {var_2} or the other way around?.")

Instead of "wikipedia", I want to use my own pdf document that is available in my local. Can anyone help me in doing this?
I have tried using the below code
from langchain.document_loaders import PyPDFium2Loader
loader = PyPDFium2Loader("hunter-350-dual-channel.pdf")
data = loader.load()

but i am not sure how to include this in the agent.