mteb/FreshStackRetrieval huggingface dataset:
I am currently trying to use the Helsinki-NLP/opus-mt-en-de and de-en models. I was trying to setup a pipeline and use both as LLMChain but I keep getting the same error:
ValueError: The following `model_kwargs` are not used by the model: ['pipeline_kwargs', 'return_full_text'] (note: typos in the generate arguments will also show up in this list)

I used the following snippet to initialise both models and ran the snippet after to test the output:
def get_translation_chains():
    _de_en_translation_prompt = PromptTemplate.from_template(
        """Translate the following text from German to English:
        {text}
        """
    )

    _en_de_translation_prompt = PromptTemplate.from_template(
        """Translate the following text from English to German:
        {text}
        """
    )

    _en_to_de_tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-de")
    _en_to_de_model = AutoModelForSeq2SeqLM.from_pretrained("Helsinki-NLP/opus-mt-en-de")
    _de_to_en_tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-de-en")
    _de_to_en_model = AutoModelForSeq2SeqLM.from_pretrained("Helsinki-NLP/opus-mt-de-en")

    _en_to_de_pipeline = pipeline(
        model=_en_to_de_model,
        tokenizer=_en_to_de_tokenizer,
        task="translation",
    )

    _de_to_en_pipeline = pipeline(
        model=_de_to_en_model,
        tokenizer=_de_to_en_tokenizer,
        task="translation",
    )

    _de_to_en_llm = HuggingFacePipeline(pipeline=_de_to_en_pipeline)
    _en_to_de_llm = HuggingFacePipeline(pipeline=_en_to_de_pipeline)

    _de_to_en_chain = LLMChain(
        prompt=_de_en_translation_prompt,
        llm=_de_to_en_llm,
    )

    _en_to_de_chain = LLMChain(
        prompt=_en_de_translation_prompt,
        llm=_en_to_de_llm,
    )

    return _en_to_de_chain, _de_to_en_chain



en_to_de_chain, de_to_en_pipeline = get_translation_chains()

print(en_to_de_chain.invoke({"text": "Hello, how are you?"}))

I am fairly new to using LLMs and both the huggingface and langchain libraries and could not find anything to give me a clue on this one.
I tried to use the pipeline with only setting the task I wanted "translation_de_to_en" and the other way around as well as using "translation" only for both default and more detailed pipeline. I also tried to set the kwargs option to None and False but with no success
I am building a very simple rag application using Langchain. The problem I'm having is that when I use ChatOpenAI and ask a question. The model doesn't make any sentences when it answers, it doesn't behave like a "chatbot" unlike llama2 for example (see images below). When I switch from ChatOpenAI to llama2, I don't touch anything in my code except to comment on the model.
My data is based on openfoodfacts, which is why I ask for specific ingredients in the question.
What's the problem and what can I do to get the same result as llama2 using ChatOpenAI ?
ChatOpenAI :

Llama2:

Code :
from fastapi import FastAPI
from langchain.vectorstores import FAISS
from langchain_community.embeddings import HuggingFaceEmbeddings
from langserve import add_routes
from langchain_community.llms import Ollama
from langchain.chat_models import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableLambda, RunnablePassthrough
from langchain.embeddings import OpenAIEmbeddings
import os

os.environ["OPENAI_API_KEY"] = "SECRET"

# model = Ollama(model="llama2")
model = ChatOpenAI(temperature=0.1)

import pandas as pd
products = pd.read_csv('./data/products.csv')
vectorstore = FAISS.from_texts(
    products['text'], embedding=OpenAIEmbeddings()
)
retriever = vectorstore.as_retriever()


app = FastAPI(
    title="LangChain Server",
    version="1.0",
    description="Spin up a simple api server using Langchain's Runnable interfaces",
)

ANSWER_TEMPLATE = """Answer the question based on the following context:
{context}

Question: {question}
"""

prompt = ChatPromptTemplate.from_template(ANSWER_TEMPLATE)

chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | model
    | StrOutputParser()
)

# Adds routes to the app for using the retriever under:
# /invoke
# /batch
# /stream
add_routes(app, chain)

if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="localhost", port=8000)
I am trying to make some queries to my CSV files using Langchain and OpenAI API. I am able to run this code, but i am not sure why the results are limited to only 4 records out of 500 rows in CSV.
I tried to print after loading from csv_loader, It shows all the records, so i am doing something wrong in embeddings/vectors. Can anyone please suggest what can i try?
    csv_loader = CSVLoader(csv_file_path)
    data = csv_loader.load()


    splitter = CharacterTextSplitter(separator = "\n",
                                chunk_size=500, 
                                chunk_overlap=0,
                                length_function=len)
    documents = splitter.split_documents(data)


    embeddings = OpenAIEmbeddings()
    vectorstore = FAISS.from_documents(documents, embeddings)
    vectorstore.save_local("faiss_index_constitution")
    persisted_vectorstore = FAISS.load_local("faiss_index_constitution", embeddings, allow_dangerous_deserialization=True)
    query = "What's the sum of amount of the transactions since 1 March 2024?"

    retriever = persisted_vectorstore.as_retriever()

    chain = RetrievalQA.from_llm(llm=model, retriever=retriever, verbose=True)


    chain_input = {"query": query, "context": None}
    result = chain(chain_input)

    return result
I am working on a chat application in Langchain, Python. The idea is that user submits some pdf files that the chat model is trained on and then asks questions from the model regarding those documents. The embeddings are stored in Chromadb vector database. So effectively a RAG-based solution.
Now, both the creation and storage of embeddings are working fine and also chat is working good. However, I am storing my custom metadata to the embeddings and some ids. The code for that is given as under:
def read_docs(pdf_file):
    pdf_loader = PyPDFLoader(pdf_file)
    pdf_documents = pdf_loader.load()

    text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
    documents = text_splitter.split_documents(pdf_documents)
    
    return documents

def generate_and_store_embeddings(documents, pdf_file, user_id):
    client = chromadb.PersistentClient(path="./trained_db")
    collection = client.get_or_create_collection("PDF_Embeddings", embedding_function=embedding_functions.OpenAIEmbeddingFunction(api_key=config["OPENAI_API_KEY"], model_name=configs.EMBEDDINGS_MODEL))
    now = datetime.now()

    #custom metadata and ids I want to store along with the embeddings for each pdf
    metadata = {"source": pdf_file.filename, "user": str(user_id), 'created_at': 
                          now.strftime("%d/%m/%Y %H:%M:%S")}
    ids = [str(uuid.uuid4()) for _ in range(len(documents))]

    try:
        vectordb = Chroma.from_documents(
                    documents,         
                    embedding=OpenAIEmbeddings(openai_api_key=config["OPENAI_API_KEY"], 
                    model=configs.EMBEDDINGS_MODEL),
                    persist_directory='./trained_db',
                    collection_name = collection.name, 
                    client = client,
                    ids = ids,
                    collection_metadata = {item: value for (item, value) in metadata.items()}
                )
        vectordb.persist()
        
    except Exception as err:
        print(f"An error occured: {err=}, {type(err)=}")
        return {"answer": "An error occured while generating embeddings. Please check terminal 
                           for more details."}
    return vectordb

Now, what I want is to retrieve those ids and metadata associated with the pdf file rather than all the ids/metadata in the collection. This is so that when a user enters the pdf file to delete the embeddings of, I can retrieve the metadata and the ids of that pdf file only so that I can use those IDs to delete the embeddings of the pdf file from the collection.
I know the vectordb._collection.get() function but it will return all the IDs.
I also used this code: print(vectordb.get(where={"source": pdf_file.filename})) but it returns

{'ids': [], 'embeddings': None, 'metadatas': [], 'documents': [], 'uris': None, 'data': None}
I am implementing RAG on a Gemma-2B-it model using langchain's HuggingFaceEmbeddings and ConversationalRetrievalChain.
When running:
chat_history = []
question = "My prompt"
result = qa.invoke({"question": question, "chat_history": chat_history})


I get
    276 
    277                 if self.pipeline.task == "text-generation":
--> 278                     text = response["generated_text"]
    279                 elif self.pipeline.task == "text2text-generation":
    280                     text = response["generated_text"]

KeyError: 'generated_text'

I don't understand why this is happening. It used to work and, today, it just stopped working. I have also tried using qa.run instead of invoke but it still raises the same exception.
I have tried changing models, devices but nothing fixes it.
One can obtain a ChatGPT response to a prompt using the following example:
from openai import OpenAI

client = OpenAI()  # requires key in OPEN_AI_KEY environment variable

completion = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a poetic assistant, skilled in explaining complex programming concepts with creative flair."},
    {"role": "user", "content": "Compose a poem that explains the concept of recursion in programming."}
  ]
)

print(completion.choices[0].message.content)

How can one continue the conversation? I've seen examples saying you just add a new message to the list of messages and re-submit:
# Continue the conversation by including the initial messages and adding a new one
continued_completion = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "system", "content": "You are a poetic assistant, skilled in explaining complex programming concepts with creative flair."},
        {"role": "user", "content": "Compose a poem that explains the concept of recursion in programming."},
        {"role": "assistant", "content": initial_completion.choices[0].message.content},  # Include the initial response
        {"role": "user", "content": "Can you elaborate more on how recursion can lead to infinite loops if not properly handled?"}  # New follow-up prompt
    ]
)

But I would imagine this means processing the previous messages all over again at every new prompt, which seems quite wasteful. Is that really the only way? Isn't there a way to keep a "session" of some sort that keeps ChatGPT's internal state and just processes a newly given prompt?
The following function was working till a few days ago but now gives this error:
ValueError: Expected EmbeddingFunction._call_ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['args', 'kwargs']) Please see https://docs.trychroma.com/embeddings for details of the EmbeddingFunction interface. Please note the recent change to the EmbeddingFunction interface: https://docs.trychroma.com/migration#migration-to-0416---november-7-2023
I am not sure what changes are necessary to work with this.
` def create_chromadb(link): 
    embedding_function = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2")
    loader = TextLoader(link)
    documents = loader.load()
    
    # Split the documents into chunks (no changes needed here)
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=500)
    chunks = text_splitter.split_documents(documents)
    
    # Update for new EmbeddingFunction definition
    # D is set to the type of documents (Text in this case)
    D = Union[str, List[str]]  # Adjust based on your document format (single string or list of strings)
    embedding_function: EmbeddingFunction[D] = embedding_function
    
    # Initialize Chroma with the embedding function and persist the database
    db = Chroma.from_documents(chunks, embedding_function, ids=None, collection_name="langchain", persist_directory="./chroma_db")
    db.persist()
    print(f"Saved {len(chunks)} chunks")
    
    return db`

docs.trychroma.com/migration#migration-to-0416---november-7-2023
I am following quick start of Langchain to call open ai for LLM.
https://python.langchain.com/docs/get_started/quickstart
While running the below python code I am getting error.
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

llm = ChatOpenAI(openai_api_key="sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are world class technical documentation writer."),
    ("user", "{input}")
])

chain = prompt | llm 
chain.invoke({"input": "how can langsmith help with testing?"})
output_parser = StrOutputParser()
chain = prompt | llm | output_parser
chain.invoke({"input": "how can langsmith help with testing?"})

I am getting below error:
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Python\Sysint_NPL2SQL\.venv\Lib\site-packages\openai\_base_client.py", line 902, in request
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Python\Sysint_NPL2SQL\.venv\Lib\site-packages\openai\_base_client.py", line 902, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "c:\Python\Sysint_NPL2SQL\.venv\Lib\site-packages\openai\_base_client.py", line 993, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
PS C:\Python\Sysint_NPL2SQL>  c:; cd 'c:\Python\Sysint_NPL2SQL'; & 'c:\Python\Sysint_NPL2SQL\.venv\Scripts\python.exe' 'c:\Users\yatanveer.singh\.vscode\extensions\ms-python.debugpy-2024.2.0-win32-x64\bundled\libs\debugpy\adapter/../..\debugpy\launcher' '64396' '--' 'C:\Python\Sysint_NPL2SQL\langchaindemo.py'
PS C:\Python\Sysint_NPL2SQL>  c:; cd 'c:\Python\Sysint_NPL2SQL'; & 'c:\Python\Sysint_NPL2SQL\.venv\Scripts\python.exe' 'c:\Users\yatanveer.singh\.vscode\extensions\ms-python.debugpy-2024.2.0-win32-x64\bundled\libs\debugpy\adapter/../..\debugpy\launcher' '64413' '--' 'C:\Python\Sysint_NPL2SQL\langchaindemo.py'

Any pointer will help.
Thanks
Yatan
I am trying to use LangChain embeddings, using the following code in Google colab:
These are the installations:
pip install pypdf
pip install -q transformers einops accelerate langchain bitsandbytes
pip install install sentence_transformers
pip3 install llama-index --upgrade
pip install llama-index-llms-huggingface
huggingface-cli login
pip install -U llama-index-core llama-index-llms-openai llama-index-embeddings-openai



Then I ran this code in the google colab:
from llama_index.core import VectorStoreIndex,SimpleDirectoryReader,ServiceContext
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core.prompts.prompts import SimpleInputPrompt


# Reading pdf
documents=SimpleDirectoryReader("/content/sample_data/Data").load_data()

#Prompt
query_wrapper_prompt=SimpleInputPrompt("
{query_str}
")

import torch
llm = HuggingFaceLLM(
    context_window=4096,
    max_new_tokens=256,
    generate_kwargs={"temperature": 0.0, "do_sample": False},
    system_prompt=system_prompt,
    query_wrapper_prompt=query_wrapper_prompt,
    tokenizer_name="meta-llama/Llama-2-7b-chat-hf",
    model_name="meta-llama/Llama-2-7b-chat-hf",
    device_map="auto",
    # uncomment this if using CUDA to reduce memory usage
    model_kwargs={"torch_dtype": torch.float16 , "load_in_8bit":True}
)

# Embeddings
from langchain.embeddings.huggingface import HuggingFaceEmbeddings
from llama_index.core import ServiceContext
from llama_index.embeddings.langchain import LangchainEmbedding

embed_model=LangchainEmbedding(
    HuggingFaceEmbeddings(model_name="sentence-transformers/all-mpnet-base-v2"))



Then I got this error:
ModuleNotFoundError: No module named 'llama_index.embeddings.langchain'
I am using latest version of llama-index
Version: 0.10.26
Can someone suggest, how to resolve this error.
I use this command 'from langchain.document_loaders import TextLoader' for import TextLoader. It used to work but now it is ERROR. It shows 'Error: No module named 'pydantic_v1.class_validators'; 'pydantic_v1' is not a package'  Anyone know how to fix it ? please !!  Using Langchain ==> langchain==0.0.266
enter image description here
I have two questions:

How could I change the distance metric directly in the function similarity_search. Because by default the function similarity_search uses euclidean distance and I want e.g. cosine. Ho could I do that?

from eurelis_langchain_solr_vectorstore import Solr

embeddings_model = OpenAIEmbeddings(model="bge-small-en")

vector_store = Solr(embeddings_model, core_kwargs={
    'page_content_field': 'content',  # field containing the text content
    'vector_field': 'content_vec',    # field containing the embeddings of the text content
    'core_name': 'default',         # core name
    'url_base': 'http://localhost:8983/solr' # base url to access solr
})

# here I want to use cosine distance metric
vector_store.similarity_search("relevant question", k=5)



How could I change the distance metric directly in as_retriever?

# here I want to use cosine distance metric
retriever = vector_store.as_retriever(search_kwargs={'k': 5})
I query a collection in a zilliz milvus db like this:
documents = vector_store.similarity_search_with_score(query)

The query is successful but in line 777 of milvus.py the value result.full_length is retrieved, which is not available:
for result in res[0]:
            data = {x: result.entity.get(x) for x in output_fields}
            doc = self._parse_document(data)
            pair = (doc, result.full_length)
            ret.append(pair)

which then leads to this exception
File "/Users/tilman/LangchainCorsera/venv/lib/python3.9/site-packages/langchain_community/vectorstores/milvus.py", line 644, in similarity_search
    res = self.similarity_search_with_score(
  File "/Users/tilman/LangchainCorsera/venv/lib/python3.9/site-packages/langchain_community/vectorstores/milvus.py", line 717, in similarity_search_with_score
    res = self.similarity_search_with_score_by_vector(
  File "/Users/tilman/LangchainCorsera/venv/lib/python3.9/site-packages/langchain_community/vectorstores/milvus.py", line 777, in similarity_search_with_score_by_vector
    pair = (doc, result.full_length)
  File "/Users/tilman/LangchainCorsera/venv/lib/python3.9/site-packages/pymilvus/client/abstract.py", line 588, in __getattr__
    raise MilvusException(message=f"Field {item} is not in the hit entity")
pymilvus.exceptions.MilvusException: <MilvusException: (code=1, message=Field full_length is not in the hit entity)>

Any clues?
I am trying to build a Chat PDF application using langchain,
During this I installed all the necessary packages, but there is one issue with this chromadb, which no matter what I do, it keeps showing the error.
I installed it, ran it many times, but I keep getting this error asking to install chromadb and
here is the screenshot of the error
repo link
I tried uninstalling and installing again, GPTed, saw issues in Github but nothing seems to help me fix the issue
I have the code:
loader = PyPDFLoader(“https://arxiv.org/pdf/2303.08774.pdf”)
data = loader.load()
docs = text_splitter1.split_documents(data)
vector_search_index = “vector_index”

vector_search = MongoDBAtlasVectorSearch.from_documents(
  documents=docs,
  embedding=OpenAIEmbeddings(disallowed_special=()),
  collection=atlas_collection,
  index_name=vector_search_index,
)

query = "What were the compute requirements for training GPT 4"
results = vector_search1.similarity_search(query)
print("result: ", results)

And in results I have every time only empty array. I don't understand what I do wrong. This is the link  on the langchain documentation with examples. Information is saved normally in database, but I cannot search info in this collection.
I'm trying to hop onto the LCEL and Langserve train but I'm having a little trouble understanding a bit of the "magic" involved with accessing variables set in the pipeline's dictionary.
The variables appear to be resolvable from prompt templates. I'd like to retrieve these values in custom functions, etc. but it's not clear to me how to access them directly. Take the following contrived example which aims to return the source document as well as the answer in the response:
class ChatResponse(BaseModel):
    answer: str
    sources: List[Document]

store = FAISS.from_texts(
    ["harrison worked at kensho"], embedding=OpenAIEmbeddings()
)
retriever = store.as_retriever()
template = """Answer the question based only on the following context:
{context}

Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)
llm = ChatOpenAI()

def format_response(answer):
    sources = [] # TODO lookup source documents (key: 'context')
    return ChatResponse(answer=answer, sources=sources)

retrieval_chain = (
        {"context": retriever, "question": RunnablePassthrough()}
        | prompt
        | llm
        | StrOutputParser()
        | RunnableLambda(format_response)
)
app = FastAPI()
add_routes(app, retrieval_chain, path="/chat", input_type=str, output_type=ChatResponse)

In format_response, I've left a TODO to lookup the source documents. I'd like to retrieve the source documents from the pipeline's context key. How would I access this key that was set from the first step of the chain?
I have been doing a POC to implement RAG driven model for my AI/ML use case.
The use case is to "Find Similar and duplicate controls by comparing each ID with every other ID, Generate similarity scores and list the pairs which exceeds a threshold of 80-87 for similar controls and exceeding above 95 for duplicate controls"
The code snippet is :
loader = CSVLoader(file_path="control.csv")
data = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks = text_splitter.split_documents(data)
vectorstore = Chroma.from_documents(documents=chunks, embedding=OpenAIEmbeddings())
retriever = vectorstore.as_retriever()
template = """You are an assistant for question-answering tasks.
Use the following pieces of retrieved context to answer the question.
If you don't know the answer, just say that you don't know.
Use three sentences maximum and keep the answer concise.
Question: {question}
Context: {context}
Answer:
"""
prompt = ChatPromptTemplate.from_template(template)
llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo",verbose=True)
rag_chain = (  {"context": retriever,  "question": RunnablePassthrough()}      | prompt      | llm     | StrOutputParser()  )
query = "FInd Similar controls by comparing each ID with every other ID in the document, combining their Name and Description. Calculate similarity scores between them and list all the pairs that is exceeding a threshold of 80-87for similar controls and above 95 for duplicate controls."
rag_chain.invoke(query)
The output i got was :
1. There are a total of 6 controls formed by comparing each ID with every other ID in the document. The similarity scores between them can be calculated and pairs exceeding a threshold of 80 can be listed in the output.
2. I don't Know
My expected outcome is to print the list of Similar and Duplicate pairs from the data , it has around 3500+ data.
But i dont find to see the expected output here ? Iam not sure where am wrong. Also would like to know if i have mentioned the right prompt for the scenario.
Also, I have tried the same prompt where i have not implemented RAG , but i could proper results , it just a connection made with Langchain and OpenAI for interaction.
I would like to know where am wrong and what needs to be corrected in order to get the right expected outcome.