mteb/FreshStackRetrieval huggingface dataset:
I am new to Langchain and followed this Retrival QA - Langchain. I have a custom prompt but when I try to pass Prompt with chain_type_kwargs its throws error in pydantic StufDocumentsChain. and on removing chain_type_kwargs itt just works.
how can pass to the prompt?
error
File /usr/local/lib/python3.11/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for StuffDocumentsChain
__root__
  document_variable_name context was not found in llm_chain input_variables: ['question'] (type=value_error)

Code
import json, os

from langchain.chains import RetrievalQA
from langchain.llms import OpenAI
from langchain.document_loaders import JSONLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.chat_models import ChatOpenAI
from langchain import PromptTemplate

from pathlib import Path
from pprint import pprint




os.environ["OPENAI_API_KEY"] = "my-key"



def metadata_func(record: dict, metadata: dict) -> dict:
    metadata["drug_name"] = record["drug_name"]

    return metadata


loader = JSONLoader(
    file_path='./drugs_data_v2.json', 
    jq_schema='.drugs[]',
    content_key="data",
    metadata_func=metadata_func)

docs = loader.load()


text_splitter = CharacterTextSplitter(chunk_size=5000, chunk_overlap=200)
texts = text_splitter.split_documents(docs)


embeddings = OpenAIEmbeddings()

docsearch = Chroma.from_documents(texts, embeddings)


template = """/
example custom prommpt

Question: {question}
Answer: 
"""

PROMPT = PromptTemplate(template=template, input_variables=['question'])


qa = RetrievalQA.from_chain_type(
        llm=ChatOpenAI(
           model_name='gpt-3.5-turbo-16k'       
    ),
    chain_type="stuff",
    chain_type_kwargs={"prompt": PROMPT},
    retriever=docsearch.as_retriever(),
)

query = "What did the president say about Ketanji Brown Jackson"
qa.run(query)
I'm creating an app with the help of Langchain and OpenAI.
I'm loading my data with JSONLoader and want to store it in a vectorstore, so I can retrieve on user request to answer questions specific to my data. The Langchain docs are describing HNSWLib as a possible store for ONLY Node.js apps.
In my understanding is that NEXT is built up on top of Node.js so it can run SS javascript, so I should be able to use it. I should also mention that the JSONLoader also only works on NodeJS, which works perfectly, so I reckon it should be all set.
I've created an API route in app/api/llm/route.ts following the docs of the new Route Handlers, and also installed the hnswlib-node package.
import { NextRequest } from 'next/server';
import { OpenAI } from 'langchain/llms/openai';
import { RetrievalQAChain } from 'langchain/chains';
import { JSONLoader } from 'langchain/document_loaders/fs/json';
import { HNSWLib } from 'langchain/vectorstores/hnswlib';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import path from 'path';

// eslint-disable-next-line @typescript-eslint/no-unused-vars, no-unused-vars
export const GET = async (req: NextRequest) => {
  const apiKey = process.env.NEXT_PUBLIC_OPENAI_API_KEY;
  const model = new OpenAI({ openAIApiKey: apiKey, temperature: 0.9, modelName: 'gpt-3.5-turbo' });
  // Initialize the LLM to use to answer the question.
  const loader = new JSONLoader(path.join(process.cwd(), '/assets/surfspots.json'));
  const docs = await loader.load();

  // Create a vector store from the documents.
  const vectorStore = await HNSWLib.fromDocuments(docs, new OpenAIEmbeddings({ openAIApiKey: apiKey }));

  // Create a chain that uses the OpenAI LLM and HNSWLib vector store.
  const chain = RetrievalQAChain.fromLLM(model, vectorStore.asRetriever());
  const res = await chain.call({
    query: 'List me all of the waves I can find in Fuerteventura',
  });
  console.log({ res });
};

Which I'm calling on the front-end inside of a client-side react component.
When I'm trying to run this code, I get the following error:
Error: Please install hnswlib-node as a dependency with, e.g. `npm install -S hnswlib-node`
    at HNSWLib.imports (webpack-internal:///(sc_server)/./node_modules/langchain/dist/vectorstores/hnswlib.js:184:19)

I tried reinstalling the package, removed node_modules and reinstall everything again, search the web for answers, etc.
Anybody worked with these libraries or have any direction I could consider to debug this?
Thank you in advance!
How should I add a field to the metadata of Langchain's Documents?
For example, using the CharacterTextSplitter gives a list of Documents:
const splitter = new CharacterTextSplitter({
  separator: " ",
  chunkSize: 7,
  chunkOverlap: 3,
});
splitter.createDocuments([text]);

A document will have the following structure:
{
  "pageContent": "blablabla",
  "metadata": {
    "name": "my-file.pdf",
    "type": "application/pdf",
    "size": 12012,
    "lastModified": 1688375715518,
    "loc": { "lines": { "from": 1, "to": 3 } }
  }
}

And I want to add a field to the metadata
I have a basic chain that classifies some text based on the Common European Framework of Reference for Languages. I'm timing the difference between normal chain.apply and chain.aapply but can't get it to work.
What am I doing wrong?
import os
from time import time

import openai
from dotenv import load_dotenv, find_dotenv
from langchain.chains import LLMChain
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate

_ = load_dotenv(find_dotenv())
openai.api_key = os.getenv('OPENAI_API_KEY')

llm = ChatOpenAI(temperature=0)

prompt = ChatPromptTemplate.from_template(
    'Classify the text based on the Common European Framework of Reference '
    'for Languages (CEFR). Give a single value: {text}',
)
chain = LLMChain(llm=llm, prompt=prompt)

texts = [
    {'text': 'Hallo, ich bin 25 Jahre alt.'},
    {'text': 'Wie geht es dir?'},
    {'text': 'In meiner Freizeit, spiele ich gerne Fussball.'}
]

start = time()
res_a = chain.apply(texts)
print(res_a)
print(f"apply time taken: {time() - start:.2f} seconds")
print()

start = time()
res_aa = chain.aapply(texts)
print(res_aa)
print(f"aapply time taken: {time() - start:.2f} seconds")

Output
[{'text': 'Based on the given text "Hallo, ich bin 25 Jahre alt," it can be classified as CEFR level A1.'}, {'text': 'A2'}, {'text': 'A2'}]
apply time taken: 2.24 seconds

<coroutine object LLMChain.aapply at 0x0000025EA95BE3B0>
aapply time taken: 0.00 seconds

C:\Users\User\AppData\Local\Temp\ipykernel_13620\1566967258.py:34: RuntimeWarning: coroutine 'LLMChain.aapply' was never awaited
  res_aa = chain.aapply(texts)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
I don't understand the following behavior of Langchain recursive text splitter. Here is my code and output.
from langchain.text_splitter import RecursiveCharacterTextSplitter
r_splitter = RecursiveCharacterTextSplitter(
    chunk_size=10,
    chunk_overlap=0,
#     separators=["\n"]#, "\n", " ", ""]
)
test = """a\nbcefg\nhij\nk"""
print(len(test))
tmp = r_splitter.split_text(test)
print(tmp)

Output
13
['a\nbcefg', 'hij\nk']

As you can see, it outputs chunks of size 7 and 5 and only splits on one of the new line characters. I was expecting output to be ['a','bcefg','hij','k']
My default assumption was that the chunk_size parameter would set a ceiling on the size of the chunks/splits that come out of the split_text method, but that's clearly not right:
from langchain.text_splitter import RecursiveCharacterTextSplitter, CharacterTextSplitter

chunk_size = 6
chunk_overlap = 2

c_splitter = CharacterTextSplitter(chunk_size=chunk_size, chunk_overlap=chunk_overlap)

text = 'abcdefghijklmnopqrstuvwxyz'

c_splitter.split_text(text)

prints: ['abcdefghijklmnopqrstuvwxyz'], i.e. one single chunk that is much larger than chunk_size=6.
So I understand that it didn't split the text into chunks because it never encountered the separator. But so then the question is what is the chunk_size even doing?
I checked the documentation page for langchain.text_splitter.CharacterTextSplitter here but did not see an answer to this question. And I asked the "mendable" chat-with-langchain-docs search functionality, but got the answer "The chunk_size parameter of the CharacterTextSplitter determines the maximum number of characters in each chunk of text."...which is not true, as the code sample above shows.
I am getting an ImportError while using GPTSimpleVectorIndex from the llama-index library. Have installed the latest version of llama-index library and trying to run it on python 3.9.
from llama_index import GPTSimpleVectorIndex, SimpleDirectoryReader, LLMPredictor, PromptHelper, ServiceContext
ImportError: cannot import name 'GPTSimpleVectorIndex' from 'llama_index' (E:\Experiments\OpenAI\data anaysis\llama-index-main\venv\lib\site-packages\llama_index\__init__.py

The source code is given below,
import os, streamlit as st

from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader, LLMPredictor, PromptHelper, ServiceContext
from langchain.llms.openai import OpenAI
I am querying a text using chatGPT. But I need chatGPT to respond with single direct answers, rather than long stories or irrelevant text. Any way to achieve this?
My code looks like:
from langchain.document_loaders import TextLoader
from langchain.vectorstores import DocArrayInMemorySearch
from langchain.indexes import VectorstoreIndexCreator

loader = TextLoader("path/to/extracted_text.txt")
loaded_text = loader.load()
# Save document text as vector.
index = VectorstoreIndexCreator(
            vectorstore_cls=DocArrayInMemorySearch
        ).from_loaders([loader])

# Query the text
response = index.query("At what time did john come home yesterday?")
print("Loaded text is:", loaded_text)
print("ChatGPT response is:", response)


>>> Loaded text is: "< a really long text > + John came home last
night at 11:30pm + < a really long text >"


>>> ChatGPT response is: "John came back yesterday at 11:30pm."

The problem is that I want a concise answer 11:30pm rather than a full sentence John came home last night at 11:30pm. Is there a way to achieve this without adding "I need a short direct response" to my query? Can I achieve a more guaranteed concise response by setting a parameter through some other means instead?
I am writing a little application in JavaScript using the LangChain library. I have the following snippet:
/* LangChain Imports */
import { OpenAI } from "langchain/llms/openai";
import { BufferMemory } from "langchain/memory";
import { ConversationChain } from "langchain/chains";

// ========================================================================================= //
  // ============= Use LangChain to send request to OpenAi API =============================== //
  // ========================================================================================= //

  const openAILLMOptions = {
    modelName: chatModel.value,
    openAIApiKey: decryptedString,
    temperature: parseFloat(temperatureValue.value),
    topP: parseFloat(topP.value),
    maxTokens: parseInt(maxTokens.value),
    stop: stopSequences.value.length > 0 ? stopSequences.value : null,
    streaming: true,
};

  const model = new OpenAI(openAILLMOptions);
  const memory = new BufferMemory();
  const chain = new ConversationChain({ llm: model, memory: memory });

  try {
    const response = await chain.call({ input: content.value, signal: signal }, undefined,
      [
        {

          handleLLMNewToken(token) {
            process.stdout.write(token);
          },
        },
      ]
    );

// handle the response

}

This does not work (I tried both using the token via TypeScript and without typing). I have scoured various forums and they are either implementing streaming with Python or their solution is not relevant to this problem. So to summarize, I can successfully pull the response from OpenAI via the LangChain ConversationChain() API call, but I can’t stream the response. Is there a solution?
When I write code in VS Code, beginning with:
import os
from langchain.chains import RetrievalQA
from langchain.llms import OpenAI
from langchain.document_loaders import TextLoader

I am met with the error: ModuleNotFoundError: No module named 'langchain'
I have updated my Python to version 3.11.4, have updated pip, and reinstalled langchain. I have also checked sys.path and the folder C:\\Python311\\Lib\\site-packages in which the Langchain folder is, is appended.
EDIT: Langchain import works when I run it in the Python console (functionality works too), but when I run the code from the VSCode run button it still provides the ModuleNotFoundError.
Has anyone else run into this issue and found a solution?
I want to experiment with adding my existing qdrant vector database to langchain for a chatGPT project. However, I cannot seem to find a way to initialise the Qdrant object without providing docs and embeddings, which seems weird to me, as I should be able to simply provide my database url since the docs and embeddings already exist in the database, like when I am interacting via the qdrant python client:
QdrantClient(host=host, port=port)

In the official langchain documentation I can only find examples where I have to provide the data when loading the object, like so:
url = "<---qdrant url here --->"
qdrant = Qdrant.from_documents(
     docs,
     embeddings,
     url,
     collection_name="my_documents",
)

Their documentation also states that:

Both Qdrant.from_texts and Qdrant.from_documents methods are great to
start using Qdrant with Langchain. In the previous versions the
collection was recreated every time you called any of them. That
behaviour has changed. Currently, the collection is going to be reused
if it already exists. Setting force_recreate to True allows to remove
the old collection and start from scratch.

Which I find strange as the collection is being reused (as I want) but i still have to provide docs and embeddings.
I have also checked the qdrants official documentation on the matter, and they provide a half solution where I "only" have to provide the embeddings:
import qdrant_client

embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-mpnet-base-v2"
)

client = qdrant_client.QdrantClient(
    "<qdrant-url>",
    api_key="<qdrant-api-key>", # For Qdrant Cloud, None for local instance
)

doc_store = Qdrant(
    client=client, collection_name="texts", 
    embeddings=embeddings,
)

If anyone has a solution for this, I would be happy to receive some help.
I'm working with AzureOpenAI and langchain, constantly getting hit by PermissionError. This mostly could be due to the proxy, but can someone please check the code --
from langchain.llms import OpenAI, AzureOpenAI
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain

llm = AzureOpenAI(openai_api_type="", openai_api_base="", deployment_name="", model_name="", openai_api_key="", openai_api_version="")

template = """"
Translate the following text from {source_lang} to {dest_lang}: {source_text}
"""

prompt_name = PromptTemplate(input_variables=["source_lang", "dest_lang", "source_text"], template=template)
chain = LLMChain(llm=llm, prompt=prompt_name)

chain.predict(source_lang="English", dest_lang="Spanish", source_text="How are you?")

chain(inputs={"source_lang": "English", "dest_lang": "Spanish", "source_text": "How are you"})

I also tried the additional openai_proxy parameter without much luck.
I am trying to get a simple vector store (chromadb) to embed texts using the add_texts method with langchain, however I get the following error despite successfully using the OpenAI package with a different simple langchain scenario:
ValueError: You must provide embeddings or a function to compute them

Code:
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma

db = Chroma()

texts = [
    """
    One of the most common ways to store and search over unstructured data is to embed it and store the resulting embedding vectors, and then at query time to embed the unstructured query and retrieve the embedding vectors that are 'most similar' to the embedded query. A vector store takes care of storing embedded data and performing vector search for you.
    """,
    """
    Today's applications are required to be highly responsive and always online. To achieve low latency and high availability, instances of these applications need to be deployed in datacenters that are close to their users. Applications need to respond in real time to large changes in usage at peak hours, store ever increasing volumes of data, and make this data available to users in milliseconds.
""",

]

db.add_texts(texts, embedding_function=OpenAIEmbeddings())
Hi i am trying to do speaker diarization with open/ai whisper model.
from langchain.llms import HuggingFacePipeline
import torch
from transformers import AutoTokenizer, WhisperProcessor,AutoModelForCausalLM, pipeline, AutoModelForSeq2SeqLM

model_id = 'openai/whisper-large-v2'
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = WhisperProcessor.from_pretrained(model_id)


pipe = pipeline(
    "automatic-speech-recognition",
    model=model, 
    tokenizer=tokenizer, 
    max_length=100
)

local_llm = HuggingFacePipeline(pipeline=pipe)

The error i am getting is " AttributeError: 'WhisperProcessor' object has no attribute 'config'"
Is there anything to change from above code?
Thanks in advance
I have the following code:
chat_history = []
embeddings = OpenAIEmbeddings()
db = FAISS.from_documents(chunks, embeddings)
qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0.1), db.as_retriever())
result = qa({"question": "What is stack overflow", "chat_history": chat_history})

The code creates embeddings, creates a FAISS in-memory vector db with some text that I have in chunks array, then it creates a ConversationalRetrievalChain, followed by asking a question.
Based on what I understand from ConversationalRetrievalChain, when asked a question, it will first query the FAISS vector db, then, if it can't find anything matching, it will go to OpenAI to answer that question.  (is my understanding correct?)
How can I detect if it actually called OpenAI to get the answer or it was able to get it from the in-memory vector DB? The result object contains question, chat_history and answer properties and nothing else.
I am trying to follow various tutorials on langchain and streamlit and I have encountered many problems regarding the names of imports. My main problem is that I can't seem to import ConversationalRetrievalChain from langchain.chains. This isn't the first case of this strange issue, for example
from langchain.chains import ConversationBufferMemory

this line of code doesn't work, and returns the error: cannot import name 'ConversationBufferMemory' from 'langchain.chains' (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/langchain/chains/init.py)
However, when I write the following code
from langchain.chains.conversation.memory import ConversationBufferMemory

It works fine. It would appear as if specifying the path to the packet I want to use in the import statement is imperative for it to work.
With this in mind I was wondering if anyone had any insight as to what path ConversationalRetrievalChain was in. I tried this, but langchain.chains.conversational_retrieval doesn't exist and many other websites like the [official langchain website] (https://python.langchain.com/docs/modules/memory/conversational_customization) have only lead me more astray.
Does anyone know where ConversationalRetrievalChain is located in Langchain version 0.0.27, or how I might go about finding it myself. Many thanks :)
What I have tried in my code:

from langchain.chains import ConversationalRetrievalChain
from langchain.chains.conversation import ConversationalRetrievalChain
from langchain.chains.conversation.memory import ConversationalRetrievalChain
langchain.chains.conversational_retrieval.base import ConversationalRetrievalChain

other things:

Installing an older version of langchain (keeps saying I need python >= 3.8.1 even though I have python 3.8.9)

Where I have gone to look:

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/langchain/chains/
Langchain documentation
I'm currently working with LangChain and using the TextLoader class to load text data from a file and utilize it within a Vectorstore index. However, I've noticed that response times to my queries are increasing as my text file grows larger. To enhance performance, I'm wondering if there are ways to expedite the response times.
Sample Code:
python

import os
import time
from langchain.document_loaders import TextLoader
from langchain.indexes import VectorstoreIndexCreator
from langchain.chat_models import ChatOpenAI
import constants

os.environ["OPENAI_API_KEY"] = constants.OPENAI_API_KEY

loader = TextLoader("all_content.txt", encoding="utf-8")

# Record the start time
start_time = time.time()

index = VectorstoreIndexCreator().from_loaders([loader])

query = "My question?"
response = index.query(query).encode('utf-8').decode('utf-8')
print(response)

# Record the end time
end_time = time.time()

# Calculate the execution time
execution_time = end_time - start_time
print(f"Execution time: {execution_time:.4f} seconds")

My Questions:

Are there ways to optimize response times when using TextLoader?

Can caching be effectively employed to reduce response times? If so, how can I integrate it into my current implementation?

Are there alternative approaches or techniques I can employ to effectively shorten response times?


I've noticed that response times increase as my text file grows, and I'm actively seeking ways to enhance the performance of my queries. Any advice or suggestions for optimizing this implementation would be greatly appreciated. Thank you in advance!
read langchain docs and tried momento cache
I'm experimenting with LangChain's AgentType.CHAT_ZERO_SHOT_REACT agent. By its name I'd assume this is an agent intended for chat use and I've given it memory but it doesn't seem able to access its memory. What else do I need to do so that this will access its memory? Or have I incorrectly assumed that this agent can handle chats?
Here is my code and sample output:
llm = ChatOpenAI(model_name="gpt-4",
                 temperature=0)

tools = load_tools(["llm-math", "wolfram-alpha", "wikipedia"], llm=llm)
memory = ConversationBufferMemory(memory_key="chat_history")

agent_test = initialize_agent(
    tools=tools, 
    llm=llm, 
    agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, 
    handle_parsing_errors=True,
    memory=memory, 
    verbose=True
)

>>> agent_test.run("What is the height of the empire state building?")
'The Empire State Building stands a total of 1,454 feet tall, including its antenna.'
>>> agent_test.run("What was the last question I asked?")
"I'm sorry, but I can't provide the information you're looking for."
I want to create a chatbot based on langchain. In the first message of the conversation, I want to pass the initial context.
What is the way to do it? I'm struggling with this, because from what I see, I can use prompt template. From their examples:
template = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.

Current conversation:
{history}
Human: {input}
AI Assistant:"""
PROMPT = PromptTemplate(input_variables=["history", "input"], template=template)
conversation = ConversationChain(
    prompt=PROMPT,
    llm=llm,
    verbose=True,
    memory=ConversationBufferMemory(ai_prefix="AI Assistant"),
)

But the issue is that my usual approach to working with the models is through the use of SystemMessage, which provides context and guidance to the bot. I am unsure if this template is the recommended way for langchain to handle system messages. If not, could you please clarify the correct method?