mteb/FreshStackRetrieval huggingface dataset:
I am experimenting with langchains and its applications, but as a newbie, I could not understand how the embeddings and indexing really work together here. I know what these two are, but I can't figure out a way to use the index that I created and saved using persist_directory.
I succesfully saved the object created by VectorstoreIndexCreator using the following code:
index = VectorstoreIndexCreator(vectorstore_kwargs={"persist_directory":"./custom_save_dir_path"}).from_loaders([loader])

but I cannot find a way to use the .pkl files created. How can I use these files in my chain to retrieve data?
Also, how does the billing in openAI work? If I cannot use any saved embeddings or index, will it re-embed all the data every time I run the code?
As a beginner, I am still learning my way around and any assistance would be greatly appreciated.
Here is the full code:
from langchain.document_loaders import CSVLoader
from langchain.indexes import VectorstoreIndexCreator
from langchain.chains import RetrievalQA
from langchain.llms import OpenAI
import os
os.environ["OPENAI_API_KEY"] = "sk-xxx"
# Load the documents
loader = CSVLoader(file_path='data/data.csv')

#creates an object with vectorstoreindexcreator
index = VectorstoreIndexCreator(vectorstore_kwargs={"persist_directory":"./custom_save_dir_path"}).from_loaders([loader])

# Create a question-answering chain using the index
chain = RetrievalQA.from_chain_type(llm=OpenAI(), chain_type="stuff", retriever=index.vectorstore.as_retriever(), input_key="question")

# Pass a query to the chain
while True:
    query = input("query: ")
    response = chain({"question": query})
    print(response['result'])
I'm using langchain to process a whole bunch of documents which are in an Mongo database.
I can load all documents fine into the chromadb vector storage using langchain. Nothing fancy being done here. This is my code:

from langchain.embeddings.openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()

from langchain.vectorstores import Chroma
db = Chroma.from_documents(docs, embeddings, persist_directory='db')
db.persist()


Now, after storing the data, I want to get a list of all the documents and embeddings WITH id's.
This is so I can store them back into MongoDb.
I also want to put them through Bertopic to get the topic categories.
Question 1 is: how do I get all documents I've just stored in the Chroma database? I want the documents, and all the metadata.
Many thanks for your help!
I am trying to query a stack of word documents using langchain, yet I get the following traceback.
May I ask what's the argument that's expected here?
Also, side question, is there a way to do such a query locally (without internet access and openai)?
Traceback:
Traceback (most recent call last):

  File C:\Program Files\Spyder\pkgs\spyder_kernels\py3compat.py:356 in compat_exec
    exec(code, globals, locals)

  File c:\data\langchain\langchaintest.py:44
    index = VectorstoreIndexCreator().from_loaders(loaders)

  File ~\AppData\Roaming\Python\Python38\site-packages\langchain\indexes\vectorstore.py:72 in from_loaders
    docs.extend(loader.load())

  File ~\AppData\Roaming\Python\Python38\site-packages\langchain\document_loaders\text.py:17 in load
    with open(self.file_path, encoding=self.encoding) as f:

OSError: [Errno 22] Invalid argument:

... where "invalid argument: " is followed by the raw text from the word document.
Code:
import os
os.environ["OPENAI_API_KEY"] = "xxxxxx"


import os
import docx
from langchain.document_loaders import TextLoader

# Function to get text from a docx file
def get_text_from_docx(file_path):
    doc = docx.Document(file_path)
    full_text = []
    for paragraph in doc.paragraphs:
        full_text.append(paragraph.text)
    
    return '\n'.join(full_text)

# Load multiple Word documents
folder_path = 'C:/Data/langchain'
word_files = [os.path.join(folder_path, file) for file in os.listdir(folder_path) if file.endswith('.docx')]

loaders = []
for word_file in word_files:
    text = get_text_from_docx(word_file)
    loader = TextLoader(text)
    loaders.append(loader)
    
    
from langchain.indexes import VectorstoreIndexCreator

index = VectorstoreIndexCreator().from_loaders(loaders)

query = "What are the main points discussed in the documents?"

responses = index.query(query)
print(responses)

results_with_source=index.query_with_sources(query)
print(results_with_source)
Getting the error while trying to run a langchain code.
ValueError: `run` not supported when there is not exactly one input key, got ['question', 'documents'].
Traceback:
File "c:\users\aviparna.biswas\appdata\local\programs\python\python37\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
File "D:\Python Projects\POC\Radium\Ana\app.py", line 49, in <module>
    answer = question_chain.run(formatted_prompt)
File "c:\users\aviparna.biswas\appdata\local\programs\python\python37\lib\site-packages\langchain\chains\base.py", line 106, in run
    f"`run` not supported when there is not exactly one input key, got ['question', 'documents']."

My code is as follows.
import os
from apikey import apikey

import streamlit as st
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain, SequentialChain
#from langchain.memory import ConversationBufferMemory
from docx import Document

os.environ['OPENAI_API_KEY'] = apikey

# App framework
st.title(' Colab Ana Answering Bot..')
prompt = st.text_input('Plug in your question here')


# Upload multiple documents
uploaded_files = st.file_uploader("Choose your documents (docx files)", accept_multiple_files=True, type=['docx'])
document_text = ""

# Read and combine Word documents
def read_docx(file):
    doc = Document(file)
    full_text = []
    for paragraph in doc.paragraphs:
        full_text.append(paragraph.text)
    return '\n'.join(full_text)

for file in uploaded_files:
    document_text += read_docx(file) + "\n\n"

with st.expander('Contextual Prompt'):
    st.write(document_text)

# Prompt template
question_template = PromptTemplate(
    input_variables=['question', 'documents'],
    template='Given the following documents: {documents}. Answer the question: {question}'
)

# Llms
llm = OpenAI(temperature=0.9)
question_chain = LLMChain(llm=llm, prompt=question_template, verbose=True, output_key='answer')

# Show answer if there's a prompt and documents are uploaded
if prompt and document_text:
    formatted_prompt = question_template.format(question=prompt, documents=document_text)
    answer = question_chain.run(formatted_prompt)
    st.write(answer['answer'])

I have gone through the documentations and even then I am getting the same error. I have already seen demos where multiple prompts are being taken by langchain.
How do i add memory to RetrievalQA.from_chain_type? or, how do I add a custom prompt to ConversationalRetrievalChain?
For the past 2 weeks ive been trying to make a chatbot that can chat over documents (so not in just a semantic search/qa so with memory) but also with a custom prompt. I've tried every combination of all the chains and so far the closest I've gotten is ConversationalRetrievalChain, but without custom prompts, and RetrievalQA.from_chain_type but without memory
I am was trying to figure out a way to use StructuredTool as a multi-input Tool and used from an Agent; for example, an ZeroShotAgent.
Sadly, looks like all the agents are defined to use Tool[] instead of StructuredTool[ ] or ObjectTool[ ]. Is there any way I can do that?
This is my code, the final Tool (QuerySpecificFieldSupabaseTool) is an StructuredTool which is incompatible with the Agents.
export class SupabaseToolkit extends Toolkit {
    tools: Tool[];
    cli: SupabaseClient
    dialect = "supabase"

    constructor(cli: SupabaseClient) {
        super()
        this.cli = cli
        this.tools = [
            new QuerySupabaseTool(cli),
            new ListTablesSupabaseTool(cli),
            new ListFieldsSupabaseTool(cli),
            new QuerySpecificFieldSupabaseTool(cli),
        ]
    }
}
I'm working with langchain and ChromaDb using python.
Now, I know how to use document loaders. For instance, the below loads a bunch of documents into ChromaDb:
from langchain.embeddings.openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()

from langchain.vectorstores import Chroma
db = Chroma.from_documents(docs, embeddings, persist_directory='db')
db.persist()

But what if I wanted to add a single document at a time? More specifically, I want to check if a document exists before I add it. This is so I don't keep adding duplicates.
If a document does not exist, only then do I want to get embeddings and add it.
How do I do this using langchain? I think I mostly understand langchain but have no idea how to do seemingly basic tasks like this.
I am using Python Flask app for chat over data. In the console I am getting streamable response directly from the OpenAI since I can enable streming with a flag streaming=True.
The problem is, that I can't "forward" the stream or "show" the strem than in my API call.
Code for the processing OpenAI and chain is:
def askQuestion(self, collection_id, question):
    collection_name = "collection-" + str(collection_id)
    self.llm = ChatOpenAI(model_name=self.model_name, temperature=self.temperature, openai_api_key=os.environ.get('OPENAI_API_KEY'), streaming=True, callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]))
    self.memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True,  output_key='answer')
    
    chroma_Vectorstore = Chroma(collection_name=collection_name, embedding_function=self.embeddingsOpenAi, client=self.chroma_client)


    self.chain = ConversationalRetrievalChain.from_llm(self.llm, chroma_Vectorstore.as_retriever(similarity_search_with_score=True),
                                                        return_source_documents=True,verbose=VERBOSE, 
                                                        memory=self.memory)
    

    result = self.chain({"question": question})
    
    res_dict = {
        "answer": result["answer"],
    }

    res_dict["source_documents"] = []

    for source in result["source_documents"]:
        res_dict["source_documents"].append({
            "page_content": source.page_content,
            "metadata":  source.metadata
        })

    return res_dict

and the API route code:
@app.route("/collection/<int:collection_id>/ask_question", methods=["POST"])
def ask_question(collection_id):
    question = request.form["question"]
    # response_generator = document_thread.askQuestion(collection_id, question)
    # return jsonify(response_generator)

    def stream(question):
        completion = document_thread.askQuestion(collection_id, question)
        for line in completion['answer']:
            yield line

    return app.response_class(stream_with_context(stream(question)))

I am testing my endpoint with curl and I am passing flag -N to curl, so I should get the streamable response, if it is possible.
When I make API call first the endpoint is waiting to process the data (I can see in my terminal in VS code the streamable answer) and when finished, I get everything displayed in one go.
I'm learning Langchain and vector databases.
Following the original documentation I can read some docs, update the database and then make a query.
https://python.langchain.com/en/harrison-docs-refactor-3-24/modules/indexes/vectorstores/examples/pinecone.html
I want to access the same index and query it again, but without re-loading the embeddings and adding the vectors again to the ddbb.
How can I generate the same docsearch object without creating new vectors?
# Load source Word doc
loader = UnstructuredWordDocumentLoader("C:/Users/ELECTROPC/utilities/openai/data_test.docx", mode="elements")
data = loader.load()

# Text splitting
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
texts = text_splitter.split_documents(data)

# Upsert vectors to Pinecone Index
pinecone.init(
    api_key=PINECONE_API_KEY,  # find at app.pinecone.io
    environment=PINECONE_API_ENV
)
index_name = "mlqai"
embeddings = OpenAIEmbeddings(openai_api_key=os.environ['OPENAI_API_KEY'])

docsearch = Pinecone.from_texts([t.page_content for t in texts], embeddings, index_name=index_name)


# Query
llm = OpenAI(temperature=0, openai_api_key=os.environ['OPENAI_API_KEY'])
chain = load_qa_chain(llm, chain_type="stuff")

query = "que sabes de los patinetes?"
docs = docsearch.similarity_search(query)
answer = chain.run(input_documents=docs, question=query)
print(answer)
I follow a YouTube LangChain tutorial where it teaches Create Your Own ChatGPT with PDF Data in 5 Minutes (LangChain Tutorial) and here is the colab notebook link provided by the author for his work below the video description. I didn't modify a lot of his codes where I just changed the OpenAPI key with my one (not free plan).


Can I know why I got this error as shown in the diagram above when I try to run the code in the cell?
I expect the FAISS vector database can be created.
Im using a conversational agent, with some tools, one of them is a calculator tool (for the sake of example).
Agent initializated as follows:
conversational_agent = initialize_agent(
        agent='chat-conversational-react-description',
        tools=[CalculatorTool()],
        llm=llm_gpt4,
        verbose=True,
        max_iterations=2,
        early_stopping_method="generate",
        memory=memory,
        # agent_kwargs=dict(output_parser=output_parser),
    )


When the CalculatorTool is being activated, it will return a string output, the agent takes that output and process it further to get to the "Final Answer" thus changing the formatting of the output from the CalculatorTool
For example, for input 10*10, the tool run() function will return 100, which will be propagated back to the agent, that will call self._take_next_step() and continue processing the output.
It will create a final output similar the result of your prompt of 10x10 is 100
I dont want the added formatting by the LLM, just the output of 100.
I want to break the chain when the CalculatorTool is done, and have it's output returned to the client as is.
I also have have tools that return serialized data, for a graph chart, having that data re-processed by next iterations of the agent will make it invalid.
I'm using langchain library to save the information of my company in a Vector Database, and when I query for information the results are great, but need a way to recover where the information are comming too - like source: "www.site.com/about" or at least "document 156". Do any of you know how to do that?
EDIT: Currently, I'm using docsearch.similarity_search(query), what only return me the page_content, but metadata came empty
I'm ingesting with this code, but I'm totally open to change.
db = ElasticVectorSearch.from_documents(
        documents,
        embeddings,
        elasticsearch_url="http://localhost:9200",
        index_name="elastic-index",
    )
I can see everything but the Embedding of the documents when I used Chroma with Langchain and OpenAI embeddings. It always show me None for that
Here is the code:
for db_collection_name in tqdm(["class1-sub2-chap3", "class2-sub3-chap4"]):
    documents = []
    doc_ids = []

    for doc_index in range(3):
        cl, sub, chap = db_collection_name.split("-")
        content = f"This is {db_collection_name}-doc{doc_index}"
        doc = Document(page_content=content, metadata={"chunk_num": doc_index, "chapter":chap, "class":cl, "subject":sub})
        documents.append(doc)
        doc_ids.append(str(doc_index))


    # # Initialize a Chroma instance with the original document
    db = Chroma.from_documents(
         collection_name=db_collection_name,
         documents=documents, ids=doc_ids,
         embedding=embeddings, 
         persist_directory="./data")
    
     db.persist()

when I do db.get(), I see everything as expected except embedding is None.
{'ids': ['0', '1', '2'],
 'embeddings': None,
 'documents': ['This is class1-sub2-chap3-doc0',
  'This is class1-sub2-chap3-doc1',
  'This is class1-sub2-chap3-doc2'],
 'metadatas': [{'chunk_num': 0,
   'chapter': 'chap3',
   'class': 'class1',
   'subject': 'sub2'},
  {'chunk_num': 1, 'chapter': 'chap3', 'class': 'class1', 'subject': 'sub2'},
  {'chunk_num': 2, 'chapter': 'chap3', 'class': 'class1', 'subject': 'sub2'}]}

My embeddings is also working fine as it returns:
len(embeddings.embed_documents(["EMBED THIS"])[0])
>> 1536

also, in my ./data directory I have Embedding file as chroma-embeddings.parquet

I tried the example with example given in document but it shows None too
# Import Document class
from langchain.docstore.document import Document

# Initial document content and id
initial_content = "This is an initial document content"
document_id = "doc1"

# Create an instance of Document with initial content and metadata
original_doc = Document(page_content=initial_content, metadata={"page": "0"})

# Initialize a Chroma instance with the original document
new_db = Chroma.from_documents(
    collection_name="test_collection",
    documents=[original_doc],
    embedding=OpenAIEmbeddings(),  # using the same embeddings as before
    ids=[document_id],
)

Here also new_db.get() gives me None
from langchain.schema import BaseMemory

class ChatMemory(BaseMemory):
   def __init__(self, user_id: UUID, type: str):
    self.user_id = user_id
    self.type = type

   # implemented abstract methods

class AnotherMem(ChatMemory):
    def __init__(self, user_id: UUID, type: str):
        super().__init__(user_id, type)

This seems simple enough - but I get an error: ValueError: "AnotherMem" object has no field "user_id". What am I doing wrong?
Note that BaseMemory is an interface.
It's not possible to pass long documents to ChatGPT directly due to its limited context size. So for example question answering or summarization of long documents is not possible at first sight. I've learned how ChatGPT can in principle "know" larger contexts -- basically by summarizing a sequence of previous contexts from the chat history -- but will this suffice to detect really long-range dependencies (bearing "meaning") inside really long texts?
LangChain seems to offer an solution, making use of OpenAI's API and  vectorstores. I'm looking for a high-level description what's going on when LangChain makes accessible long documents or even corpora of long documents to ChatGPT and then makes use of ChatGPT's NLP abilities by clever automated prompting, e.g. question answering or summarization. Let's assume that the documents are already formatted as LangChain Document objects.
I am working with LangChain for the first time. Due to data security, I want to be sure about the storage of  langchain's vector store storage. I am using HNSWLib vector store, which mentions it is an in-memory store. What does it mean? Does Langchain/vector stores store any data in its servers?
https://js.langchain.com/docs/modules/indexes/vector_stores/integrations/hnswlib
https://github.com/nmslib/hnswlib
I tried executing a langchain agent. I want to save the output from verbose into a variable, but all I can access from the agent.run is only the final answer.
How can I save the verbose output to a variable so that I can use later?
My code:
import json
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.llms import OpenAI
from langchain.agents import Tool
from langchain.utilities import PythonREPL

llm = OpenAI(temperature=0.1)

## Define Tools
python_repl = PythonREPL()

tools = load_tools(["python_repl", "llm-math"], llm=llm)

agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)

response = agent.run("What is 3^2. Use calculator to solve.")

I tried accessing the response from the agent, but it's only the final answer instead of the verbose output.
printing response gives only 9. But I would like the verbose process like:
> Entering new AgentExecutor chain...
 I need to use the calculator to solve this.
Action: Calculator
Action Input: 3^2
Observation: Answer: 9
Thought: I now know the final answer.
Final Answer: 9
Whats the recommended way to define an output schema for a nested json, the method I use doesn't feel ideal.
# adding to planner -> from langchain.experimental.plan_and_execute import load_chat_planner

refinement_response_schemas = [
        ResponseSchema(name="plan", description="""{'1': {'step': '','tools': [],'data_sources': [],'sub_steps_needed': bool},
 '2': {'step': '','tools': [<empty list>],'data_sources': [<>], 'sub_steps_needed': bool},}"""),] #define json schema in description, works but doesn't feel proper
    
refinement_output_parser = StructuredOutputParser.from_response_schemas(refinement_response_schemas)
refinement_format_instructions = refinement_output_parser.get_format_instructions()

refinement_output_parser.parse(output)

gives:
{'plan': {'1': {'step': 'Identify the top 5 strikers in La Liga',
   'tools': [],
   'data_sources': ['sports websites', 'official league statistics'],
   'sub_steps_needed': False},
  '2': {'step': 'Identify the top 5 strikers in the Premier League',
   'tools': [],
   'data_sources': ['sports websites', 'official league statistics'],
   'sub_steps_needed': False},
    ...
  '6': {'step': 'Given the above steps taken, please respond to the users original question',
   'tools': [],
   'data_sources': [],
   'sub_steps_needed': False}}}

it works but I want to know if theres a better way to go about this.
I have a simple Langchain chatbot using GPT4ALL that's being run in a singleton class within my Django server.
Here's the simple code:
gpt4all_path = './models/gpt4all_converted.bin'
llama_path = './models/ggml_model_q4_0.bin'

embeddings = LlamaCppEmbeddings(model_path=llama_path)

print("Initializing Index...")
vectordb = FAISS.from_documents(docs, embeddings)
print("Initialzied Index!!!")

This code runs fine when used inside the manage.py shell separately but the class instantiation fails to create a FAISS index with the same code. It keeps printing the llama_print_timings   43000ms with the ms increasing on every print message.
Can someone help me out?
I use the following line to add langchain documents to a chroma database: Chroma.from_documents(docs, embeddings, ids=ids, persist_directory='db')
when ids are duplicates, I get this error: chromadb.errors.IDAlreadyExistsError
how do I catch the error? (duplicate ids are expected - I expect Chorma to not add them)
I've tried identifying the error in langchain documentation. Not sure how to catch it.