Langchain QA over large documents results in Rate limit errors
Labels: name: area: embeddings, name: area: doc loader, name: area: vector store, name: auto:bug, name: area: models

### System Info

Langchain Version: 0.0.258
Python Version 3.8.10
Ubuntu 20.04.5 LTS

We are following the instructions from the blog posted at https://python.langchain.com/docs/use_cases/question_answering/

We find that this works on small documents/directories.  However when we run it on larger data sets, we get rate limit errors as below:

```
Retrying langchain.embeddings.openai.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-text-embedding-ada-002 in organization org-RVhTeiJGcKtuLfYUSO6rLABk on tokens per min. Limit: 1000000 / min. Current: 899517 / min. Contact us through our help center at help.openai.com if you continue to have issues..
Retrying langchain.embeddings.openai.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-text-embedding-ada-002 in organization org-RVhTeiJGcKtuLfYUSO6rLABk on tokens per min. Limit: 1000000 / min. Current: 887493 / min. Contact us through our help center at help.openai.com if you continue to have issues..
```

Since we have a paid account with OpenAI we doubt we are running into any actual issues with OpenAI.  Looking at their dashboird, we see we are well under any limits.

Our full code that demonstrates this issue is posted below:

```
import os
import sys
import environ

from langchain.document_loaders import DirectoryLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma

from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI
from langchain.prompts import PromptTemplate

env = environ.Env()
environ.Env.read_env()

# Load API key and document location
OPENAI_API_KEY = env('OPENAI_API_KEY')

if OPENAI_API_KEY == "":
        print("Missing OpenAPI key")
        exit()

print("Using OpenAPI with key ["+OPENAI_API_KEY+"]")

path = sys.argv[1]
if path == "":
        print("Missing document path")
        exit()

# Document loading
loader = DirectoryLoader(path, glob="*")
data = loader.load()

# Text splitting
text_splitter = RecursiveCharacterTextSplitter(chunk_size = 500, chunk_overlap = 0)
all_splits = text_splitter.split_documents(data)

# Create retriever
vectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())

# Connect to LLM for generation

template = """Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
Use three sentences maximum and keep the answer as concise as possible.
Always say "thanks for asking!" at the end of the answer.
{context}
Question: {question}
Helpful Answer:"""
QA_CHAIN_PROMPT = PromptTemplate.from_template(template)

llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)
qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectorstore.as_retriever(),
    chain_type_kwargs={"prompt": QA_CHAIN_PROMPT}
)

# prompt loop
def get_prompt():
        print("Type 'exit' to quit")

        while True:
                prompt = input("Enter a prompt: ")

                if prompt.lower() == 'exit':
                        print('Exiting...')
                        break
                else:
                        try:
                                result = qa_chain({"query": prompt})
                                print(result["result"])
                        except Exception as e:
                                print(e)

get_prompt()
```



### Who can help?

@eyurtsev @hwchase17 @agola11

### Information

- [x] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

You can run the code posted above against any document/directory that is large... try with something over 50MB

### Expected behavior

A correct chat reply