Use also bigger files as input to get answers?
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

Hello - i try vectorize a local txt-file and get an answer out of it using the following code:

```
from langchain.document_loaders import DirectoryLoader, TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.llms import CTransformers
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.prompts import PromptTemplate 
from langchain.chains import RetrievalQA
import os

### Interpret content of local files
text_loader_kwargs={'autodetect_encoding': True}
loader = TextLoader("url.txt", encoding="utf8")
documents = loader.load()

splitter = RecursiveCharacterTextSplitter(chunk_size=500,
                                          chunk_overlap=50)
texts = splitter.split_documents(documents)
embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-MiniLM-L6-v2",
    model_kwargs={'device': 'cpu'})

# create and save the local database
db = FAISS.from_documents(texts, embeddings)
db.save_local("robalf")


### prepare AI with local files
# prepare the template we will use when prompting the AI
template = """Use the following text to answer the user's question.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
Context: {context}
Question: {question}
Only return the helpful answer below and nothing else.
Helpful answer:
"""

# load the language model
fn = os.path.join("G:\DEV\TEMPDEV", "llama-2-7b-chat.ggmlv3.q8_0.bin")
llm = CTransformers(model=fn,
                    model_type='llama',
                    config={'max_new_tokens': 1024, 'temperature': 0.01})

# load the interpreted information from the local database
embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-MiniLM-L6-v2",
    model_kwargs={'device': 'cpu'})
db = FAISS.load_local("robalf", embeddings)

# prepare a version of the llm pre-loaded with the local content
retriever = db.as_retriever(search_kwargs={'k': 2})
prompt = PromptTemplate(
    template=template,
    input_variables=['context', 'question'])
qa_llm = RetrievalQA.from_chain_type(llm=llm,
                                     chain_type='stuff',
                                     retriever=retriever,
                                     return_source_documents=True,
                                     chain_type_kwargs={'prompt': prompt})

# ask the AI chat about information in our local files
prompt = "Write a maximum of 20 products produced or sold by the company in the text below. "
output = qa_llm({'query': prompt})
print(output["result"])
```

When i run this with a small url.txt everything works fine and i get an answer to the text.
But when i use a bigger url.txt i get the following error:

```
Number of tokens (513) exceeded maximum context length (512).
Number of tokens (514) exceeded maximum context length (512).
Number of tokens (515) exceeded maximum context length (512).
Number of tokens (516) exceeded maximum context length (512).
Number of tokens (517) exceeded maximum context length (512).
Number of tokens (518) exceeded maximum context length (512).
Number of tokens (519) exceeded maximum context length (512).
Number of tokens (520) exceeded maximum context length (512).
Number of tokens (521) exceeded maximum context length (512).
Number of tokens (522) exceeded maximum context length (512).
```

How can i use also bigger files as input and get answers?




### Suggestion:

How can i use also bigger files as input and get answers?