RetrievalQA for document comparison is not working for similar type of questions.
Labels: name: area: agent, name: auto:bug

### System Info

pydantic==1.10.12
langchain==0.0.271

System: MacOS Ventura 13.5 (22G74)

Python 3.9.6

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Here is the main python file:

```
from dotenv import load_dotenv
load_dotenv()

from langchain.chat_models import ChatOpenAI
from langchain.document_loaders import PyPDFLoader
from langchain.agents import Tool
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import FAISS
from langchain.document_loaders import PyPDFLoader
from langchain.chains import RetrievalQA
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from pydantic import BaseModel, Field

class DocumentInput(BaseModel):
    question: str = Field()  

llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613")

tools = []
files = [
    {
        "name": "alphabet-earnings",
        "path": "https://abc.xyz/investor/static/pdf/2023Q1_alphabet_earnings_release.pdf",
    },
    {
        "name": "tesla-earnings",
        "path": "https://digitalassets.tesla.com/tesla-contents/image/upload/IR/TSLA-Q1-2023-Update",
    },
]

for file in files:
    print(f"Loading {file['name']} with path {file['path']}")
    loader = PyPDFLoader(file["path"])
    pages = loader.load_and_split()
    text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
    docs = text_splitter.split_documents(pages)
    embeddings = OpenAIEmbeddings()
    retriever = FAISS.from_documents(docs, embeddings).as_retriever()

    # Wrap retrievers in a Tool
    tools.append(
        Tool(
            args_schema=DocumentInput,
            name=file["name"],
            description=f"useful when you want to answer questions about {file['name']}",
            func=RetrievalQA.from_chain_type(llm=llm, retriever=retriever),
        )
    )

agent = initialize_agent(
    agent=AgentType.OPENAI_FUNCTIONS,
    tools=tools,
    llm=llm,
    verbose=True,
)

comparison = agent({"input": "which one has higher earning?"})
print(comparison)

print('-------------------------------')

comparison = agent({"input": "did alphabet or tesla have more revenue?"})
print(comparison)
```

The python version is 3.9.6 and pydantic==1.10.12 and langchain==0.0.271
Run the code using python main.py

Here is the output:

```
Loading alphabet-earnings with path https://abc.xyz/investor/static/pdf/2023Q1_alphabet_earnings_release.pdf
Loading tesla-earnings with path https://digitalassets.tesla.com/tesla-contents/image/upload/IR/TSLA-Q1-2023-Update


> Entering new AgentExecutor chain...
Which companies' earnings are you referring to?

> Finished chain.
{'input': 'which one has higher earning?', 'output': "Which companies' earnings are you referring to?"}
-------------------------------


> Entering new AgentExecutor chain...

Invoking: `alphabet-earnings` with `{'question': 'revenue'}`


{'query': 'revenue', 'result': 'The revenue for Alphabet Inc. for the quarter ended March 31, 2023, was $69,787 million.'}
Invoking: `tesla-earnings` with `{'question': 'revenue'}`


{'query': 'revenue', 'result': 'Total revenue for Q1-2023 was $23.3 billion.'}Alphabet Inc. had more revenue than Tesla. Alphabet's revenue for the quarter ended March 31, 2023, was $69,787 million, while Tesla's total revenue for Q1-2023 was $23.3 billion.

> Finished chain.
{'input': 'did alphabet or tesla have more revenue?', 'output': "Alphabet Inc. had more revenue than Tesla. Alphabet's revenue for the quarter ended March 31, 2023, was $69,787 million, while Tesla's total revenue for Q1-2023 was $23.3 billion."}
```

### Expected behavior

I expect that for the question 'which one has higher earning?' also gives a good answer just like it did when the question asked was 'did alphabet or tesla have more revenue?' instead.

I was following this guide: https://python.langchain.com/docs/integrations/toolkits/document_comparison_toolkit