Simirality search not working for Conversational Retrieval Agents
Labels: name: area: agent, name: area: vector store, name: auto:bug

### System Info

Hello! I'm using Conversational Retrieval Agents in a Nodejs environment with Faiss vectorStore. The problem is that the agent does not search for simurality. Vectors are normally well created and saved in a folder by Faiss.
It should be noted that the chatbot communicates well but does not take vectors into account.

Here are the snippets of my code : 

**creation and export of the chain**

`const llm = new ChatOpenAI({
  temperature: 0,
  maxTokens: 300,
  modelName: "gpt-4"
});
const memory = new OpenAIAgentTokenBufferMemory({
  llm: llm,
  memoryKey: "chat_history",
  returnMessages: true,
  inputKey:"input",
  outputKey:"output",
});

const makeChain = async (vectorstore,basePrompt) => {
  
  const retriever = vectorstore.asRetriever();

const tool = createRetrieverTool(retriever, {
  name: "search_state_of_union",
  description: "Searches and returns documents regarding the state-of-the-union.",
});
const tools = [tool]

  const chain = await initializeAgentExecutorWithOptions(tools,llm,{
    verbose: true,
    agentType: "openai-functions",
    memory:memory,
    returnIntermediateSteps: true,
    agentArgs: {
      prefix:basePrompt +` `+`Do your best to answer the questions. Feel free to use any tools available to look up relevant information, only if necessary.`,
    },
  });
  return chain;
};
`
**Using of the chain**

 ` const vectorStorePath = process.env.VECTOR_STORE_PATH || path.join(__dirname, '..', 'vectorStore');
                  const directory = path.join(vectorStorePath, chatbot.name);
                      console.log(directory);
                      const vectorStore=await FaissStore.load(
                        directory,
                        new OpenAIEmbeddings()
                      );
                      /**const resultTwo = await vectorStore.similaritySearch("nestor", 1);
                      console.log(resultTwo); */
                      // Sanitize the message
                      const sanitizedMessage = message.trim().replaceAll('\n', ' ')
                      
                      //create chain
                      const chain = await makeChain(vectorStore, chatbot.basePrompt);
                      //Ask a question using chat history
                      const response = await chain.call({
                        input: sanitizedMessage,
                        chat_history: chatHistory || []
                      });

`

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

**Create and export the chain**
`const llm = new ChatOpenAI({
  temperature: 0,
  maxTokens: 300,
  modelName: "gpt-4"
});
const memory = new OpenAIAgentTokenBufferMemory({
  llm: llm,
  memoryKey: "chat_history",
  returnMessages: true,
  inputKey:"input",
  outputKey:"output",
});

const makeChain = async (vectorstore,basePrompt) => {
  
  const retriever = vectorstore.asRetriever();

const tool = createRetrieverTool(retriever, {
  name: "search_state_of_union",
  description: "Searches and returns documents regarding the state-of-the-union.",
});
const tools = [tool]

  const chain = await initializeAgentExecutorWithOptions(tools,llm,{
    verbose: true,
    agentType: "openai-functions",
    memory:memory,
    returnIntermediateSteps: true,
    agentArgs: {
      prefix:basePrompt +` `+`Do your best to answer the questions. Feel free to use any tools available to look up relevant information, only if necessary.`,
    },
  });
  return chain;
};
`
**Using of the chain**

 ` const vectorStorePath = process.env.VECTOR_STORE_PATH || path.join(__dirname, '..', 'vectorStore');
                  const directory = path.join(vectorStorePath, chatbot.name);
                      console.log(directory);
                      const vectorStore=await FaissStore.load(
                        directory,
                        new OpenAIEmbeddings()
                      );
                      /**const resultTwo = await vectorStore.similaritySearch("nestor", 1);
                      console.log(resultTwo); */
                      // Sanitize the message
                      const sanitizedMessage = message.trim().replaceAll('\n', ' ')
                      
                      //create chain
                      const chain = await makeChain(vectorStore, chatbot.basePrompt);
                      //Ask a question using chat history
                      const response = await chain.call({
                        input: sanitizedMessage,
                        chat_history: chatHistory || []
                      });

`

### Expected behavior

Help me find a solution