Repository: langchain
Issue: 429 Resource Exhausted error when using gemini-1.5-pro with langchain
Labels: 🤖:bug,🔌: google,🔌: neo4j

### Checked other resources

- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that this is a bug in LangChain rather than my code.
- [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

### Example Code

``` python import json
import gradio as gr
import typing_extensions
import os

from langchain_google_genai import ChatGoogleGenerativeAI
from langchain.prompts.prompt import PromptTemplate
from langchain_community.graphs import Neo4jGraph
from langchain.chains import GraphCypherQAChain
from langchain.memory import ConversationBufferMemory

# process of getting credentials
def get_credentials():
    google_api_key = os.getenv("GOOGLE_API_KEY") # get json credentials stored as a string
    if google_api_key is None:
        raise ValueError("Provide your Google API Key")

    return google_api_key

# pass
os.environ["GOOGLE_API_KEY"]= get_credentials()

NEO4J_URI = os.getenv("NEO4J_URI")
NEO4J_USERNAME = os.getenv("NEO4J_USERNAME")
NEO4J_PASSWORD = os.getenv("NEO4J_PASSWORD")
  
CYPHER_GENERATION_TEMPLATE = """You are an expert Neo4j Cypher translator who understands the question in english and convert to Cypher strictly based on the Neo4j Schema provided and following the instructions below:
1. Generate Cypher query compatible ONLY for Neo4j Version 5
2. Do not use EXISTS, SIZE keywords in the cypher. Use alias when using the WITH keyword
3. Please do not use same variable names for different nodes and relationships in the query.
4. Use only Nodes and relationships mentioned in the schema
5. Always enclose the Cypher output inside 3 backticks
6. Always do a case-insensitive and fuzzy search for any properties related search. Eg: to search for a Company name use `toLower(c.name) contains 'neo4j'`
7. Candidate node is synonymous to Manager
8. Always use aliases to refer the node in the query
9. 'Answer' is NOT a Cypher keyword. Answer should never be used in a query.
10. Please generate only one Cypher query per question. 
11. Cypher is NOT SQL. So, do not mix and match the syntaxes.
12. Every Cypher query always starts with a MATCH keyword.
13. Always do fuzzy search for any properties related search. Eg: when the user asks for "matrix" instead of "the matrix", make sure to search for a Movie name using use `toLower(c.name) contains 'matrix'` 
Schema:
{schema}
Samples:
Question: List down 5 movies that released after the year 2000
Answer: MATCH (m:Movie)  WHERE m.released > 2000  RETURN m LIMIT 5
Question: Get all the people who acted in a movie that was released after 2010
Answer: MATCH (p:Person)-[r:ACTED_IN]->(m:Movie)  WHERE m.released > 2010  RETURN p,r,m
Question: Name the Director of the movie Apollo 13
Answer: MATCH (m:Movie)<-[:DIRECTED]-(p:Person)  WHERE toLower(m.title) contains "apollo 13"  RETURN p.name
Question: {question}
Answer: 
"""

CYPHER_GENERATION_PROMPT = PromptTemplate(
    input_variables=["schema","question"], validate_template=True, template=CYPHER_GENERATION_TEMPLATE
)

CYPHER_QA_TEMPLATE = """You are an assistant that helps to form nice and human understandable answers.
The information part contains the provided information that you must use to construct an answer.
The provided information is authoritative, you must never doubt it or try to use your internal knowledge to correct it.
Make the answer sound as a response to the question. Do not mention that you based the result on the given information.
Here are two examples:

Question: List down 5 movies that released after the year 2000
Context:[movie:The Matrix Reloaded, movie:The Matrix Revolutions, movie:Something's Gotta Give, movie:The Polar Express, movie:RescueDawn]
Helpful Answer: The Matrix Reloaded, The Matrix Revolutions, Something's Gotta Give, The Polar Express and RescueDawn are the movies released after the year 2000.

Question: Who is the director of the movie V for Vendetta
Context:[person:James Marshall]
Helpful Answer: James Marshall is the director of the movie V for Vendetta.

If the provided information is empty, say that you don't know the answer.
Final answer should be easily readable and structured.
Information:
{context}

Question: {question}
Helpful Answer:"""

CYPHER_QA_PROMPT = PromptTemplate(
    input_variables=["context", "question"], template=CYPHER_QA_TEMPLATE
)

graph = Neo4jGraph(
    url=NEO4J_URI, 
    username=NEO4J_USERNAME, 
    password=NEO4J_PASSWORD,
    enhanced_schema=True
)

chain = GraphCypherQAChain.from_llm(
    ChatGoogleGenerativeAI(model='gemini-1.5-pro', max_output_tokens=8192, temperature=0.0),
    graph=graph,
    cypher_prompt=CYPHER_GENERATION_PROMPT,
    qa_prompt=CYPHER_QA_PROMPT,
    verbose=True,
    validate_cypher=True
)

memory = ConversationBufferMemory(memory_key = "chat_history", return_messages = True)

def chat_response(input_text,history):
    try:
        return str(chain.invoke(input_text)['result'])
        
    except Exception as e:  # Catch specific exceptions or log the error
        print(f"An error occurred: {e}")
        return "I'm sorry, there was an error retrieving the information you requested."

interface = gr.ChatInterface(fn = chat_response,
                             title = "Movies Chatbot",
                             theme = "soft",
                             chatbot = gr.Chatbot(height=430),
                             undo_btn = None,
                             clear_btn = "\U0001F5D1 Clear Chat",
                             examples = ["List down 5 movies that released after the year 2000",
                                         "Get all the people who acted in a movie that was released after 2010",
                                         "Name the Director of the movie Apollo 13",
                                         "Who are the actors in the movie V for Vendetta"])

# Launch the interface
interface.launch(share=True)

```

### Error Message and Stack Trace (if applicable)

===== Application Startup at 2024-05-28 17:43:47 =====

Caching examples at: '/home/user/app/gradio_cached_examples/14'
Caching example 1/4


> Entering new GraphCypherQAChain chain...
Generated Cypher:

MATCH (m:Movie)  WHERE m.released > 2000  RETURN m LIMIT 5

Full Context:
[{'m': {'tagline': 'Free your mind', 'title': 'The Matrix Reloaded', 'released': 2003}}, {'m': {'tagline': 'Everything that has a beginning has an end', 'title': 'The Matrix Revolutions', 'released': 2003}}, {'m': {'title': "Something's Gotta Give", 'released': 2003}}, {'m': {'tagline': 'This Holiday Season… Believe', 'title': 'The Polar Express', 'released': 2004}}, {'m': {'tagline': "Based on the extraordinary true story of one man's fight for freedom", 'title': 'RescueDawn', 'released': 2006}}]

> Finished chain.
Caching example 2/4


> Entering new GraphCypherQAChain chain...
Generated Cypher:
cypher
MATCH (p:Person)-[r:ACTED_IN]->(m:Movie) WHERE m.released > 2010 RETURN p, r, m

Full Context:
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 2.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
[{'p': {'born': 1960, 'name': 'Hugo Weaving'}, 'r': ({'born': 1960, 'name': 'Hugo Weaving'}, 'ACTED_IN', {'tagline': 'Everything is connected', 'title': 'Cloud Atlas', 'released': 2012}), 'm': {'tagline': 'Everything is connected', 'title': 'Cloud Atlas', 'released': 2012}}, {'p': {'born': 1956, 'name': 'Tom Hanks'}, 'r': ({'born': 1956, 'name': 'Tom Hanks'}, 'ACTED_IN', {'tagline': 'Everything is connected', 'title': 'Cloud Atlas', 'released': 2012}), 'm': {'tagline': 'Everything is connected', 'title': 'Cloud Atlas', 'released': 2012}}, {'p': {'born': 1966, 'name': 'Halle Berry'}, 'r': ({'born': 1966, 'name': 'Halle Berry'}, 'ACTED_IN', {'tagline': 'Everything is connected', 'title': 'Cloud Atlas', 'released': 2012}), 'm': {'tagline': 'Everything is connected', 'title': 'Cloud Atlas', 'released': 2012}}, {'p': {'born': 1949, 'name': 'Jim Broadbent'}, 'r': ({'born': 1949, 'name': 'Jim Broadbent'}, 'ACTED_IN', {'tagline': 'Everything is connected', 'title': 'Cloud Atlas', 'released': 2012}), 'm': {'tagline': 'Everything is connected', 'title': 'Cloud Atlas', 'released': 2012}}]

> Finished chain.
Caching example 3/4


> Entering new GraphCypherQAChain chain...
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 2.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 4.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 8.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 16.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 32.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
Generated Cypher:
cypher
MATCH (m:Movie)<-[:DIRECTED]-(p:Person) WHERE toLower(m.title) CONTAINS 'apollo 13' RETURN p.name

Full Context:
[{'p.name': 'Ron Howard'}]

> Finished chain.
Caching example 4/4


> Entering new GraphCypherQAChain chain...
Generated Cypher:
cypher
MATCH (p:Person)-[r:ACTED_IN]->(m:Movie) WHERE toLower(m.title) contains "v for vendetta" RETURN p

Full Context:
[{'p': {'born': 1960, 'name': 'Hugo Weaving'}}, {'p': {'born': 1981, 'name': 'Natalie Portman'}}, {'p': {'born': 1946, 'name': 'Stephen Rea'}}, {'p': {'born': 1940, 'name': 'John Hurt'}}, {'p': {'born': 1967, 'name': 'Ben Miles'}}]
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 2.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 4.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 8.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 16.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..
Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 32.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..

> Finished chain.
Running on local URL:  http://0.0.0.0:7860
/usr/local/lib/python3.10/site-packages/gradio/blocks.py:2368: UserWarning: Setting share=True is not supported on Hugging Face Spaces
  warnings.warn(

To create a public link, set `share=True` in `launch()`.

### Description

I am trying to use Google Gemini 1.5 Pro API key from Google AI Studio in the above code and getting the error:
```Retrying langchain_google_genai.chat_models._chat_with_retry.<locals>._chat_with_retry in 2.0 seconds as it raised ResourceExhausted: 429 Resource has been exhausted (e.g. check quota)..```

This doesn't seem right as the API call was made only twice. I tried switching to ```gemini-1.5-flash``` and it seems to work fine. I am assuming this has something to relate to gemini 1.5 pro's implementation with langchain. Quoting one of the replies in a "somewhat" similar [issue](https://www.googlecloudcommunity.com/gc/AI-ML/Gemini-API-429-Resource-has-been-exhausted-e-g-check-quota/m-p/728855):

> My theory is that the gemini-pro-1.5-latest endpoint has some sort of other limit, that we as users can't see when using the "generativeai" python SDK. The only thing that shows up in metrics is failed API calls, but NOT limit hits. 
The way around this, I believe, would be to directly use the Vertex SDK directly, not the GenAI API.

### System Info

```
neo4j-driver
gradio
langchain==0.1.20
langchain_google_genai
langchain-community
```