[Request] Note that the 'base branch' needs to match the repository's 'default branch'
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

I have an extremely simple setup where I created a GH app following the [Github App Quickstart Guide](https://docs.github.com/en/apps/creating-github-apps/writing-code-for-a-github-app/quickstart). I then start a blank repo with the template `README.md` (just the name of the repo in a title).

The agent can successfully create a pull request from one request where I ask the agent to make an ascii art of a whale & insert a new section in the README.md, but after manually merging that pull request myself the agent keeps incorrectly finding the initial value of the `README.md` with (what I'm assuming is) an incorrect commit hash that's never existed in [my repo](https://github.com/Zak-Alford/gh-test-repo2):

```
> Entering new AgentExecutor chain...
First, I need to read the current README.md to know its contents. Then, I will update the file with the riddle. Finally, I will create a pull request.
Action: Read File
Action Input: README.md
Observation: # gh-test-repo2
Thought:The file opens with a header for the repository. Now, I will add the riddle to the 'Weekly Challenge' section.
Action: Update File
Action Input: README.md

OLD <<<<
# gh-test-repo2
>>>> OLD

NEW <<<<
# gh-test-repo2

## Weekly Challenge

Riddle: What has keys but can't open locks?
>>>> NEW
Observation: Unable to update file due to error:
409 {"message": "README.md does not match 87f560c4c6fd9eecce7374b3ac8f5517a975bf63", "documentation_url": "https://docs.github.com/rest/repos/contents#create-or-update-file-contents"}
Thought:It seems like there was a conflict when trying to update the file. The file might have been updated after I last read it. I will try to read the file again and then update it.
Action: Read File
Action Input: README.md
Observation: # gh-test-repo2
Thought:The file content hasn't changed, so I'll attempt to update it again.
Action: Update File
Action Input: README.md

OLD <<<<
# gh-test-repo2
>>>> OLD

NEW <<<<
# gh-test-repo2

## Weekly Challenge

Riddle: What has keys but can't open locks?
>>>> NEW
Observation: Unable to update file due to error:
409 {"message": "README.md does not match 87f560c4c6fd9eecce7374b3ac8f5517a975bf63", "documentation_url": "https://docs.github.com/rest/repos/contents#create-or-update-file-contents"}
Thought:There seems to be a problem with updating the file. Maybe the file was updated by someone else in the meantime, or there's an issue with the way I'm trying to update it. I'll try to create a new file with the riddle instead, and then make a pull request to merge it with the README.md.
Action: Create File
Action Input: WeeklyChallenge.md

## Weekly Challenge

Riddle: What has keys but can't open locks?
Observation: Unable to make file due to error:
404 {"message": "Not Found", "documentation_url": "https://docs.github.com/rest/repos/contents#get-repository-content"}
Thought:The creation of the file failed, perhaps the filepath or file name is incorrect. I will try to create the pull request directly and mention the changes proposed in the pull request description.
Action: Create Pull Request
Action Input: Update README.md with Weekly Challenge

In this pull request, I propose adding the following riddle to the 'Weekly Challenge' section of the README.md:

Riddle: What has keys but can't open locks?
Observation: Unable to make pull request due to error:
422 {"message": "Validation Failed", "errors": [{"resource": "PullRequest", "code": "custom", "message": "No commits between test and bot-branch"}], "documentation_url": "https://docs.github.com/rest/pulls/pulls#create-a-pull-request"}
Thought:It seems like the pull request failed because there are no commits between the test and the bot-branch. This could be because I haven't successfully updated or created any files, so there's no changes to commit. I'll try to update the README.md file again, making sure to include the entire content of the file in the OLD <<<< and >>>> OLD tags, in case that was the issue.
Action: Update File
Action Input: README.md

OLD <<<<
# gh-test-repo2
>>>> OLD

NEW <<<<
# gh-test-repo2

## Weekly Challenge

Riddle: What has keys but can't open locks?
>>>> NEW
Observation: Unable to update file due to error:
409 {"message": "README.md does not match 87f560c4c6fd9eecce7374b3ac8f5517a975bf63", "documentation_url": "https://docs.github.com/rest/repos/contents#create-or-update-file-contents"}
Thought:It seems like the error persists despite multiple attempts. There might be an issue with the GitHub repository or the API. At this point, I am unable to complete the task as instructed. 
Final Answer: I was unable to complete the task due to an error while updating the README.md file.

> Finished chain.
```

Code I'm using to initialize the GH agent:
```
import os
from langchain.agents import AgentType
from langchain.agents import initialize_agent
from langchain.agents.agent_toolkits.github.toolkit import GitHubToolkit
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI
from langchain.utilities.github import GitHubAPIWrapper
import json
import custom_tools

# List of required environment variables:
env_vars = [
    "GITHUB_APP_ID",
    "GITHUB_APP_PRIVATE_KEY",
    "GITHUB_REPOSITORY",
    "GITHUB_BRANCH",
    "GITHUB_BASE_BRANCH",
    "OPENAI_API_KEY",
]

# Check your json file for key values
with open("envvars.json", "r") as f:
    env_var_values = json.load(f)

for var in env_vars:
    # Check that each key exists. If it doesn't, set it to be "" and then complain later
    if env_var_values.get(var, "") != "": 
        os.environ[var] = env_var_values[var]
    else: # Complaint line
        raise Exception(f"The environment variable {var} was not set. You must set this value to continue.")
    
bot_branch = os.environ["GITHUB_BRANCH"]
gh_base_branch = os.environ["GITHUB_BASE_BRANCH"]
    
llm = ChatOpenAI(model="gpt-4")
github = GitHubAPIWrapper()
toolkit = GitHubToolkit.from_github_api_wrapper(github)

tools = []

# unwanted_tools = ['Get Issue','Delete File']
unwanted_tools = []
for tool in toolkit.get_tools():
    if tool.name not in unwanted_tools:
        tools.append(tool)


agent = initialize_agent(
    tools=tools, llm=llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True
)

# request = f"Draw an ascii art whale and add it once to the 'Nature' section of the README.md file. Make a pull request back to {gh_base_branch}."

request = f"Do a git pull first. Then write a riddle and it once to the 'Weekly  Challenge' section of the README.md file. Don't write the answer to the riddle. Make a pull request back to {gh_base_branch}."

agent.run(request)
```
scrubbed JSON file I'm using to grab env vars from:
```
{
    "GITHUB_APP_ID": "<my-app-id>",
    "GITHUB_APP_PRIVATE_KEY": "<my-path-to-pem>",
    "GITHUB_REPOSITORY": "Zak-Alford/gh-test-repo2",
    "GITHUB_BRANCH": "bot-branch",
    "GITHUB_BASE_BRANCH": "test",
    "OPENAI_API_KEY": "<my-openai-api-key>"
}
```

### Who can help?

@hw

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Copy my script included above and name the file
2. Run `py (or python3) <filename>.py`

### Expected behavior

The github agent should find the latest commit that it itself committed to the branch before creating a new commit with changes.