Huggingface Agents Tutorial has an error when using Huggingface models
Author: mohit-raghavendra
Pull request: no

### System Info

Transformers v4.29.0

### Who can help?

@ArthurZucker 
@younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://colab.research.google.com/drive/1c7MHD-T1forUPGcC_jlwsIptOzpG3hSj

### Expected behavior

I believe it is supposed to generate the results that are in the notebook but it returns the error: 

ValueError: Error 422: {'error': 'Input validation error: `inputs` must have less than 1024 tokens. Given: 1545', 'error_type': 'validation'}

Comments:
Hey! I don't have an openAI key, could you try with the latest version of transformers and / or share the exact traceback as where this failed> 

I tied running the notebook with the main version of Transformers, with the Huggingface StarCoder model. 

I set the Huggingface token (hf_xyz) as follows:
![image](https://github.com/huggingface/transformers/assets/42749143/d917969f-2ef1-4adc-b0bb-7ba6e2349d61)

This is the error I am getting now.

![image](https://github.com/huggingface/transformers/assets/42749143/5b979014-1041-44fa-8e73-bfae84092946)


You cannot use a huggingface token for the openai API 😓 You need to also pass your openAI api token to the agent like it's done in the notebook I think

Isn't StarCoder an HF model? Do I need an OpenAI key to use even that?

Right sorry for the confusion, your colab is not defaulting on starcoder. 
Not 100% sure what's at play, maybe @Wauplin for the colab integration? 🤗 

Hi @mohit-raghavendra could you regenerate a token in https://huggingface.co/settings/tokens and save it in your colab secrets? This issue can happen if the token is outdated/revoked for some reason.

I tried it with a new token, same error. 

@mohit-raghavendra 
I had the same issue with the Error 400 token for the Huggingface StarCoder model. It was resolved by passing the token directly to the agent initialization like:

`agent = HfAgent(url_endpoint="https://api-inference.huggingface.co/models/bigcode/starcoder", token='hf_my_token')`

@Wauplin @ArthurZucker 
There is a new error with the StarCoder agent:
![image](https://github.com/huggingface/transformers/assets/54349415/dd940949-4a49-443a-846c-5bc89ac40155)

To my mind, it seems like there is a strict limitation on `max_new_tokens` [here](https://github.com/huggingface/transformers/blob/a7cab3c283312b8d4de5df3bbe719971e24f4281/src/transformers/tools/agents.py#L640C40-L640C40)

Thanks for the workaround @dashapetr 

Regarding the other issue, could you open in a separate issue so that we can discuss it further? It is just to avoid mixing topics. Thanks in advance!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.