KeyError: 'rope_type'
Author: pseudotensor
Pull request: no
Labels: name: bug

### System Info

transformers 4.31.1 is broken, 4.24.4 was ok

### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoConfig

model = 'deepseek-ai/deepseek-coder-33b-instruct'
model = 'togethercomputer/Llama-2-7B-32K-Instruct'

AutoConfig.from_pretrained(model)
```

for either model, gives now:
```
Traceback (most recent call last):
  File "/home/jon/h2ogpt/checkropetype.py", line 6, in <module>
    AutoConfig.from_pretrained(model, rope_scaling=1)
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 996, in from_pretrained
    return config_class.from_dict(config_dict, **unused_kwargs)
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/configuration_utils.py", line 772, in from_dict
    config = cls(**config_dict)
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/models/llama/configuration_llama.py", line 192, in __init__
    rope_config_validation(self)
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/modeling_rope_utils.py", line 546, in rope_config_validation
    validation_fn(config)
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/transformers/modeling_rope_utils.py", line 379, in _validate_linear_scaling_rope_parameters
    rope_type = rope_scaling["rope_type"]
KeyError: 'rope_type'
```

### Expected behavior

no errors

Comments:
Hi! Can confirm that I encountered this too, with `deepseek-coder-1.3b-base`.

I've encountered this issue both on 4.43.1 and 4.43.0.

![image](https://github.com/user-attachments/assets/f540b8ed-8870-4f9b-9d03-ce847f259c30)


It seems like the issue lies in the following sections still indexing `"rope_type"` instead of `config.rope_scaling["type"]`. 

https://github.com/huggingface/transformers/blob/c85510f958e6955d88ea1bafb4f320074bfbd0c1/src/transformers/modeling_rope_utils.py#L369-L516


Indeed we will have a patch today to fix this

Hi, I'm trying to use deepseek-ai/deepseek-coder-33b-instruct with TGI via docker, but I'm getting similar errors in the docker logs:

```python
2024-09-22 09:50:15 2024-09-22T13:50:15.480520Z  INFO download: text_generation_launcher: Successfully downloaded weights for deepseek-ai/deepseek-coder-33b-instruct
2024-09-22 09:50:15 2024-09-22T13:50:15.480769Z  INFO shard-manager: text_generation_launcher: Starting shard rank=0
2024-09-22 09:50:19 2024-09-22T13:50:19.254647Z ERROR text_generation_launcher: Error when initializing model
2024-09-22 09:50:19 Traceback (most recent call last):
2024-09-22 09:50:19   File "/opt/conda/bin/text-generation-server", line 8, in <module>
2024-09-22 09:50:19     sys.exit(app())
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/typer/main.py", line 311, in __call__
2024-09-22 09:50:19     return get_command(self)(*args, **kwargs)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
2024-09-22 09:50:19     return self.main(*args, **kwargs)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/typer/core.py", line 778, in main
2024-09-22 09:50:19     return _main(
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/typer/core.py", line 216, in _main
2024-09-22 09:50:19     rv = self.invoke(ctx)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
2024-09-22 09:50:19     return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
2024-09-22 09:50:19     return ctx.invoke(self.callback, **ctx.params)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
2024-09-22 09:50:19     return __callback(*args, **kwargs)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/typer/main.py", line 683, in wrapper
2024-09-22 09:50:19     return callback(**use_params)  # type: ignore
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/text_generation_server/cli.py", line 118, in serve
2024-09-22 09:50:19     server.serve(
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/text_generation_server/server.py", line 297, in serve
2024-09-22 09:50:19     asyncio.run(
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/asyncio/runners.py", line 44, in run
2024-09-22 09:50:19     return loop.run_until_complete(main)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
2024-09-22 09:50:19     self.run_forever()
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
2024-09-22 09:50:19     self._run_once()
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/asyncio/base_events.py", line 1909, in _run_once
2024-09-22 09:50:19     handle._run()
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/asyncio/events.py", line 80, in _run
2024-09-22 09:50:19     self._context.run(self._callback, *self._args)
2024-09-22 09:50:19 > File "/opt/conda/lib/python3.10/site-packages/text_generation_server/server.py", line 231, in serve_inner
2024-09-22 09:50:19     model = get_model(
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/text_generation_server/models/__init__.py", line 706, in get_model
2024-09-22 09:50:19     return FlashCausalLM(
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/text_generation_server/models/flash_causal_lm.py", line 878, in __init__
2024-09-22 09:50:19     config = config_class.from_pretrained(
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 996, in from_pretrained
2024-09-22 09:50:19     return config_class.from_dict(config_dict, **unused_kwargs)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/configuration_utils.py", line 772, in from_dict
2024-09-22 09:50:19     config = cls(**config_dict)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/configuration_llama.py", line 192, in __init__
2024-09-22 09:50:19     rope_config_validation(self)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_rope_utils.py", line 546, in rope_config_validation
2024-09-22 09:50:19     validation_fn(config)
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_rope_utils.py", line 379, in _validate_linear_scaling_rope_parameters
2024-09-22 09:50:19     rope_type = rope_scaling["rope_type"]
2024-09-22 09:50:19 KeyError: 'rope_type'
2024-09-22 09:50:19 2024-09-22T13:50:19.786989Z ERROR shard-manager: text_generation_launcher: Shard complete standard error output:
2024-09-22 09:50:19 
2024-09-22 09:50:19 2024-09-22 13:50:16.492 | INFO     | text_generation_server.utils.import_utils:<module>:75 - Detected system cuda
2024-09-22 09:50:19 Traceback (most recent call last):
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/bin/text-generation-server", line 8, in <module>
2024-09-22 09:50:19     sys.exit(app())
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/text_generation_server/cli.py", line 118, in serve
2024-09-22 09:50:19     server.serve(
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/text_generation_server/server.py", line 297, in serve
2024-09-22 09:50:19     asyncio.run(
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/asyncio/runners.py", line 44, in run
2024-09-22 09:50:19     return loop.run_until_complete(main)
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
2024-09-22 09:50:19     return future.result()
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/text_generation_server/server.py", line 231, in serve_inner
2024-09-22 09:50:19     model = get_model(
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/text_generation_server/models/__init__.py", line 706, in get_model
2024-09-22 09:50:19     return FlashCausalLM(
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/text_generation_server/models/flash_causal_lm.py", line 878, in __init__
2024-09-22 09:50:19     config = config_class.from_pretrained(
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 996, in from_pretrained
2024-09-22 09:50:19     return config_class.from_dict(config_dict, **unused_kwargs)
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/configuration_utils.py", line 772, in from_dict
2024-09-22 09:50:19     config = cls(**config_dict)
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/configuration_llama.py", line 192, in __init__
2024-09-22 09:50:19     rope_config_validation(self)
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_rope_utils.py", line 546, in rope_config_validation
2024-09-22 09:50:19     validation_fn(config)
2024-09-22 09:50:19 
2024-09-22 09:50:19   File "/opt/conda/lib/python3.10/site-packages/transformers/modeling_rope_utils.py", line 379, in _validate_linear_scaling_rope_parameters
2024-09-22 09:50:19     rope_type = rope_scaling["rope_type"]
2024-09-22 09:50:19 
2024-09-22 09:50:19 KeyError: 'rope_type'
2024-09-22 09:50:19  rank=0
2024-09-22 09:50:19 2024-09-22T13:50:19.886203Z ERROR text_generation_launcher: Shard 0 failed to start
2024-09-22 09:50:19 2024-09-22T13:50:19.886384Z  INFO text_generation_launcher: Shutting down shards
2024-09-22 09:50:19 Error: ShardCannotStart
```

Hey @shoang22, this is linked to text-generation-inference, not transformers; do you mind opening an issue there? Thanks!