There is a bug when using the neftune_noise_alpha feature in transformers v4.35.0.
Author: CaC033
Pull request: no

### System Info

- `transformers` version: 4.35.0
- Platform: Linux-4.19.91-014.kangaroo.alios7.x86_64-x86_64-with-glibc2.17
- Python version: 3.8.13
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=lm_datasets["train"],
        eval_dataset=lm_datasets["validation"],
        tokenizer=tokenizer,
        data_collator=default_data_collator,
        neftune_noise_alpha=0.1,
    )

### Expected behavior

[INFO|modeling_utils.py:3118] 2023-11-06 09:56:03,877 >> loading weights file /mnt/workspace/peipao/jichunengli/Qwen-14B-Chat/model.safetensors.index.json
[INFO|configuration_utils.py:791] 2023-11-06 09:56:03,879 >> Generate config GenerationConfig {}

Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:18<00:00,  1.24s/it]
Traceback (most recent call last):
  File "/mnt/workspace/peipao/jichunengli/test_nefqwen_hf/ds_train_huggingface_llama.py", line 325, in <module>
    main()
  File "/mnt/workspace/peipao/jichunengli/test_nefqwen_hf/ds_train_huggingface_llama.py", line 290, in main
    model.gradient_checkpointing_enable()
  File "/opt/conda/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1872, in gradient_checkpointing_enable
    self._set_gradient_checkpointing(enable=True, gradient_checkpointing_func=gradient_checkpointing_func)
TypeError: _set_gradient_checkpointing() got an unexpected keyword argument 'enable'
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:20<00:00,  1.34s/it]
Traceback (most recent call last):
  File "/mnt/workspace/peipao/jichunengli/test_nefqwen_hf/ds_train_huggingface_llama.py", line 325, in <module>
    main()
  File "/mnt/workspace/peipao/jichunengli/test_nefqwen_hf/ds_train_huggingface_llama.py", line 290, in main
    model.gradient_checkpointing_enable()
  File "/opt/conda/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1872, in gradient_checkpointing_enable
    self._set_gradient_checkpointing(enable=True, gradient_checkpointing_func=gradient_checkpointing_func)
TypeError: _set_gradient_checkpointing() got an unexpected keyword argument 'enable'
Loading checkpoint shards:  80%|███████████████████████████████████████████████████████████████████████████████████████▏                     | 12/15 [00:23<00:04,  1.59s/it]WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2378 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2379 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2382 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2383 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2384 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 2385 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 2 (pid: 2380) of binary: /opt/conda/bin/python3.8
Traceback (most recent call last):
  File "/opt/conda/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/run.py", line 794, in main
    run(args)
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/run.py", line 785, in run
    elastic_launch(
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
/mnt/workspace/peipao/jichunengli/test_nefqwen_hf/ds_train_huggingface_llama.py FAILED
------------------------------------------------------------

Comments:
cc @muellerz @pacman100 

Hello @amyeroberts , tagging @younesbelkada would added the feature. Untagging myself.

The error is 

```bash
TypeError: _set_gradient_checkpointing() got an unexpected keyword argument 'enable'
```

This is because you are using a model that uses code on the Hub feature that used the old gradient checkpointing logic that has been deprecated. We do not support it again thanks to https://github.com/huggingface/transformers/pull/27610. Please refer to my comment in https://github.com/huggingface/transformers/pull/27610#issue-2002672467 for more details. 

Closing this issue as the issue is unrelated to Neftune, the issue is also fixed if you switch to transformers main

```bash
pip install -U git+https://github.com/huggingface/transformers.git
```