AttributeError: 'InternLMTokenizer' object has no attribute 'sp_model'
Author: KnutJaegersberg
Pull request: no

### System Info

- `transformers` version: 4.34.0.dev0
- Platform: Linux-5.10.194-1-MANJARO-x86_64-with-glibc2.38
- Python version: 3.9.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): 2.10.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.6.9 (cpu)
- Jax version: 0.4.8
- JaxLib version: 0.4.7
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


### Who can help?

@abhishekkrthakur

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Update autotrain-advanced and run autotrain-setup
2. Try to fine tune the new internlm-20b with peft in 8 bit
3. Using the command line tool
![image](https://github.com/huggingface/transformers/assets/17965169/e75b026f-9773-4c3d-b90d-ac106a264a48)


### Expected behavior

Fine tune the model. 

Comments:
same error

Hey! You are probably using `main` which introduced #23909. The InternLM repo relies on `"transformers_version": "4.33.1",` as you can see from the `config.json`! Don't worry, internLM will be add to transformers super soon see #26302 

Thank you. After installing 4.33.1, I get another exception. Perhaps it is because autotrain setup updates to the newest versions? I already tried an older peft version, but that did not help. There seems to be a new target_modules argument in autotrain advanced. Wonder how to use that. 


> ERROR   train has failed due to an exception:
> ERROR   Traceback (most recent call last):
  File "/home/knut/transformers/lib/python3.9/site-packages/autotrain/utils.py", line 280, in wrapper
    return func(*args, **kwargs)
  File "/home/knut/transformers/lib/python3.9/site-packages/autotrain/trainers/clm/__main__.py", line 152, in train
    model = get_peft_model(model, peft_config)
  File "/home/knut/transformers/lib/python3.9/site-packages/peft/mapping.py", line 98, in get_peft_model
    return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name)
  File "/home/knut/transformers/lib/python3.9/site-packages/peft/peft_model.py", line 893, in __init__
    super().__init__(model, peft_config, adapter_name)
  File "/home/knut/transformers/lib/python3.9/site-packages/peft/peft_model.py", line 112, in __init__
    self.base_model = PEFT_TYPE_TO_MODEL_MAPPING[peft_config.peft_type](
  File "/home/knut/transformers/lib/python3.9/site-packages/peft/tuners/lora.py", line 180, in __init__
    self.add_adapter(adapter_name, self.peft_config[adapter_name])
  File "/home/knut/transformers/lib/python3.9/site-packages/peft/tuners/lora.py", line 192, in add_adapter
    config = self._prepare_lora_config(config, model_config)
  File "/home/knut/transformers/lib/python3.9/site-packages/peft/tuners/lora.py", line 434, in _prepare_lora_config
    raise ValueError("Please specify `target_modules` in `peft_config`")
ValueError: Please specify `target_modules` in `peft_config`

check init function of InternLMTokenizer, then, move super init code to the end of the function.

It works! I used the pull request and moved the init function. Also manually specified the target modules, didn't know how to do that before. I guess I just have to grab the linear layers from print(model)

> Hey! You are probably using `main` which introduced #23909. The InternLM repo relies on `"transformers_version": "4.33.1",` as you can see from the `config.json`! Don't worry, internLM will be add to transformers super soon see #26302

Any chance you could give me a hint as to what changes I need to make to InternLM to make it work with main branch? Sorry to be greedy but I want the best of both worlds!

Hi @jph00! We're working on making proper library ports of InternLM (by converting them to LLaMA checkpoints, the model their code is based on). These should be available in a couple of days.

If you really want to get it working on `main` right now, though, the underlying problem is caused by the base tokenizer `__init__()` looking for `self.sp_model` before the child class has created that attribute. Moving the call to `super().__init__()` to a line after the creation of `self.sp_model` in `tokenization_internlm.py` should resolve the issue.

Message ID: ***@***.***>Awesome - thanks so much I'll try that. :D

the port internlm as llama branch has been deleted. 
I'm not sure what the 'canon' way to use or fine tune in autotrain-advanced this model is now. 
Got newest transformers version, but I seem to have to use the adjusted tokenization_internlm file in the folder of internlm. I tried to fine tune the model before but it kept stopping at 11 %, weirdly merging the model and acting as if it finished, but it could never have been as fast as that. 

This is my trainer_state.json. I have the feeling it does not count the epochs i a right way. 
The model that comes out is not useless, also those from earlier runs. It stops prematurely it seems. the run I share first apparently almost finished, it had a larger batch size, but also a smaller dataset. It's like your training has to outspeed the bug. ;) 

{
  "best_metric": null,
  "best_model_checkpoint": null,
  "epoch": 4.5464684014869885,
  "eval_steps": 500,
  "global_step": 735,
  "is_hyper_param_search": false,
  "is_local_process_zero": true,
  "is_world_process_zero": true,
  "log_history": [
    {
      "epoch": 0.2,
      "learning_rate": 1.1777777777777778e-05,
      "loss": 1.6026,
      "step": 53
    },
    {
      "epoch": 0.39,
      "learning_rate": 2.3555555555555556e-05,
      "loss": 1.4208,
      "step": 106
    },
    {
      "epoch": 1.04,
      "learning_rate": 2.9404958677685954e-05,
      "loss": 1.392,
      "step": 159
    },
    {
      "epoch": 1.24,
      "learning_rate": 2.8090909090909092e-05,
      "loss": 1.2228,
      "step": 212
    },
    {
      "epoch": 1.44,
      "learning_rate": 2.6776859504132234e-05,
      "loss": 1.3286,
      "step": 265
    },
    {
      "epoch": 2.09,
      "learning_rate": 2.546280991735537e-05,
      "loss": 1.2465,
      "step": 318
    },
    {
      "epoch": 2.29,
      "learning_rate": 2.4148760330578513e-05,
      "loss": 1.1794,
      "step": 371
    },
    {
      "epoch": 2.48,
      "learning_rate": 2.2834710743801655e-05,
      "loss": 1.2161,
      "step": 424
    },
    {
      "epoch": 3.13,
      "learning_rate": 2.1520661157024793e-05,
      "loss": 1.2076,
      "step": 477
    },
    {
      "epoch": 3.33,
      "learning_rate": 2.0206611570247934e-05,
      "loss": 1.1163,
      "step": 530
    },
    {
      "epoch": 3.53,
      "learning_rate": 1.8892561983471072e-05,
      "loss": 1.2432,
      "step": 583
    },
    {
      "epoch": 4.18,
      "learning_rate": 1.7578512396694214e-05,
      "loss": 1.2346,
      "step": 636
    },
    {
      "epoch": 4.38,
      "learning_rate": 1.6264462809917355e-05,
      "loss": 1.2157,
      "step": 689
    }
  ],
  "logging_steps": 53,
  "max_steps": 1345,
  "num_train_epochs": 5,
  "save_steps": 500,
  "total_flos": 3.528343138861056e+17,
  "trial_name": null,
  "trial_params": null
}


Another run, different dataset: 

{
  "best_metric": null,
  "best_model_checkpoint": null,
  "epoch": 2.108743232346504,
  "eval_steps": 500,
  "global_step": 2832,
  "is_hyper_param_search": false,
  "is_local_process_zero": true,
  "is_world_process_zero": true,
  "log_history": [
    {
      "epoch": 1.09,
      "learning_rate": 1.9992322456813822e-05,
      "loss": 1.1724,
      "step": 1736
    }
  ],
  "logging_steps": 1736,
  "max_steps": 26043,
  "num_train_epochs": 3,
  "save_steps": 500,
  "total_flos": 1.265552258531328e+18,
  "trial_name": null,
  "trial_params": null
}

Yet another: 

{
  "best_metric": null,
  "best_model_checkpoint": null,
  "epoch": 4.11,
  "eval_steps": 500,
  "global_step": 660,
  "is_hyper_param_search": false,
  "is_local_process_zero": true,
  "is_world_process_zero": true,
  "log_history": [
    {
      "epoch": 1.09,
      "learning_rate": 1.2e-05,
      "loss": 1.3913,
      "step": 240
    },
    {
      "epoch": 3.07,
      "learning_rate": 2.4e-05,
      "loss": 1.1288,
      "step": 480
    }
  ],
  "logging_steps": 240,
  "max_steps": 6000,
  "num_train_epochs": 5,
  "save_steps": 500,
  "total_flos": 2.9493802635264e+17,
  "trial_name": null,
  "trial_params": null
}


but it merges the model, it finishes the script, despite apparently it does not finish the fine tuning process. 

the trainer state files are from the last checkpoint. the saved, merged model files are a few minutes younger than that. I dont think it finished. 

I found an unmerged pull request with files that seem to load internlm-7b as llama. 


https://huggingface.co/internlm/internlm-7b/discussions/4/files#d2h-846292

I copied the files to my internlm20b local copy, but I get this exception:

2023-10-15 09:16:02 ERROR:Failed to load the model.
Traceback (most recent call last):
  File "/run/media/knut/HD/text-generation-webui/modules/ui_model_menu.py", line 201, in load_model_wrapper
    shared.model, shared.tokenizer = load_model(shared.model_name, loader)
  File "/run/media/knut/HD/text-generation-webui/modules/models.py", line 87, in load_model
    tokenizer = load_tokenizer(model_name, model)
  File "/run/media/knut/HD/text-generation-webui/modules/models.py", line 106, in load_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(
  File "/home/knut/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 735, in from_pretrained
    tokenizer_class = get_class_from_dynamic_module(class_ref, pretrained_model_name_or_path, **kwargs)
  File "/home/knut/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 497, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module.replace(".py", ""))
  File "/home/knut/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 199, in get_class_in_module
    module = importlib.import_module(module_path)
  File "/home/knut/miniconda3/envs/textgen/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/knut/.cache/huggingface/modules/transformers_modules/deacon-20b2/tokenization_internlm.py", line 18, in <module>
    from transformers.tokenization_utils import LlamaTokenizer


Same for baichuan2 model: AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'

Hi @lucasjinreal - in all cases this is caused by the custom modelling code not being compatible with the latest versions of `transformers`. You should file an issue on the model repos and tell them to rearrange the tokenizer `__init__` so that `self.sp_model` is created before calling `super().__init__()`!

@Rocketknight1 tansformers should consider back compatible I think, lots of code are broke by this changes, and if they apply new changes, might not compatible to old ones too.

A good lib should always consider back compatible, this is why tensorflow dying.

Hey, backward compatibility with remote code is not really possible, it's one of the downside of having code on the hub, you have to maintain it as well

> transformers_version": "4.33.1

`pip install transformers_version==4.33.1` fixed my issue.

This should be fixed by https://github.com/InternLM/InternLM/pull/419

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.