fix get_keys_to_not_convert() to return correct modules for full precision inference
Author: ranchlai
Pull request: yes

# What does this PR do?
This PR fixes `get_keys_to_not_convert()` and returns the correct modules that should not be quantized for numerical stability reasons. 

For GPT2 and bloom, both the current(1st output) and new version(2nd output) work well.
GPT2
```
>>> filtered_module_names_old is ['transformer.wte', 'lm_head', 'lm_head'] # both are correct, duplicated though

>>>  filtered_module_names is ['lm_head', 'transformer.wte'] # both are correct
```
bloom
```
>>> filtered_module_names_old is ['lm_head', 'transformer.word_embeddings', 'lm_head'] # both are correct, duplicated though
>>> filtered_module_names is ['transformer.word_embeddings', 'lm_head'] # both are correct
```
But for ChatGLM and MPT(and possibly others), this PR finds the correct modules while the current code doesn't.
MPT
```
>>> filtered_module_names_old is ['transformer.wte', 'transformer.output_wte', 'transformer'] # this is wrong
>>> filtered_module_names is ['transformer.output_wte', 'transformer.wte'] # this is correct
```
ChatGLM2
```
>>> filtered_module_names_old is ['transformer'] # this is wrong
>>> filtered_module_names is ['transformer.output_layer'] # this is correct
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @younesbelkada for a first look

@younesbelkada Looks like that you have fixed this in #25047 two days ago. Great work. My bad for not using latest code. 


Alright. I am diving deeper into the code.

Could you try this code: 
```python
from transformers import AutoConfig, MptForCausalLM
from transformers import AutoModelForCausalLM # many people use this for convenience

from transformers.utils.bitsandbytes import get_keys_to_not_convert

from accelerate import init_empty_weights

config = AutoConfig.from_pretrained("mosaicml/mpt-7b", trust_remote_code=True)

with init_empty_weights():
    model = AutoModelForCausalLM.from_config(config, trust_remote_code=True)


print(get_keys_to_not_convert(model))

>>> []  # this is because there is problem checking "if (not has_tied_params) and is_base_model" for remote code. 



with init_empty_weights():
    model = MptForCausalLM(config) # 
>>> AttributeError: 'MPTConfig' object has no attribute 'hidden_size'
 
print(get_keys_to_not_convert(model))
```
See my [another issue for checking tied parameters](https://github.com/huggingface/accelerate/issues/1761)
 

if you comment out the following lines:
```python
    if (not has_tied_params) and is_base_model:
        return []
```

you will get the following : 
```
>> ['transformer.norm_f']
```
But the new fix still get: 
```
>>  ['transformer.wte']
```

So maybe it is a good practice to utilize get_output_embeddings() in both find tied_parameters and checking lm_head? 
@sgugger @younesbelkada any comments? See again [here](https://github.com/huggingface/accelerate/issues/1761)



Sure, I am happy to add the test. 
Actually I have done tests similar to yours above, for the following models:
```python
model_names = [
     "mosaicml/mpt-7b",
    "mosaicml/mpt-7b-storywriter",
    "mosaicml/mpt-7b-8k-chat",
    "THUDM/chatglm2-6b",
    "THUDM/chatglm-6b",
    "lmsys/longchat-7b-16k",
    "lmsys/fastchat-t5-3b-v1.0",
    "TheBloke/koala-13B-HF",
    "WizardLM/WizardLM-13B-V1.0",
    "OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5",
    "nomic-ai/gpt4all-13b-snoozy",
    "WizardLM/WizardLM-13B-V1.0",
    "stabilityai/FreeWilly2",
    "OpenAssistant/llama2-13b-orca-8k-3319",
    "JosephusCheung/Guanaco",
    "lmsys/vicuna-7b-v1.3",
    "baichuan-inc/Baichuan-7B",
    "openlm-research/open_llama_3b",
    "EleutherAI/gpt-j-6b",
    "facebook/opt-1.3b",
    "tiiuae/falcon-7b",
    "tiiuae/falcon-40b-instruct",
   ```
Maybe too many. I will stick to your choice of models. 

> 
Added test. Please would you kindly review. @younesbelkada 


> 

Thanks a lot. any problems please let me know 

Ok thanks, I will keep you updated

Indeed there is not `output_embeddings` for `AutoModelForSequenceClassification`. 
We can pass the test the model by adding
 ```python
if output_embeddings is None: 
        output_embeddings = list(model.modules())[-1]
```
Do you think we we can add this? 

One more thing, I am thinking,  is it important to keep classification head (not originally `lm_head`) in full precision, as the matrix is not that large(thus quantization error might be not that large too)?  Or, is it useful to quantize classification models to 8bits in the first place, as they are usually not that large. 




Make sense.  Minimum change is much safer. Will update the code.

Thanks very much @ranchlai !!

@younesbelkada You are very welcome. I have tried to keep minimum change and fix the mpt problem. Please check if that's  what you mean. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25105). All of your documentation changes will be reflected on that endpoint.