Unable to run compute_transition_scores : facing 'CodeT5pConfig' object has no attribute 'vocab_size' error
Author: MansiShinde
Pull request: no

### System Info

Hello Team , your help is appreciated in the below issue. 
I am running the below code snippet on Google Colab. 

- `transformers` version: 4.30.2
- Platform: Linux-5.15.109+-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): 2.12.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.7.0 (gpu)
- Jax version: 0.4.13
- JaxLib version: 0.4.13
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

@gante

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
from transformers import  AutoModelForSeq2SeqLM, AutoTokenizer

DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
beam_size = 5
max_len = 500

model = "Salesforce/codet5p-2b"
tokenizer = AutoTokenizer.from_pretrained(model)
model = AutoModelForSeq2SeqLM.from_pretrained(model,
                                                trust_remote_code=True,
                                                torch_dtype=torch.float16,
                                                low_cpu_mem_usage=True)

model.eval()
model.to(DEVICE)

prompt = "\n\n\ndef sum_squares(lst):\n    \"\"\"\"\n    This function will take a list of integers. For all entries in the list, the function shall square the integer entry if its index is a \n    multiple of 3 and will cube the integer entry if its index is a multiple of 4 and not a multiple of 3. The function will not \n    change the entries in the list whose indexes are not a multiple of 3 or 4. The function shall then return the sum of all entries. \n    \n    Examples:\n    For lst = [1,2,3] the output should be 6\n    For lst = []  the output should be 0\n    For lst = [-1,-5,2,-1,-5]  the output should be -126\n    \"\"\"\n"

prompt = prompt.replace('    ', '\t')
prompt_batch_decoder = [prompt]

encoding_decoder = tokenizer(prompt_batch_decoder, return_tensors="pt", truncation=True, max_length=max_len).to(DEVICE) 
input_ids=encoding_decoder['input_ids']

with torch.no_grad():
  gen_tokens = model.generate(**encoding_decoder,
                            decoder_input_ids=encoding_decoder['input_ids'],
                            max_length=max_len,
                            num_beams= beam_size,
                            do_sample=False,
                            num_return_sequences=1,
                            return_dict_in_generate=True,
                            output_scores=True,
                            early_stopping=True)

transition_scores = model.compute_transition_scores(gen_tokens.sequences, gen_tokens.scores,gen_tokens.beam_indices, normalize_logits=True)
```

### Expected behavior

transition_scores are computed

Comments:
#### Facing the below error:
```
AttributeError                            Traceback (most recent call last)
[<ipython-input-22-821b8f7c4aac>](https://localhost:8080/#) in <cell line: 1>()
----> 1 transition_scores = model.compute_transition_scores(gen_tokens.sequences, gen_tokens.scores,gen_tokens.beam_indices, normalize_logits=True)

1 frames
[/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py](https://localhost:8080/#) in __getattribute__(self, key)
    259         if key != "attribute_map" and key in super().__getattribute__("attribute_map"):
    260             key = super().__getattribute__("attribute_map")[key]
--> 261         return super().__getattribute__(key)
    262 
    263     def __init__(self, **kwargs):

AttributeError: 'CodeT5pConfig' object has no attribute 'vocab_size'
```

I checked the config file of CodeT5p-2B model on https://huggingface.co/Salesforce/codet5p-2b/blob/main/config.json, it has a vocab_size attribute, I believe this is the right config file to check. I am not sure what is causing this error. 

Could you please help?

Hey @MansiShinde, the issue is that the `CodeT5pConfig` class defined [here](https://huggingface.co/Salesforce/instructcodet5p-16b/blob/70bb08afa3d6f081b347e67752ca8e031a35ac4a/configuration_codet5p.py#L71-L90) does not have a `vocab_size` attribute but rather `encoder` and `decoder` attributes of type `CodeT5pModuleConfig` which then holds the vocab size attribute. You should be able calculate transition scores by using the model's decoder like this:
```
transition_scores = model.decoder.compute_transition_scores(gen_tokens.sequences, gen_tokens.scores,gen_tokens.beam_indices, normalize_logits=True)
```

> Hey @MansiShinde, the issue is that the `CodeT5pConfig` class defined [here](https://huggingface.co/Salesforce/instructcodet5p-16b/blob/70bb08afa3d6f081b347e67752ca8e031a35ac4a/configuration_codet5p.py#L71-L90) does not have a `vocab_size` attribute but rather `encoder` and `decoder` attributes of type `CodeT5pModuleConfig` which then holds the vocab size attribute. You should be able calculate transition scores by using the model's decoder like this:
> 
> ```
> transition_scores = model.decoder.compute_transition_scores(gen_tokens.sequences, gen_tokens.scores,gen_tokens.beam_indices, normalize_logits=True)
> ```


Ohh okay got it. Thanks for the help @fadynakhla !!


Thank you for jumping in, @fadynakhla 🚀