Electra raises UnboundLocalError: local variable 'seq_length' referenced before assignment when inputs are pre-computed embeddings
Author: sararb
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

 -`transformers` version: 4.9.2
- Platform: Linux-4.15.0-15-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): 2.6.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
Models:
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj

## Information

Model I am using (ELECTRA):

The problem arises when using:
* [ ] my own modified scripts: (give details below)
I am pre-training the ELECTRA model for session-based recommendation task and directly feeding the inputs embeddings instead of their ids.

## To reproduce

Steps to reproduce the behavior:

1.  Load ELECTRA model from config : 

``` 
transformers.MODEL_MAPPING[transformers.ElectraConfig(hidden_size=d_model,
            embedding_size=d_model,
            num_hidden_layers=n_layer,
            num_attention_heads=n_head,...)] 
```

2. Apply the model to pre-computed embeddings :  
``` model(inputs_embeds=inputs) ```

3. The error raised is : 
```
 def forward(
        self,
        input_ids=None,
        attention_mask=None,
        token_type_ids=None,
        position_ids=None,
        head_mask=None,
        inputs_embeds=None,
        output_attentions=None,
        output_hidden_states=None,
        return_dict=None,
    ):
        output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions
        output_hidden_states = (
            output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
        )
        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
    
        if input_ids is not None and inputs_embeds is not None:
            raise ValueError("You cannot specify both input_ids and inputs_embeds at the same time")
        elif input_ids is not None:
            input_shape = input_ids.size()
            batch_size, seq_length = input_shape
        elif inputs_embeds is not None:
            input_shape = inputs_embeds.size()[:-1]
        else:
            raise ValueError("You have to specify either input_ids or inputs_embeds")
    
        device = input_ids.device if input_ids is not None else inputs_embeds.device
    
        if attention_mask is None:
            attention_mask = torch.ones(input_shape, device=device)
        if token_type_ids is None:
            if hasattr(self.embeddings, "token_type_ids"):
>               buffered_token_type_ids = self.embeddings.token_type_ids[:, :seq_length]
E               UnboundLocalError: local variable 'seq_length' referenced before assignment

/opt/conda/lib/python3.8/site-packages/transformers/models/electra/modeling_electra.py:869: UnboundLocalError
```

## Expected behavior

- The seq_len value should also be computed when inputs are pre-computed embeddings instead of raw ids.  

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
That's indeed a small bug. It can be fixed as follows:

```diff
if input_ids is not None and inputs_embeds is not None:
            raise ValueError("You cannot specify both input_ids and inputs_embeds at the same time")
        elif input_ids is not None:
            input_shape = input_ids.size()
-            batch_size, seq_length = input_shape
        elif inputs_embeds is not None:
            input_shape = inputs_embeds.size()[:-1]
        else:
            raise ValueError("You have to specify either input_ids or inputs_embeds")
    
+     batch_size, seq_length = input_shape  
      device = input_ids.device if input_ids is not None else inputs_embeds.device   
```

Btw, I love Github's abilities to showcase this haha. Mind opening a PR to fix this?

Sure, I opened a PR  #13128. Thank you for your reply ! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.