TypeError: `OPTModel.forward()` got an unexpected keyword argument 'labels'
Author: deven367
Pull request: no

### System Info

transformers version: 4.35.2
python version: 3.10.9
os version: SUSE Linux Enterprise Server 15 SP3

### Who can help?

@ArthurZucker @younesbelkada The forward pass of the base `OPTModel` does not have a `labels` argument. The other class `OPTforCausalLM` does have a `labels` argument in its forward. The reason I wish to use the `OPTModel` is that I want the network to be initialized with random weights.

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Here is my training loop

```py
import torch
from fastcore.xtras import globtastic
from torch.utils.data import DataLoader, Dataset
from transformers import AutoModelForCausalLM, AutoTokenizer, OPTConfig, OPTModel

class CharRNNDataset(Dataset):
    def __init__(self, data, block_size):
        self.data = data
        self.block_size = block_size

    def __len__(self):
        return len(self.data) - self.block_size

    def __getitem__(self, idx):
        chunk = self.data[idx : idx + self.block_size + 1]
        x = chunk[:-1]
        y = chunk[1:]
        return x, y


def main():
    torch.manual_seed(0)
    tokenizer = AutoTokenizer.from_pretrained("facebook/opt-125m")
    print("Loading data...")
    files = globtastic("/N/slate/demistry/babylm_10M", file_glob="*.train").sorted()
    print("Data loaded!")
    all_text = ""
    for f in files:
        with open(f, "r") as f:
            all_text += f.read()

    print("Tokenizing data...")
    tokens = tokenizer.encode(all_text, return_tensors="pt")
    print(tokens.shape)
    tokens = tokens.squeeze()
    print(tokens.shape)
    print("Data tokenized!")

    print("Creating dataset and dataloader...")
    dataset = CharRNNDataset(tokens, block_size=2048)
    dataloader = DataLoader(dataset, batch_size=8)
    print("Dataset and dataloader created!")
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

    print("Training model...")
    configuration = OPTConfig()
    model = OPTModel(configuration)

    # model = AutoModelForCausalLM.from_config("facebook/opt-125m")
    model.to(device)
    print("Model loaded!")
    optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)

    for i in range(1, 11):
        losses, perplexities = [], []
        for inner, (x, y) in enumerate(dataloader):
            x = x.to(device)
            y = y.to(device)
            outputs = model(x, labels=y)
            loss = outputs.loss
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            losses.append(loss.item())
            perplexities.append(torch.exp(loss).item())

            if inner % 100 == 0:
                print(
                    f"Epoch {i} Batch {inner} Loss {loss.item()} Perplexity {torch.exp(loss).item()}"
                )
            # print("Epoch", i, "Batch", inner, "Loss", loss.item(), "Perplexity", torch.exp(loss).item())

        print(f"Epoch {i} loss: {sum(losses) / len(losses)}")
        print(f"Epoch {i} perplexity: {sum(perplexities) / len(perplexities)}")
    print("Training complete!")


if __name__ == "__main__":
    main()

```

Code works as expected if I change the model to 
```
model = AutoModelForCausalLM.from_config("facebook/opt-125m")
```

### Expected behavior

Model training should work as expected.

Comments:
Hi,

The Transformers library is designed this way, `xxxModel` classes define a "base" Transformer, without any head on top, and then you have the `xxxForCausalLM` (or `xxxForImageClassification`, `xxxForSequenceClassification`, etc.) classes which define a base Transformer with a linear head on top. Only the latter accept a labels argument, since that is required to compute the loss between the logits predicted by the classifier and the ground truth targets.

Hence `OPTModel` does not accept `labels`, so you can do `model = OPTForCausalLM(config)` to initialize it with randomly initialized weights.

Thanks for the quick reply @NielsRogge. I'll go ahead and close the issue.