error AttributeError: 'tuple' object has no attribute 'logits'
Author: TzurV
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

AttributeError                            Traceback (most recent call last)
<ipython-input-4-594fed3b7299> in <module>()
      6 input = tokenizer.encode(sequence, return_tensors="pt")
      7 mask_token_index = torch.where(input == tokenizer.mask_token_id)[1]
----> 8 token_logits = model(input).logits
      9 mask_token_logits = token_logits[0, mask_token_index, :]
     10 top_5_tokens = torch.topk(mask_token_logits, 5, dim=1).indices[0].tolist()

AttributeError: 'tuple' object has no attribute 'logits'     

- `transformers` version:
Successfully installed sacremoses-0.0.43 sentencepiece-0.1.91 tokenizers-0.8.1rc2 transformers-3.3.1
- Platform: 
Masked Language Modeling - Colab PyTorch 
- Python version:
Python 3.6.9
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [V ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Summary of the tasks Open Colab Pytorch
2. Masked Language Modeling example
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Can you replace:

```python
token_logits = model(input).logits
```
by

```python
token_logits = model(input, return_dict=True).logits
```

and see if the error persists? 

Can you give a link to the example, so that we can fix the code snippet? 

Hi Patrick,

The suggested change fixed the problem.

Thank you.
Tzur


On Wed, 7 Oct 2020 at 18:34, Patrick von Platen <notifications@github.com>
wrote:

> Can you give a link to the example, so that we can fix the code snippet?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/7638#issuecomment-705087242>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEVYDXL4DBQIJO4MPX3QZS3SJSRBXANCNFSM4SHNAGUQ>
> .
>


-- 
Email: tzurvaich@gmail.com
Home: +44 (0) 1480 839198
Mobile: +44 (0) 7825 363873
Israel: +972 (0) 3 7201013
Address:
23 Old Pinewood Way,
Papworth Everard
Cambridge CB23 3GT
UK


@patrickvonplaten The same issue is on this page https://huggingface.co/transformers/training.html

In these 2 lines 
`outputs = model(input_ids, attention_mask=attention_mask, labels=labels)`

`outputs = model(input_ids, attention_mask=attention_mask)`

Thanks for the help!!


I fixed this problem by update transfromers from 3.0.2 to 4.23.1(the latest version in 2022.10.16)

import torch
from torch.utils.data import DataLoader, TensorDataset
from transformers import DistilBertTokenizer, DistilBertForQuestionAnswering, AdamW

# Assuming you have already preprocessed the questions and answers
questions = [
    "How do I initiate a policy action?",
    "What is the difference between policy and procedure?",
    "On average, how long does it take to complete a policy action?", 
    "What happens to the policy once I submit my draft to UPO?",
    "Who do I talk to if I have a question about the content of a policy?", 
    "Where do I find the previously approved versions of the policy?", 
    "What is the legal sufficiency review?", 
    "Substantive vs. non-substantive change?", 
    "How do I know when the policy was last updated?", 
    "Are there any rules about posting UNT policy on our departmental web page?"
]  

answers = [
    "To initiate a policy action, contact the University Policy Office at policy@unt.edu.", 
    "A policy is a governing principle that mandates or constrains actions. Procedures outline the steps necessary to implement the policy.", 
    " ",
    " ",
    " ",
    " ",
    " ",
    " ",
    " ",
    " ",
  # Rest of answers
]

# Make sure questions and answers have the same length
assert len(questions) == len(answers), "Mismatch in the number of questions and answers"
labels = torch.tensor([1] * len(questions))  # Assuming all questions have correct answers

def preprocess(questions, answers):
    tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
    
    # Combine questions and answers into a list of strings
    inputs = [f"{q} {a}" for q, a in zip(questions, answers)]
    
    # Tokenize the combined strings
    tokenized_inputs = tokenizer(inputs, return_tensors='pt', padding=True, truncation=True)
    
    # Create tensors for input_ids, attention_mask
    input_ids = tokenized_inputs['input_ids']
    attention_mask = tokenized_inputs['attention_mask']
    
    return input_ids, attention_mask

input_ids, attention_mask = preprocess(questions, answers)

# Create a DataLoader
dataset = TensorDataset(input_ids, attention_mask, labels)
dataloader = DataLoader(dataset, batch_size=8, shuffle=True)

# Modify the labels to represent whether each question has an answer or not
# For simplicity, let's assume 1 represents having an answer and 0 represents not having an answer
labels = torch.tensor([1 if a.strip() else 0 for a in answers])

# Update the training loop
def train(model, dataloader, lr=5e-5, epochs=3):
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model.to(device)
    
    optimizer = AdamW(model.parameters(), lr=lr)
    criterion = torch.nn.BCEWithLogitsLoss()  # Assuming binary classification (answer or not)
    
    for epoch in range(epochs):
        model.train()
        total_loss = 0.0
        
        for batch in dataloader:
            input_ids, attention_mask, labels = batch
            input_ids, attention_mask, labels = input_ids.to(device), attention_mask.to(device), labels.to(device)
            
            optimizer.zero_grad()
            
            outputs = model(input_ids, attention_mask=attention_mask)
            logits = outputs.logits
            
            loss = criterion(logits.squeeze(), labels.float())
            total_loss += loss.item()
            
            loss.backward()
            optimizer.step()
        
        print(f"Epoch {epoch + 1}, Loss: {total_loss / len(dataloader)}")

# Instantiate the model
bert_model = DistilBertForQuestionAnswering.from_pretrained('distilbert-base-uncased')

# Train the model
train(bert_model, dataloader)

# Save the trained model
torch.save(bert_model.state_dict(), 'saved_model.pth')

# Load the trained model
loaded_model = DistilBertForQuestionAnswering.from_pretrained('distilbert-base-uncased')
loaded_model.load_state_dict(torch.load('saved_model.pth'))
loaded_model.eval()

# Prediction function
def get_answer(question, model, tokenizer):
    inputs = tokenizer(question, return_tensors='pt')
    input_ids = inputs['input_ids']
    attention_mask = inputs['attention_mask']
    
    with torch.no_grad():
        outputs = model(input_ids, attention_mask=attention_mask)
    
    start_logits, end_logits = outputs.start_logits, outputs.end_logits
    start_idx = torch.argmax(start_logits)
    end_idx = torch.argmax(end_logits)
    
    answer = tokenizer.decode(input_ids[0, start_idx:end_idx+1], skip_special_tokens=True)
    
    return answer

# Example usage
question_to_predict = "How do I initiate a policy action?"
answer = get_answer(question_to_predict, loaded_model, DistilBertTokenizer.from_pretrained('distilbert-base-uncased'))
print("Answer:", answer)


for the above code i am getting the bellow error

Some weights of DistilBertForQuestionAnswering were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['qa_outputs.bias', 'qa_outputs.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_18184\3727763023.py in <module>
     93 
     94 # Train the model
---> 95 train(bert_model, dataloader)
     96 
     97 # Save the trained model

~\AppData\Local\Temp\ipykernel_18184\3727763023.py in train(model, dataloader, lr, epochs)
     79 
     80             outputs = model(input_ids, attention_mask=attention_mask)
---> 81             logits = outputs.logits
     82 
     83             loss = criterion(logits.squeeze(), labels.float())

AttributeError: 'QuestionAnsweringModelOutput' object has no attribute 'logits'


你好，我已经收到您的邮件~

Can any one fix the error, please?

Hey, feel free to check this: https://github.com/younesbelkada/transformers/blob/587b8e6ce3742063e835c33d239a0a400c69d631/src/transformers/modeling_outputs.py#L1046 there is not `logits` key. Adapt your script accordingly 🤗