forward() got an unexpected keyword argument 'output_all_encoded_layers'
Author: gogokre
Pull request: no
Labels: name: Usage

I am getting an error such as forward () got an unexpected keyword argument 'output_all_encoded_layers', how can I fix it?

class BertBinaryClassifier(nn.Module):
    def __init__(self, dropout=0.1):
        super(BertBinaryClassifier, self).__init__()

        self.bert = BertModel.from_pretrained('bert-base-uncased')

        self.dropout = nn.Dropout(dropout)
        self.linear = nn.Linear(768, 1)
        self.sigmoid = nn.Sigmoid()
    
    def forward(self, tokens, masks=None):
        _, pooled_output = self.bert(tokens, attention_mask=masks, output_all_encoded_layers=False)
        dropout_output = self.dropout(pooled_output)
        linear_output = self.linear(dropout_output)
        proba = self.sigmoid(linear_output)
        return proba

bert_clf = BertBinaryClassifier()
bert_clf = bert_clf.cuda()

x = torch.tensor(train_tokens_ids[:3]).to(device)
y, pooled = bert_clf.bert(x, output_all_encoded_layers=False)
x.shape, y.shape, pooled.shape

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-22-def915600e8d> in <module>()
      1 x = torch.tensor(train_tokens_ids[:3]).to(device)
----> 2 y, pooled = bert_clf.bert(x, output_all_encoded_layers=False)
      3 x.shape, y.shape, pooled.shape

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    530             result = self._slow_forward(*input, **kwargs)
    531         else:
--> 532             result = self.forward(*input, **kwargs)
    533         for hook in self._forward_hooks.values():
    534             hook_result = hook(self, input, result)

TypeError: forward() got an unexpected keyword argument 'output_all_encoded_layers'

Comments:
This should be added to the initializer, not the forward method. Also, the correct parameter is `output_hidden_states`. You can also be more explicit by changing the config:

```python
model_config = AutoConfig.from_pretrained('bert-base-uncased', output_hidden_states=True)
self.bert = AutoModel.from_pretrained('bert-base-uncased', config=model_config)
```

> This should be added to the initializer, not the forward method. Also, the correct parameter is `output_hidden_states`. You can also be more explicit by changing the config:
> 
> ```python
> model_config = AutoConfig.from_pretrained('bert-base-uncased', output_hidden_states=True)
> self.bert = AutoModel.from_pretrained('bert-base-uncased', config=model_config)
> ```

Thank you very much for answering. 
Sorry, ...can you please write the entire code?

In your code, replace

```python
self.bert = BertModel.from_pretrained('bert-base-uncased')
```

with

```python
model_config = BertConfig.from_pretrained('bert-base-uncased', output_hidden_states=True)
self.bert = BertModel.from_pretrained('bert-base-uncased', config=model_config)
```

and don't forget to import BertConfig at the top.

In the future, please format your code correctly. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks