Bert Tensor Dimensions
Author: halidziya
Pull request: no
Labels: name: wontfix

## 🐛 Bug

My code was working on 2.1 it throws an error in 2.2:    

    transformers\modeling_tf_bert.py:777 call
        outputs = self.bert(inputs, **kwargs)
    transformers\modeling_tf_bert.py:512 call
        attention_mask = tf.fill(input_shape, 1)
    tensorflow_core\python\ops\array_ops.py:171 fill
        result = gen_array_ops.fill(dims, value, name=name)
    tensorflow_core\python\ops\gen_array_ops.py:3602 fill
        "Fill", dims=dims, value=value, name=name)
    tensorflow_core\python\framework\op_def_library.py:545 _apply_op_helper
        (input_name, err))

    ValueError: Tried to convert 'dims' to a tensor and failed. Error: Cannot convert a partially known TensorShape to a Tensor: (None, 72)

<!-- Important information -->

Model I am using (Bert, XLNet....): BERT

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [ ] the official example scripts: (give details)
* [X] my own modified scripts: predict function

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details)

## Environment

* OS:
* Python version: 3.7.4
* PyTorch version: - 
* Tensorflow :  2.0.0
* Using GPU : Yes
* Distributed of parallel setup : No
* Any other relevant information: 

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
Hi, thanks for letting us know. Is there any way you could provide a script that reproduces the error in a few lines so that we may see what is wrong on our end?

model.predict function fails with output_hidden_states=True in constructor

I'm failing to reproduce what you're mentioning with the following snippet:

```py
from transformers import TFBertModel, BertTokenizer, BertConfig
import tensorflow as tf

config = BertConfig.from_pretrained("bert-base-cased", output_hidden_states=True)
model = TFBertModel.from_pretrained("bert-base-cased", config=config)

tok = BertTokenizer.from_pretrained("bert-base-cased")
text = tok.encode("Ain't this [MASK] best thing you've ever seen?")

inputs = tf.constant(text)
outputs = model.predict(inputs)

print(outputs)
```
Is there any way you could provide a script that reproduces the error in a few lines so that we may see what is wrong on our end?


With this piece of code you've posted, I'm encountered the same problem highlighted by @halidziya .

**ENVIRONMENT**:
- Python 3.6.9
- OS: Ubuntu 16.04 ('Linux-4.15.0-70-generic-x86_64-with-debian-buster-sid')
- Transformers: 2.2.2 (installed with `pip install transformers` the day after the release)
- PyTorch: 1.3.1
- TensorFlow: 2.0.0

The stack trace is reported below:
```
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers import TFBertModel, BertTokenizer, BertConfig
/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
2019-12-03 09:46:35.606174: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-03 09:46:35.610775: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3408000000 Hz
2019-12-03 09:46:35.611320: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55fa0a097860 executing computations on platform Host. Devices:
2019-12-03 09:46:35.611341: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
>>> import tensorflow as tf
>>> config = BertConfig.from_pretrained("bert-base-cased", output_hidden_states=True)
>>> model = TFBertModel.from_pretrained("bert-base-cased", config=config)
>>> tok = BertTokenizer.from_pretrained("bert-base-cased")
>>> text = tok.encode("Ain't this [MASK] best thing you've ever seen?")
>>> inputs = tf.constant(text)
>>> outputs = model.predict(inputs)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 909, in predict
    use_multiprocessing=use_multiprocessing)
  File "/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_arrays.py", line 715, in predict
    x, check_steps=True, steps_name='steps', steps=steps)
  File "/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 2419, in _standardize_user_data
    all_inputs, y_input, dict_inputs = self._build_model_with_inputs(x, y)
  File "/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 2622, in _build_model_with_inputs
    self._set_inputs(cast_inputs)
  File "/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 2709, in _set_inputs
    outputs = self(inputs, **kwargs)
  File "/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 842, in __call__
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "/home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/autograph/impl/api.py", line 237, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in converted code:
    relative to /home/vidiemme/anaconda3/envs/huggingface/lib/python3.6/site-packages:

    transformers/modeling_tf_bert.py:684 call  *
        outputs = self.bert(inputs, **kwargs)
    tensorflow_core/python/keras/engine/base_layer.py:842 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    transformers/modeling_tf_bert.py:512 call  *
        attention_mask = tf.fill(input_shape, 1)
    tensorflow_core/python/ops/array_ops.py:171 fill
        result = gen_array_ops.fill(dims, value, name=name)
    tensorflow_core/python/ops/gen_array_ops.py:3602 fill
        "Fill", dims=dims, value=value, name=name)
    tensorflow_core/python/framework/op_def_library.py:545 _apply_op_helper
        (input_name, err))

    ValueError: Tried to convert 'dims' to a tensor and failed. Error: Cannot convert a partially known TensorShape to a Tensor: (None, 1)
```

> I'm failing to reproduce what you're mentioning with the following snippet:
> 
> ```python
> from transformers import TFBertModel, BertTokenizer, BertConfig
> import tensorflow as tf
> 
> config = BertConfig.from_pretrained("bert-base-cased", output_hidden_states=True)
> model = TFBertModel.from_pretrained("bert-base-cased", config=config)
> 
> tok = BertTokenizer.from_pretrained("bert-base-cased")
> text = tok.encode("Ain't this [MASK] best thing you've ever seen?")
> 
> inputs = tf.constant(text)
> outputs = model.predict(inputs)
> 
> print(outputs)
> ```
> 
> Is there any way you could provide a script that reproduces the error in a few lines so that we may see what is wrong on our end?

Cannot reproduce on release 2.2.1.
Can you check with the latest release or master?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.