Dict in the first positional arguments
Author: celsofranssa
Pull request: no
Labels: name: TensorFlow

name: Usage

Could someone help me figure out what is wrong in the TFBertModel below?

```python
features = next(iter(dataset))
features
```
which prints:

```python
{'attention_mask': <tf.Tensor: shape=(16,), dtype=int32, numpy=array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], dtype=int32)>,
 'input_ids': <tf.Tensor: shape=(16,), dtype=int32, numpy=
 array([  101, 13366,  2131,  1035,  6819,  2094,  1035,  2013,  1035,
        24471,  2140,  1006, 24471,  2140,  1007,   102], dtype=int32)>}
```
In turn, I loaded the `TFBertModel` and following the[ documentation page](https://huggingface.co/transformers/model_doc/bert.html#transformers.TFBertForPreTraining) I tried to use `features` as input:

```python
model = TFBertModel.from_pretrained('bert-base-uncased')
model(features)
```
But I'm getting the following error:

```python
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-19-8bcadb504daf> in <module>()
----> 1 model(features)

8 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_bert.py in call(self, inputs, **kwargs)
    706         last_hidden_states = outputs[0]  # The last hidden-state is the first element of the output tuple
    707         """
--> 708         outputs = self.bert(inputs, **kwargs)
    709         return outputs
    710 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    966           with base_layer_utils.autocast_context_manager(
    967               self._compute_dtype):
--> 968             outputs = self.call(cast_inputs, *args, **kwargs)
    969           self._handle_activity_regularization(inputs, outputs)
    970           self._set_mask_metadata(inputs, outputs, input_masks)

/usr/local/lib/python3.6/dist-packages/transformers/modeling_tf_bert.py in call(self, inputs, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, training)
    545         # this attention mask is more simple than the triangular masking of causal attention
    546         # used in OpenAI GPT, we just need to prepare the broadcast dimension here.
--> 547         extended_attention_mask = attention_mask[:, tf.newaxis, tf.newaxis, :]
    548 
    549         # Since attention_mask is 1.0 for positions we want to attend and 0.0 for

/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py in _slice_helper(tensor, slice_spec, var)
    982         ellipsis_mask=ellipsis_mask,
    983         var=var,
--> 984         name=name)
    985 
    986 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py in strided_slice(input_, begin, end, strides, begin_mask, end_mask, ellipsis_mask, new_axis_mask, shrink_axis_mask, var, name)
   1148       ellipsis_mask=ellipsis_mask,
   1149       new_axis_mask=new_axis_mask,
-> 1150       shrink_axis_mask=shrink_axis_mask)
   1151 
   1152   parent_name = name

/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_array_ops.py in strided_slice(input, begin, end, strides, begin_mask, end_mask, ellipsis_mask, new_axis_mask, shrink_axis_mask, name)
  10155         pass  # Add nodes to the TensorFlow graph.
  10156     except _core._NotOkStatusException as e:
> 10157       _ops.raise_from_not_ok_status(e, name)
  10158   # Add nodes to the TensorFlow graph.
  10159   if begin_mask is None:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
   6651   message = e.message + (" name: " + name if name is not None else "")
   6652   # pylint: disable=protected-access
-> 6653   six.raise_from(core._status_to_exception(e.code, message), None)
   6654   # pylint: enable=protected-access
   6655 

/usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: Index out of range using input dim 1; input has only 1 dims [Op:StridedSlice] name: tf_bert_model/bert/strided_slice/
```
note: edited to correct typo.

Comments:
I would assume that you need to unpack the dictionary as you pass it to the model:

```python
model(**features)
```

EDIT: I was wrong, since the TF version should be used differently than the PT version.

Thank you @BramVanroy, but another error arose:

```python
model(**features)
```

```python
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-20-6248eef0b628> in <module>()
----> 1 model(**features)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    798     else:
    799       raise ValueError(
--> 800           'The first argument to `Layer.call` must always be passed.')
    801 
    802     call_context = base_layer_utils.call_context()

ValueError: The first argument to `Layer.call` must always be passed.

---------------------------------------------------------------------------
NOTE: Current TensorFlow version is 2.2.0-rc2. To use TF 1.x instead,
restart your runtime (Ctrl+M .) and run "%tensorflow_version 1.x" before
you run "import tensorflow".
---------------------------------------------------------------------------
```

I believe the error comes from the fact that you''re lacking a dimension in your features. All inputs to the model should have a shape of `[batch_size, sequence_length]`, whereas from your output:

```py
{
    'attention_mask': <tf.Tensor: shape=(16,), dtype=int32, numpy=array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], dtype=int32)>,
    'input_ids': <tf.Tensor: shape=(16,), dtype=int32, numpy=array([  101, 13366,  2131,  1035,  6819,  2094,  1035,  2013,  1035, 24471,  2140,  1006, 24471,  2140,  1007,   102], dtype=int32)>
}
```

Your tensors are of shape `[sequence_length]`. You can unsqueeze those to add a batch dimension (or simply batch them, since you're already making use of the attention mask), and it should work.

@LysandreJik,
Yes, it was really missing a dimension in `features`. 
Using `features = tokenized_dataset.batch(2)` produces the desired input to transformer:

```python
({'attention_mask': <tf.Tensor: shape=(2, 16), dtype=int32, numpy=
  array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
         [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]], dtype=int32)>,
  'input_ids': <tf.Tensor: shape=(2, 16), dtype=int32, numpy=
  array([[  101, 13366,  2131,  1035,  6819,  2094,  1035,  2013,  1035,
          24471,  2140,  1006, 24471,  2140,  1007,   102],
         [  101, 13366,  8254,  2050,  1035, 20950,  1035,  2000,  1035,
          24471,  2140,  1035,  2862,  1006, 20950,   102]], dtype=int32)>},
 {'attention_mask': <tf.Tensor: shape=(2, 16), dtype=int32, numpy=
  array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0],
         [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]], dtype=int32)>,
  'input_ids': <tf.Tensor: shape=(2, 16), dtype=int32, numpy=
  array([[  101, 27059,  2678,  8909,  2013, 24471,  2140,  1012,   102,
              0,     0,     0,     0,     0,     0,     0],
         [  101,  2358,  2099,  1011,  1028,  2862, 10463, 20950,  2000,
          24471,  2140,  2862,  1012,  2013, 12170,   102]], dtype=int32)>})
```
Thank you.