WARNING:tensorflow:AutoGraph
Author: tide90
Pull request: no

Since v4.2 I get those strange outputs while finetuning a TFBert Model:

Using

`bert_model = TFBertForSequenceClassification.from_pretrained('bert-base-cased')`

```
WARNING:tensorflow:AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7fb902ce88d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: <cyfunction Socket.send at 0x7fb920685d90> is not a module, class, method, function, traceback, frame, or code object
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING: AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7fb902ce88d0>> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: <cyfunction Socket.send at 0x7fb920685d90> is not a module, class, method, function, traceback, frame, or code object
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert

The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`
```

I saw that the outputs are now different with return_dict=True in the new version. But I can still use model.predict (using TFBert within a keras model) to get the scores (seems working though)? So I wonder how does this correlate with returning dict when just using the model output via `model(input)`? So using TF Bert with predict gives still old behaviour?

I still get the normal TFSequenceClassifierOutput with the  TFBertForSequenceClassification Model? What are now the exact changes with v4?

Would be the training results actually be different with the version 4?

Also, how can I disable above messages?


Comments:
Hello !

You can safely ignore those warnings, no worries.

@jplu Thanks. Might the results of training a model may different to new version (like because of the new kind of tokenizer)?

But "why" do I get the same output as in v3, since in the doc is stated that the ouput structure somehow chnaged and you cannot do unpacking like 

`a, b, c = model(input)
`

But which stills works.

How can I ignore (disable) this messages? I tried a lot, but nothing worked!



In graph mode you cannot get tuples anymore, the dict output is forced and you cannot disable this message for now. This will be possible in a future release, as it will be displayed only when you set yourself the `output_attentions`, `output_hidden_states` or `return_dict` yourself in the method call while running your model in graph mode.

@jplu Thanky. What do you mean by graph mode? As stated above, I still get the tuples as output?

You are not getting tuples, by doing:
```
a, b, c = model(input)
```
You are getting the keys of the dict.

By graph mode, I mean TensorFlow graph mode, and not eager mode.

@jplu Ah ok, that is why I still get the tuple, because by default in tf2 eager mode is activated, right?

Yes eager mode is activated by default, and no you don't get tuples, you get a dict because `return_dict` is set to `True` is all the configs by default.

Ôk, but then I am confused why above unpacked worked for, although return dict is true?

Because you can unpack a dict, and you get the keys of the dict.

This issue has been stale for 1 month.