TFDistilBERT ValueError when loading a saved model and running model.predict(), same with any sequence classification model in tensorflow
Author: brandonbell11
Pull request: no
Labels: name: wontfix

This issue happens when I save and reload a model. I am trying to distinguish between fake text and real text, and everything works just fine. 
When I save and reload the model elsewhere, model.predict() gives me a value error, and I have to run model.fit() AGAIN otherwise it continues to raise a ValueError.

> ValueError: Please provide model inputs as a list or tuple of 2 or 3 elements: (input, target) or (input, target, sample_weights) Received tf.Tensor([100], shape=(1,), dtype=int64)

Here is the code that works:
```
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = TFDistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased', num_labels=2)

real_path = '/data/brabel1/vtj/4_facebook/model_data/cleaned_messages.txt'
fake_path = '/data/brabel1/vtj/4_facebook/model_data/fake.txt'
real = open(real_path, 'r')
fake = open(fake_path, 'r')

real_input_ids = tf.keras.preprocessing.sequence.pad_sequences([tokenizer.encode(line) for line in real.readlines()],
                          maxlen=256, dtype="int", truncating="post", padding="post")
fake_input_ids = tf.keras.preprocessing.sequence.pad_sequences([tokenizer.encode(line) for line in fake.readlines()],
                          maxlen=256, dtype="int", truncating="post", padding="post")

FILE_NAMES=[real_input_ids, fake_input_ids]

def labeler(example, index):
    return example, tf.cast(index, tf.int64)

labeled_data_sets = []

for i, file_name in enumerate(FILE_NAMES):
    lines_dataset = tf.data.Dataset.from_tensor_slices(file_name)
    labeled_dataset = lines_dataset.map(lambda ex: labeler(ex, i))
    labeled_data_sets.append(labeled_dataset)

BUFFER_SIZE = 100000
BATCH_SIZE = 32
TAKE_SIZE = 1800

all_labeled_data = labeled_data_sets[0]

for labeled_dataset in labeled_data_sets[1:]:
    all_labeled_data = all_labeled_data.concatenate(labeled_dataset)

all_labeled_data = all_labeled_data.shuffle(
    BUFFER_SIZE, reshuffle_each_iteration=False)

train_data = all_labeled_data.skip(TAKE_SIZE).shuffle(BUFFER_SIZE)
train_data = train_data.padded_batch(BATCH_SIZE, padded_shapes=([-1],[]))

test_data = all_labeled_data.take(TAKE_SIZE)
test_data = test_data.padded_batch(BATCH_SIZE, padded_shapes=([-1],[]))

optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=loss, metrics=[metric])

model.fit(train_data, validation_data=test_data, epochs=5)

model.predict(tokenizer.encode(["this is a test sentence, no value errors here!"]))
```

HOWEVER, the following saving and reloading of the model results in a ValueError:
```
model.save_pretrained('saved_models/fucky_bert')
del model
model = TFDistilBertForSequenceClassification.from_pretrained('saved_models/fucky_bert')
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=loss, metrics=[metric])

model.predict(tokenizer.encode(["Why am I getting a value error now???"]))
```

The only thing I've found that works is to train this loaded model for a single epoch, and then no value error.

What is going on here?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.