Saving and loading the model from local throws error
Author: SrikanthChellappa
Pull request: no

### System Info

Python version 3.11.3
Transformer version  4.34.1

### Who can help?

@Rocketknight1 @gan

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Can you pls assist me on how to save the model with .keras or .h5 extensions to load it again. I am able to save the model as .keras buty it throws error when i load it again. Pls see the code used as below. Kindly assist

I was able to save my model earlier using model.save('FlanT5-Chatbot_model.keras')

When i tried loading the model again as below
from tensorflow.keras.models import load_model
model=load_model('FlanT5-Chatbot_model.keras')

I am getting "ModuleNotFoundError" error. Error stack is given below

Error Stack
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\serialization_lib.py:800, in _retrieve_class_or_fn(name, registered_name, module, obj_type, full_config, custom_objects)
    799 try:
--> 800     mod = importlib.import_module(module)
    801 except ModuleNotFoundError:

File C:\ProgramData\anaconda3\Lib\importlib\__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1206, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1178, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1128, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File <frozen importlib._bootstrap>:1206, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1178, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1128, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File <frozen importlib._bootstrap>:1206, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1178, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1142, in _find_and_load_unlocked(name, import_)

ModuleNotFoundError: No module named 'transformers.models'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
Cell In[4], line 2
      1 from tensorflow.keras.models import load_model
----> 2 model=load_model('FlanT5-Chatbot_model.keras')

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_api.py:254, in load_model(filepath, custom_objects, compile, safe_mode, **kwargs)
    249     if kwargs:
    250         raise ValueError(
    251             "The following argument(s) are not supported "
    252             f"with the native Keras format: {list(kwargs.keys())}"
    253         )
--> 254     return saving_lib.load_model(
    255         filepath,
    256         custom_objects=custom_objects,
    257         compile=compile,
    258         safe_mode=safe_mode,
    259     )
    261 # Legacy case.
    262 return legacy_sm_saving_lib.load_model(
    263     filepath, custom_objects=custom_objects, compile=compile, **kwargs
    264 )

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:281, in load_model(filepath, custom_objects, compile, safe_mode)
    278             asset_store.close()
    280 except Exception as e:
--> 281     raise e
    282 else:
    283     return model

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:246, in load_model(filepath, custom_objects, compile, safe_mode)
    244 # Construct the model from the configuration file in the archive.
    245 with ObjectSharingScope():
--> 246     model = deserialize_keras_object(
    247         config_dict, custom_objects, safe_mode=safe_mode
    248     )
    250 all_filenames = zf.namelist()
    251 if _VARS_FNAME + ".h5" in all_filenames:

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\serialization_lib.py:705, in deserialize_keras_object(config, custom_objects, safe_mode, **kwargs)
    702     if obj is not None:
    703         return obj
--> 705 cls = _retrieve_class_or_fn(
    706     class_name,
    707     registered_name,
    708     module,
    709     obj_type="class",
    710     full_config=config,
    711     custom_objects=custom_objects,
    712 )
    714 if isinstance(cls, types.FunctionType):
    715     return cls

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\serialization_lib.py:802, in _retrieve_class_or_fn(name, registered_name, module, obj_type, full_config, custom_objects)
    800     mod = importlib.import_module(module)
    801 except ModuleNotFoundError:
--> 802     raise TypeError(
    803         f"Could not deserialize {obj_type} '{name}' because "
    804         f"its parent module {module} cannot be imported. "
    805         f"Full object config: {full_config}"
    806     )
    807 obj = vars(mod).get(name, None)
    809 if obj is None:
    810     # Special case for keras.metrics.metrics

TypeError: Could not deserialize class 'TFT5ForConditionalGeneration' because its parent module transformers.models.t5.modeling_tf_t5 cannot be imported. Full object config: {'module': 'transformers.models.t5.modeling_tf_t5', 'class_name': 'TFT5ForConditionalGeneration', 'config': {'vocab_size': 32128, 'd_model': 768, 'd_kv': 64, 'd_ff': 2048, 'num_layers': 12, 'num_decoder_layers': 12, 'num_heads': 12, 'relative_attention_num_buckets': 32, 'relative_attention_max_distance': 128, 'dropout_rate': 0.1, 'classifier_dropout': 0.0, 'layer_norm_epsilon': 1e-06, 'initializer_factor': 1.0, 'feed_forward_proj': 'gated-gelu', 'use_cache': True, 'dense_act_fn': 'gelu_new', 'is_gated_act': True, 'return_dict': True, 'output_hidden_states': False, 'output_attentions': False, 'torchscript': False, 'torch_dtype': None, 'use_bfloat16': False, 'tf_legacy_loss': False, 'pruned_heads': {}, 'tie_word_embeddings': False, 'is_encoder_decoder': True, 'is_decoder': False, 'cross_attention_hidden_size': None, 'add_cross_attention': False, 'tie_encoder_decoder': False, 'max_length': 20, 'min_length': 0, 'do_sample': False, 'early_stopping': False, 'num_beams': 1, 'num_beam_groups': 1, 'diversity_penalty': 0.0, 'temperature': 1.0, 'top_k': 50, 'top_p': 1.0, 'typical_p': 1.0, 'repetition_penalty': 1.0, 'length_penalty': 1.0, 'no_repeat_ngram_size': 0, 'encoder_no_repeat_ngram_size': 0, 'bad_words_ids': None, 'num_return_sequences': 1, 'chunk_size_feed_forward': 0, 'output_scores': False, 'return_dict_in_generate': False, 'forced_bos_token_id': None, 'forced_eos_token_id': None, 'remove_invalid_values': False, 'exponential_decay_length_penalty': None, 'suppress_tokens': None, 'begin_suppress_tokens': None, 'architectures': ['T5ForConditionalGeneration'], 'finetuning_task': None, 'id2label': {'0': 'LABEL_0', '1': 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_1': 1}, 'tokenizer_class': None, 'prefix': None, 'bos_token_id': None, 'pad_token_id': 0, 'eos_token_id': 1, 'sep_token_id': None, 'decoder_start_token_id': 0, 'task_specific_params': {'summarization': {'early_stopping': True, 'length_penalty': 2.0, 'max_length': 200, 'min_length': 30, 'no_repeat_ngram_size': 3, 'num_beams': 4, 'prefix': 'summarize: '}, 'translation_en_to_de': {'early_stopping': True, 'max_length': 300, 'num_beams': 4, 'prefix': 'translate English to German: '}, 'translation_en_to_fr': {'early_stopping': True, 'max_length': 300, 'num_beams': 4, 'prefix': 'translate English to French: '}, 'translation_en_to_ro': {'early_stopping': True, 'max_length': 300, 'num_beams': 4, 'prefix': 'translate English to Romanian: '}}, 'problem_type': None, '_name_or_path': 'google/flan-t5-base', 'transformers_version': '4.34.1', 'model_type': 't5', 'n_positions': 512, 'output_past': True}, 'registered_name': 'TFT5ForConditionalGeneration', 'compile_config': {'optimizer': {'module': 'transformers.optimization_tf', 'class_name': 'AdamWeightDecay', 'config': {'name': 'AdamWeightDecay', 'learning_rate': 1.9999999494757503e-05, 'decay': 0.0, 'beta_1': 0.8999999761581421, 'beta_2': 0.9990000128746033, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}, 'registered_name': 'AdamWeightDecay'}, 'loss': {'module': 'builtins', 'class_name': 'function', 'config': 'dummy_loss', 'registered_name': 'function'}, 'metrics': None, 'loss_weights': None, 'weighted_metrics': None, 'run_eagerly': None, 'steps_per_execution': None, 'jit_compile': None}}

​

### Expected behavior

Should be able to load the model and continue with predictions

Comments:
Sorry Pls ignore the above error stack and consider the below one.

I am getting ValueError and the updated error stack is given below

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[2], line 2
      1 from tensorflow.keras.models import load_model
----> 2 model=load_model('FlanT5-Chatbot_model.keras')

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_api.py:254, in load_model(filepath, custom_objects, compile, safe_mode, **kwargs)
    249     if kwargs:
    250         raise ValueError(
    251             "The following argument(s) are not supported "
    252             f"with the native Keras format: {list(kwargs.keys())}"
    253         )
--> 254     return saving_lib.load_model(
    255         filepath,
    256         custom_objects=custom_objects,
    257         compile=compile,
    258         safe_mode=safe_mode,
    259     )
    261 # Legacy case.
    262 return legacy_sm_saving_lib.load_model(
    263     filepath, custom_objects=custom_objects, compile=compile, **kwargs
    264 )

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:281, in load_model(filepath, custom_objects, compile, safe_mode)
    278             asset_store.close()
    280 except Exception as e:
--> 281     raise e
    282 else:
    283     return model

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:269, in load_model(filepath, custom_objects, compile, safe_mode)
    266 else:
    267     asset_store = None
--> 269 _load_state(
    270     model,
    271     weights_store=weights_store,
    272     assets_store=asset_store,
    273     inner_path="",
    274     visited_trackables=set(),
    275 )
    276 weights_store.close()
    277 if asset_store:

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:457, in _load_state(trackable, weights_store, assets_store, inner_path, skip_mismatch, visited_trackables)
    455 for child_attr, child_obj in _walk_trackable(trackable):
    456     if _is_keras_trackable(child_obj):
--> 457         _load_state(
    458             child_obj,
    459             weights_store,
    460             assets_store,
    461             inner_path=tf.io.gfile.join(inner_path, child_attr),
    462             skip_mismatch=skip_mismatch,
    463             visited_trackables=visited_trackables,
    464         )
    465     elif isinstance(child_obj, (list, dict, tuple, set)):
    466         _load_container_state(
    467             child_obj,
    468             weights_store,
   (...)
    472             visited_trackables=visited_trackables,
    473         )

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:466, in _load_state(trackable, weights_store, assets_store, inner_path, skip_mismatch, visited_trackables)
    457     _load_state(
    458         child_obj,
    459         weights_store,
   (...)
    463         visited_trackables=visited_trackables,
    464     )
    465 elif isinstance(child_obj, (list, dict, tuple, set)):
--> 466     _load_container_state(
    467         child_obj,
    468         weights_store,
    469         assets_store,
    470         inner_path=tf.io.gfile.join(inner_path, child_attr),
    471         skip_mismatch=skip_mismatch,
    472         visited_trackables=visited_trackables,
    473     )

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:534, in _load_container_state(container, weights_store, assets_store, inner_path, skip_mismatch, visited_trackables)
    532 else:
    533     used_names[name] = 0
--> 534 _load_state(
    535     trackable,
    536     weights_store,
    537     assets_store,
    538     inner_path=tf.io.gfile.join(inner_path, name),
    539     skip_mismatch=skip_mismatch,
    540     visited_trackables=visited_trackables,
    541 )

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:466, in _load_state(trackable, weights_store, assets_store, inner_path, skip_mismatch, visited_trackables)
    457     _load_state(
    458         child_obj,
    459         weights_store,
   (...)
    463         visited_trackables=visited_trackables,
    464     )
    465 elif isinstance(child_obj, (list, dict, tuple, set)):
--> 466     _load_container_state(
    467         child_obj,
    468         weights_store,
    469         assets_store,
    470         inner_path=tf.io.gfile.join(inner_path, child_attr),
    471         skip_mismatch=skip_mismatch,
    472         visited_trackables=visited_trackables,
    473     )

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:534, in _load_container_state(container, weights_store, assets_store, inner_path, skip_mismatch, visited_trackables)
    532 else:
    533     used_names[name] = 0
--> 534 _load_state(
    535     trackable,
    536     weights_store,
    537     assets_store,
    538     inner_path=tf.io.gfile.join(inner_path, name),
    539     skip_mismatch=skip_mismatch,
    540     visited_trackables=visited_trackables,
    541 )

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:457, in _load_state(trackable, weights_store, assets_store, inner_path, skip_mismatch, visited_trackables)
    455 for child_attr, child_obj in _walk_trackable(trackable):
    456     if _is_keras_trackable(child_obj):
--> 457         _load_state(
    458             child_obj,
    459             weights_store,
    460             assets_store,
    461             inner_path=tf.io.gfile.join(inner_path, child_attr),
    462             skip_mismatch=skip_mismatch,
    463             visited_trackables=visited_trackables,
    464         )
    465     elif isinstance(child_obj, (list, dict, tuple, set)):
    466         _load_container_state(
    467             child_obj,
    468             weights_store,
   (...)
    472             visited_trackables=visited_trackables,
    473         )

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\saving\saving_lib.py:435, in _load_state(trackable, weights_store, assets_store, inner_path, skip_mismatch, visited_trackables)
    428             warnings.warn(
    429                 f"Could not load weights in object {trackable}. "
    430                 "Skipping object. "
    431                 f"Exception encountered: {e}",
    432                 stacklevel=2,
    433             )
    434     else:
--> 435         trackable.load_own_variables(weights_store.get(inner_path))
    437 if hasattr(trackable, "load_assets") and assets_store:
    438     if skip_mismatch:

File ~\AppData\Roaming\Python\Python311\site-packages\keras\src\engine\base_layer.py:3531, in Layer.load_own_variables(self, store)
   3529 all_vars = self._trainable_weights + self._non_trainable_weights
   3530 if len(store.keys()) != len(all_vars):
-> 3531     raise ValueError(
   3532         f"Layer '{self.name}' expected {len(all_vars)} variables, "
   3533         "but received "
   3534         f"{len(store.keys())} variables during loading. "
   3535         f"Expected: {[v.name for v in all_vars]}"
   3536     )
   3537 for i, v in enumerate(all_vars):
   3538     # TODO(rchao): check shapes and raise errors.
   3539     v.assign(store[f"{i}"])

ValueError: Layer 'SelfAttention' expected 0 variables, but received 1 variables during loading. Expected: []
      

Hi @SrikanthChellappa, thanks for raising this issue! 

The errors your receiving are all from within the keras library, not `transformers` and the code example provided uses just tensorflow/keras functionality. As such, this isn't an issue for this repo. 

If you do wish to use a model from the transformers library, the recommended way to load and save models in transformers is by using the `from_pretrained` and `save_pretrained` methods. This will save out all the necessary files, including the weights and model config e.g.: 

```py
import BertConfig, BertModel

# Create a model
config = BertConfig(num_attention_heads=4)
model = BertModel(config)

# Save out the model
model.save_pretrained("my_model_name")

# Load a saved model
model = BertModel.from_pretrained("my_model_name")
```

If you want to know more, I suggest looking through the [quicktour in the docs](https://huggingface.co/docs/transformers/quicktour).

Thanks @amyeroberts 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.