nvidia

/

Hymba-1.5B-Base

like 123

Follow

NVIDIA 5.89k

When loading the model in transformers with

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("nvidia/Hymba-1.5B-Base", trust_remote_code=True)

attempting to debug or print model.config results in the error:

>>> model.config
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/transformers/configuration_utils.py", line 778, in __repr__
    return f"{self.__class__.__name__} {self.to_json_string()}"
                                        ^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/transformers/configuration_utils.py", line 899, in to_json_string
    return json.dumps(config_dict, indent=2, sort_keys=True) + "\n"
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/envs/py3.11/lib/python3.11/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
          ^^^^^^^^^^^
  File "/root/miniconda3/envs/py3.11/lib/python3.11/json/encoder.py", line 202, in encode
    chunks = list(chunks)
             ^^^^^^^^^^^^
  File "/root/miniconda3/envs/py3.11/lib/python3.11/json/encoder.py", line 432, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/root/miniconda3/envs/py3.11/lib/python3.11/json/encoder.py", line 406, in _iterencode_dict
    yield from chunks
  File "/root/miniconda3/envs/py3.11/lib/python3.11/json/encoder.py", line 354, in _iterencode_dict
    items = sorted(dct.items())
            ^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

the issue is more apparent when using model.config.to_dict() and inspecting the conv_dim key:

'conv_dim': {'0': 3200, '1': 3200, '2': 3200, '3': 3200, '4': 3200, '5': 3200, '6': 3200, '7': 3200, '8': 3200, '9': 3200, '10': 3200, '11': 3200, '12': 3200, '13': 3200, '14': 3200, '15': 3200, '16': 3200, '17': 3200, '18': 3200, '19': 3200, '20': 3200, '21': 3200, '22': 3200, '23': 3200, '24': 3200, '25': 3200, '26': 3200, '27': 3200, '28': 3200, '29': 3200, '30': 3200, '31': 3200, 0: 3200, 1: 3200, 2: 3200, 3: 3200, 4: 3200, 5: 3200, 6: 3200, 7: 3200, 8: 3200, 9: 3200, 10: 3200, 11: 3200, 12: 3200, 13: 3200, 14: 3200, 15: 3200, 16: 3200, 17: 3200, 18: 3200, 19: 3200, 20: 3200, 21: 3200, 22: 3200, 23: 3200, 24: 3200, 25: 3200, 26: 3200, 27: 3200, 28: 3200, 29: 3200, 30: 3200, 31: 3200}

here you can see the layer indexes end up as both int and strings leading to the error above.

Thanks for pointing it out and fixing it.