projecte-aina/FLOR-6.3B · [SOLVED] "KeyError: 'architectures'" when trying to convert the model using convert-hf-to-gguf.py from llama.cpp

When trying to convert using the following command

python3 convert-hf-to-gguf.py --outtype f16 /media/psf/2TB/Software/AI/Models/FLOR-6.3B

It outputs the following error message:

Loading model: FLOR-6.3B
Traceback (most recent call last):
  File "/home/christopher/llama.cpp/convert-hf-to-gguf.py", line 1054, in <module>
    model_class = Model.from_model_architecture(hparams["architectures"][0])
                                                ~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: 'architectures'

I solved it with this suggestion https://github.com/ggerganov/llama.cpp/issues/4877

Basicaly I had to add a few lines to the model's config.json file, which should look like this:

{
    "architectures": [
        "BloomForCausalLM"
    ],
    "vocab_size": 50257,
    "hidden_size": 4096,
    "tie_word_embeddings": true,
    "n_layer": 30,
    "hidden_dropout": 0.0,
    "layer_norm_epsilon": 1e-05,
    "n_head": 32,
    "attention_dropout": 0.0,
    "model_type": "bloom"
}