adept

/

fuyu-8b

like 957

Using more or less this code (transformers@caa0ff0bf104ee68ea46432b298a534e9a4a02a4):

from transformers import FuyuForCausalLM, AutoTokenizer
…
FuyuForCausalLM.from_pretrained("adept/fuyu-8b")

The checkpoint seems to download ok, but doesn't load:

Some weights of FuyuForCausalLM were not initialized from the model checkpoint at adept/fuyu-8b and are newly initialized: ['language_model.model.layers.27.self_attn.query_key_value.bias', 'language_model.model.layers.20.input_layernorm.weight', 'language_model.model.layers.10.self_attn.k_layernorm.weight', 'language_model.model.layers.21.mlp.dense_h_to_4h.weight', 'language_model.model.layers.3.input_layernorm.weight', 'language_model.model.layers.6.self_attn.dense.bias', 'language_model.model.layers.2.self_attn.query_key_value.weight', 'language_model.model.layers.11.self_attn.query_key_value.weight', 'language_model.model.layers.30.input_layernorm.weight', 'language_model.model.layers.22.mlp.dense_4h_to_h.bias', 'language_model.model.layers.16.post_attention_layernorm.bias'…