openaccess-ai-collective
/

manticore-13b

Text Generation Transformers PyTorch

English llama Inference Endpoints text-generation-inference

Model card Files Files and versions Community

manticore-13b / config.json

winglian's picture

inital commit of things without the model

7ef44a9 11 months ago

raw history blame contribute delete

No virus

582 Bytes

	{
	"_name_or_path": "huggyllama/llama-13b",
	"architectures": [
	"LlamaForCausalLM"
	],
	"bos_token_id": 1,
	"eos_token_id": 2,
	"hidden_act": "silu",
	"hidden_size": 5120,
	"initializer_range": 0.02,
	"intermediate_size": 13824,
	"max_position_embeddings": 2048,
	"max_sequence_length": 2048,
	"model_type": "llama",
	"num_attention_heads": 40,
	"num_hidden_layers": 40,
	"pad_token_id": 0,
	"rms_norm_eps": 1e-06,
	"tie_word_embeddings": false,
	"torch_dtype": "bfloat16",
	"transformers_version": "4.30.0.dev0",
	"use_cache": true,
	"vocab_size": 32000
	}