pszemraj
/

opt-350m-email-generation

Text Generation Transformers PyTorch Safetensors

opt generated_from_trainer custom-license no-commercial email auto-complete Inference Endpoints text-generation-inference

Model card Files Files and versions Community

opt-350m-email-generation / config.json

pszemraj's picture

Change BOS token from 0 to 2 as BOS token is equal to EOS for OPT. See: https://github.com/huggingface/transformers/issues/17431 (#2)

c4d485b almost 2 years ago

raw history blame contribute delete

No virus

649 Bytes

	{
	"_name_or_path": "facebook/opt-350m",
	"activation_dropout": 0.0,
	"activation_function": "relu",
	"architectures": [
	"OPTForCausalLM"
	],
	"attention_dropout": 0.0,
	"bos_token_id": 2,
	"do_layer_norm_before": false,
	"dropout": 0.1,
	"eos_token_id": 2,
	"ffn_dim": 4096,
	"hidden_size": 1024,
	"init_std": 0.02,
	"layerdrop": 0.0,
	"max_position_embeddings": 2048,
	"model_type": "opt",
	"num_attention_heads": 16,
	"num_hidden_layers": 24,
	"pad_token_id": 1,
	"prefix": "</s>",
	"torch_dtype": "float32",
	"transformers_version": "4.19.2",
	"use_cache": false,
	"vocab_size": 50265,
	"word_embed_proj_dim": 512
	}