Model Card: llama2_7b_PCG

Model Details

• Developed by: Sanjay Shrestha (sasudo2)
• Model Type: Causal Language Model (LlamaForCausalLM) based on the Llama architecture.
• Model Size: ~7 Billion parameters ($\approx 13.5\text{ GB}$ safetensors weights).
• Language(s): English (with fine-tuned adaptation tracks).
• License: Apache-2.0

Model Architecture & Configuration

The model inherits standard Llama structural parameters with custom decoding configurations defined in its config.json and generation_config.json:

• Hidden Size ($hidden_size$): 4096
• Number of Hidden Layers ($num_hidden_layers$): 32
• Attention Heads ($num_attention_heads$): 32 (with 32 Key-Value heads)
• Intermediate Size ($intermediate_size$): 11,008
• Max Position Embeddings: 4096 tokens
• Activation Function: Silu (hidden_act: silu)
• Precision/Data Type: float16

Tokenization & Prompt Template

• Tokenizer Class: LlamaTokenizer (backed by tokenizer.json).
• Vocabulary Size: 32,000 tokens.
• Chat Template: Uses a Llama-2 instruction template format configured in tokenizer_config.json and chat_template.jinja, structuring inputs with standard system tags (<<SYS>>), beginning-of-sequence tokens (<s>), and instruction wrappers ([INST] ... [/INST]).

Generation Parameters

The model's default sampling behavior is tuned for creative code generation or domain-specific generation tasks:

• Sampling: Enabled (do_sample: true)
• Temperature: 0.6
• Top-P (Nucleus Sampling): 0.9
• Max Length: 4096 tokens
• BOS / EOS Tokens: ID 1 (<s>) and ID 2 (</s>)

---

Quick Usage Example

You can load and interact with this model locally or via Python using the transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sasudo2/llama2_7b_PCG"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype="auto")

inputs = tokenizer("Write a Python script for...", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))