dkudos/cinimod-devops

Cinimod DevOps 1.5B

A 1.5B parameter (781M active) Llama-3 style language model, trained from scratch on DevOps and cloud infrastructure data. Designed for answering technical questions about Kubernetes, Terraform, AWS, Docker, CI/CD, Linux administration, and cloud operations.

Model Details

Parameter	Value
Architecture	LlamaForCausalLM (custom)
Parameters	~781M active (1.5B config name)
Hidden Size	1536
Layers	24
Attention Heads	24 (Q) / 6 (KV) - GQA
Intermediate Size	4096
Vocab Size	65,536 (BPE)
Context Window	1,024 tokens (training)
RoPE Theta	500,000
Training Dtype	FP32 (full precision)
Inference Dtype	FP16
Tokenizer	Custom BPE, 65K vocab

Training Details

Pretraining

• Data: ~50GB curated DevOps corpus (FineWeb-Edu subset, GitHub DevOps repos, cloud provider documentation)
• Steps: 60,423
• Batch Size: 8 (2 GPUs x 1 per GPU x 4 grad accum)
• Learning Rate: Peak 3e-4, cosine decay
• Optimizer: DeepSpeed ZeRO-2
• Sequence Length: 1,024
• Hardware: 2x Tesla V100 16GB
• Training Time: ~6 days

Architecture Notes

• No RoPE scaling - previous linear scaling (factor=256) broke positional encoding during training, pinning loss at unigram floor
• Gradient checkpointing enabled for memory efficiency
• Tied word embeddings for parameter savings

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('dkudos/cinimod-devops')
model = AutoModelForCausalLM.from_pretrained(
    'dkudos/cinimod-devops',
    torch_dtype='auto',
    device_map='auto',
    trust_remote_code=True
)

prompt = 'How do I set up a Kubernetes Deployment?'
inputs = tokenizer(prompt, return_tensors='pt').to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

OpenAI-Compatible Server

python scripts/serve_model.py --model_path dkudos/cinimod-devops --tokenizer_path tokenizer --port 8080

Capabilities

The model was trained on a broad DevOps corpus covering:

• Kubernetes: Pods, Deployments, StatefulSets, Services, Ingress, networking, storage, RBAC, Helm, operators, troubleshooting
• Terraform: IaC, state management, modules, providers, workspaces, import, best practices
• AWS: EC2, S3, VPC, RDS, EKS, Lambda, IAM, CloudWatch, cost optimization
• Docker: Images, containers, volumes, networking, Compose, multi-stage builds, Swarm
• CI/CD: GitHub Actions, Jenkins, GitLab CI, ArgoCD, deployment strategies, pipelines
• Linux: System administration, networking, security, performance monitoring, systemd
• Monitoring: Prometheus, Grafana, Alertmanager, Loki, Elasticsearch, distributed tracing
• Security: Zero trust, vulnerability scanning, incident response, penetration testing
• Networking: DNS, load balancing, CDN, SSL/TLS, troubleshooting

Limitations

• Context window: 1,024 tokens training context (RoPE theta 500K allows extrapolation to ~256K at inference, but quality may degrade)
• Parameter count: 781M params - smaller than typical production models; complex reasoning may be limited
• Training data: ~50GB - curated but limited compared to frontier models
• No instruction tuning: This is the base model. Chat-tuned versions available separately.
• Hallucination: May produce incorrect technical details, especially for niche topics or version-specific APIs
• No code execution: Cannot run commands or verify solutions - always test in a safe environment

Ethical Considerations

This model is designed as a technical assistant for DevOps professionals. Always verify generated configurations and commands before applying to production systems. Infrastructure changes can have significant consequences if applied without review.

Citation

@misc{cinimod-devops-1b5,
  title={Cinimod DevOps 1.5B},
  author={Dominic Kaiser},
  year={2026},
  url={https://huggingface.co/dkudos/cinimod-devops}
}