Neurologist-7B-Instruct

A 7B parameter instruction-tuned language model fine-tuned for technical reasoning, code generation, and systems-level problem solving.

Model Description

Neurologist-7B-Instruct is a fine-tune of NousResearch/Hermes-3-Llama-3.1-8B trained on a curated mix of:

• OpenOrca filtered for technical/scientific content
• Custom dataset of systems programming Q&A (DMA, kernel modules, hardware interfaces)
• Code instruction pairs focused on C++, Rust, Python, and CUDA

Training was done on a single 9950X + RTX setup using QLoRA (4-bit) with the following configuration:

Parameter	Value
Base model	Hermes-3-Llama-3.1-8B
Method	QLoRA (4-bit NF4)
LoRA rank	64
LoRA alpha	128
Learning rate	2e-4
Epochs	3
Batch size	4 (gradient accumulation 8)
Context length	8192
Training time	~18 hours

Intended Use

• Technical Q&A and reasoning
• Code generation (C++, Rust, Python, CUDA)
• Systems architecture discussion
• Hardware/low-level programming assistance

Usage

Benchmarks

Benchmark	Score
MMLU (5-shot)	63.8
HellaSwag	79.2
ARC-Challenge	54.1
HumanEval	41.5
MBPP	48.2

Limitations

This model inherits the limitations of its base model. It may occasionally generate incorrect code or technical information. Always verify generated code before use in production systems.

Citation