Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,116 @@
+---
+license: apache-2.0
+language:
+- en
+tags:
+- text-generation
+- cli
+- shell
+- command-line
+- sft
+- instruction-following
+pipeline_tag: text-generation
+widget:
+- text: "Instruction: List all files in the current directory\nCommand:"
+  example_title: List files
+- text: "Instruction: Find all Python files\nCommand:"
+  example_title: Find Python files
+- text: "Instruction: Show disk usage\nCommand:"
+  example_title: Disk usage
+---
+
+# Tiny-LLM CLI SFT (54M)
+
+A **54 million parameter** language model fine-tuned for CLI command generation.
+
+## Model Description
+
+This model is a Supervised Fine-Tuned (SFT) version of [jonmabe/tiny-llm-54m](https://huggingface.co/jonmabe/tiny-llm-54m), trained to generate Unix/Linux shell commands from natural language instructions.
+
+### Training Data
+
+- **Geddy's NL2Bash dataset**: ~2,300 natural language to bash command pairs
+- **NL2Bash benchmark**: Standard benchmark for command translation
+- **Synthetic examples**: Additional generated pairs
+- **Total**: ~13,000 training pairs
+
+### Training Details
+
+| Parameter | Value |
+|-----------|-------|
+| Base Model | tiny-llm-54m |
+| Training Steps | 2,000 |
+| Best Checkpoint | Step 1,000 |
+| Best Val Loss | 1.2456 |
+| Learning Rate | 5e-5 |
+| Batch Size | 16 |
+| Hardware | NVIDIA RTX 5090 |
+| Training Time | ~9 minutes |
+
+## Architecture
+
+- **Parameters**: 54.93M
+- **Layers**: 12
+- **Hidden Size**: 512
+- **Attention Heads**: 8
+- **Intermediate Size**: 1408
+- **Max Position**: 512
+- **Vocabulary**: 32,000 tokens
+- **Features**: RoPE, RMSNorm, SwiGLU, Weight Tying
+
+## Usage
+
+### Prompt Format
+
+```
+Instruction: <natural language description>
+Command:
+```
+
+### Example
+
+```python
+from model import TinyLLM
+import torch
+
+# Load model
+checkpoint = torch.load("best_model.pt", map_location="cpu")
+model = TinyLLM(checkpoint["config"]["model"])
+model.load_state_dict(checkpoint["model_state_dict"])
+model.eval()
+
+# Generate
+prompt = "Instruction: Find all Python files modified in the last day\nCommand:"
+# ... tokenize and generate
+```
+
+## Limitations
+
+⚠️ **Known Issues:**
+- Tokenizer decode shows raw BPE tokens (Ġ = space, Ċ = newline)
+- Model generates fragments of correct commands but output can be noisy
+- Needs more training steps for reliable generation
+- Small model size limits command complexity
+
+## Improvement Plan
+
+1. **Fix tokenizer decode** - Proper BPE to text conversion
+2. **Longer training** - 5,000-10,000 steps
+3. **Data quality** - Curate cleaner training pairs
+4. **Lower LR** - More stable convergence with 1e-5
+
+## License
+
+Apache 2.0
+
+## Citation
+
+```bibtex
+@misc{tiny-llm-cli-sft-2026,
+  author = {Jon Mabe},
+  title = {Tiny-LLM CLI SFT: Small Language Model for Command Generation},
+  year = {2026},
+  publisher = {HuggingFace},
+  url = {https://huggingface.co/jonmabe/tiny-llm-cli-sft}
+}
+```