Focus V1 Instruct
Focus V1 Instruct is an instruction-tuned language model developed by BRSX Labs. It is based on Focus V1 Base and was adapted through supervised fine-tuning (SFT) to generate assistant-style responses and follow instructions.
The model was pretrained primarily on programming and software-related content, alongside text from other topics and task types. It is intended for experimentation with small language models, English instruction following, programming-related tasks, and text generation.
Model Details
- Developer: BRSX Labs
- Model family: Focus V1
- Model type: Decoder-only Transformer
- Parameter count: Approximately 100 million
- Language: English
- Context length: 2,048 tokens
- Pretraining: 6 billion tokens
- Post-training: Supervised fine-tuning (SFT) on 150,000 examples
- License: Apache-2.0
Architecture
| Property | Value |
|---|---|
| Transformer layers | 12 |
Hidden size (d_model) |
768 |
| Attention mechanism | Grouped-Query Attention (GQA) |
| Query heads | 12 |
| Key/value heads | 4 |
| Context length | 2,048 tokens |
Base Model
Focus V1 Instruct is based on BRSX Labs / Focus V1 Base.
Intended Uses
- English instruction following and text generation
- Programming and software-development-related tasks
- Generating code examples and working with technical text
- Research and experimentation with small language models
- Further fine-tuning and model-development experiments
Limitations
Focus V1 Instruct is a small language model. It may produce incorrect, incomplete, or inconsistent answers, especially on complex reasoning, advanced mathematics, long conversations, and tasks requiring specialized knowledge. Generated code and factual claims should be checked before use in critical applications.
The model's stated pretraining token count and parameter count do not by themselves guarantee performance on any particular benchmark or task. Evaluate it on your own use cases before deployment.
Training
The base model was pretrained on 6 billion tokens, with the training data consisting primarily of programming and software-related content as well as material from other topics. Focus V1 Instruct was then adapted using supervised fine-tuning (SFT) with 150,000 training examples.
Detailed dataset composition, training hyperparameters, and benchmark results are not specified in this model card.
License
This model is released under the Apache-2.0 license. Review the repository's license file and the licenses and terms of any training data before redistributing or using the model.
Developer
BRSX Labs
Website: https://brsxlabs.com
- Downloads last month
- 2