Edit model card


CodeMate-v0.1 is an intelligent programming assistant developed by CodeMate. This model aims to assist users in generating high-quality code solutions for programming problems. Please note that this model is currently in version 0.1.

Model Details

  • Training Data: Exclusively fine-tuned on a proprietary dataset of 1.8 billion tokens of high-quality programming problems and solutions.

  • The dataset was generated manually and is internal to CodeMate.

  • Training Techniques: The model was fine-tuned using Flash Attention 2, trained over 15 hours on 40 A100-80GB GPUs.

  • A sequence length of 8096 tokens was used during training.

  • Multilingual Support: CodeMate-v0.1 is proficient in multiple programming languages, including Python, C/C++, TypeScript, Java, and more.

How to Get Started with the Model

Make sure to install Transformers from the main git branch:

pip install git+https://github.com/huggingface/transformers.git

How to Prompt the Model

This model accepts prompts in the Alpaca/Vicuna instruction format. For example:

### System Prompt
You are an intelligent programming assistant.

### User Message
Implement a linked list in C++

### Assistant

Load the Model:

To load the model, utilize the following Python script:

from transformers import AutoTokenizer, AutoModelForCausalLM

# Initialize the model
model_path = "codemateai/CodeMate-v0.1"
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_path)

# ... generate response ...

Bias, Risks, and Limitations

This model has undergone very limited testing. CodeMate recommends additional safety testing before any real-world deployments.

For more information and updates, visit the CodeMate website.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 58.39
AI2 Reasoning Challenge (25-Shot) 55.55
HellaSwag (10-Shot) 78.03
MMLU (5-Shot) 55.31
TruthfulQA (0-shot) 48.64
Winogrande (5-shot) 72.61
GSM8k (5-shot) 40.18
Downloads last month
Model size
33.7B params
Tensor type
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Evaluation results