File size: 1,684 Bytes
e7a02df 5bef32f e7a02df 39391f2 5bef32f 5382601 5bef32f 009da66 5bef32f e7a02df 39391f2 d4a0bc6 5bef32f 39391f2 5bef32f 39391f2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
---
language:
- code
license: bigcode-openrail-m
datasets:
- bigcode/the-stack-dedup
pipeline_tag: text-generation
tags:
- code
- shader
base_model: bigcode/santacoder
widget:
- text: void mainImage( out vec4 fragColor, in vec2 fragCoord )
example_title: mainImage
group: Shadertoy
model-index:
- name: santacoder-finetuned-the-stack-glsl
results:
- task:
type: text-generation
name: ShaderEval
dataset:
type: Vipitis/Shadertoys-fine
name: Shadertoys-fine
config: return_completion
revision: 0.0.2
metrics:
- type: exact_match
value: 0.380
name: 300 samples, greedy decoding
verified: false
---
[Santacoder](https://huggingface.co/bigcode/santacoder) finetuned on [The-Stack-dedup (GLSL subset)](https://huggingface.co/datasets/bigcode/the-stack-dedup/tree/main/data/glsl) for 1000 steps with a batch size of 2 and full sequence length of 2048.
adapted finetuning script found [here](./train.py)
### Finetuning parameters
```sh
python3 train.py --model_path "bigcode/santacoder" \
--dataset_name "bigcode/the-stack-dedup" \
--subset "data/glsl" \
--data_column "content" \
--split "train" \
--seq_length 2048 \
--max_steps 1000 \
--batch_size 2 \
--gradient_accumulation_steps 4 \
--learning_rate 5e-5 \
--num_warmup_steps 100 \
--eval_freq 100 \
--save_freq 100 \
--log_freq 1 \
--output_dir "checkpoint_dir" \
--no_fp16
```
Main purpose of this model is to explore if finetuning models improves performance on [ShaderEval](https://huggingface.co/spaces/Vipitis/ShaderEval), which reached 0.380 with 300 samples.
License carried over from model, and the finetuning dataset holds the same license. |