improved README
Browse files
README.md
CHANGED
|
@@ -1,4 +1,6 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
license: bigcode-openrail-m
|
| 3 |
datasets:
|
| 4 |
- bigcode/the-stack-dedup
|
|
@@ -6,11 +8,62 @@ datasets:
|
|
| 6 |
pipeline_tag: text-generation
|
| 7 |
tags:
|
| 8 |
- code
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
---
|
| 10 |
|
| 11 |
-
[Santacoder](https://huggingface.co/bigcode/santacoder) finetuned on [
|
| 12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
-
|
|
|
|
| 15 |
|
| 16 |
License carried over from model, however training data has an undefied license. Check details in [Shadertoys](https://huggingface.co/datasets/Vipitis/Shadertoys).
|
|
|
|
| 1 |
---
|
| 2 |
+
language:
|
| 3 |
+
- code
|
| 4 |
license: bigcode-openrail-m
|
| 5 |
datasets:
|
| 6 |
- bigcode/the-stack-dedup
|
|
|
|
| 8 |
pipeline_tag: text-generation
|
| 9 |
tags:
|
| 10 |
- code
|
| 11 |
+
- shader
|
| 12 |
+
widget:
|
| 13 |
+
- text: void mainImage( out vec4 fragColor, in vec2 fragCoord )
|
| 14 |
+
example_title: mainImage
|
| 15 |
+
group: Shadertoy
|
| 16 |
+
model-index:
|
| 17 |
+
- name: santacoder-finetuned-the-stack-glsl
|
| 18 |
+
results:
|
| 19 |
+
- task:
|
| 20 |
+
type: text-generation
|
| 21 |
+
name: ShaderEval
|
| 22 |
+
dataset:
|
| 23 |
+
type: Vipitis/Shadertoys-fine
|
| 24 |
+
name: Shadertoys-fine
|
| 25 |
+
config: return_completion
|
| 26 |
+
revision: 0.0.2
|
| 27 |
+
metrics:
|
| 28 |
+
- type: exact_match
|
| 29 |
+
value: 0.567
|
| 30 |
+
name: 300 samples, greedy decoding
|
| 31 |
+
verified: false
|
| 32 |
+
- type: exact_match
|
| 33 |
+
value: 0.59749
|
| 34 |
+
name: all samples, greedy decoding
|
| 35 |
+
verified: false
|
| 36 |
---
|
| 37 |
|
| 38 |
+
[Santacoder](https://huggingface.co/bigcode/santacoder) finetuned on [The-Stack-deddup (GLSL subset)](https://huggingface.co/datasets/bigcode/the-stack-dedup/tree/main/data/glsl) for 1000 steps with a batch size of 2 and full sequence length of 2048.
|
| 39 |
+
adapted finetuning script found [here](./train.py)
|
| 40 |
+
|
| 41 |
+
### Finetuning parameters
|
| 42 |
+
```sh
|
| 43 |
+
python3 train.py --model_path "bigcode/santacoder" \
|
| 44 |
+
--dataset_name "Vipitis/Shadertoys-fine" \
|
| 45 |
+
--data_column "code" \
|
| 46 |
+
--split "train" \
|
| 47 |
+
--seq_length 2048 \
|
| 48 |
+
--max_steps 1000 \
|
| 49 |
+
--batch_size 2 \
|
| 50 |
+
--gradient_accumulation_steps 4 \
|
| 51 |
+
--learning_rate 5e-5 \
|
| 52 |
+
--num_warmup_steps 100 \
|
| 53 |
+
--eval_freq 100 \
|
| 54 |
+
--save_freq 100 \
|
| 55 |
+
--log_freq 1 \
|
| 56 |
+
--output_dir "checkpoint_dir" \
|
| 57 |
+
--no_fp16
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Main purpose of this model is to explore if finetuning models improves performance on [ShaderEval](https://huggingface.co/spaces/Vipitis/ShaderEval), which reached 0.567 with 300 samples and 0.59749 on all samples.
|
| 63 |
+
|
| 64 |
+
### Disclaimer
|
| 65 |
|
| 66 |
+
While the train/test split is held out, there is a lot of data contamination. The model results can't be trusted for this simple benchmark.
|
| 67 |
+
Better tasks for the benchmark will be developed and tested against these models.
|
| 68 |
|
| 69 |
License carried over from model, however training data has an undefied license. Check details in [Shadertoys](https://huggingface.co/datasets/Vipitis/Shadertoys).
|