vicenherrera
/

experiment-peft-starcoder-lora-a100

Model card Files Files and versions Metrics Training metrics Community

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

RESEARCH EXPERIMENT, DO NOT USE

This model is a cybersecurity research experiment, do not use.

base_model: bigcode/starcoderbase-1b library_name: peft license: bigcode-openrail-m tags: - generated_from_trainer model-index: - name: peft-starcoder-lora-a100 results: []

peft-starcoder-lora-a100

This model is a fine-tuned version of bigcode/starcoderbase-1b on vicenherrera/experiment-hf-stack-v1 dataset. It achieves the following results on the evaluation set:

Loss: 1.0542

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0005
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 30
training_steps: 2000

Training results

Training Loss	Epoch	Step	Validation Loss
0.9713	0.05	100	0.9314
0.9809	0.1	200	0.9649
0.6441	0.15	300	0.9632
0.9071	0.2	400	0.9757
0.9452	0.25	500	0.9758
0.5982	0.3	600	1.0048
0.8401	0.35	700	1.0010
0.8431	0.4	800	1.0066
0.5342	0.45	900	1.0253
0.6812	0.5	1000	1.0291
0.7754	0.55	1100	1.0255
0.4703	0.6	1200	1.0409
0.7191	0.65	1300	1.0441
0.6917	0.7	1400	1.0457
0.5528	0.75	1500	1.0506
0.6626	0.8	1600	1.0513
0.5795	0.85	1700	1.0541
0.5888	0.9	1800	1.0568
0.6415	0.95	1900	1.0565
0.5606	1.0	2000	1.0542

Framework versions

PEFT 0.11.1
Transformers 4.41.2
Pytorch 2.3.0+cu121
Datasets 2.20.0
Tokenizers 0.19.1

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model's library.