Hugging Face's logo

aengusl
/
llama2-7b-sft-lora

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files Metrics Community
aengusl commited on
Commit
40cb824
•
1 Parent(s): e4acb3d

Model save

Browse files
Files changed (7) hide show
  1. README.md +7 -6
  2. adapter_config.json +2 -2
  3. runs/Jun11_18-42-24_f18dbaae1158/events.out.tfevents.1718131535.f18dbaae1158.11913.0 +3 -0
  4. step_0/README.md +0 -476
  5. step_0/adapter_config.json +2 -2
  6. step_0/training_args.bin +1 -1
  7. training_args.bin +1 -1
README.md CHANGED
@@ -16,7 +16,7 @@ model-index:
16
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
  should probably proofread and complete it, then remove this comment. -->
18
 
19
- [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/quirky_lats_at_mats/harmbench_adv_training/runs/02q555gr)
20
  # llama2-7b-sft-lora
21
 
22
  This model is a fine-tuned version of [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) on the generator dataset.
@@ -39,13 +39,14 @@ More information needed
39
 
40
  The following hyperparameters were used during training:
41
  - learning_rate: 2e-05
42
- - train_batch_size: 2
43
- - eval_batch_size: 4
44
  - seed: 4
45
  - distributed_type: multi-GPU
46
- - num_devices: 2
47
- - total_train_batch_size: 4
48
- - total_eval_batch_size: 8
 
49
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
50
  - lr_scheduler_type: cosine
51
  - training_steps: 2000
 
16
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
  should probably proofread and complete it, then remove this comment. -->
18
 
19
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/quirky_lats_at_mats/harmbench_adv_training/runs/qcw9ss2y)
20
  # llama2-7b-sft-lora
21
 
22
  This model is a fine-tuned version of [meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) on the generator dataset.
 
39
 
40
  The following hyperparameters were used during training:
41
  - learning_rate: 2e-05
42
+ - train_batch_size: 4
43
+ - eval_batch_size: 8
44
  - seed: 4
45
  - distributed_type: multi-GPU
46
+ - num_devices: 8
47
+ - gradient_accumulation_steps: 4
48
+ - total_train_batch_size: 128
49
+ - total_eval_batch_size: 64
50
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
  - lr_scheduler_type: cosine
52
  - training_steps: 2000
adapter_config.json CHANGED
@@ -17,10 +17,10 @@
17
  "revision": null,
18
  "target_modules": [
19
  "o_proj",
20
- "q_proj",
21
- "v_proj",
22
  "down_proj",
23
  "k_proj",
 
 
24
  "up_proj"
25
  ],
26
  "task_type": "CAUSAL_LM"
 
17
  "revision": null,
18
  "target_modules": [
19
  "o_proj",
 
 
20
  "down_proj",
21
  "k_proj",
22
+ "q_proj",
23
+ "v_proj",
24
  "up_proj"
25
  ],
26
  "task_type": "CAUSAL_LM"
runs/Jun11_18-42-24_f18dbaae1158/events.out.tfevents.1718131535.f18dbaae1158.11913.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d71495c5608a5b0e967c050afd6d8a363ac7c8529b9b64a59563e6b6e900628
3
+ size 5063
step_0/README.md CHANGED
@@ -203,480 +203,4 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
203
  ### Framework versions
204
 
205
 
206
- - PEFT 0.6.1
207
- ## Training procedure
208
-
209
-
210
- ### Framework versions
211
-
212
-
213
- - PEFT 0.6.1
214
- ## Training procedure
215
-
216
-
217
- ### Framework versions
218
-
219
-
220
- - PEFT 0.6.1
221
- ## Training procedure
222
-
223
-
224
- ### Framework versions
225
-
226
-
227
- - PEFT 0.6.1
228
- ## Training procedure
229
-
230
-
231
- ### Framework versions
232
-
233
-
234
- - PEFT 0.6.1
235
- ## Training procedure
236
-
237
-
238
- ### Framework versions
239
-
240
-
241
- - PEFT 0.6.1
242
- ## Training procedure
243
-
244
-
245
- ### Framework versions
246
-
247
-
248
- - PEFT 0.6.1
249
- ## Training procedure
250
-
251
-
252
- ### Framework versions
253
-
254
-
255
- - PEFT 0.6.1
256
- ## Training procedure
257
-
258
-
259
- ### Framework versions
260
-
261
-
262
- - PEFT 0.6.1
263
- ## Training procedure
264
-
265
-
266
- ### Framework versions
267
-
268
-
269
- - PEFT 0.6.1
270
- ## Training procedure
271
-
272
-
273
- ### Framework versions
274
-
275
-
276
- - PEFT 0.6.1
277
- ## Training procedure
278
-
279
-
280
- ### Framework versions
281
-
282
-
283
- - PEFT 0.6.1
284
- ## Training procedure
285
-
286
-
287
- ### Framework versions
288
-
289
-
290
- - PEFT 0.6.1
291
- ## Training procedure
292
-
293
-
294
- ### Framework versions
295
-
296
-
297
- - PEFT 0.6.1
298
- ## Training procedure
299
-
300
-
301
- ### Framework versions
302
-
303
-
304
- - PEFT 0.6.1
305
- ## Training procedure
306
-
307
-
308
- ### Framework versions
309
-
310
-
311
- - PEFT 0.6.1
312
- ## Training procedure
313
-
314
-
315
- ### Framework versions
316
-
317
-
318
- - PEFT 0.6.1
319
- ## Training procedure
320
-
321
-
322
- ### Framework versions
323
-
324
-
325
- - PEFT 0.6.1
326
- ## Training procedure
327
-
328
-
329
- ### Framework versions
330
-
331
-
332
- - PEFT 0.6.1
333
- ## Training procedure
334
-
335
-
336
- ### Framework versions
337
-
338
-
339
- - PEFT 0.6.1
340
- ## Training procedure
341
-
342
-
343
- ### Framework versions
344
-
345
-
346
- - PEFT 0.6.1
347
- ## Training procedure
348
-
349
-
350
- ### Framework versions
351
-
352
-
353
- - PEFT 0.6.1
354
- ## Training procedure
355
-
356
-
357
- ### Framework versions
358
-
359
-
360
- - PEFT 0.6.1
361
- ## Training procedure
362
-
363
-
364
- ### Framework versions
365
-
366
-
367
- - PEFT 0.6.1
368
- ## Training procedure
369
-
370
-
371
- ### Framework versions
372
-
373
-
374
- - PEFT 0.6.1
375
- ## Training procedure
376
-
377
-
378
- ### Framework versions
379
-
380
-
381
- - PEFT 0.6.1
382
- ## Training procedure
383
-
384
-
385
- ### Framework versions
386
-
387
-
388
- - PEFT 0.6.1
389
- ## Training procedure
390
-
391
-
392
- ### Framework versions
393
-
394
-
395
- - PEFT 0.6.1
396
- ## Training procedure
397
-
398
-
399
- ### Framework versions
400
-
401
-
402
- - PEFT 0.6.1
403
- ## Training procedure
404
-
405
-
406
- ### Framework versions
407
-
408
-
409
- - PEFT 0.6.1
410
- ## Training procedure
411
-
412
-
413
- ### Framework versions
414
-
415
-
416
- - PEFT 0.6.1
417
- ## Training procedure
418
-
419
-
420
- ### Framework versions
421
-
422
-
423
- - PEFT 0.6.1
424
- ## Training procedure
425
-
426
-
427
- ### Framework versions
428
-
429
-
430
- - PEFT 0.6.1
431
- ## Training procedure
432
-
433
-
434
- ### Framework versions
435
-
436
-
437
- - PEFT 0.6.1
438
- ## Training procedure
439
-
440
-
441
- ### Framework versions
442
-
443
-
444
- - PEFT 0.6.1
445
- ## Training procedure
446
-
447
-
448
- ### Framework versions
449
-
450
-
451
- - PEFT 0.6.1
452
- ## Training procedure
453
-
454
-
455
- ### Framework versions
456
-
457
-
458
- - PEFT 0.6.1
459
- ## Training procedure
460
-
461
-
462
- ### Framework versions
463
-
464
-
465
- - PEFT 0.6.1
466
- ## Training procedure
467
-
468
-
469
- ### Framework versions
470
-
471
-
472
- - PEFT 0.6.1
473
- ## Training procedure
474
-
475
-
476
- ### Framework versions
477
-
478
-
479
- - PEFT 0.6.1
480
- ## Training procedure
481
-
482
-
483
- ### Framework versions
484
-
485
-
486
- - PEFT 0.6.1
487
- ## Training procedure
488
-
489
-
490
- ### Framework versions
491
-
492
-
493
- - PEFT 0.6.1
494
- ## Training procedure
495
-
496
-
497
- ### Framework versions
498
-
499
-
500
- - PEFT 0.6.1
501
- ## Training procedure
502
-
503
-
504
- ### Framework versions
505
-
506
-
507
- - PEFT 0.6.1
508
- ## Training procedure
509
-
510
-
511
- ### Framework versions
512
-
513
-
514
- - PEFT 0.6.1
515
- ## Training procedure
516
-
517
-
518
- ### Framework versions
519
-
520
-
521
- - PEFT 0.6.1
522
- ## Training procedure
523
-
524
-
525
- ### Framework versions
526
-
527
-
528
- - PEFT 0.6.1
529
- ## Training procedure
530
-
531
-
532
- ### Framework versions
533
-
534
-
535
- - PEFT 0.6.1
536
- ## Training procedure
537
-
538
-
539
- ### Framework versions
540
-
541
-
542
- - PEFT 0.6.1
543
- ## Training procedure
544
-
545
-
546
- ### Framework versions
547
-
548
-
549
- - PEFT 0.6.1
550
- ## Training procedure
551
-
552
-
553
- ### Framework versions
554
-
555
-
556
- - PEFT 0.6.1
557
- ## Training procedure
558
-
559
-
560
- ### Framework versions
561
-
562
-
563
- - PEFT 0.6.1
564
- ## Training procedure
565
-
566
-
567
- ### Framework versions
568
-
569
-
570
- - PEFT 0.6.1
571
- ## Training procedure
572
-
573
-
574
- ### Framework versions
575
-
576
-
577
- - PEFT 0.6.1
578
- ## Training procedure
579
-
580
-
581
- ### Framework versions
582
-
583
-
584
- - PEFT 0.6.1
585
- ## Training procedure
586
-
587
-
588
- ### Framework versions
589
-
590
-
591
- - PEFT 0.6.1
592
- ## Training procedure
593
-
594
-
595
- ### Framework versions
596
-
597
-
598
- - PEFT 0.6.1
599
- ## Training procedure
600
-
601
-
602
- ### Framework versions
603
-
604
-
605
- - PEFT 0.6.1
606
- ## Training procedure
607
-
608
-
609
- ### Framework versions
610
-
611
-
612
- - PEFT 0.6.1
613
- ## Training procedure
614
-
615
-
616
- ### Framework versions
617
-
618
-
619
- - PEFT 0.6.1
620
- ## Training procedure
621
-
622
-
623
- ### Framework versions
624
-
625
-
626
- - PEFT 0.6.1
627
- ## Training procedure
628
-
629
-
630
- ### Framework versions
631
-
632
-
633
- - PEFT 0.6.1
634
- ## Training procedure
635
-
636
-
637
- ### Framework versions
638
-
639
-
640
- - PEFT 0.6.1
641
- ## Training procedure
642
-
643
-
644
- ### Framework versions
645
-
646
-
647
- - PEFT 0.6.1
648
- ## Training procedure
649
-
650
-
651
- ### Framework versions
652
-
653
-
654
- - PEFT 0.6.1
655
- ## Training procedure
656
-
657
-
658
- ### Framework versions
659
-
660
-
661
- - PEFT 0.6.1
662
- ## Training procedure
663
-
664
-
665
- ### Framework versions
666
-
667
-
668
- - PEFT 0.6.1
669
- ## Training procedure
670
-
671
-
672
- ### Framework versions
673
-
674
-
675
- - PEFT 0.6.1
676
- ## Training procedure
677
-
678
-
679
- ### Framework versions
680
-
681
-
682
  - PEFT 0.6.1
 
203
  ### Framework versions
204
 
205
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
206
  - PEFT 0.6.1
step_0/adapter_config.json CHANGED
@@ -17,10 +17,10 @@
17
  "revision": null,
18
  "target_modules": [
19
  "o_proj",
20
- "q_proj",
21
- "v_proj",
22
  "down_proj",
23
  "k_proj",
 
 
24
  "up_proj"
25
  ],
26
  "task_type": "CAUSAL_LM"
 
17
  "revision": null,
18
  "target_modules": [
19
  "o_proj",
 
 
20
  "down_proj",
21
  "k_proj",
22
+ "q_proj",
23
+ "v_proj",
24
  "up_proj"
25
  ],
26
  "task_type": "CAUSAL_LM"
step_0/training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:06351975e0d120c1c01dbb9fa183965eea7c2de2240a97325a85bec8a38b0b06
3
  size 6200
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77c67c523d97535db753f471f6b99a6884de1dc4b62d9f2b8e04f5f00c0cb563
3
  size 6200
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:06351975e0d120c1c01dbb9fa183965eea7c2de2240a97325a85bec8a38b0b06
3
  size 6200
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77c67c523d97535db753f471f6b99a6884de1dc4b62d9f2b8e04f5f00c0cb563
3
  size 6200