Hugging Face's logo

AlignmentResearch
/
robust_llm_w9a5ielg_from_EleutherAI_pythia-14m

Text Classification
Transformers
TensorBoard
Safetensors
gpt_neox
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
michalzajac commited on
Commit
6fd34da
•
1 Parent(s): 287647a

End of training

Browse files
Files changed (37) hide show
  1. README.md +52 -0
  2. config.json +32 -0
  3. model.safetensors +3 -0
  4. runs/Mar05_12-06-06_devbox-bx8b2/events.out.tfevents.1709640369.devbox-bx8b2.2843.0 +3 -0
  5. runs/Mar05_12-07-30_devbox-bx8b2/events.out.tfevents.1709640453.devbox-bx8b2.4376.0 +3 -0
  6. runs/Mar05_12-40-23_devbox-bx8b2/events.out.tfevents.1709642427.devbox-bx8b2.21277.0 +3 -0
  7. runs/Mar05_12-41-15_devbox-bx8b2/events.out.tfevents.1709642477.devbox-bx8b2.22145.0 +3 -0
  8. runs/Mar05_12-42-28_devbox-bx8b2/events.out.tfevents.1709642584.devbox-bx8b2.23072.0 +3 -0
  9. runs/Mar05_13-57-23_devbox-bx8b2/events.out.tfevents.1709647044.devbox-bx8b2.54312.0 +3 -0
  10. runs/Mar05_13-58-25_devbox-bx8b2/events.out.tfevents.1709647106.devbox-bx8b2.55435.0 +3 -0
  11. runs/Mar05_13-59-41_devbox-bx8b2/events.out.tfevents.1709647182.devbox-bx8b2.56466.0 +3 -0
  12. runs/Mar05_14-00-54_devbox-bx8b2/events.out.tfevents.1709647255.devbox-bx8b2.57642.0 +3 -0
  13. runs/Mar05_14-16-30_devbox-bx8b2/events.out.tfevents.1709648192.devbox-bx8b2.65865.0 +3 -0
  14. runs/Mar05_14-18-17_devbox-bx8b2/events.out.tfevents.1709648298.devbox-bx8b2.67243.0 +3 -0
  15. runs/Mar05_14-27-41_devbox-bx8b2/events.out.tfevents.1709648868.devbox-bx8b2.70600.0 +3 -0
  16. runs/Mar05_14-29-02_devbox-bx8b2/events.out.tfevents.1709648945.devbox-bx8b2.72037.0 +3 -0
  17. runs/Mar05_14-33-39_devbox-bx8b2/events.out.tfevents.1709649221.devbox-bx8b2.74191.0 +3 -0
  18. runs/Mar05_14-34-59_devbox-bx8b2/events.out.tfevents.1709649301.devbox-bx8b2.75186.0 +3 -0
  19. runs/Mar05_14-36-18_devbox-bx8b2/events.out.tfevents.1709649381.devbox-bx8b2.76279.0 +3 -0
  20. runs/Mar05_14-39-09_devbox-bx8b2/events.out.tfevents.1709649552.devbox-bx8b2.77152.0 +3 -0
  21. runs/Mar05_14-40-24_devbox-bx8b2/events.out.tfevents.1709649628.devbox-bx8b2.79027.0 +3 -0
  22. runs/Mar05_14-44-14_devbox-bx8b2/events.out.tfevents.1709649860.devbox-bx8b2.81658.0 +3 -0
  23. runs/Mar05_14-46-49_devbox-bx8b2/events.out.tfevents.1709650024.devbox-bx8b2.82944.0 +3 -0
  24. runs/Mar05_14-50-11_devbox-bx8b2/events.out.tfevents.1709650228.devbox-bx8b2.84985.0 +3 -0
  25. runs/Mar05_14-55-24_devbox-bx8b2/events.out.tfevents.1709650525.devbox-bx8b2.87673.0 +3 -0
  26. runs/Mar05_14-56-07_devbox-bx8b2/events.out.tfevents.1709650568.devbox-bx8b2.88617.0 +3 -0
  27. runs/Mar05_14-57-17_devbox-bx8b2/events.out.tfevents.1709650638.devbox-bx8b2.89893.0 +3 -0
  28. runs/Mar05_15-06-05_devbox-bx8b2/events.out.tfevents.1709651172.devbox-bx8b2.93566.0 +3 -0
  29. runs/Mar05_15-08-00_devbox-bx8b2/events.out.tfevents.1709651297.devbox-bx8b2.95010.0 +3 -0
  30. runs/Mar05_16-02-41_devbox-bx8b2/events.out.tfevents.1709654561.devbox-bx8b2.115460.0 +3 -0
  31. runs/Mar05_16-05-01_devbox-bx8b2/events.out.tfevents.1709654702.devbox-bx8b2.117458.0 +3 -0
  32. runs/Mar05_16-13-50_devbox-bx8b2/events.out.tfevents.1709655230.devbox-bx8b2.121943.0 +3 -0
  33. runs/Mar05_16-22-28_devbox-bx8b2/events.out.tfevents.1709655748.devbox-bx8b2.125728.0 +3 -0
  34. runs/Mar05_16-23-34_devbox-bx8b2/events.out.tfevents.1709655815.devbox-bx8b2.126691.0 +3 -0
  35. runs/Mar06_08-44-42_devbox-bx8b2/events.out.tfevents.1709714683.devbox-bx8b2.136096.0 +3 -0
  36. runs/Mar06_09-21-56_devbox-bx8b2/events.out.tfevents.1709716917.devbox-bx8b2.141499.0 +3 -0
  37. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: EleutherAI/pythia-14m
3
+ tags:
4
+ - generated_from_trainer
5
+ model-index:
6
+ - name: robust_llm_w9a5ielg_from_EleutherAI_pythia-14m
7
+ results: []
8
+ ---
9
+
10
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
11
+ should probably proofread and complete it, then remove this comment. -->
12
+
13
+ # robust_llm_w9a5ielg_from_EleutherAI_pythia-14m
14
+
15
+ This model is a fine-tuned version of [EleutherAI/pythia-14m](https://huggingface.co/EleutherAI/pythia-14m) on an unknown dataset.
16
+
17
+ ## Model description
18
+
19
+ More information needed
20
+
21
+ ## Intended uses & limitations
22
+
23
+ More information needed
24
+
25
+ ## Training and evaluation data
26
+
27
+ More information needed
28
+
29
+ ## Training procedure
30
+
31
+ ### Training hyperparameters
32
+
33
+ The following hyperparameters were used during training:
34
+ - learning_rate: 1e-05
35
+ - train_batch_size: 1
36
+ - eval_batch_size: 1
37
+ - seed: 42
38
+ - distributed_type: multi-GPU
39
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
40
+ - lr_scheduler_type: linear
41
+ - num_epochs: 1
42
+
43
+ ### Training results
44
+
45
+
46
+
47
+ ### Framework versions
48
+
49
+ - Transformers 4.37.2
50
+ - Pytorch 2.2.0
51
+ - Datasets 2.17.0
52
+ - Tokenizers 0.15.2
config.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "EleutherAI/pythia-14m",
3
+ "architectures": [
4
+ "GPTNeoXForSequenceClassification"
5
+ ],
6
+ "attention_bias": true,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 0,
9
+ "classifier_dropout": 0.1,
10
+ "eos_token_id": 0,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout": 0.0,
13
+ "hidden_size": 128,
14
+ "initializer_range": 0.02,
15
+ "intermediate_size": 512,
16
+ "layer_norm_eps": 1e-05,
17
+ "max_position_embeddings": 2048,
18
+ "model_type": "gpt_neox",
19
+ "num_attention_heads": 4,
20
+ "num_hidden_layers": 6,
21
+ "pad_token_id": 0,
22
+ "problem_type": "single_label_classification",
23
+ "rope_scaling": null,
24
+ "rotary_emb_base": 10000,
25
+ "rotary_pct": 0.25,
26
+ "tie_word_embeddings": false,
27
+ "torch_dtype": "float32",
28
+ "transformers_version": "4.37.2",
29
+ "use_cache": false,
30
+ "use_parallel_residual": true,
31
+ "vocab_size": 50304
32
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f33c02d984a6885b5ba295395ae8977dedeb9a407cdb909cfb630afc94563b33
3
+ size 30524712
runs/Mar05_12-06-06_devbox-bx8b2/events.out.tfevents.1709640369.devbox-bx8b2.2843.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6915bad27ca64454ab0d9092bfd8876b65e08beb041d59c40d0a216ffa4132bd
3
+ size 9222
runs/Mar05_12-07-30_devbox-bx8b2/events.out.tfevents.1709640453.devbox-bx8b2.4376.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e89d1b29a0dbcea078c4f2183d20d1ec74bb44d2872eae67fb0fbc3fa6df0d33
3
+ size 7071
runs/Mar05_12-40-23_devbox-bx8b2/events.out.tfevents.1709642427.devbox-bx8b2.21277.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:502ec5f75f10f982f7b7a0bb5d242a9d32159d425dcb7a0be1e6de587cca16dd
3
+ size 4952
runs/Mar05_12-41-15_devbox-bx8b2/events.out.tfevents.1709642477.devbox-bx8b2.22145.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8899090c735c5bbc151d6027f7eea61276352b39010c9e9057c568ee51c3632d
3
+ size 5300
runs/Mar05_12-42-28_devbox-bx8b2/events.out.tfevents.1709642584.devbox-bx8b2.23072.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abcb74a8aee2c4adae5e59edee2fb5b6d74222cf53d54378f8426f3d12cfdd28
3
+ size 4952
runs/Mar05_13-57-23_devbox-bx8b2/events.out.tfevents.1709647044.devbox-bx8b2.54312.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8068c20f2024dcac656729d36fd206d5444af5b77914e031e1d901a0954c28f
3
+ size 4525
runs/Mar05_13-58-25_devbox-bx8b2/events.out.tfevents.1709647106.devbox-bx8b2.55435.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f129433fd03a9d53112efd35464a503cdb9cd3000fff04ffbfbf640f9c22a84
3
+ size 4525
runs/Mar05_13-59-41_devbox-bx8b2/events.out.tfevents.1709647182.devbox-bx8b2.56466.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:94aac111e185e184d3a345bc54e4365b15810b6fd7e9acc30ecdf344695ed2eb
3
+ size 4545
runs/Mar05_14-00-54_devbox-bx8b2/events.out.tfevents.1709647255.devbox-bx8b2.57642.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a72ef2e82671191cf5edc9f79e61cee8d1cd66adaf8d65bfb2dda0d137e1cfa
3
+ size 4842
runs/Mar05_14-16-30_devbox-bx8b2/events.out.tfevents.1709648192.devbox-bx8b2.65865.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:913f0495fcd6a4e60cf498b5365497fd74642a47683169d4808dd55e5269ec89
3
+ size 4525
runs/Mar05_14-18-17_devbox-bx8b2/events.out.tfevents.1709648298.devbox-bx8b2.67243.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36565d35ef32433934445dc5199d9c313fbce7c4487c6d30f22fca967ef3ff93
3
+ size 4525
runs/Mar05_14-27-41_devbox-bx8b2/events.out.tfevents.1709648868.devbox-bx8b2.70600.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:462bcd42f25961c24c5407f2aa89e8e794c3ab5baba517389bb9bdb7e72ff79a
3
+ size 4136
runs/Mar05_14-29-02_devbox-bx8b2/events.out.tfevents.1709648945.devbox-bx8b2.72037.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1823a4fb65901503b6c33f9abda9bfcbb2e23bd0b090251bae41b6df03fb7032
3
+ size 4647
runs/Mar05_14-33-39_devbox-bx8b2/events.out.tfevents.1709649221.devbox-bx8b2.74191.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:015bf5ccea7ea8804fb1e8ec711248f67a1c962929241b0bf8a3c17cd37d2c45
3
+ size 4647
runs/Mar05_14-34-59_devbox-bx8b2/events.out.tfevents.1709649301.devbox-bx8b2.75186.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:577c693de9211daa5f8a101f191880c499b5f9d19b8571767bca499d65f02cb7
3
+ size 4647
runs/Mar05_14-36-18_devbox-bx8b2/events.out.tfevents.1709649381.devbox-bx8b2.76279.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5bdd7463c5dc45e054008e97499147236862225ed2306bf86390b1a4a4429014
3
+ size 4668
runs/Mar05_14-39-09_devbox-bx8b2/events.out.tfevents.1709649552.devbox-bx8b2.77152.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:055ed3364af288cfcc8719ba5fe1121e3acffe533f851c56dc78cab937b7012c
3
+ size 4664
runs/Mar05_14-40-24_devbox-bx8b2/events.out.tfevents.1709649628.devbox-bx8b2.79027.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d71d2452ed38d1eff06b0aa2473fe33fd0b6d1e2294d270b0e74084068b66fd6
3
+ size 4664
runs/Mar05_14-44-14_devbox-bx8b2/events.out.tfevents.1709649860.devbox-bx8b2.81658.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7a062f78385e476348d5c3ef6a3687552bda200c73f44bc29509b34d966a39c
3
+ size 4662
runs/Mar05_14-46-49_devbox-bx8b2/events.out.tfevents.1709650024.devbox-bx8b2.82944.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:15f1be984cde5fb193a1c5e3d30c45352b50874d35ee3626e3d2d9f8dcea0a60
3
+ size 4666
runs/Mar05_14-50-11_devbox-bx8b2/events.out.tfevents.1709650228.devbox-bx8b2.84985.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8e64b4d9ef15cc821be1d02b4ac816b37c0dfafcc55636208a6c614d03074bc
3
+ size 4664
runs/Mar05_14-55-24_devbox-bx8b2/events.out.tfevents.1709650525.devbox-bx8b2.87673.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97e222c0a6f213a575a75ca264efab389db640b7df7233def9931702f23498a5
3
+ size 4136
runs/Mar05_14-56-07_devbox-bx8b2/events.out.tfevents.1709650568.devbox-bx8b2.88617.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e55a8937dd518fa389c5b6b1d1c261191c91ea78590a57de7b855c55d28d4a3
3
+ size 4650
runs/Mar05_14-57-17_devbox-bx8b2/events.out.tfevents.1709650638.devbox-bx8b2.89893.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dfb438549ee973ff86f7ea0a40edd2ba8a2868c9270496aec755ddd63573308f
3
+ size 4996
runs/Mar05_15-06-05_devbox-bx8b2/events.out.tfevents.1709651172.devbox-bx8b2.93566.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:37e9c02f3a20ce08a80d27c028256d0de58aee462bc1292ae5d4df1642546d50
3
+ size 4504
runs/Mar05_15-08-00_devbox-bx8b2/events.out.tfevents.1709651297.devbox-bx8b2.95010.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4996d6d52d2398b1771257192968f3671815cb0b8d01a79349052c71adc98004
3
+ size 4852
runs/Mar05_16-02-41_devbox-bx8b2/events.out.tfevents.1709654561.devbox-bx8b2.115460.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e8f189045dc8c4386abfa00252dfe41f5f8740939f7b34d5ef0adf3c9d5b7e6
3
+ size 4829
runs/Mar05_16-05-01_devbox-bx8b2/events.out.tfevents.1709654702.devbox-bx8b2.117458.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84422ad640f4b49057a584ff13131d8acf1d0a3f581a244c76bfe58206d51be6
3
+ size 5382
runs/Mar05_16-13-50_devbox-bx8b2/events.out.tfevents.1709655230.devbox-bx8b2.121943.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee728f3f2ab57e7cc28f3713b9f141480f4adf8e730ae356859ad2201d433c41
3
+ size 5382
runs/Mar05_16-22-28_devbox-bx8b2/events.out.tfevents.1709655748.devbox-bx8b2.125728.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db2a1d92da3e9c1e8c8eace26e53a4601f7fe614202a66381b70488f3fbc9061
3
+ size 5382
runs/Mar05_16-23-34_devbox-bx8b2/events.out.tfevents.1709655815.devbox-bx8b2.126691.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2c459539b1929d47f3243f720409331565c2ec01a7608f1e9e769af8cefd8276
3
+ size 5382
runs/Mar06_08-44-42_devbox-bx8b2/events.out.tfevents.1709714683.devbox-bx8b2.136096.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0bf1862ed1ad43b330ee10b1bbb9d780df7e1f40af739ceb2afde4f81256e813
3
+ size 4830
runs/Mar06_09-21-56_devbox-bx8b2/events.out.tfevents.1709716917.devbox-bx8b2.141499.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1cbd935d39b27347fde7a27517084f1c0a0c4ee6b71666d9f16e265bd322bbba
3
+ size 4830
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:824149c3664c1b997b6b1dc3e2b409948805179addd4ee7fbab4e30d4ac98c0e
3
+ size 4728