End of training

README.md

@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [bigcode/starcoderbase-1b](https://huggingface.co/bigcode/starcoderbase-1b) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.1132
+- Loss: 0.7359
 
 ## Model description
 
@@ -51,26 +51,26 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 0.2847        | 0.05  | 100  | 0.3010          |
-| 0.2397        | 0.1   | 200  | 0.3370          |
-| 0.1288        | 0.15  | 300  | 0.5087          |
-| 0.0894        | 0.2   | 400  | 0.6274          |
-| 0.067         | 0.25  | 500  | 0.7248          |
-| 0.0489        | 0.3   | 600  | 0.7530          |
-| 0.03          | 0.35  | 700  | 0.8735          |
-| 0.0192        | 0.4   | 800  | 0.9347          |
-| 0.0143        | 0.45  | 900  | 0.9769          |
-| 0.0127        | 0.5   | 1000 | 1.0044          |
-| 0.0114        | 0.55  | 1100 | 1.0451          |
-| 0.0108        | 0.6   | 1200 | 1.0593          |
-| 0.0101        | 0.65  | 1300 | 1.0556          |
-| 0.0092        | 0.7   | 1400 | 1.0834          |
-| 0.0093        | 0.75  | 1500 | 1.1055          |
-| 0.0092        | 0.8   | 1600 | 1.0918          |
-| 0.0079        | 0.85  | 1700 | 1.1194          |
-| 0.0089        | 0.9   | 1800 | 1.1114          |
-| 0.0086        | 0.95  | 1900 | 1.1126          |
-| 0.0079        | 1.0   | 2000 | 1.1132          |
+| 0.2429        | 0.05  | 100  | 0.2525          |
+| 0.2099        | 0.1   | 200  | 0.2812          |
+| 0.0957        | 0.15  | 300  | 0.4394          |
+| 0.0277        | 0.2   | 400  | 0.5758          |
+| 0.015         | 0.25  | 500  | 0.6307          |
+| 0.0144        | 0.3   | 600  | 0.6582          |
+| 0.0122        | 0.35  | 700  | 0.6811          |
+| 0.0105        | 0.4   | 800  | 0.6984          |
+| 0.0116        | 0.45  | 900  | 0.7030          |
+| 0.0101        | 0.5   | 1000 | 0.7078          |
+| 0.0097        | 0.55  | 1100 | 0.7047          |
+| 0.0091        | 0.6   | 1200 | 0.7144          |
+| 0.0087        | 0.65  | 1300 | 0.7196          |
+| 0.0075        | 0.7   | 1400 | 0.7318          |
+| 0.0082        | 0.75  | 1500 | 0.7242          |
+| 0.008         | 0.8   | 1600 | 0.7289          |
+| 0.0078        | 0.85  | 1700 | 0.7322          |
+| 0.0074        | 0.9   | 1800 | 0.7398          |
+| 0.0075        | 0.95  | 1900 | 0.7349          |
+| 0.0073        | 1.0   | 2000 | 0.7359          |
 
 
 ### Framework versions

final_checkpoint/adapter_config.json

@@ -19,9 +19,9 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "q_attn",
     "c_attn",
     "c_fc",
-    "q_attn",
     "c_proj"
   ],
   "task_type": "CAUSAL_LM"

final_checkpoint/adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:02dd3ca2da00926adac92ab39fcd96d14ebf2770cb56681cad289f0a05ce9e23
+oid sha256:241b4951484eab7c19e78dce7f8f0cf2cfc92637ff5541638adcf2a1cbe04198
 size 88891680