Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-Nemo-Instruct-2407
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-Nemo-Instruct-2407",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:944dbd3d27a1ca739d18de361d0269d6b4d16ed9b3dcf2f8734e4ab22cca5d3f
+size 228140600

checkpoint-12012/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-Nemo-Instruct-2407
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

checkpoint-12012/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-Nemo-Instruct-2407",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-12012/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3154d50b4a649b62190b8b3d5dbf22992cdbd455abab62a627d8d295b00a8472
+size 228140600

checkpoint-12012/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:11607d950fc245bff619782e6fd5dc895105f9e9418f8884f4134e3df8c99a56
+size 456467058

checkpoint-12012/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18fb00a102fa960c222752cee5d49e8c97ed9d571273fd4be7a85619ea02f5e3
+size 14244

checkpoint-12012/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7b08f4f2c3b78e708db3e24614e146a59ed831dcdaf958fd5c53e98184a2a60
+size 1064

checkpoint-12012/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-12012/trainer_state.json

@@ -0,0 +1,117 @@
+{
+  "best_metric": 1.016390323638916,
+  "best_model_checkpoint": "/home/ankur/projects/llm_test/Akshay_Work/mistral_12B_snippets_v1/checkpoint-5149",
+  "epoch": 6.998640512720916,
+  "eval_steps": 3000,
+  "global_step": 12012,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.9998057875315596,
+      "eval_loss": 1.1702903509140015,
+      "eval_runtime": 1084.2637,
+      "eval_samples_per_second": 2.364,
+      "eval_steps_per_second": 2.364,
+      "step": 1716
+    },
+    {
+      "epoch": 1.747912215964265,
+      "grad_norm": 0.8725250363349915,
+      "learning_rate": 0.00015795285251073526,
+      "loss": 1.1644,
+      "step": 3000
+    },
+    {
+      "epoch": 1.9996115750631192,
+      "eval_loss": 1.0563772916793823,
+      "eval_runtime": 1084.6886,
+      "eval_samples_per_second": 2.363,
+      "eval_steps_per_second": 2.363,
+      "step": 3432
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.016390323638916,
+      "eval_runtime": 1085.9027,
+      "eval_samples_per_second": 2.36,
+      "eval_steps_per_second": 2.36,
+      "step": 5149
+    },
+    {
+      "epoch": 3.4958244319285297,
+      "grad_norm": 0.9269574880599976,
+      "learning_rate": 0.00010540706335991588,
+      "loss": 0.6414,
+      "step": 6000
+    },
+    {
+      "epoch": 3.9998057875315594,
+      "eval_loss": 1.0506994724273682,
+      "eval_runtime": 1098.0815,
+      "eval_samples_per_second": 2.334,
+      "eval_steps_per_second": 2.334,
+      "step": 6865
+    },
+    {
+      "epoch": 4.999611575063119,
+      "eval_loss": 1.1407289505004883,
+      "eval_runtime": 1094.1268,
+      "eval_samples_per_second": 2.343,
+      "eval_steps_per_second": 2.343,
+      "step": 8581
+    },
+    {
+      "epoch": 5.243736647892795,
+      "grad_norm": 1.0000396966934204,
+      "learning_rate": 5.2843747261414424e-05,
+      "loss": 0.3137,
+      "step": 9000
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.2437316179275513,
+      "eval_runtime": 1079.3569,
+      "eval_samples_per_second": 2.375,
+      "eval_steps_per_second": 2.375,
+      "step": 10298
+    },
+    {
+      "epoch": 6.991648863857059,
+      "grad_norm": 0.6523720026016235,
+      "learning_rate": 2.804311629129787e-07,
+      "loss": 0.1744,
+      "step": 12000
+    },
+    {
+      "epoch": 6.998640512720916,
+      "eval_loss": 1.377004623413086,
+      "eval_runtime": 1079.4922,
+      "eval_samples_per_second": 2.374,
+      "eval_steps_per_second": 2.374,
+      "step": 12012
+    }
+  ],
+  "logging_steps": 3000,
+  "max_steps": 12012,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 7,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.4877646120458547e+18,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-12012/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6892de9a1785cda2ea7862ad4d23341eaa7c9604e85375138dff05f213d18964
+size 5496

checkpoint-5149/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-Nemo-Instruct-2407
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

checkpoint-5149/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-Nemo-Instruct-2407",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-5149/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:944dbd3d27a1ca739d18de361d0269d6b4d16ed9b3dcf2f8734e4ab22cca5d3f
+size 228140600

checkpoint-5149/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eca50808b695312e91e460ff479ef7544db59eb927b2389709b1822b8e6a3f66
+size 456467058

checkpoint-5149/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79affeab5ad310f214e73e8ce137139c339e46722a35cc86801ecac6b3597f33
+size 14244

checkpoint-5149/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6ab42d03ca5e4fdd94665cf4ceec561a198b1d34fb60cd6278a1d3e2d3526ea
+size 1064

checkpoint-5149/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-5149/trainer_state.json

@@ -0,0 +1,64 @@
+{
+  "best_metric": 1.016390323638916,
+  "best_model_checkpoint": "/home/ankur/projects/llm_test/Akshay_Work/mistral_12B_snippets_v1/checkpoint-5149",
+  "epoch": 3.0,
+  "eval_steps": 3000,
+  "global_step": 5149,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.9998057875315596,
+      "eval_loss": 1.1702903509140015,
+      "eval_runtime": 1084.2637,
+      "eval_samples_per_second": 2.364,
+      "eval_steps_per_second": 2.364,
+      "step": 1716
+    },
+    {
+      "epoch": 1.747912215964265,
+      "grad_norm": 0.8725250363349915,
+      "learning_rate": 0.00015795285251073526,
+      "loss": 1.1644,
+      "step": 3000
+    },
+    {
+      "epoch": 1.9996115750631192,
+      "eval_loss": 1.0563772916793823,
+      "eval_runtime": 1084.6886,
+      "eval_samples_per_second": 2.363,
+      "eval_steps_per_second": 2.363,
+      "step": 3432
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 1.016390323638916,
+      "eval_runtime": 1085.9027,
+      "eval_samples_per_second": 2.36,
+      "eval_steps_per_second": 2.36,
+      "step": 5149
+    }
+  ],
+  "logging_steps": 3000,
+  "max_steps": 12012,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 7,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.376899283834982e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-5149/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6892de9a1785cda2ea7862ad4d23341eaa7c9604e85375138dff05f213d18964
+size 5496

events.out.tfevents.1721457517.BVR-Compute-4.940826.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1227be1454fc2d77ad22af137bf37446686cdb089a96af15c31fcae9518bc19f
+size 5831

events.out.tfevents.1721457714.BVR-Compute-4.942824.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bded85e9c19ebb213d83457648ae81a67c8bae05a135e24f5664065e7a587281
+size 5831

events.out.tfevents.1721457751.BVR-Compute-4.943299.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6cd1ba928ce97f0c8b350da63f47b62f3d82812ced965bdd248d987c21054cf
+size 5831

events.out.tfevents.1721457918.BVR-Compute-4.944852.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25ef32ecfee071f80a8a37ac656bc114068488a39cedf0050c6fd2af2ffdd912
+size 5831

events.out.tfevents.1721457952.BVR-Compute-4.945279.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88a44d908b8af546ce263e0efa22f1d3b3f98f82426b9c2fe8326356db94e6b8
+size 5831

events.out.tfevents.1721457979.BVR-Compute-4.945665.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b7d1d0ad897e66294584496dd2aa58b0e26eae4d7ae6e75c6903c5fa3f9a0d7
+size 5831

events.out.tfevents.1721458111.BVR-Compute-4.947008.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f456e64b20468f96b62c09f4aa902721f9317daee9ed80a3ebdf704a4dce884
+size 5831

events.out.tfevents.1721458147.BVR-Compute-4.947435.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95d524516e54a4e1c36c1e2f30e4d263c0e258af374e36f86d6318fe647e8a11
+size 5831

events.out.tfevents.1721458192.BVR-Compute-4.947972.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:895f659a7cf33f29984a1f29466888475033ddb3c83717083b77071061017191
+size 5831

events.out.tfevents.1721458230.BVR-Compute-4.948617.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b6986803ff6cf77eacdeec6b086b6870da4b54183f24c28e53d48325f4e65d1
+size 5831

events.out.tfevents.1721458281.BVR-Compute-4.949189.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:286ec9944305fb964f99ab39620e89263b3bc11ec573afc493d0b48f679881f5
+size 5831

events.out.tfevents.1721458314.BVR-Compute-4.949591.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77e60e76812534527150f20b815560fa39a22d62c953bc00d81ae64970a700da
+size 5831

events.out.tfevents.1721458349.BVR-Compute-4.950019.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fea7c4310c0e561904ac82f9b956436c2bc4f3b8f868bb457c0aeb6572fbde4f
+size 5831

events.out.tfevents.1721458387.BVR-Compute-4.950481.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08dc6b5bb0c37e18f069dd77af776efbe30d0b8253f3aeb5494d6206b4956095
+size 5831

events.out.tfevents.1721458419.BVR-Compute-4.950875.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f476cdfdfb4249cada968001f2334ecef3ca830a80f93e3490c5d3fb9e94a06
+size 5831

events.out.tfevents.1721458449.BVR-Compute-4.951256.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a08a2257f732fafb1644d3768ca6443c8bcee5f3de2c54f42105c3f730d9d259
+size 5831

events.out.tfevents.1721458474.BVR-Compute-4.951616.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a69c1bf06ea9420eb6889993d80fb17fec387dc407f9e4212604cf227d984e3
+size 5832

events.out.tfevents.1721458505.BVR-Compute-4.952036.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38409810fadb5c25e4765fad55d068f1277f6e4fdc9ec34d97aaff66a4252b79
+size 5831

events.out.tfevents.1721458538.BVR-Compute-4.952441.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a743a7e9aeb4d53153845181876b1c522a7d8fc6718b3e83fc7ebea2a1fbbcd
+size 5832

events.out.tfevents.1721458584.BVR-Compute-4.952956.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:887fd8b97f42e614e5ed4930a5ae60813bad57fcadf3844f7f9abea1b109d7f2
+size 5832

events.out.tfevents.1721458676.BVR-Compute-4.953845.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d511c5ba2078c34436417503dbfa16673d12e11b107b19628320791d11fc271
+size 5832

events.out.tfevents.1721458911.BVR-Compute-4.956269.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c91e5357b98841bc5a8892f25a4c3ac2c45bb9d69ad8ccb95f18c16c350c5a9
+size 5832

events.out.tfevents.1721458952.BVR-Compute-4.956784.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9c61cdce25241906eb39be8a2af9dddb253b5bf18102bc19cef1c2c4f2867d0
+size 5832

events.out.tfevents.1721459049.BVR-Compute-4.957720.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb9770f9fbfa328a2617f331a59825b5ee7b1cc4b6f43850c7b339dbcd964383
+size 5832

events.out.tfevents.1721459100.BVR-Compute-4.958254.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df3485e9b83ef97f07a1cf162fb608113c8a7fbf6d458f72b06169842205e242
+size 5832