m00bs/llama-3-8b-inst-CausalRelationship-finetune

README.md

@@ -1,7 +1,7 @@
 ---
 base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
 library_name: peft
-license: llama3.1
+license: llama3
 tags:
 - trl
 - sft
@@ -21,61 +21,17 @@ This model is a fine-tuned version of [unsloth/llama-3-8b-Instruct-bnb-4bit](htt
 
 ## Model description
 
-This fine-tuning model is a large language model using the unsloth library, which focuses on memory efficiency and speed. 
-It demonstrates data preparation, model configuration with LoRA, training with SFTTrainer, and inference with optimized settings.
-The unsloth models, especially the 4-bit quantized versions, enable efficient and faster training and inference, making them suitable for various AI and ML applications.
+More information needed
 
-## How to use
+## Intended uses & limitations
 
-1. **Install Required Libraries**
+More information needed
 
-```python
-import torch
-from transformers import AutoModelForCausalLM, AutoTokenizer
-from unsloth import FastLanguageModel
-from unsloth.chat_templates import get_chat_template
-from peft import PeftModel, PeftConfig
-```
+## Training and evaluation data
 
+More information needed
 
-2. **Load the Model and Tokenizer**
-
-```python
-# Load the tokenizer
-tokenizer = AutoTokenizer.from_pretrained("m00bs/llama-3-8b-inst-CausalRelationship-finetune-tokenizer")
-
-# Load the model
-config = PeftConfig.from_pretrained("m00bs/outputs")
-base_model = AutoModelForCausalLM.from_pretrained("unsloth/llama-3-8b-Instruct-bnb-4bit")
-model = PeftModel.from_pretrained(base_model, "m00bs/outputs")
-
-# Move model to GPU if available
-device = "cuda" if torch.cuda.is_available() else "cpu"
-model.to(device)
-
-```
-
-3. **Prepare Inputs**
-
-```python
-# Prepare the input text
-input_text = """As a finance expert, answer the following question about the following market event about Market Event:
-Given that China's full reopening announcement on December 26, 2022 caused an immediate jump in Chinese stock prices, What was the impact of China's full reopening announcement on December 26, 2022 on Chinese stock prices?"""
-
-# Tokenize the input text
-inputs = tokenizer(input_text, return_tensors="pt").to(device)
-```
-
-4. **Run Inference**
-
-```python
-# Generate the response
-outputs = model.generate(**inputs, max_new_tokens=300, use_cache=True)
-
-# Decode the output
-response = tokenizer.batch_decode(outputs, skip_special_tokens=True)
-```
-
+## Training procedure
 
 ### Training hyperparameters
 
@@ -92,6 +48,8 @@ The following hyperparameters were used during training:
 - training_steps: 60
 - mixed_precision_training: Native AMP
 
+### Training results
+
 
 
 ### Framework versions

adapter_config.json

@@ -20,12 +20,12 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "q_proj",
+    "gate_proj",
     "up_proj",
-    "k_proj",
-    "v_proj",
     "down_proj",
-    "gate_proj",
+    "v_proj",
     "o_proj"
   ],
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1293e3aa6a548d7963e6a612b7388c75df99b7bb2ec8a3f5ef87e9d8c9c6ef9a
+oid sha256:2033567a1cd1794d0e2e1d2465a8f1e8dd5f528622c8d444b2994036159acb6e
 size 167832240

llama-3-8b-News-Finetune/adapter_config.json

@@ -20,12 +20,12 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "q_proj",
+    "gate_proj",
     "up_proj",
-    "k_proj",
-    "v_proj",
     "down_proj",
-    "gate_proj",
+    "v_proj",
     "o_proj"
   ],
   "task_type": "CAUSAL_LM",

llama-3-8b-News-Finetune/adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1293e3aa6a548d7963e6a612b7388c75df99b7bb2ec8a3f5ef87e9d8c9c6ef9a
+oid sha256:2033567a1cd1794d0e2e1d2465a8f1e8dd5f528622c8d444b2994036159acb6e
 size 167832240

llama-3-8b-News-Finetune/training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:96c1a0dd34487c1b98c5d2abf8922a1949e7bc203665b4126ae0ade39a0e583b
+oid sha256:959ffb565f674276e9449f64a7bc2c4f9d5872b07e98baf993264f115025f8ca
 size 5176

runs/Aug05_03-50-11_c76be924f45b/events.out.tfevents.1722829823.c76be924f45b.163.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:41acc89c27abb18c81b0335ad1b94320c6234552fff746f57672912def65182c
+size 18168

runs/Aug05_04-06-13_c76be924f45b/events.out.tfevents.1722830785.c76be924f45b.163.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4aeb6e69add0d994f3dcf27912ac842116dcd08f7190674009be136d10c5e3b2
+size 18168

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:96c1a0dd34487c1b98c5d2abf8922a1949e7bc203665b4126ae0ade39a0e583b
+oid sha256:959ffb565f674276e9449f64a7bc2c4f9d5872b07e98baf993264f115025f8ca
 size 5176