Create README.md

README.md

@@ -0,0 +1,98 @@
+# Model Card
+
+## Model Details
+
+- **Model Name:** Alpaca69B/llama-2-7b-absa-semeval-2016
+- **Base Model:** NousResearch/Llama-2-7b-chat-hf
+- **Fine-Tuned On:** Alpaca69B/semeval2016-full-absa-reviews-english-translated-resampled-train
+- **Fine-Tuning Techniques:** LoRA attention, 4-bit precision base model loading, gradient checkpointing, etc.
+- **Training Resources:** Low resource usage
+
+## Model Description
+
+This model is an aspect based sentiment analysis model fine-tuned from the Llama-2-7b-chat model on an adjusted semeval-2016 dataset.
+
+## Fine-Tuning Techniques
+
+### LoRA Attention
+- LoRA attention dimension: 64
+- Alpha parameter for LoRA scaling: 16
+- Dropout probability for LoRA layers: 0.1
+
+### bitsandbytes (4-bit precision)
+- Activated 4-bit precision base model loading
+- Compute dtype for 4-bit base models: "float16"
+- Quantization type: "nf4"
+- Nested quantization for 4-bit base models: Disabled
+
+### TrainingArguments
+- Output directory: "./results"
+- Number of training epochs: 1
+- Enabled fp16/bf16 training: False
+- Batch size per GPU for training: 4
+- Batch size per GPU for evaluation: 4
+- Gradient accumulation steps: 1
+- Enabled gradient checkpointing: True
+- Maximum gradient norm (gradient clipping): 0.3
+- Initial learning rate: 2e-4
+- Weight decay: 0.001
+- Optimizer: paged_adamw_32bit
+- Learning rate scheduler: cosine
+- Maximum training steps: -1 (overrides num_train_epochs)
+- Ratio of steps for linear warmup: 0.03
+- Group sequences into batches with the same length: True
+- Save checkpoint every X update steps: 0 (disabled)
+- Log every X update steps: 25
+
+### SFT (Sequence-level Fine-Tuning)
+- Maximum sequence length: Not specified
+- Packing multiple short examples in the same input sequence: False
+- Load the entire model on GPU 0
+
+## Evaluation
+
+The model's performance and usage can be observed in the provided [Google Colab notebook](https://colab.research.google.com/drive/1ArLpQFfXJiHcAT3VuYZndDqvskM6SkOM?usp=sharing).
+
+## Model Usage
+
+To use the model, follow the provided code snippet:
+
+```python
+from transformers import AutoTokenizer
+import transformers
+import torch
+
+model = "Alpaca69B/llama-2-7b-absa-semeval-2016"
+tokenizer = AutoTokenizer.from_pretrained(model)
+pipeline = transformers.pipeline(
+    "text-generation",
+    model=model,
+    torch_dtype=torch.float16,
+    device_map="auto",
+)
+
+def process_user_prompt(input_sentence):
+    sequences = pipeline(
+        f'### Human: {input_sentence} ### Assistant: aspect: ',
+        do_sample=True,
+        top_k=10,
+        num_return_sequences=1,
+        eos_token_id=tokenizer.eos_token_id,
+        max_length=200,
+    )
+    result_dict = process_output(sequences[0]['generated_text'])
+    return result_dict
+
+def process_output(output):
+    # ... (provided code for processing output)
+
+output = process_user_prompt('the first thing that attracts attention is the warm reception and the smiling receptionists.')
+print(output)
+```
+
+## Fine-Tuning Details
+
+Details of the fine-tuning process are available in the [fine-tuning Colab notebook](https://colab.research.google.com/drive/1PQfBsDyM8TSSBchL6PPA4o6rOyLFLUnu?usp=sharing).
+
+
+**Note:** Ensure that you have the necessary dependencies and resources before running the model.