matiusX/lamma-legis-ufam

Model Details

Model Description

This model is a fine-tuned version of LLaMA 3 utilizing the Quantized Low-Rank Adaptation (QLoRA) technique. It is designed to answer questions related to the academic legislation of the Universidade Federal do Amazonas (UFAM). The training process involved generating a synthetic dataset of questions and answers based on the legislation, which includes various resolutions and norms provided by UFAM.

• Developed by: Matheus dos Santos Palheta
• Model type: More Information Needed
• Language(s) (NLP): Portuguese, English
• License: MIT
• Finetuned from model: unsloth/llama-3-8b-bnb-4bit

Model Sources [optional]

• Repository: [More Information Needed]

Uses

This model is intended for use by anyone with questions about UFAM's legislation. It is especially designed for students, professors, and administrative staff who need quick and accurate answers regarding academic policies and regulations. The model aims to support these groups by providing reliable information, thereby facilitating a better understanding of the rules and guidelines that govern their academic and professional activities at UFAM.

Direct Use

This model can be directly used to answer questions regarding UFAM's academic legislation without additional fine-tuning.

Downstream Use

The model can be integrated into larger ecosystems or applications, particularly those focusing on academic information systems, legal information retrieval, or automated student support systems from UFAM.

Out-of-Scope Use

This model is not suitable for general-purpose question answering beyond the scope of UFAM's academic legislation. It should not be used for legal advice or any critical decision-making processes outside its trained domain.

Bias, Risks, and Limitations

While the model has been fine-tuned for accuracy in the context of UFAM's legislation, it may still exhibit biases present in the training data. Additionally, the model's performance is constrained by the quality and comprehensiveness of the synthetic dataset generated.

How to Get Started with the Model

Use the code below to get started with the model.

!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps "xformers<0.0.27" "trl<0.9.0" peft accelerate bitsandbytes

from datasets import load_dataset
from datasets import Dataset
import pandas as pd

from unsloth import FastLanguageModel
import torch
max_seq_length = 2048 
dtype = None 
load_in_4bit = True 
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "matiusX/lamma-legis-ufam",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)
FastLanguageModel.for_inference(model)

prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

inputs = tokenizer(
[
    prompt.format(
        contexto, # contexto
        pergunta, # pergunta
        "", # resposta - deixar em branco
    )
], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)
tokenizer.batch_decode(outputs)

from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer, skip_prompt=True)
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 128)

Training Details

Training Data

The training data for this model is based on the academic legislation of UFAM. It includes a wide range of documents, such as resolutions and norms, which have been pre-processed and structured to create a synthetic dataset of questions and answers. For more details on the dataset, including the pre-processing and filtering steps, please refer to the Dataset Card available here.

Training Procedure

Training Hyperparameters

• Training regime: Mixed precision (fp16)
• LoRA configuration:
  • Alpha: 16
  • Dropout: 0
  • Target modules: down_proj, up_proj, q_proj, gate_proj, v_proj, o_proj, k_proj

Speeds, Sizes, Times [optional]

• Global Step: 60
• Metrics:
  • Train Runtime: 1206.8508 seconds
  • Train Samples per Second: 0.398
  • Train Steps per Second: 0.05
  • Total FLOPs: 4.451323701362688e+16
  • Train Loss: 0.9744117197891077

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Contact

matheus.palheta@icomp.ufam.edu.br

Framework versions

• PEFT 0.12.0