YulaGeoLLM

YulaGeoLLM is a LoRA adapter for Microsoft Phi-3 Mini 4K Instruct, fine-tuned for question answering about mineral exploration geology. Its intended languages are English and Spanish. The training dataset is documented as English; Spanish performance has not been separately evaluated.

This repository contains adapter weights, not a standalone base model. Load it with microsoft/Phi-3-mini-4k-instruct.

Model details

• Developed by: Thanatos0512
• Model type: PEFT LoRA adapter for causal language modeling
• Intended task: Mineral exploration geology question answering
• Languages: English and Spanish (Spanish performance has not been evaluated)
• License: MIT for this adapter repository
• Base model: microsoft/Phi-3-mini-4k-instruct
• Adapter repository: Thanatos0512/YulaGeoLLM
• Training dataset: AshkanTaghipour/mineral-exploration-geology-qa

The base model is distributed under MIT. The training dataset declares Apache-2.0; consult its dataset card for the dataset terms and provenance.

Intended use

YulaGeoLLM is intended to generate draft answers to questions about mineral exploration geology. The source dataset focuses on Western Australian geology and mineral exploration topics. Use the model as an aid for exploration, study, and technical writing, with qualified geological review.

Do not use its output as the sole basis for drilling programs, exploration budgets, mineral resource or reserve statements, engineering or site-safety decisions, environmental approvals, or investment decisions. Verify factual claims against current primary sources and project data.

Limitations, risks, and recommendations

• The model may produce incorrect, incomplete, or unsupported geological claims. It does not provide verified source citations with its answers.
• The training dataset is small and focused on mineral exploration in Western Australia; performance on other regions, geological specialties, and question styles is unknown.
• The dataset card identifies English as its language. Spanish is an intended language for this adapter, but Spanish training coverage and performance have not been established.
• No formal evaluation was performed for this adapter. The reported training loss is not a measure of generalization or answer quality.
• Review generated answers with a qualified geologist and consult primary geological reports, maps, assay data, and applicable reporting codes before acting on them.

Getting started

Install the libraries used to load the adapter:

pip install torch transformers==4.46.2 peft==0.13.2 accelerate==1.1.1

Load the base model and adapter, then submit a geology question:

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE_MODEL_ID = "microsoft/Phi-3-mini-4k-instruct"
ADAPTER_ID = "Thanatos0512/YulaGeoLLM"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_ID)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL_ID,
    device_map="auto",
    torch_dtype="auto",
)
model = PeftModel.from_pretrained(base_model, ADAPTER_ID)
model.eval()

messages = [
    {
        "role": "user",
        "content": "What geological evidence should guide follow-up work at a mineral exploration target?",
    }
]
input_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    generated = model.generate(
        input_ids=input_ids,
        max_new_tokens=256,
        do_sample=False,
        eos_token_id=tokenizer.eos_token_id,
    )

answer = tokenizer.decode(
    generated[0, input_ids.shape[-1]:],
    skip_special_tokens=True,
)
print(answer)

The adapter weights do not include the base model. The base model must be downloaded separately and its license and usage terms followed.

Training details

Training data

The reported training source is Mineral Exploration Geology QA Dataset. Its dataset card describes expert-reviewed question-answer and chain-of-thought records focused on Western Australian mineral exploration, based on approximately 300 WAMEX exploration reports. The card lists 479 training records (352 QA and 127 chain-of-thought), 26 test records (19 QA and 7 chain-of-thought), and 15 separate hallucination traps. The dataset declares English and Apache-2.0.

The fine-tuning workflow used the training split. No formal evaluation was run on the dataset test examples or hallucination traps. The exact field selection and handling of reasoning text in this adapter's training examples were not recorded in the supplied materials.

Training procedure and hyperparameters

The adapter was fine-tuned using supervised fine-tuning with a 4-bit quantized base model and LoRA adapters (QLoRA-style training).

Setting	Value
Base model	microsoft/Phi-3-mini-4k-instruct
Task	Causal language modeling (CAUSAL_LM)
Quantization	4-bit NF4 with double quantization
Quantized compute dtype	bfloat16
LoRA rank (r)	8
LoRA alpha	16
LoRA dropout	0.05
Target modules	qkv_proj, gate_up_proj, down_proj, o_proj
Bias	none
Epochs	10
Learning rate	3e-4
Per-device batch size	16
Gradient accumulation steps	1
Maximum sequence length	64 tokens
Sequence packing	enabled
Optimizer	paged_adamw_8bit
Gradient checkpointing	enabled
Training steps	220
Recorded training loss	0.8188

The logged training run reports approximately 1,602 seconds of runtime on a Tesla T4 in Google Colab. The training loss is reported for transparency; it is not a held-out evaluation metric.

Framework versions

The training notebook specifies Transformers 4.46.2, PEFT 0.13.2, Accelerate 1.1.1, TRL 0.12.1, Datasets 3.1.0, and bitsandbytes 0.50.2.

Evaluation

No formal evaluation was performed. No held-out metrics, language-specific results, or expert review results are reported for YulaGeoLLM. The recorded training loss of 0.8188 describes the training run only and should not be interpreted as evidence of answer quality or generalization.

Environmental impact

• Hardware: Tesla T4 (as reported in the training notebook)
• Cloud environment: Google Colab
• Reported runtime: approximately 0.45 hours
• Carbon emissions: not measured

Technical specifications

YulaGeoLLM is a PEFT LoRA adapter for the causal language modeling head of Phi-3 Mini 4K Instruct. The saved repository contains adapter weights and configuration; inference requires the base model.

Citation and references

• Base model: Phi-3-mini-4k-instruct
• Training dataset: Mineral Exploration Geology QA Dataset

Contact

For questions about this adapter, contact Thanatos0512 on Hugging Face.