snowflake-Lora-512

Model description

Model with Trained Lora adapter based on "dragonkue/snowflake-arctic-embed-l-v2.0-ko"

Model Details

Base Model

• Base model: "dragonkue/snowflake-arctic-embed-l-v2.0-ko"

LoRA Configuration

LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=["query", "value"],
    lora_dropout=0.1,
    bias="none",
    task_type=TaskType.FEATURE_EXTRACTION,
)

Training Loss Function

losses.CachedGISTEmbedLoss(
    model=model,
    guide=guide_model,
    temperature=0.2,
    mini_batch_size=128,
    margin_strategy="absolute",
    margin=0.1,
    contrast_positives=False
)

Batch size : 512

Training Datasets

AI Hub Dataset

• 기계 독해
• Language: Korean

---

Usage

python

Usage code here

from sentence_transformers import SentenceTransformer

LORA_MODEL : "skdiwlsdn5/Snowflake-Lora-512"

model = SentenceTransformer(
    LORA_MODEL
)