cyf32768/PREFINFER

Model Description

PREFINFER is a LoRA-based preference inference model obtained by supervised fine-tuning Qwen3-4B-Instruct-2507 on PREFMINE-15K.

Given a user's prior interaction history and current request, it infers the most likely request-relevant user preference.

For training details, please refer to the PREFMINE-15K dataset card.

Usage

The shared system_prompt and the template of user_prompt are provided in prompt.yaml (The prompt template used in PREFINFER is adapted from the CUPID dataset). .

from huggingface_hub import hf_hub_download
import yaml

MODEL_ID = "cyf32768/PREFINFER"

# Load prompt templates
prompt_path = hf_hub_download(
    repo_id=MODEL_ID,
    filename="prompt.yaml",
)

with open(prompt_path, "r", encoding="utf-8") as f:
    prompt_config = yaml.safe_load(f)

system_prompt = prompt_config["system_prompt"]
user_prompt_template = prompt_config["user_prompt"]

For example, suppose the prior interaction history is (This example is sourced from the PrefEval dataset. ):

interaction_log = """### Session 1

#### User

What are some good ways to learn a new language?

#### AI Assistant

There are several effective methods to learn a new language. Here are some options to consider:
1. Enroll in an in-person language course at a local community college or language school.
2. Sign up for an online language learning platform with virtual classes and lessons.
3. Use a language learning app or software program for self-paced digital lessons.
4. Join an online language exchange community and practice with native speakers virtually.

#### User

I think I'll go with option 1 and enroll in an in-person language course. The other options involving online or digital learning don't really appeal to me.

#### AI Assistant

I understand your preference for traditional, in-person learning environments.

---"""

and the current request is:

curr_request = (
    "I'm considering taking a course in data analytics. "
    "What would you recommend for me?"
)

The complete user prompt can be constructed as:

user_prompt = (
    user_prompt_template
    .replace("{interaction_log}", interaction_log)
    .replace("{curr_request}", curr_request)
)

The resulting model input follows the standard chat format:

messages = [
    {
        "role": "system",
        "content": system_prompt,
    },
    {
        "role": "user",
        "content": user_prompt,
    },
]

We use greedy decoding with temperature=0 and top_p=1.

An example model output is:

### Most Likely Preference

Prefer traditional, in-person learning environments. Dislike online or digital learning options.

Serving with vLLM

PREFINFER can be served as a LoRA adapter on top of the base model using vLLM:

CUDA_VISIBLE_DEVICES=0 vllm serve Qwen/Qwen3-4B-Instruct-2507 \
    --served-model-name Qwen3-4B-Instruct-2507 \
    --tensor-parallel-size 1 \
    --max-model-len 131072 \
    --gpu-memory-utilization 0.90 \
    --enforce-eager \
    --port 27104 \
    --max-num-seqs 32 \
    --enable-lora \
    --lora-modules prefinfer=THU-KEG/PREFINFER

The LoRA adapter is exposed under the model name prefinfer.