DomSense（域策）— Domain-Aware MDP Decision Model

🤗 Hugging Face Hub

English | 中文

---

English

DomSense (short for Domain Sense) is a lightweight, domain-aware Markov Decision Process (MDP) decision model. It is designed for diverse real-world decision-making scenarios — medical triage, financial planning, customer service routing, etc. Given a scene description and a list of candidate actions, it outputs the optimal action choice, Q-value estimates, state transition probabilities, and decision confidence.

Model Architecture

DomSense adopts a four-stage pipeline:

1. LightweightTextEncoder — UTF-8 byte-level character embedding + lightweight CNN, maps input text to semantic vectors. No pretrained language model required; self-contained and low-latency.
2. DomainMoE — Optional sparse mixture-of-experts routing that automatically dispatches inputs to the appropriate domain expert sub-network. Enables domain-aware representation enhancement.
3. MDPHead — Multi-step look-ahead via bootstrapped value iteration, modeling the state-action value function Q(s,a) and state transition probability P(s'|s,a).
4. DecisionHeads — Fuses Q-values and MDP results to output final action selection probabilities, confidence, and expected return.

Key Features

• Automatic domain routing — No need to specify domain labels manually; the model automatically determines which domain the input belongs to and activates the corresponding expert
• Multi-step look-ahead — N-step value propagation via differentiable Bellman iterations
• Lightweight & self-contained — Only requires PyTorch + Transformers; no external LM API needed
• Discrete action space — Supports any number of candidate actions (fixed at training, variable at inference)

Quick Start

from modeling_domsense import SystemOneModelForDecision

# Load model directly from Hugging Face Hub
model = SystemOneModelForDecision.from_pretrained("LIJINGHAI111/DomSense")
model.eval()

# Prepare input
scene_text = "A patient with persistent low-grade fever and cough for three days"
action_options = ["Self-medicate at home", "Visit fever clinic immediately", "Rest and hydrate"]

# Inference
out = model.forward([scene_text], action_texts=[action_options])

# Parse results
choice_idx = out.choices[0].item()
confidence = out.confidences[0].item()
q_values = out.q_values[0].tolist()
probs = out.choice_probs[0].tolist()

print(f"Choice: {action_options[choice_idx]}  Confidence={confidence:.4f}")
print(f"Action probs: {probs}")
print(f"Q-values: {q_values}")

Output Fields

Field	Type	Description
choices	LongTensor	Index of the chosen optimal action
choice_probs	FloatTensor	Selection probability for each action
transition_probs	FloatTensor	State transition matrix [batch, n_actions, 2]
expected_returns	FloatTensor	Expected return value
q_values	FloatTensor	Q-value estimate for each action
confidences	FloatTensor	Decision confidence score
embeddings	FloatTensor	Scene text embedding vector

Model Weights

This repository includes a set of pretrained weights (saved_model/) using the lightweight encoder backend. The weights are in standard Hugging Face safetensors format and can be loaded directly via from_pretrained.

Specifications:

• Parameters: ~1.1M
• Encoder: LightweightTextEncoder (byte-level character embeddings)
• Domain experts: configured at training time
• Output format: HF ModelOutput compatible

Converting from Training Checkpoints

If you have your own training checkpoints, use the conversion script:

python scripts/convert_to_hf.py --checkpoint path/to/checkpoint.pt --output ./my_model

Repository Structure

huggingface_repo/
├── configuration_domsense.py   # HF configuration class
├── modeling_domsense.py        # HF model class + core kernel
├── tokenization_domsense.py    # HF tokenizer wrapper
├── __init__.py                 # AutoXxx registration
├── src_models/                 # Original model modules (reference)
├── saved_model/                # Converted HF-format weights
├── scripts/
│   ├── convert_to_hf.py        # checkpoint → HF format converter
│   └── inference_example.py    # Inference example
└── upload_to_hub.py            # Upload helper script

---

中文

DomSense（Domain Sense，中文名「域策」）是一种轻量级、领域感知的马尔可夫决策过程（MDP）决策模型。它针对多样化的现实决策场景（医疗、金融、客服等）设计，能够在给定场景描述和候选动作列表时，输出最优动作选择、Q 值估计、状态转移概率和置信度。

模型架构

DomSense 采用四阶段流水线：

1. 编码器（LightweightTextEncoder）：UTF-8 字节级字符嵌入 + 轻量 CNN 编码器，将输入文本映射为语义向量。无需预训练语言模型，自包含、低延迟。
2. 领域 MoE（DomainMoE）：可选的稀疏专家混合路由，自动将输入分发到对应的领域专家子网络。支持领域感知的表示增强。
3. MDP 前瞻（MDPHead）：基于 Bootstrapped Value Iteration 的多步前瞻，建模状态-动作价值函数 Q(s,a) 和状态转移概率 P(s'|s,a)。
4. 决策头（DecisionHeads）：从 Q 值和 MDP 结果中融合输出最终的动作选择概率、置信度和期望回报。

关键技术特点

• 领域自动路由：无需手动指定领域标签，模型自动判断输入属于哪个领域并激活对应专家
• 多步前瞻：通过可微分的 Bellman 迭代进行 N 步价值传播
• 轻量自包含：仅依赖 PyTorch + Transformers，无需外部语言模型 API
• 离散动作空间：支持任意数量的候选动作（训练时固定，推理时可变）

快速使用

from modeling_domsense import SystemOneModelForDecision

# 直接从 Hugging Face Hub 加载模型
model = SystemOneModelForDecision.from_pretrained("LIJINGHAI111/DomSense")
model.eval()

# 准备输入
scene_text = "患者持续低烧三天伴随咳嗽，请选择下一步处置"
action_options = ["建议自行服药观察", "立即前往发热门诊", "多喝水并休息"]

# 推理
out = model.forward([scene_text], action_texts=[action_options])

# 解析结果
choice_idx = out.choices[0].item()
confidence = out.confidences[0].item()
q_values = out.q_values[0].tolist()
probs = out.choice_probs[0].tolist()

print(f"选择: {action_options[choice_idx]}  置信度={confidence:.4f}")
print(f"各选项概率: {probs}")
print(f"Q 值: {q_values}")

输出字段说明

字段	类型	说明
choices	LongTensor	选择的最优动作索引
choice_probs	FloatTensor	各动作的选择概率
transition_probs	FloatTensor	状态转移概率矩阵 [batch, n_actions, 2]
expected_returns	FloatTensor	期望回报值
q_values	FloatTensor	各动作的 Q 值估计
confidences	FloatTensor	决策置信度
embeddings	FloatTensor	场景文本的嵌入向量

模型权重

本仓库附带了一份训练好的权重（saved_model/），使用 lightweight 编码器后端。权重格式为标准 Hugging Face safetensors 格式，可直接通过 from_pretrained 加载。

权重规格：

• 参数量：约 1.1M
• 编码器：LightweightTextEncoder（字节级字符嵌入）
• 领域专家数：由训练配置决定
• 输出格式：HF ModelOutput 兼容

从训练 checkpoint 转换

如果你有自己的训练 checkpoint，可以用转换脚本将其转换为 HF 格式：

python scripts/convert_to_hf.py --checkpoint path/to/checkpoint.pt --output ./my_model

转换后即可用 from_pretrained 加载。

仓库结构

huggingface_repo/
├── configuration_domsense.py   # HF 配置类
├── modeling_domsense.py        # HF 模型类 + 原始内核
├── tokenization_domsense.py    # HF Tokenizer 包装
├── __init__.py                 # AutoXxx 注册
├── src_models/                 # 原始模型模块（参考）
├── saved_model/                # 已转换的 HF 格式权重
├── scripts/
│   ├── convert_to_hf.py        # checkpoint → HF 格式转换
│   └── inference_example.py    # 推理示例
└── upload_to_hub.py            # 上传辅助脚本

---

License / 许可证

Apache 2.0