CodesBee/DeQARAG

DeQA-RAG question decomposer

This repository contains the trained LoRA adapter used by DeQA-RAG to turn an original question into useful supporting questions. It requires the Qwen3-4B-Instruct-2507 base model, using base revision cdbee75f17c01a7cc42f958dc650907174af0554.

The adapter contains 66,060,288 parameters stored as 504 float32 tensors. It applies ordinary LoRA with rank 32, alpha 64 and dropout 0.05 to the attention and MLP projections. The base weights are loaded separately in bfloat16.

Input and output

Use the included decomposition.txt as the system message and the original question as the user message, with the base tokenizer's native chat template. Preserve the prompt text, including its final newline.

The output is a JSON array of supporting-question strings. Each question preserves its identifying description so it can be retrieved independently. The output contains no answers or references such as #1. An empty array is allowed. DeQA-RAG separately retrieves and answers the original question after the supporting questions.

Use greedy generation: do_sample=False, num_beams=1, max_new_tokens=1024, seed 1729, and SDPA attention. The DeQA-RAG loader explicitly sets its generation configuration to avoid inheriting sampling defaults.

Loading

After installing the DeQA-RAG project with its model dependencies:

from deqa.decompose import Decomposer

decompose = Decomposer("CodesBee/DeQARAG")
questions = decompose("Who owns the company where the composer of Song S works?")
print(questions)

The underlying PEFT loading pattern is:

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B-Instruct-2507",
    revision="cdbee75f17c01a7cc42f958dc650907174af0554",
    dtype=torch.bfloat16, attn_implementation="sdpa",
).to("cuda")
model = PeftModel.from_pretrained(base, "CodesBee/DeQARAG").eval()

Training and evaluation scope

The project training examples combine BREAK-high sources, HotpotQA, 2WikiMultiHopQA, MuSiQue and explicitly authored synthetic coverage examples. The project author led the design, development and review of the supporting-question annotations.

The released weights continue an existing ordinary LoRA for 40 additional optimizer updates. This continuation combines existing decomposition examples with native coverage questions and explicitly marked synthetic examples.

This adapter was developed for the project's English multi-hop QA experiments. Generated questions can omit prerequisites or change the meaning of a relation. DeQA-RAG's evaluation package includes the original saved supporting questions for its fixed evaluation pool; use those files when reproducing that evaluation.

The adapter is released under Apache-2.0. The base model's Apache-2.0 license also applies to its original materials.