krishgolcha/Qwen3-4B-Customer-Support

Qwen3 4B Customer Support

A customer support adaptation project built around Qwen3-4B-Instruct-2507.

Release status

This repository contains a training recipe, not a trained checkpoint. Training has not been run. No adapter weights, benchmark results, or performance improvements are claimed. Use the upstream Qwen model for base-model inference until a verified adapter is released.

Intended scope

English customer support responses for orders, delivery, account access, refunds, and escalation. This is a response-generation project; it does not implement backend tools or perform real customer account actions.

Data

Source: Bitext customer support dataset, copyright Bitext Innovations, 2024, CDLA-Sharing-1.0. The source card reports 26,872 question/answer pairs. This is an upstream count, not a processed count. The preparation command downloads the complete source snapshot, preserves its README/license files, normalizes instruction/response pairs, removes duplicate user instructions, and creates deterministic 90/5/5 train/validation/test splits. It records the resolved source commit and SHA-256 hashes in data/manifest.json. Actual counts are generated only when preparation runs. Data is not yet mirrored in this repo. Generated source and derived data retain their source license; the Apache-2.0 metadata applies to original code, not Bitext data.

Quick start

On a CUDA machine with sufficient GPU memory, create a Python environment and install:

pip install torch transformers datasets peft accelerate huggingface_hub safetensors
python train.py prepare
python train.py train

The default recipe uses BF16 LoRA, batch size 1, gradient accumulation 16, sequence length 1024, one epoch, rank 16, alpha 32, dropout 0.05, learning rate 2e-4. These are proposed settings, not completed-run parameters. Memory needs depend on GPU and package versions. Training requires CUDA and BF16 support. The script saves the adapter, tokenizer, training state, package versions and validation loss locally. It never starts paid cloud jobs or uploads automatically. After reviewing artifacts:

hf auth login
hf upload krishgolcha/Qwen3-4B-Customer-Support output/customer-support .
hf upload krishgolcha/Qwen3-4B-Customer-Support data data

Review source license obligations before redistributing data. Preserve the source notices and license in data/source.

Evaluation plan

Validation loss is calculated during training. The test split remains held out. Before release, compare the adapter against the base model on answer correctness, policy grounding, escalation, privacy, and false action claims. Check paraphrase leakage between splits; exact instruction deduplication does not remove semantic duplicates. No evaluation has been performed.

Limitations

Synthetic support data contains placeholders and generic policies. Responses can invent policy or imply actions were completed. Ground production answers in approved policies, authenticate users, verify tool results, and route sensitive requests to human support. No production readiness is claimed.

Repository contents

• train.py: complete data preparation and LoRA training entry point.
• README.md: status, source attribution, proposed recipe, and release instructions.

License

Original repository code: Apache-2.0 (https://www.apache.org/licenses/LICENSE-2.0). Base model: see the upstream Qwen license. Source and derived Bitext data: CDLA-Sharing-1.0 (https://cdla.dev/sharing-1-0/).