MThien213/deepparse-iot-r1-8b

DeepParse R1-8B adapter, fine-tuned for IoT logs

This is a LoRA adapter for deepseek-ai/DeepSeek-R1-Distill-Llama-8B that does the job of the mask synthesiser in DeepParse (DeepParse: Hybrid Log Parsing with LLM-Synthesized Regex Masks, Shetaia et al.). You show it a few dozen raw log lines, it answers with a Python list of regexes for the variable parts, and DeepParse applies those regexes once as masks before its Drain stage parses the whole stream.

The authors released an adapter trained on LogHub-2k (NightBaRron1412/deepparse-r1-8b). This one is trained on the same 14 LogHub-2k systems plus a synthetic IoT fleet corpus (sensors, cameras, MQTT, OTA, device twins, kernel lines), so it also knows the shapes that LogHub never shows: values after key=, quoted values, dash-joined device ids like TH-75230 or gw-edge-53, MAC addresses, long hex digests, 1280x720 resolutions, MCC/MNC codes.

Usage

git clone https://github.com/NightBaRron1412/DeepParse && cd DeepParse
pip install -e ".[hf]"
export HF_TOKEN=...   # the base model is gated

deepparse synth --dataset <system> --mode hf --k 50 --adapter <this-repo-id> --out masks.json
deepparse eval  --config configs/eval_16_datasets.yaml --deterministic

bf16 needs about 16 GB of VRAM for the base model. The file deepparse_finetune_config.json shipped here says load_in_4bit: true; a DeepParse build that reads it loads the base in NF4 and fits in about 5.5 GB.

Training

Base	DeepSeek-R1-Distill-Llama-8B, NF4 4-bit, bf16 compute
LoRA	r 8, alpha 32, dropout 0.01, on q/k/v/o/gate/up/down
Optimiser	paged AdamW 8-bit, lr 2e-4, cosine, 5 % warm-up
Epochs	5, effective batch 16, 280 steps, completion-only loss
Data	893 examples: 50 entropy-greedy lines per LogHub-2k system (605) + 6 lines per template of the IoT corpus (288)
Hardware	one RTX 4060 Laptop (8 GB), about 50 minutes

Results

Protocol as in the paper: LogHub-2k corrected (14 of 16 systems, Android and Windows have no corrected release), k = 50 entropy-greedy sample, greedy decoding, Drain depth 5 / similarity 0.4 / max children 100, GA and PA over all 2,000 lines, one seed. The same mask bundles are scored by two parsers: the Drain implementation that ships with the DeepParse repository ("repo engine") and the open-source drain3 library behind the same mask-first preprocessing ("Drain3"). The paper's own numbers come from an extended Drain3 that the repository does not contain. The IoT fleet is a 2,000-line synthetic corpus with 48 templates, not part of the paper.

Grouping accuracy

Dataset	Paper	Authors' adapter, repo engine	This adapter, repo engine	Authors' adapter, Drain3	This adapter, Drain3
Apache	1.000	1.000	1.000	1.000	1.000
BGL	0.964	0.483	0.483	0.971	0.971
HDFS	1.000	0.998	0.998	0.998	0.998
HPC	0.973	0.873	0.873	0.887	0.887
Hadoop	0.985	0.948	0.750	0.945	0.945
HealthApp	0.868	1.000	1.000	0.902	0.902
Linux	0.956	0.178	0.172	0.671	0.693
Mac	0.795	0.775	0.794	0.731	0.731
OpenSSH	0.661	0.488	0.488	0.789	0.789
OpenStack	0.990	0.513	0.513	0.807	0.807
Proxifier	1.000	0.001	0.001	0.017	0.017
Spark	0.987	0.730	0.730	0.924	0.924
Thunderbird	0.971	0.651	0.645	0.950	0.961
Zookeeper	0.999	0.989	0.989	0.967	0.967
IoT fleet	n/a	0.827	0.916	0.888	0.888
Mean, 14 LogHub	0.939	0.688	0.674	0.826	0.828

Parsing accuracy

Dataset	Paper	Authors' adapter, repo engine	This adapter, repo engine	Authors' adapter, Drain3	This adapter, Drain3
Apache	1.000	1.000	1.000	1.000	1.000
BGL	0.983	0.471	0.423	0.559	0.526
HDFS	1.000	0.647	0.647	0.647	0.647
HPC	0.996	0.875	0.875	0.886	0.886
Hadoop	0.983	0.107	0.178	0.107	0.359
HealthApp	0.997	0.864	0.873	0.766	0.775
Linux	0.971	0.000	0.161	0.000	0.224
Mac	0.738	0.426	0.522	0.402	0.461
OpenSSH	0.994	0.248	0.244	0.538	0.538
OpenStack	0.998	0.350	0.350	0.179	0.179
Proxifier	1.000	0.000	0.000	0.000	0.000
Spark	0.995	0.213	0.704	0.378	0.869
Thunderbird	0.976	0.437	0.467	0.165	0.197
Zookeeper	1.000	0.967	0.967	0.945	0.945
IoT fleet	n/a	0.136	0.905	0.096	0.865
Mean, 14 LogHub	0.974	0.472	0.529	0.470	0.543

What the numbers say: grouping is set by the parser, not by the adapter. With Drain3 both adapters land at the same GA, and the repo engine costs about 15 points on LogHub. On parsing this adapter is ahead of the authors' on both parsers, mostly because it does not emit catch-all patterns such as [0-9a-fA-F]+ (which zero Linux PA and cut Spark for the authors' bundle). On IoT logs it is the only one that works. Neither adapter reaches the paper's numbers on either parser.

Limitations

• The Drain port in the public repository buckets lines on their first five literal tokens and only treats pure-digit tokens as wildcards. On systems whose variables sit in the first tokens (Linux, Proxifier, OpenSSH, BGL) grouping stays low whatever masks you give it; Drain3 recovers most of that.
• IoT conventions: units stay static (rsrp=<*>dBm), a dash-joined id is one variable, ip:port is one variable. LogHub writes ip:port as <*>:<*>, which costs some PA there.
• The IoT corpus is synthetic. Try a sample of your own fleet before relying on it.
• It only answers the DeepParse prompt with a regex list; it is not a chat model.

License

Apache 2.0 for the adapter and this card. The base model keeps the DeepSeek licence.