ssg1/doc-compass

Doc Compass

Routes a plain-language health concern to a kind of doctor to book. It suggests a door to knock on; it does not diagnose and is not medical advice.

This repo holds the app code (doccompass/) and its routers, so the demo notebook can run without access to the project's GitHub repo.

What is here

File	What it is
checkpoints/distilroberta_stage1/	distilbert/distilroberta-base fine-tuned (all weights) to route concerns to 12 labels
checkpoints/tfidf_stage1.joblib	TF-IDF + logistic regression router, trained from scratch on the same data
doccompass/	emergency rules, routers, explanation and the Gradio app

Labels: Dermatology, Orthopedics, ENT, Gastroenterology, Neurology, Urology, Ob-Gyn, Mental health, Cardiology, Eye care, Dentistry, Start with a GP.

Training data

Stage 1 only: 6,252 unique comments from Patient Comments and Specialist Types (Mendeley Data, DOI 10.17632/2twgjzpn82.2, CC BY 4.0), with its 68 symptom categories remapped to the labels above. These are short, one-sentence comments, so treat results on this set as a sanity check only.

Results

On 938 held-out comments from that same public set, the fine-tuned router scores 94.3% top-1 (95% CI 92.7%–95.7%), 99.0% top-3 and 0.908 macro-F1. The comments are short and similar to each other, so this is an easy score: it says how well the router fits the public data, not how it does on real posts.

Limits

• English only, and trained on short comments. It can be confidently wrong on wording the public set doesn't cover: "my gums bleed when I brush" is routed to Ob-Gyn.
• The emergency check is a short list of written rules and will miss emergencies phrased in ways the rules don't anticipate. In an emergency call 911.
• Not clinically validated.