zipqa-intent-150k — natural-language questions over a US ZIP / county / city table, with a 148k-parameter model

A from-scratch, 148,195-parameter character-level parser that turns a plain-English or Hinglish question about a US ZIP-code table into a structured query, plus a deterministic engine that answers it from the table. The model decides what is being asked; the engine guarantees every fact comes from a row. It never hallucinates a county.

"60614 ka county aur area code batao"
   → parser : intent=lookup  wants=[COUNTY_NAME, AERA_CODE]  slots=[(ZIP, "60614")]
   → engine : ZIP 60614 is in Cook County and has area code 773.

Trained on CPU (2 cores) in ~2.5 hours. Inference is ~2 ms for the model and ~100 ms for a fuzzy name lookup over 36k place names. Weights: 594 kB.

The table is not included. The model was trained against a licensed USPS-derived dataset (99,214 rows: ZIP, STATE, STATE_NAME, CITY, CITY_ABB, LATITUDE, LONGITUDE, COUNTY_CODE, COUNTY_NAME, AERA_CODE, ZIP_TYPE, LAST_LINE_INDICATOR, FACILITY_CODE, FINANCE_NUMBER, CBSA, CBSA_TITLE, CBSA_STATUS). Any CSV with those columns works; the free HUD USPS ZIP crosswalk or the Census ZCTA relationship files can be mapped to it. The engine builds its indexes from whatever CSV you give it.

---

1. Why not just train a model that memorises the table?

That was the first idea — published as sraivante/zip-county-city-seq2seq-21m — and it is the wrong tool. A ~40k-ZIP table is ~100k rows of digits and proper nouns with no pattern to generalise from. A small transformer trained to emit "72013 ARECIBO" after "zip 612" is gradient descent used as a lossy hash map: it confuses 612/613/614 (adjacent numbers, same county — great for generalisation, fatal for exact recall), invents plausible FIPS codes and coordinates, and has to be retrained on every table refresh.

What people actually need is: understand the question, look up the facts. Those are different problems with different right-sized solutions. Understanding ~7 question shapes over ~12 columns is a 150k-parameter job. Looking up facts is a dictionary.

2. Architecture

question (≤96 chars, lower-cased)
   │
   ▼  char embedding (52 symbols → 48-d)
   │
   ▼  1-layer bidirectional GRU, hidden 128  ─────────────┐
   │                                                       │
   ├─► per-character head → BIO slot tags                  │  (ZIP, CITY, COUNTY, STATE, CBSA, AREACODE, LAT, LON)
   │                                                       │
   └─► sentence vector (final states + mean pool) ─────────┤
         ├─► softmax → intent   (lookup, list, count, check, nearest, top, unknown)
         └─► sigmoid → wants    (COUNTY_NAME, COUNTY_CODE, STATE, CITY, ZIP, AERA_CODE,
                                 LATLON, CBSA_TITLE, CBSA, ZIP_TYPE, ALL — multi-label)

Three heads, one loss (CE + BCE + CE). Char-level because "Mayagez", "mayagüez" and "MAYAGUEZ" must land on the same embedding and unseen place names must not become <unk>. A 2-layer GRU was 2× slower on CPU for no accuracy gain.

3. Training data: generated from the table itself

No human-labelled questions exist for this, so the corpus is synthesised from the table — the city, county and state names are the entity vocabulary, so coverage is exhaustive by construction. 1,000,000 examples:

intent	share	how it is generated
lookup	42%	55 question frames × 11 want-columns × random entity; 20% compound ("county and area code")
list	14%	"zips in X", "neighbourhoods of X", "which states have a Springfield"
check	13%	"is X in Y", "are X and Y in the same county"
count	11%	"how many zips in X", "kitne county hai"
unknown	8%	weather, population, greetings, SQL, gibberish, questions about places in the table that the table can't answer
nearest	6%	"closest city to 18.44, -66.40" with random coordinates in the table's bounding box
top	6%	"which county has the most zips", "top 5 counties in Texas by zip codes", "sabse zyada…"

Every example is built from labelled segments, so character-level slot tags survive the noise that is then applied: keyboard-neighbour typos, dropped/swapped/doubled letters (35% of entity tokens, 12% of frame tokens), random casing, dropped neighbourhood prefixes ("Urb Monaco 3" → "Monaco 3"), un-padded ZIPs ("603" for "00603"), chatty prefixes and suffixes ("bhai", "pls", "batao"), and Hinglish frames ("{E} kis county me hai", "{E} ka {T} batao").

Three generator decisions turned out to matter more than anything in the model:

1. State qualifiers must be true. 35% of city/county mentions carry a state ("austin tx", "portland, maine"), tagged STATE — but only a state that place actually exists in. A first run without this read "springfield MA" as a metro-area question because it had never seen a state code after a city.
2. Word-like state codes are never bare surfaces. IN OR ME OK HI DE LA OH PA MA CO ID AL MO NE MD MS AR MI are English (or Spanish) words. A model that learns "miami in" → Indiana is worse than useless; the generator only emits these as full state names, and the parser accepts the 2-letter form only when written in capitals (or, for the rarer ones like co, ma, immediately after a place name).
3. The unknown class must not contain in-table entities with answerable phrasings. "Tell me about {E}" was originally an unknown example; it is a legitimate lookup (ALL). Conversely, foreign places ("pin code of Varanasi") were removed from the unknown templates — the model should not learn that "Varanasi" means unknown, it should learn that unresolvable means unknown, which the engine decides.

4. Training

python3 prepare_data.py raw_dump.csv table.csv        # clean  (see §6)
python3 gen_corpus.py   table.csv corpus.jsonl 1000000
python3 train.py        corpus.jsonl parser.pt 2      # AdamW 2e-3, OneCycle, bs 256, ~75 min/epoch on 2 CPU cores

Validation on a 2% held-out split of the generated corpus:

epoch	intent acc	wants exact-match	slot tag acc (per char)
1	99.93%	97.56%	99.02%
2	99.98%	98.78%	99.22%

Those numbers are against generated questions and are mostly a sanity check. The number that matters is below.

5. Evaluation on questions a person typed

eval_full.py holds 48 questions written without looking at the templates — English, Hinglish, typos, compound asks, ambiguous names, out-of-scope requests — each with the substring the answer must contain. Final score: 48/48.

which county is 90210 in               → ZIP 90210 is in Los Angeles County.
Bevrly Hills california zip            → Beverly Hills has ZIP codes 90209, 90210, 90211, 90212, 90213.
which county is springfield in         → There are 26 places called Springfield (AR, CO, GA, …). Which state do you mean?
county of springfield MA               → Springfield is in Hampden County.
what county is las vegas in            → (assuming Las Vegas, NV; it also exists in NM) Las Vegas is in Clark County.
county of St. Louis MO                 → Saint Louis is in Saint Louis City County, Saint Louis County.
Cook County IL me kitne zip hai        → Cook County has 228 ZIP codes.
how many states have a washington county → There is a Washington County in 31 states: Alabama (AL), …
top 3 states by number of zip codes    → 1. Texas (2598); 2. California (2589); 3. Pennsylvania (2172).
is 10001 in the new york metro         → Yes, ZIP 10001 is in the New York-Newark-Jersey City metro area.
county of 09002                        → ZIP 09002 is a military (APO/FPO/DPO) address in Armed Forces Europe (AE)…
population of los angeles county       → I can only answer questions about this ZIP table: …

6. What the full table forced us to fix (none of it in the model)

Going from a 1,000-row sample to 99,214 rows broke nothing in the parser and almost everything in the engine. This is the central lesson of the project: the model was never the bottleneck.

problem at scale	fix
The dump had stray " characters inside CITY_ABB and CBSA_TITLE; a standard CSV parser silently dropped 1,158 rows	no field contains a comma, so: strip all quotes, split on commas
CBSA_TITLE was double-encoded mojibake (San GermÃÆ'…)	repair latin1→utf8 twice, fold to ASCII, override 4 titles by CBSA code
ZIP stored as int: 00601 → 601	zero-pad on load; accept both forms on input
31 Washington Counties, 26 Springfields, 6,543 city names in more than one state	state scoping: a STATE slot narrows; else if one state holds ≥80% of the ZIPs (New York, Los Angeles) assume it and say so; else ask "which state?"
Names that are a city and a county and a state (Miami, Vermont, Washington, Nevada)	decided from cue words ("county", "city") and position, not the tagger alone
A city called Many (LA) and a city called Tell (TX)	a single-word gazetteer hit is rejected if the word occurs in any question template
"st louis" exists literally in MS and TX; Missouri's is "SAINT LOUIS"	st/ste/mt/ft/pt aliases, applied when the state filter comes back empty
difflib fuzzy matching: ~2 s per query over 36k names	rapidfuzz WRatio: ~100 ms; plus a ratio≥80 floor so "varanasi" stops matching "Varna"
Military APO/FPO rows: no county, coordinates 0,0	answered as military addresses; skipped by nearest
USPS label "cities" (BUSINESS REPLY MAIL, BAR CODE CHURCH STREET…)	kept in the table, excluded from question generation
"chicago metro" vs title "Chicago-Naperville-Elgin"	CBSA prefix match wins over the exact city hit
low.find("or") matched the "or" inside "portland"	whole-word span expansion

7. Every question type, verified against one row

Row: 60614, IL, Illinois, CHICAGO, 41.9238, -87.6492, 17031, COOK, 773, standard, CBSA 16980 Chicago-Naperville-Elgin. Each line below was actually run against the published weights.

Field lookups — answered

• which county is 60614 in → Cook County · what state is 60614 in → Illinois (IL) · what city is zip 60614 → Chicago
• area code of 60614 → 773 · coordinates / lat long for 60614 → 41.9238, -87.6492
• what metro area is 60614 in → Chicago-Naperville-Elgin · cbsa code for 60614 → 16980 · county fips code of 60614 → 17031
• what type of zip is 60614 / is 60614 a po box zip → standard ZIP
• tell me about 60614 → full record · where is 60614 / 60614 kahan hai → Cook County, Illinois
• 60614 ka county aur area code batao → Cook County and area code 773 (compound)
• 60614 -> city → Chicago (terse forms work)

Same row from the city / county side — answered

• zip codes of chicago illinois → 90 ZIPs · how many zips in chicago → 90 · chicago il area code → 773, 312, 847, 708
• which county is chicago in → Cook · what metro is chicago in → Chicago-Naperville-Elgin · cook county illinois fips → 17031
• neighbourhoods in 60614 → Chicago, Lincoln Park · how many places are in 60614 → 2
• which state is cook county in → (assuming Cook, IL; also GA, MN) Illinois · zips in cook county il → 228, first 40 listed

Checks — answered

• is 60614 in cook county / in illinois / in the chicago metro / in chicago → Yes
• is 60614 in dupage county → No — it is in Cook · 60614 and 90210 same state? → No. IL vs CA
• are 60614 and 60657 in the same county → Yes, Cook · do 60614 and 60601 share a metro area → Yes

Geography — answered

• nearest zip to 41.92, -87.65 → 60614 (0.4 km) · what city is at 41.9238, -87.6492 → Chicago · which county is at 41.92,-87.65 → Cook

Aggregations — answered

• which county in illinois has the most zips → Cook (228) · top 3 cities in cook county by zip codes → Chicago (90), Schaumburg (8), Palatine (7)
• which city in illinois has the fewest zips → (one of many with 1) · how many states have a chicago → 1 · which states have a cook county → IL, GA, MN

Follow-ups — answered (one turn of memory)

• "which county is 60614 in" → "and its area code?" → 773 · "aur uska metro?" → Chicago-Naperville-Elgin · "what about 60657" → full record

Refused — not in the table (scope message)

• population · weather · time zone · median income · is it safe · crime · schools · hospitals · distance between 60614 and 60601 · driving time · who delivers mail · 60614 ka pin code india

Refused — columns deliberately not exposed (explicit message)

• finance number · facility code · city abbreviation · CBSA status · last-line indicator (internal USPS bookkeeping)

Cannot do — honest limits

• Ranges and arithmetic: "zips between 60601 and 60610", "average zips per county in illinois" → answers the nearest supported thing instead of refusing
• Filters combining two attributes: "which zips in chicago are po box only", "zips with area code 773 in cook county" → one filter is applied, the other dropped
• Comparatives: "60614 vs 60615 which is bigger" → nonsense answer
• Three-way compound wants: "county, state and metro for 60614" → usually returns one or two of the three (the corpus only has two-way compounds)
• Lowercase in / or / me / ok / hi / de / oh / la / pa are never read as states — "portland or" needs OR or "oregon"
• Memory is one turn deep and entity-only; no "the second one" / "the other county"
• Answers are templated English; fluent prose would need a small seq2seq writer fed the engine's facts

8. Files

file	role
model.safetensors, config.json	the parser (148,195 params)
model.py / schema.py	architecture, labels
ask.py	QA class: parse → gazetteer backstop → scope guards → engine; CLI and interactive loop
engine.py	entity resolution (exact → abbreviation → fuzzy → cross-type), state scoping, the 6 intents, answer templates
gen_corpus.py, train.py, prepare_data.py	reproduce everything from a raw dump
eval_full.py, eval_questions.py	hand-typed test sets
example.py	10-line usage

pip install torch rapidfuzz safetensors
python3 example.py            # expects table.csv next to it
python3 ask.py                # interactive

9. Retraining on your own table

The parser learns question shapes, not facts, so it transfers to any table with these columns unchanged — the engine rebuilds its indexes from the CSV. Retrain only if your place-name distribution is very different (non-US scripts, say): gen_corpus.py reads its entity vocabulary from the table, so the four commands in §4 are the whole recipe. Adding a new question type = a few dozen frames in gen_corpus.py + a method in engine.py + retrain (~2.5 h CPU).

10. Lineage

Predecessor: sraivante/zip-county-city-seq2seq-21m, a 31M-parameter seq2seq that tried to memorise the table directly — 12.1% complete answers on fresh prompts at v0.3.0. This repo replaces it: 200× fewer parameters, 48/48 on hand-typed questions, zero hallucinated facts.

Same recipe as sraivante/rx-intent-1m (drug-name lookup): a tiny intent parser in front of a deterministic, typo-tolerant lookup engine. The thesis both models test is that for "answer questions about this dataset", a sub-megabyte model plus a dictionary beats a large model plus RAG on correctness, latency and cost — as long as you are honest about the question shapes you support and refuse the rest.