NagaYu/deference-keigo

Deference — Japanese honorific (keigo) error detection

Detects the direction of deference in Japanese business writing and points to the passage of the Council for Cultural Affairs' Keigo no Shishin (敬語の指針, 2007) that the judgement rests on.

A token-classification model over BIO × error type, trained on a corpus constructed from the norm by rule — no LLM was used to create the errors.

• Base model: xlm-roberta-base
• Parameters: 277.5M (0.277B)
• Inference: CPU, ~26 ms per message after int8 quantisation
• Code, evaluation and Gradio app: https://github.com/NagaYu/deference

The problem this addresses

Whether a Japanese honorific is appropriate often cannot be decided from the string. The guidelines define sonkeigo as raising the one who acts and kenjougo I as raising the one the act is directed to (Ch.2 Sec.1, pp.14-15). So 「お持ちします」 is appropriate when the writer carries something and inappropriate when the reader does (Ch.3 Sec.2-2 Q11, p.37). The verdict also flips with the audience (Ch.3 Sec.3-2 Q25, pp.44-45).

The audience is therefore a required input, encoded as a prefix.

Input format

[社外][書き手:自分側][相手:貴社] <message body>

The prefix carries the audience and the standpoints; character offsets of predictions are mapped back onto the body. Use the deference package rather than building the prefix by hand:

from deference import Deference, MailContext, Audience, Person, Party

df = Deference(engine="neural", model_dir="<path to this checkpoint>")
ctx = MailContext(
    audience=Audience.EXTERNAL,
    persons=(Person("佐藤", Party.SELF_GROUP), Person("田中", Party.ADDRESSEE)),
)
for f in df.check(open("mail.txt").read(), ctx).reportable:
    print(f.span.text, f.message, f.citation.render("en"))

Raw transformers use is possible but you lose the citation, the correction candidates and the variation filter:

from transformers import AutoTokenizer, AutoModelForTokenClassification
tok = AutoTokenizer.from_pretrained("NagaYu/deference-keigo")
model = AutoModelForTokenClassification.from_pretrained("NagaYu/deference-keigo")

Results

Validation split of NagaYu/deference-keigo-corpus.

metric	P	R	F1
overall (micro)	0.986	0.989	0.988
direction errors	0.975	0.981	0.978

error type	key	needs context	P	R	F1	n
Respectful/humble mix-up (direction)	direction_swap	yes	0.995	1.000	0.998	218
Doubled honorific (nijuu keigo)	double_keigo	no	1.000	1.000	1.000	179
'go-...-sareru' form	go_sareru	no	1.000	1.000	1.000	161
'o/go-...-dekiru' form	ogo_dekiru	no	1.000	1.000	1.000	135
Respectful form for oneself	self_sonkeigo	yes	0.881	0.987	0.931	75
Ill-formed honorific chain	bad_keigo_link	yes	1.000	1.000	1.000	27
Respectful form for one's own side (uchi)	uchi_sonkeigo	yes	0.938	0.652	0.769	23
Inserted 'sa' (sa-ire kotoba)	sa_insertion	no	1.000	1.000	1.000	21

Against other tools

Full pipeline (rules + this model), 559 texts with errors, 152 without, 53 variation cases:

condition	detection (type)	direction errors	no-over-flag	median
textlint (4 JA presets)	0.0%	0.0%	94.3%	2.26 ms
surface rule set	42.9%	31.7%	90.6%	0.01 ms
Deference	97.9%	97.4%	100.0%	31.33 ms
Deference (int8)	98.0%	97.6%	100.0%	25.55 ms

On error types visible in the surface pattern (doubled honorifics, sa-insertion, the go-...-sareru form) the rule set also reaches 100% — no advantage is claimed there. The difference is confined to types that need context.

Variation is not error

Expressions whose acceptability is genuinely divided are held apart and not reported. The guidelines warn against treating usage uniformly by gender or generation (Ch.1 Sec.2-2, p.8), list doubled honorifics established by custom (お伺いする, お召し上がりになる, お見えになる; Ch.2 Sec.2-6(2), p.30), and say that tolerance for させていただく varies by individual (Ch.3 Sec.2-6 Q18).

The training data includes those cases as negatives, and the pipeline applies a variation filter on top of the model's output.

Constrained by the norm

For error types whose truth can be checked on the surface, a model prediction is verified against the rules and dropped if it cannot hold. The model read 「ご報告させていただきます」 as sa-insertion, but 報告する is a suru verb whose causative is 報告させる — the reading is impossible, so it is discarded.

Correction candidates are likewise restricted to forms the rules can generate, so a suggestion can never itself introduce a new divergence.

Limitations

• Dialects and spoken language are out of scope. Sa-insertion is common in speech; it is treated here from the standpoint of written norms, and the citation says the report does not cover it.
• Where the actor cannot be read from the text, no direction judgement is made.
• Trained on synthetic sentences bounded by the generator's lexicon (49 verbs, 35 suppletive forms). Scores here do not by themselves establish performance on real correspondence.
• uchi_sonkeigo has the smallest support and the weakest score of the context-dependent types; treat its output with more caution than the rest.
• This is information, not a verdict. Output is phrased as how the guidelines organise the matter, never as a judgement on the writer's Japanese.

Training

python scripts/build_dataset.py --output data/hf
python scripts/train.py --data data/hf --output checkpoints/deference-base     --epochs 3 --batch-size 8 --lr 3e-05     --max-length 192

Variation samples are fed as negatives — that is the mechanism by which the model learns not to flag divided usage.

Source

Adapted from the Council for Cultural Affairs' report Keigo no Shishin (敬語の指針), Agency for Cultural Affairs, 2007. https://www.bunka.go.jp/seisaku/bunkashingikai/kokugo/hokoku/pdf/keigo_tosin.pdf

Content on the Agency's site follows the MEXT website terms of use, stated to be compatible with CC BY and permitting adaptation provided the source is credited; where the material has been edited or adapted that must be stated. This project keeps only short quotations as the basis for each note and does not redistribute the report in full.

Citation

@misc{deference_keigo,
  title  = {Deference: detecting the direction of deference in Japanese honorifics},
  author = {NagaYu},
  year   = {2026},
  note   = {Adapted from Keigo no Shishin, Agency for Cultural Affairs},
  url    = {https://huggingface.co/NagaYu/deference-keigo}
}