  The use of pretrained masked language models (MLMs) has drastically improved
the performance of zero anaphora resolution (ZAR). We further expand this
approach with a novel pretraining task and finetuning method for Japanese ZAR.
Our pretraining task aims to acquire anaphoric relational knowledge necessary
for ZAR from a large-scale raw corpus. The ZAR model is finetuned in the same
manner as pretraining. Our experiments show that combining the proposed methods
surpasses previous state-of-the-art performance with large margins, providing
insight on the remaining challenges.
