SlowGuess/ACL_2020_OCR huggingface dataset:
Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142-147. Association for Computational Linguistics.
Xuan Wang, Yu Zhang, Xiang Ren, Yuhao Zhang, Marinka Zitnik, Jingbo Shang, Curtis Langlotz, and Jiawei Han. 2019. Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics, 35(10):1745-1752.
Zhenghui Wang, Yanru Qu, Liheng Chen, Jian Shen, Weinan Zhang, Shaodian Zhang, Yimei Gao, Gen Gu, Ken Chen, and Yong Yu. 2018. Label-aware double transfer learning for cross-specialty medical named entity recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1-15. Association for Computational Linguistics.
Huiyun Yang, Shujian Huang, Xinyu Dai, and Jiajun Chen. 2019. Fine-grained knowledge fusion for sequence labeling domain adaptation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4188-4197. Association for Computational Linguistics.
Jie Yang, Shuailong Liang, and Yue Zhang. 2018. Design challenges and misconceptions in neural sequence labeling. In Proceedings of the 27th International Conference on Computational Linguistics,
pages 3879-3889. Association for Computational Linguistics.
Jie Yang and Yue Zhang. 2018. Ncrf++: An open-source neural sequence labeling toolkit. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics-System Demonstrations, pages 74-79. Association for Computational Linguistics.
Zhilin Yang, Ruslan Salakhutdinov, and William W. Cohen. 2017. Transfer learning for sequence tagging with hierarchical recurrent networks. In International Conference on Learning Representations.
Joey Tianyi Zhou, Hao Zhang, Di Jin, Hongyuan Zhu, Meng Fang, Rick Siow Mong Goh, and Kenneth Kwok. 2019. Dual adversarial neural transfer for low-resource named entity recognition. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3461-3471. Association for Computational Linguistics.
Proof. Given the precondition  $\epsilon_{\alpha}(h^{*}) = \alpha \epsilon_{T}(h_{1}) + (1 - \alpha)\epsilon_{S}(h_{2})$ , we use the triangle inequality as follows:
$$
\begin{array}{l} \epsilon_ {T} (h _ {1}) - \epsilon_ {\alpha} (h ^ {*}) \leq \left| \epsilon_ {\alpha} (h ^ {*}) - \epsilon_ {T} (h _ {1}) \right| \\ = (1 - \alpha) \left| \epsilon_ {S} \left(h _ {2}\right) - \epsilon_ {T} \left(h _ {1}\right) \right| \\ \leq (1 - \alpha) \left[ \left| \epsilon_ {S} \left(h _ {2}\right) - \epsilon_ {S} \left(h _ {1}, h _ {2}\right) \right| \right. \\ + \left| \epsilon_ {S} (h _ {1}, h _ {2}) - \epsilon_ {T} (h _ {1}, h _ {2}) \right| \\ \left. + \left| \epsilon_ {T} \left(h _ {1}, h _ {2}\right) - \epsilon_ {T} \left(h _ {1}\right) \right| \right] \\ \end{array}
$$
The triangle inequality in Crammer et al. (2008) states that for a class of models  $\mathcal{F}$  and expected error function  $\epsilon$  if for all  $g_{1}, g_{2}, g_{3} \in \mathcal{F}$ , we have  $\epsilon(g_{1}, g_{2}) \leq \epsilon(g_{1}, g_{3}) + \epsilon(g_{2}, g_{3})$ . Following the above formula and the definition of  $d_{\mathcal{H}\Delta \mathcal{H}}(\cdot, \cdot)$ , we can further obtain:
$$
\begin{array}{l} \epsilon_ {T} (h _ {1}) - \epsilon_ {\alpha} (h ^ {*}) \\ \leq (1 - \alpha) \left[ \epsilon_ {S} \left(h _ {1}\right) + \epsilon_ {T} \left(h _ {2}\right) \right. \\ \left. + \left| \epsilon_ {S} \left(h _ {1}, h _ {2}\right) - \epsilon_ {T} \left(h _ {1}, h _ {2}\right) \right| \right] \\ \leq (1 - \alpha) \left[ \epsilon_ {S} \left(h _ {1}\right) + \epsilon_ {T} \left(h _ {2}\right) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} \left(\mathcal {D} _ {S}, \mathcal {D} _ {T}\right) \right] \\ \end{array}
$$
Given the precondition  $\epsilon_{\alpha}(h^{*}) = \alpha \epsilon_{T}(h_{1}) + (1 - \alpha)\epsilon_{S}(h_{2})$ , we consider two UDA settings: (i) domain  $T$  with hypothesis  $h_1$  as the source; (ii) domain  $S$  with hypothesis  $h_2$  as the source. Using Theorem 1 under  $\alpha = 0$ , we can obtain:
$$
\begin{array}{l} \epsilon_ {S} (h _ {1}) \leq \epsilon_ {T} (h _ {1}) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {S}, \mathcal {D} _ {T}) + \lambda_ {1} \\ \epsilon_ {T} (h _ {2}) \leq \epsilon_ {S} (h _ {2}) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {S}, \mathcal {D} _ {T}) + \lambda_ {2} \\ \end{array}
$$
5916
As the common setting of transfer learning, we set  $1 > \alpha \geq \frac{1}{2}$  and then  $\frac{\alpha}{1 - \alpha} \geq 1$ , further obtaining:
$$
\epsilon_ {S} (h _ {1}) \leq \frac {\alpha}{1 - \alpha} \epsilon_ {T} (h _ {1}) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {S}, \mathcal {D} _ {T}) + \lambda_ {1}
$$
Using these conclusions to the previous inequalities, we have:
$$
\begin{array}{l} (1 - \alpha) \Big [ \epsilon_ {S} (h _ {1}) + \epsilon_ {T} (h _ {2}) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {S}, \mathcal {D} _ {T}) \Big ] \\ \leq \alpha \epsilon_ {T} (h _ {1}) + (1 - \alpha) \epsilon_ {S} (h _ {2}) \\ + (1 - \alpha) \left[ \frac {3}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {S}, \mathcal {D} _ {T}) + \lambda_ {1} + \lambda_ {2} \right] \\ \end{array}
$$
Setting  $\lambda^{*} = \lambda_{1} + \lambda_{2}$ , which is the shared error of ideal joint hypothesis and use the precondition,  $\epsilon_{\alpha}(h^{*}) = \alpha \epsilon_{T}(h_{1}) + (1 - \alpha)\epsilon_{S}(h_{2})$ , we have
$$
\begin{array}{l} \epsilon_ {T} (h _ {1}) - \epsilon_ {\alpha} (h ^ {*}) \\ \leq (1 - \alpha) \bigg [ \epsilon_ {S} (h _ {1}) + \epsilon_ {T} (h _ {2}) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {S}, \mathcal {D} _ {T}) \bigg ] \\ \leq \epsilon_ {\alpha} (h ^ {*}) + (1 - \alpha) \left[ \frac {3}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {S}, \mathcal {D} _ {T}) + \lambda^ {*} \right] \\ \end{array}
$$
Finally, we obtain the Lemma 1:
$$
\begin{array}{l} \epsilon_ {T} (h _ {1}) \leq 2 \epsilon_ {\alpha} (h ^ {*}) \\ + (1 - \alpha) \left[ \frac {3}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {S}, \mathcal {D} _ {T}) + \lambda^ {*} \right] \\ \end{array}
$$
5917