SlowGuess/NAACL_2021_OCR huggingface dataset:
Language Technologies, Volume 1 (Long and Short Papers), pages 3081-3091, Minneapolis, Minnesota. Association for Computational Linguistics.
Tao Gui, Qi Zhang, Haoran Huang, Minlong Peng, and Xuanjing Huang. 2017. Part-of-speech tagging for twitter with adversarial neural networks. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2411-2420, Copenhagen, Denmark. Association for Computational Linguistics.
Han Guo, Ramakanth Pasunuru, and Mohit Bansal. 2020. Multi-source domain adaptation for text classification via distancenet-bandits. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 7830-7838. AAAI Press.
Jiang Guo, Darsh Shah, and Regina Barzilay. 2018. Multi-source domain adaptation with mixture of experts. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4694-4703, Brussels, Belgium. Association for Computational Linguistics.
Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don't stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342-8360, Online. Association for Computational Linguistics.
Xiaochuang Han and Jacob Eisenstein. 2019. Unsupervised domain adaptation of contextualized embeddings for sequence labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4238-4248, Hong Kong, China. Association for Computational Linguistics.
Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The  $90\%$  solution. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pages 57-60, New York City, USA. Association for Computational Linguistics.
Ann Irvine, John Morgan, Marine Carpuat, Hal Daume III, and Dragos Munteanu. 2013. Measuring machine translation errors in new domains. Transactions of the Association for Computational Linguistics, 1:429-440.
James J. Jiang. 2007. A literature survey on domain adaptation of statistical classifiers.
Young-Bum Kim, Karl Stratos, and Dongchan Kim. 2017. Adversarial adaptation of synthetic or stale data. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1297-1307, Vancouver, Canada. Association for Computational Linguistics.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Yitong Li, Timothy Baldwin, and Trevor Cohn. 2018. What's in a domain? learning domain-robust text representations using adversarial training. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 474-479, New Orleans, Louisiana. Association for Computational Linguistics.
Le Liu, Yu Hong, Hao Liu, Xing Wang, and Jian-min Yao. 2014. Effective selection of translation model training data. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 569-573, Baltimore, Maryland. Association for Computational Linguistics.
Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2017. Adversarial multi-task learning for text classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1-10, Vancouver, Canada. Association for Computational Linguistics.
Yajuan Lü, Jin Huang, and Qun Liu. 2007. Improving statistical machine translation performance by training data selection and optimization. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 343-350, Prague, Czech Republic. Association for Computational Linguistics.
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579-2605.
Anna Margolis. 2011. A literature review of domain adaptation with unlabeled data.
Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pages 6294-6305.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems
- Volume 2, NIPS'13, page 3111-3119, Red Hook, NY, USA. Curran Associates Inc.
Robert C. Moore and William Lewis. 2010. Intelligent selection of language model training data. In Proceedings of the ACL 2010 Conference Short Papers, pages 220-224, Uppsala, Sweden. Association for Computational Linguistics.
Joakim Nivre, Marie-Catherine de Marneffé, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016. Universal dependencies v1: A multilingual treebank collection. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 1659-1666, Portoorž, Slovenia. European Language Resources Association (ELRA).
Minlong Peng, Qi Zhang, Yu-gang Jiang, and Xuanjing Huang. 2018. Cross-domain sentiment classification with target domain specific information. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2505-2513, Melbourne, Australia. Association for Computational Linguistics.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
Barbara Plank, Anders Johannsen, and Anders Søgaard. 2014. Importance weighting and unsupervised domain adaptation of POS taggers: a negative result. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 968-973, Doha, Qatar. Association for Computational Linguistics.
Barbara Plank and Gertjan van Noord. 2011. *Effective measures of domain similarity for parsing*. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1566–1576, Portland, Oregon, USA. Association for Computational Linguistics.
Natalia Ponomareva and Mike Thelwall. 2012. Biographies or blenders: Which resource is best for cross-domain sentiment analysis? In International Conference on Intelligent Text Processing and Computational Linguistics, pages 488-499. Springer.
Alan Ramponi and Barbara Plank. 2020. Neural unsupervised domain adaptation in NLP - A survey. CoRR, abs/2006.00632.
Sujith Ravi, Kevin Knight, and Radu Soricut. 2008. Automatic prediction of parser accuracy. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 887-896, Honolulu, Hawaii. Association for Computational Linguistics.
Robert Remus. 2012. Domain adaptation using domain similarity- and domain complexity-based instance selection for cross-domain sentiment analysis. In 12th IEEE International Conference on Data Mining Workshops, ICDM Workshops, Brussels, Belgium, December 10, 2012, pages 717-723. IEEE Computer Society.
Jonathan S. Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, and Nir Shavit. 2020. A constructive prediction of the generalization error across scales. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
Peter Rousseeuw. 1987. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math., 20(1):53-65.
Sebastian Ruder, Parsa Ghaffari, and John G. Breslin. 2017. Data selection strategies for multi-domain sentiment analysis. ArXiv, abs/1702.02426.
Sebastian Ruder and Barbara Plank. 2017. Learning to select data for transfer learning with Bayesian optimization. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 372-382, Copenhagen, Denmark. Association for Computational Linguistics.
Victor Sanh, Lysandre Debut, Julien Chaumont, and Thomas Wolf. 2019. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/1910.01108.
Darsh Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, and Preslav Nakov. 2018. Adversarial domain adaptation for duplicate question detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1056-1063, Brussels, Belgium. Association for Computational Linguistics.
Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. 2018. Wasserstein distance guided representation learning for domain adaptation. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 4058-4065. AAAI Press.
Catarina Cruz Silva, Chao-Hong Liu, Alberto Poncelas, and Andy Way. 2018. Extracting in-domain training corpora for neural machine translation using data selection methods. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 224-231, Brussels, Belgium. Association for Computational Linguistics.
Lucia Specia, Kashif Shah, Jose G.C. de Souza, and Trevor Cohn. 2013. QuEst - a translation quality estimation framework. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 79-84, Sofia, Bulgaria. Association for Computational Linguistics.
Jacob Steinhardt and Percy S Liang. 2016. Unsupervised risk estimation using only conditional independence structure. In Advances in Neural Information Processing Systems, volume 29, pages 3657-3665. Curran Associates, Inc.
Baochen Sun, Jiashi Feng, and Kate Saenko. 2017. Correlation alignment for unsupervised domain adaptation. In Gabriela Csurka, editor, Domain Adaptation in Computer Vision Applications, Advances in Computer Vision and Pattern Recognition, pages 153-171. Springer.
Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142-147.
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. CoRR, abs/1412.3474.
Vincent Van Asch and Walter Daelemans. 2010. Using domain similarity for performance estimation. In Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing, pages 31-36, Uppsala, Sweden. Association for Computational Linguistics.
Yogarshi Vyas, Xing Niu, and Marine Carpuat. 2018. Identifying semantic divergences in parallel text without annotations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1503-1515, New Orleans, Louisiana. Association for Computational Linguistics.
Rui Wang, Andrew Finch, Masao Utiyama, and Eichiro Sumita. 2017. Sentence embedding for neural machine translation domain adaptation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 560-566, Vancouver, Canada. Association for Computational Linguistics.
Yong Wang, Longyue Wang, Shuming Shi, Victor O.K. Li, and Zhaopeng Tu. 2019. Go from the general to the particular: Multi-domain translation with domain transformation networks. ArXiv, abs/1911.09912.
Zhenghui Wang, Yanru Qu, Liheng Chen, Jian Shen, Weinan Zhang, Shaodian Zhang, Yimei Gao, Gen Gu, Ken Chen, and Yong Yu. 2018. Label-aware double transfer learning for cross-specialty medical named entity recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1-15, New Orleans, Louisiana. Association for Computational Linguistics.
Bonnie Webber. 2009. Genre distinctions for discourse in the Penn TreeBank. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 674-682, Suntec, Singapore. Association for Computational Linguistics.
Marlies van der Wees, Arianna Bisazza, and Christof Monz. 2017. Dynamic data selection for neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1400-1410, Copenhagen, Denmark. Association for Computational Linguistics.
Guillaume Wisniewski and François Yvon. 2019. How Bad are PoS Tagger in Cross-Corpora Settings? Evaluating Annotation Divergence in the UD Project. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 218-227, Minneapolis, Minnesota. Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R'emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface's transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771.
Dustin Wright and Isabelle Augenstein. 2020. Transformer based multi-source domain adaptation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7963-7974, Online. Association for Computational Linguistics.
Yuan Wu and Yuhong Guo. 2020. Dual adversarial colearning for multi-domain text classification. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 6438-6445. AAAI Press.
Mengzhou Xia, Antonios Anastasopoulos, Ruochen Xu, Yiming Yang, and Graham Neubig. 2020. Predicting performance for natural language processing tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8625-8646, Online. Association for Computational Linguistics.
Brian Xu, Mitra Mohtarami, and James R. Glass. 2019. Adversarial domain adaptation for stance detection. ArXiv, abs/1902.02401.
Keiji Yasuda, Ruiqiang Zhang, Hirofumi Yamamoto, and Eiichiro Sumita. 2008. Method of selecting training data to build a compact and efficient translation model. In Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-II.
Michihiro Yasunaga, Jungo Kasai, and Dragomir Radev. 2018. Robust multilingual part-of-speech tagging via adversarial training. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 976-986, New Orleans, Louisiana. Association for Computational Linguistics.
Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschlager, and Susanne Saminger-Platz. 2017. Central moment discrepancy (CMD) for domain-invariant representation learning. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Jiali Zeng, Jinsong Su, Huating Wen, Yang Liu, Jun Xie, Yongjing Yin, and Jianqiang Zhao. 2018. Multidomain neural machine translation with word-level domain context discrimination. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 447-457, Brussels, Belgium. Association for Computational Linguistics.
Bing Zhao, Matthias Eck, and Stephan Vogel. 2004. Language model adaptation for statistical machine translation via structured query models. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics, pages 411-417, Geneva, Switzerland. COLING.
Joey Tianyi Zhou, Hao Zhang, Di Jin, Hongyuan Zhu, Meng Fang, Rick Siow Mong Goh, and Kenneth Kwok. 2019. Dual adversarial neural transfer for low-resource named entity recognition. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3461-3471, Florence, Italy. Association for Computational Linguistics.
Suyang Zhu, Shoushan Li, and Guodong Zhou. 2019. Adversarial attention modeling for multidimensional emotion regression. In Proceedings of
the 57th Annual Meeting of the Association for Computational Linguistics, pages 471-480, Florence, Italy. Association for Computational Linguistics.
Yftah Ziser and Roi Reichart. 2017. Neural structural correspondence learning for domain adaptation. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 400-410, Vancouver, Canada. Association for Computational Linguistics.
Yftah Ziser and Roi Reichart. 2018. Pivot based language modeling for improved neural domain adaptation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1241-1251, New Orleans, Louisiana. Association for Computational Linguistics.
Yftah Ziser and Roi Reichart. 2019. Task refinement learning for improved accuracy and stability of unsupervised domain adaptation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5895-5906, Florence, Italy. Association for Computational Linguistics.
Bowei Zou, Zengzhuang Xu, Yu Hong, and Guodong Zhou. 2018. Adversarial feature adaptation for cross-lingual relation classification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 437-448, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
A Domain Divergence Measures
This section provides the necessary background on different kinds of divergence measures used in the literature. They can be either information-theoretic – which measure the distance between two probability distributions, geometric - which measure the distance between two vectors in a space, or higher-order which capture similarity in a projected space and consider higher order moments of random variables.
A.1 Information-Theoretic Measures
Let  $P$  and  $Q$  be two probability distributions. These information-theoretic measures are used to capture differences between  $P$  and  $Q$ .
Kullback-Leibler Divergence (KL-Div)  $Q$  is called the reference probability distribution. More precisely, KL is defined if only for all  $Q(x)$  st  $Q(x) = 0$ ,  $P(x)$  is also 0; and undefined if  $\exists x$ ,  $Q(x) = 0$  and  $P(x) > 0$ .
$$
D _ {K L} (P | | Q) = \sum_ {x} P (x) \log \left(\frac {P (x)}{Q (x)}\right) \tag {1}
$$
Renyi Divergence (Renyi-Div) Renyi Divergence is a generalisation of the KL Divergence and is also called  $\alpha$ -power divergence:
$$
D _ {\alpha} (P | | Q) = \frac {1}{\alpha - 1} \log \left(\sum_ {x} \frac {P (x) ^ {\alpha}}{Q (x) ^ {\alpha - 1}}\right) \tag {2}
$$
Here  $\alpha \geq 0$  and  $\alpha \neq 1$ . Renyi divergence is equivalent to KL divergence in the limit where  $\alpha \to 1$ .
Jensen Shannon Divergence (JS-Div) Jensen Shannon divergence (JS-divergence) is a symmetric version of KL-Divergence. It has many advantages. The square root of the Jensen Shannon Divergence is a metric and it can be used for non-continuous probabilities:
$$
\begin{array}{c} D _ {J S} (P | | Q) = \frac {1}{2} D _ {K L} (P | | M) + \frac {1}{2} D _ {K L} (Q | | M) \\ M = \frac {1}{2} (P + Q) \end{array} \tag {3}
$$
Entropy-Related - (CE) Let,  $H_{T}$ ,  $H_{S}$  assign entropy to a sentence using a language model trained on the target and source domain, respectively. If  $s$  is a text segment from the source domain, then the difference in entropy, as shown below, gives the similarity of a source domain segment to the target domain. Some works just use  $H_{T}$ , ignoring  $H_{S}$ . MT related work (Moore and Lewis, 2010), consider only the source language. Axelrod et al. (2011) extend to consider both the source and the target language of machine translation, which performs better for data selection. We present these variations in the formulae below and attribute the same name CE to both these variations in the literature review.
$$
D _ {C E} = H _ {T} (s) - H _ {S} (s) \tag {4}
$$
$$
\begin{array}{l} D _ {C E} = \left[ H _ {T} ^ {s r c - l a n g} (s) - H _ {S} ^ {s r c - l a n g} (s) \right] \tag {5} \\ + [ H _ {T} ^ {t r g - l a n g} (s) - H _ {S} ^ {t r g - l a n g} (s) ] \\ \end{array}
$$
A.2 Geometric Measures
Let  $\vec{p}$  and  $\vec{q}$  be two vectors in  $\mathbb{R}^n$ . Domain adaptation works use geometric metrics for continuous representations like word vectors.
Cosine Similarity (Cos): It calculates the cosine of the angle between vectors. To measure the cosine distance between two points, we use  $1 - \cos$
$$
\cos (\vec {p}, \vec {q}) = \frac {\vec {p} . \vec {q}}{\| p \| \cdot \| q \|} \tag {6}
$$
$l_{p}$ -norm (Norm): Euclidean distance or  $l_{2}$  distance measures the straight line distance between vectors and Manhattan or  $l_{1}$  measures the sum of the difference between their projections.
$$
d _ {2} (p, q) = \sqrt {\sum_ {i = 1} ^ {n} \left(p _ {i} - q _ {i}\right) ^ {2}} \tag {7}
$$
$$
d _ {1} (p, q) = \sum_ {i = 1} ^ {n} \left| p _ {i} - q _ {i} \right| \tag {8}
$$
A.3 Higher-Order Measures
$\mathcal{H}$ -divergence and Proxy-A-Distance (PAD): Ben-David et al. (2010) state that the error of a machine learning classifier in a target domain is bound by its performance on the source domain and the  $\mathcal{H}$ -divergence between the source and the
$^{4}$ KL divergence is asymmetric and cannot be considered a metric
target distributions.  $\mathcal{H}$ -divergence is expensive to calculate. An approximation of  $\mathcal{H}$  is called Proxy-A-Distance. This definition has been adopted from (Elsahar and Galle, 2019). Here  $G: \mathcal{X} \to [0,1]$  is a supervised machine learning model that classifies examples to the source and target domains,  $D_s, D_t$ .  $|D|$  is the size of the training data and  $\mathbb{1}$  is an indicator function:
$$
P A D = 1 - 2 \epsilon \left(G _ {d}\right) \tag {9}
$$
$$
\epsilon \left(G _ {d}\right) = 1 - \frac {1}{| D |} \sum_ {x _ {i} \in D _ {s}, D _ {t}} \left| G \left(x _ {i}\right) - \mathbb {1} \left(x _ {i} \in D _ {s}\right) \right| \tag {10}
$$