SlowGuess/ACL_2020_OCR huggingface dataset:
Non-discrimination Learning Theorem 2 shows that, we can obtain the unbiased loss of the non-discrimination distribution  $\widehat{\mathcal{D}}$  by adding proper instance weights to the samples from the discrimination distribution  $\mathcal{D}$ . In other words, non-discrimination models can be trained with the instance weights  $w = \frac{Q(y)}{P(y|z)}$ . As the discrimination distribution is directly observable, estimating  $P(y|z)$  is not hard. In practice, we can train classifiers and use cross predictions to estimate  $P(y|z)$  in the original datasets. Since  $Q(y)$  is only a real number indicating the prior probability of  $Y \in [0,1]$  on distribution  $\widehat{\mathcal{D}}$ , we do not specifically make an assumption on it. Intuitively, setting  $Q(Y) = P(Y)$  can be a good choice. Considering an non-discrimination dataset where  $P(Y|Z) = P(Y)$ , the calculated weights  $\frac{Q(y)}{P(y|z)}$  should be the same for all samples when we set  $Q(Y) = P(Y)$ , and thus have little impacts on trained models.
We present the step-by-step procedure for nondiscrimination learning in Algorithm 1. Note that the required data is only the biased dataset, and a pre-defined set of demographic identity-terms, with which we can extract  $\{x,y,z\}$  for all the samples.
In this section, we present the experimental results for non-discrimination learning. We demonstrate that our method can effectively mitigate the impacts of unintended discriminatory biases in datasets.
5.1 Dataset Usage
We evaluate our methods on three datasets, including the Sexist Tweets dataset, the Toxicity Comments dataset, and the Jigsaw Toxicity dataset.
Sexist Tweets We use the Sexist Tweets dataset released by Waseem and Hovy (2016); Waseem (2016), which is for abusive language detection
Dataset	Size	Positives	avg. Length
Sexist Tweets	12,097	24.7%	14.7
Toxicity Comments	159,686	9.6%	68.2
Jigsaw Toxicity	1,804,874	8.0%	51.3
Table 2: Statistics of the three datasets for evaluation.
task<sup>5</sup>. The dataset consists of tweets annotated by experts as "exist" or "normal." We process the dataset as to how Park et al. (2018) does. It is reported that the dataset has an unintended gender bias so that models trained in this dataset may consider "You are a good woman." as "exist." We randomly split the dataset in a ratio of  $8:1:1$  for training-validation-testing and use this dataset to evaluate our method's effectiveness on mitigating gender discrimination.
Toxicity Comments Another choice is the Toxicity Comments dataset released by Dixon et al. (2018), in which texts are extracted from Wikipedia Talk Pages and labeled by human raters as either toxic or non-toxic. It is found that in this dataset, some demographic identity-terms (e.g., "gay", "black") appear disproportionately among labels. As a result, models trained in this dataset can be discriminatory among groups. We adopt the split released by Dixon et al. (2018) and use this dataset to evaluate our method's effectiveness on mitigating discrimination towards minority groups.
Jigsaw Toxicity We also tested a recently released large-scale dataset Jigsaw Toxicity from Kaggle<sup>6</sup>, in which it is found that some frequently attacked identities are associated with toxicity. Sentences in the dataset are extracted from the Civil Comment platform and annotated with toxicity and identities mentioned in every sentence. We randomly split the dataset into  $80\%$  for training,  $10\%$  for validation and testing respectively. The dataset is used to evaluate our method's effectiveness on large-scale datasets.
The statistic of the three datasets is shown as in Table 2.
5.2 Evaluation Scheme
Apart from the original testing set of each dataset, we use the Identity Phrase Templates Test Sets (IPTTS) described in Section 3.3 to evaluate
Unfortunately, due to the rules of Twitter, some TweetIDs got expired, so we cannot collect the exact same dataset as Park et al. (2018).
$^{6}$ https://www.kaggle.com/c/jigsaw-unintended-bias-intoxicity-classification
the models as mentioned in Section 3.3. For experiments with the Sexist Tweets dataset, we generate IPTTS following Park et al. (2018). For experiments with Toxicity Comments datasets and Jigsaw Toxicity, we use the IPTTS released by Dixon et al. (2018). Details about the IPTTS generation are introduced in Appendix C.
For metrics, we use FPED and FNED in IPTTS to evaluate how discriminatory the models are, and lower scores indicate better equality. However, as mentioned in Section 3.3, these two metrics are not enough since models can achieve low FPED and FNED by making trivial predictions in IPTTS. So we use AUC in both the original testing set and IPTTS to reflect the trade-off between the debiasing effect and the accuracy of models. We also report the significance test results under confidence levels of 0.05 for Sexist Tweets dataset and Jigsaw Toxicity dataset<sup>7</sup>.
For baselines, we compare with the gender-swapping method proposed by Park et al. (2018) for the Sexist Tweets dataset, as there are only two demographics groups (male and female) provided by the dataset, it's practical for swapping. For the other two datasets, there are 50 demographics groups, and we compare them with data supplementation proposed by Dixon et al. (2018).
5.3 Experiment Setup
To generate the weights, we use Random Forest Classifiers to estimate  $P(y|z)$  following Algorithm 1. We simply set  $Q(Y) = P(Y)$  to partial out the influence of the prior probability of  $Y$ . The weights are used as the sample weights to the loss functions during training and validation.
For experiments with the Sexist Tweets dataset, we extract the gender identity words (released by Zhao et al. (2018)) in every sentence and used them as  $Z$ . For experiments with Toxicity Comments dataset, we take the demographic group identity words (released by Dixon et al. (2018)) contained in every sentence concatenated with the lengths of sentences as  $Z$ , just the same as how Dixon et al. (2018) chose the additional sentence for data supplement. For experiments with the Jigsaw Toxicity dataset, the provided identity attributes of every sentence and lengths of sentences are used as  $Z$ .
For experiments with the Toxicity Comments dataset, to compare with the results released by
Model	Orig. AUC	IPTTS AUC	FPED	FNED
Baseline	0.920	0.673	0.147	0.204
Swap	0.911†	0.651	0.047†	0.050†
Weight	0.897†‡	0.686‡	0.057†	0.086†‡
Baseline+	0.900	0.624	0.049	0.099
Swap+	0.890†	0.611	0.008†	0.013†
Weight+	0.881†‡	0.647†‡	0.007†	0.024†‡
$\dagger p <   0.05$  compared with Baseline
$\ddagger p < 0.05$  compared with Swap
Table 3: Experimental results with Sexist Tweets dataset. “+” refers to models using debiased word embeddings.
Model	Orig. AUC	IPTTS AUC	FPED	FNED
Baseline	0.960	0.952	7.413	3.673
Supplement	0.959	0.960	5.294	3.073
Weight	0.956	0.961	4.798	2.491
The results of Baseline and Supplement are taken from Dixon et al. (2018)
Table 4: Experimental results with Toxicity Comments dataset.
Model	Orig. AUC	IPTTS AUC	FPED	FNED
Baseline	0.928	0.993	3.088	3.317
Supplement	0.928	0.999†	0.180†	3.111
Weight	0.922†‡	0.999†	0.085†	2.538
$\dagger p <   0.05$  compared with Baseline
$\ddagger p <   0.05$  compared with Supplement
Table 5: Experimental results with Jigsaw Toxicity dataset.
Dixon et al. (2018), we use their released codes, where a three-layer Convolutional Neural Network (CNN) model is used. For experiments with Sexist Tweets dataset and Jigsaw Toxicity dataset, as our method is model-agnostic, we simply implement a one-layer LSTM with a dimensionality of 128 using Keras and Tensorflow backend. $^{8}$
For all models, pre-trained GloVe word embeddings (Pennington et al., 2014) are used. We also report results when using gender-debiased pretrained embeddings (Bolukbasi et al., 2016) for experiments with Sexist Tweets. All the reported results are the average numbers of ten runs with different random initializations.
In this section, we present and discuss the experimental results. As expected, training with calculated weights can effectively mitigate the impacts of the unintended bias in the datasets.
Sexist Tweets Tabel 3 reports the results on Socialist Tweets dataset. Baseline refers to vanilla mod
<sup>7</sup>As we use some results from Dixon et al. (2018) directly, we don't report the significance test results for Toxicity Comments dataset.
<sup>8</sup>Codes are publicly available at https://github.com/ghzhang233/Non-Discrimination-Learning-for-Text-Classification.
els. Swap refers to models trained and validated with 2723 additional gender-swapped samples to balance the identity-terms across labels (Park et al., 2018). Weight refers to models trained and validated with calculated weights. “+” refers to models using debiased word embeddings.
Regarding the results with the GloVe word embeddings, we can find that Weight performs significantly better than Baseline under FPED and FNED, which demonstrate that our method can effectively mitigate the discrimination of models. Swap outperforms Weight in FPED and FNED, but our method achieves significantly higher IPTTS AUC. We notice that Swap even performs worse in terms of IPTTS AUC than Baseline (although the difference is not significant at 0.05), which implies that cost for the debiasing effect of Swap is the loss of models' accuracy, and this can be ascribed to the gender-swapping assumptions as mentioned in Section 3.4. We also notice that both Weight and Swap have lower Orig. AUC than Baseline and this can be ascribed to that the unintended bias pattern is mitigated.
Regarding the results with the debiased word embeddings, the conclusions remain largely unchanged, while Weight get a significant improvement over Baseline in terms of IPTTS AUC. Besides, compared with GloVe embeddings, we can find that debiased embeddings can effectively improve FPED and FNED, but Orig. AUC and IPTTS AUC also drop.
Toxicity Comments Table 4 reports the results on Toxicity Comments dataset. Baseline refers to vanilla models. Supplement refers to models trained and validated with 4620 additional samples to balance the identity-terms across labels (Dixon et al., 2018). Weight refers to models trained and validated with calculated instance weights.
From the table, we can find that Weight outperforms Baseline in terms of IPTTS AUC, FPED, and FNED, and also gives slightly better debiasing performance compared with Supplement, which demonstrate that the calculated weights can effectively make models more non-discriminatory. Meanwhile, Weight performs similarly in Orig. AUC to all the other methods, indicating that our method does not hurt models' generalization ability very much.
In general, the results demonstrate that our method can provide a better debiasing effect without additional data, and avoiding the high cost of
Figure 1: Comparison for the evaluation results of Baseline and Weight for sentences containing a selection of specific identities in IPTTS in Jigsaw Toxicity dataset, in which  $\Delta \mathrm{FPR}_z = \mathrm{FPR}_z - \mathrm{FPR}_{\mathrm{overall}}$  and  $\Delta \mathrm{FNR}_z = \mathrm{FNR}_z - \mathrm{FNR}_{\mathrm{overall}}$ . Values closer to 0 indicate better equality. Best viewed in color.
extra data collection and annotation makes it more practical for adoptions.
Jigsaw Toxicity Table 5 reports the results on Jigsaw Toxicity dataset. Baseline refers to vanilla models. Supplement refers to models trained and validated with 15249 additional samples extracted from Toxicity Comments to balance the identity-terms across labels. Weight refers to models trained with calculated weights.
Similar to results on Toxicity Comments, we find that both Weight and Supplement perform significantly better than Baseline in terms of IPTTS AUC and FPED, and the results of Weight and Supplement are comparable. On the other hand, we notice that Weight and Supplement improve FNED slightly, while the differences are not statistically significant at confidence level 0.05.
To gain better knowledge about the debiasing effects, we further visualize the evaluation results on the Jigsaw Toxic dataset for sentences containing some specific identity-terms in IPTTS in Figure 1, where  $\Delta \mathrm{FPR}_z$  and  $\Delta \mathrm{FNR}_z$  are presented. Based on the definition of FPED and FNED, values closer to 0 indicate better equality. We can find that Baseline trained in the original biased dataset can discriminate against some demographic groups.
For example, sentences containing identity words like "gay", "homosexual" and "lesbian" are more likely to be falsely judged as "toxic" as indicated by  $\Delta$ FPR, while ones with words like "straight" are more likely to be falsely judged as "not toxic" as indicated by  $\Delta$ FNR. We can also notice that Weight performs more consistently among most identities in both FPR and FNR. For instance,  $\Delta$ FPR of the debiased model in samples with "gay", "homosexual" and "lesbian" significantly come closer to 0, while  $|\Delta$ FNR| also drop for "old" and "straight".
We also note that  $\mathrm{FPR}_{\mathrm{overall}}$  and  $\mathrm{FPR}_{\mathrm{overall}}$  of Weight are significantly better than the results of Baseline, i.e.,  $\mathrm{FPR}_{\mathrm{overall}}$  results are 0.001 and 0.068 for Weight and Baseline respectively, and  $\mathrm{FNR}_{\mathrm{overall}}$  results are 0.061 and 0.068 for Weight and Baseline respectively, representing that Weight is both more accurate and more non-discriminatory on the IPTTS set.
In this paper, we focus on the unintended discrimination bias in existing text classification datasets. We formalize the problem as a kind of selection bias from the non-discrimination distribution to the discrimination distribution and propose a debiasing training framework that does not require any extra resources or annotations. Experiments show that our method can effectively alleviate discrimination. It's worth mentioning that our method is general enough to be applied to other tasks, as the key idea is to obtain the loss on the non-discrimination distribution, and we leave this to future works.
Conghui Zhu and Tiejun Zhao are supported by National Key R&D Program of China (Project No. 2017YFB1002102).
Peter C Austin and Elizabeth A Stuart. 2015. Moving towards best practice when using inverse probability of treatment weighting (iptw) using the propensity score to estimate causal treatment effects in observational studies. Statistics in medicine, 34(28):3661-3679.
Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. 2007. Analysis of representations for domain adaptation. In Advances in neural information processing systems, pages 137-144.
Alex Beutel, Ed H Chi, Jilin Chen, and Zhe Zhao. 2017. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligram, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, pages 4349-4357.
Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Nuanced metrics for measuring unintended bias with real data for text classification. In *Companion Proceedings of The 2019 World Wide Web Conference*, pages 491-500. ACM.
Toon Calders and Sicco Verwer. 2010. Three naive bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery, 21(2):277-292.
Aylin Caliskan-Islam, Joanna J. Bryson, and Arvind Narayanan. 2016. Semantics derived automatically from language corpora necessarily contain human biases. Science, 356(6334):183-186.
Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. *Big data*, 5(2):153-163.
Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 67-73. ACM.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214-226. ACM.
Wei Fan, Ian Davidson, Bianca Zadrozny, and Philip S Yu. 2005. An improved categorization of classifier's sensitivity on sample selection bias. In Fifth IEEE International Conference on Data Mining (ICDM'05), pages 4-pp. IEEE.
Sorelle A Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2016. On the (im) possibility of fairness. arXiv preprint arXiv:1609.07236.
Moritz Hardt, Eric Price, Nati Srebro, et al. 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pages 3315-3323.
James J Heckman. 1979. Sample selection bias as a specification error. *Econometrica: Journal of the econometric society*, pages 153-161.
Jing Jiang and ChengXiang Zhai. 2007. Instance weighting for domain adaptation in nlp. In Proceedings of the 45th annual meeting of the association of computational linguistics, pages 264-271.
Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased learning-to-rank with biased feedback. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pages 781-789. ACM.
Svetlana Kiritchenko and Saif Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 43-53.
Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2799-2804.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543.
Paul R Rosenbaum and Donald B Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55.
Donald B Rubin. 1976. Inference and missing data. Biometrika, 63(3):581-592.
Matthias Schonlau, Arthur Van Soest, Arie Kapteyn, and Mick Couper. 2009. Selection bias in web surveys and the use of propensity scores. Sociological Methods & Research, 37(3):291-318.
Hidetoshi Shimodaira. 2000. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference, 90(2):227-244.
Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. In Proceedings of the 57th Annual Meeting of the Association of Computational Linguistics, pages 1630-1640.
Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Marc Najork. 2018. Position bias estimation for unbiased learning to rank in personal search. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 610-618. ACM.
Zeerak Waseem. 2016. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the first workshop on NLP and computational social science, pages 138-142.
Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In Proceedings of the NAACL student research workshop, pages 88-93.
Bianca Zadrozny. 2004. Learning and evaluating classifiers under sample selection bias. In Proceedings of the twenty-first international conference on Machine learning, page 114. ACM.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335-340. ACM.
Guanhua Zhang, Bing Bai, Jian Liang, Kun Bai, Shiyu Chang, Mo Yu, Conghui Zhu, and Tiejun Zhao. 2019. Selection bias explorations and debias methods for natural language sentence matching datasets. In Proceedings of the 57th Annual Meeting of the Association of Computational Linguistics, pages 4418-4429.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979-2989.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, volume 2.
A Proof for the Criterion Consistency Theorem
Proof. Here we present the proof for Theorem 1. For the Statistical Parity criterion,
$$
\begin{array}{l} \Pr (\widehat {Y} | Z) = \sum_ {y \in [ 0, 1 ]} \Pr (\widehat {Y}, Y = y | Z) \\ = \sum_ {y \in [ 0, 1 ]} \Pr (\widehat {Y} | Y = y, Z) \Pr (Y = y | Z) \\ = \sum_ {y \in [ 0, 1 ]} \Pr (\widehat {Y} | Y = y) \Pr (Y = y) \\ = \Pr (\widehat {Y}). \\ \end{array}
$$
For the Predictive Parity criterion,
$$
\begin{array}{l} \Pr (Y | \widehat {Y} = 1, Z) = \frac {\Pr (\widehat {Y} = 1 | Y , Z) \Pr (Y | Z)}{\Pr (\widehat {Y} = 1 | Z)} \\ = \frac {\operatorname * {P r} (\widehat {Y} = 1 | Y) \operatorname * {P r} (Y)}{\operatorname * {P r} (\widehat {Y} = 1)} \\ = \Pr (Y | \widehat {Y} = 1). \\ \end{array}
$$
B Discussion about Assumption 4
We show that even if the assumption does not hold, we can still make models fit  $Q(Y|X)$  with calculated weights when  $Z$  is contained in  $X$ , which is the common setting in practical.
We firstly present the equation of the weights  $w$  without the assumption  $P(Z) = Q(Z)$ .
$$
\begin{array}{l l} w = \frac {Q (y)}{P (y | z)} & \Bigg \downarrow 1 \\ = \frac {Q (y)}{Q (y | z , S = 1)} & \Bigg \downarrow b a y e s \\ = \frac {Q (y)}{Q (S = 1 | z , y) Q (y | z) / Q (S = 1 | z)} & \Bigg \downarrow 2, 3 \\ = \frac {Q (S = 1 | z)}{Q (S = 1 | x , z , y)} & \Bigg \downarrow b a y e s \\ = \frac {Q (S = 1)}{Q (S = 1 | x , z , y)} \cdot \frac {Q (z | S = 1)}{Q (Z)} & \Bigg \downarrow 1, b a y e s \\ = \frac {Q (x , z , y)}{P (x , z , y)} \cdot \frac {P (z)}{Q (z)} & \Bigg \downarrow 1, b a y e s \end{array}
$$
After applying these weights to every sample in the dataset, we can get a new distribution defined as below,
$$
P ^ {*} (x, y, z _ {x}) = \frac {w _ {x , y , z _ {x}} \cdot P (x , y , z _ {x})}{\int w _ {x ^ {\prime} , y ^ {\prime} , z _ {x} ^ {\prime}} \cdot d P \left(x ^ {\prime} , y ^ {\prime} , z _ {x} ^ {\prime}\right)}.
$$
in which we use  $P^{*}(\cdot)$  to represent the probability in the obtained distribution. As  $Z$  is contained in  $X$ , we use  $Z_{X}$  to represent the specific  $Z$  contained in every  $X$ .
$$
\begin{array}{l} P ^ {*} (y | x) = \frac {P ^ {*} (x , z _ {x} , y)}{\sum_ {y ^ {\prime}} P ^ {*} (x , z _ {x} , y ^ {\prime})} \\ = \frac {P (x , z _ {x} , y) \cdot \frac {Q (x , z _ {x} , y)}{P (x , z _ {x} , y)} \cdot \frac {P (z _ {x})}{Q (z _ {x})}}{\sum_ {y ^ {\prime}} P (x , z _ {x} , y ^ {\prime}) \cdot \frac {Q (x , z _ {x} , y ^ {\prime})}{P (x , z _ {x} , y ^ {\prime})} \cdot \frac {P (z _ {x})}{Q (z _ {x})}} \\ = \frac {\frac {P (z _ {x})}{Q (z _ {x})} \cdot Q (x , z _ {x} , y)}{\frac {P (z _ {x})}{Q (z _ {x})} \cdot \sum_ {y ^ {\prime}} Q (x , z _ {x} , y ^ {\prime})} \\ = Q (y | x, z _ {x}) \\ = Q (y | x), \\ \end{array}
$$
$$
\begin{array}{l} P ^ {*} (x) = \sum_ {y} P ^ {*} (x, z _ {x}, y) \\ = \sum_ {y} \frac {w _ {x , z _ {x} , y} \cdot P (x , z _ {x} , y)}{\int w _ {x ^ {\prime} , z _ {x ^ {\prime}} , y ^ {\prime}} \cdot d P \left(x ^ {\prime} , z _ {x ^ {\prime}} , y ^ {\prime}\right)} \\ \propto \sum_ {y} \frac {Q (x , z _ {x} , y)}{P (x , z _ {x} , y)} \cdot \frac {P (z _ {x})}{Q (z _ {x})} \cdot P (x, z _ {x}, y) \\ = \frac {P (z _ {x})}{Q (z _ {x})} \cdot Q (x, z _ {x}) \\ = \frac {P (z _ {x})}{Q (z _ {x})} \cdot Q (x). \\ \end{array}
$$