## 4 Experiments
### Comparing Explicit and Implicit Bias  
**Results**: In the first group of experiments, we investigated the performance of our method of distinguishing explicit and implicit bias. We calculated the explicit and implicit bias separately with our method on each corpus for each concept pair. The significance of the difference between explicit and implicit bias pairs was tested with a permutation test using no difference as the null hypothesis. The results are shown in Figure 4, where the explicit and implicit bias of each concept pair on each corpus is compared using sentiment scores calculated with Equations (4) and (5). Positive and negative values indicate the bias preferring the right and left concept of this column, respectively (e.g., in the first column, Insect is the left concept, Flower is the right concept). The red star indicates that the difference between this pair of explicit/implicit biases is significant (\(p\) = 0.0001). \"Afri-A\" = \"Afri-American\", \"Euro-A\" = \"Euro-American\". For example, in English Wiki, people prefer to Afri-American in explicit bias and prefer to Euro-American in implicit bias.  
**Discussion**: In Figure 4, we have four main observations:  
1. **In general:** implicit bias can be significantly different from explicit bias, which agrees with the IAT observation. The difference is sensitive to the sociality of concepts, scenarios, and cultures, which agrees with the psychological assumptions of implicit cognition: the difference between explicit and implicit bias can be explained by the social desire [5].
2. **On concepts**: in most cases, the difference between implicit and explicit bias is more significant on social and controversial concept pairs (i.e., \"Afri-American vs. Euro-American\" and \"China vs. America\") than the difference on unsocial concept pairs (i.e., \"Insect vs. Flower\" and \"Weapon vs. Instrument\"). This observation agrees with the psychological assumption that for social and controversial concepts, people are more motivated to adjust their explicit bias to adapt to social desire, which leads to greater differences between implicit and explicit bias.
3. **On scenario:** for controversial concept pairs, implicit and explicit biases are closer to each other in social media than in Wikipedia. From social desire, users are more strongly required to conform to mainstream social values in Wikipedia than in social media.  
Figure 4: Compare explicit and implicit bias of each concept pair on each corpus.  
* **On language (culture):** though on both two English corpora, explicit and implicit bias on _\"Afri-American_ vs. _Euro-American_\" have opposite polarity (consistent with IAT reports), but this does not happen in Chinese corpora. Similarly, explicit and implicit bias on _\"China_ vs. _America_\" are opposite on Chinese corpora instead of on English corpora. From social desire, the importance of consistency with mainstream social values can vary across countries or cultures for a concept pair.  
From the view of the relationship with previous work, our experimental results line with psychological observations in general. Our innovation lies in the automation of explicit and implicit bias analysis by correlating concepts from psychological measurements with elements of automated sentiment analysis. Furthermore, we addressed the shortcomings of the previous word vector-based automated analysis method, which was unable to distinguish between explicit and implicit biases. The limitation of our approach is that further explanation of the difference in specific concept pairs still needs help from psychology scientists in future work. This explanation might achieve new findings considering the absence of observations on a massive population in the current psychological study on implicit bias.