Surveying the Dead Minds: Historical-Psychological Text Analysis with Contextualized Construct Representation ( CCR ) for Classical C hinese (2024)

Abstract: In this work, we develop a pipeline for historical-psychological text analysis in classical Chinese. Humans have produced texts in various languages for thousands of years; however, most of the computational literature is focused on contemporary languages and corpora. The emerging field of historical psychology relies on computational techniques to extract aspects of psychology from historical corpora using new methods developed in natural language processing (NLP). The present pipeline, called Contextualized Construct Representations (CCR), combines expert knowledge in psychometrics (i.e., psychological surveys) with text representations generated via Transformer-based language models to measure psychological constructs such as traditionalism, norm strength, and collectivism in classical Chinese corpora. Considering the scarcity of available data, we propose an indirect supervised contrastive learning approach and build the first Chinese historical psychology corpus (C-HI-PSY) to fine-tune pre-trained models. We evaluate the pipeline to demonstrate its superior performance compared with other approaches. The CCR method outperforms word-embedding-based approaches across all of our tasks and exceeds prompting with GPT-4 in most tasks. Finally, we benchmark the pipeline against objective, external data to further verify its validity.

## introduction humans have been producing written language for thousands of years. historical populations have expressed their norms, values, stories, songs, and more in these texts. such historical corpora represent a rich yet underexplored source of psychological data that contains the thoughts, feelings, and actions of people who lived in the past (jackson et al. @xcite . the emerging field of "historical psychology" has been developed to understand how different aspects of psychology vary over historical time and how the origins of our contemporary psychology are rooted in historical processes @xcite @xcite ). since we cannot access "dead minds" directly but can access their textual remains, natural language processing (nlp) is the primary method to extract aspects of psychology from historical corpora. previous works, however, are often monolingual and in english @xcite . in addition, much of the literature at the intersection of psychology and nlp has relied on bag-ofwords or word embedding models, focusing on non-contextual word meanings rather than a holistic approach to language modeling. recently, more research attention in the nlp community has been directed to historical and ancient languages @xcite , including but not limited to english @xcite , latin @xcite , ancient @xcite , and ancient hebrew @xcite . while all these languages have historical significance, classical chinese is particularly important in the quantitative study of history. china has a long history spanning thousands of years, largely recorded in classical chinese. the language served as a medium for expressing and disseminating influential philosophical and religious ideas. confucianism, daoism, and later buddhism (through translations from sanskrit) all found expression in classical chinese, profoundly shaping chinese thought, ethics, governance, and norms. as more resources become readily available for classical chinese, scholars of ancient china can test more specific hypotheses using computational methods @xcite @xcite . due to its historical significance and geographical coverage, classical chinese represents one of the most important languages in historical psychology @xcite . prior work in social science has often relied on bag-of-words approaches @xcite or bottom-up techniques such as topic modeling @xcite . in the nlp community, various transformer-based models for classical chinese have been developed @xcite @xcite , primarily for tasks like punctuation pre-diction @xcite , poem generation @xcite , and translation @xcite . however, they have not been applied to theorydriven psychological text analysis for extracting psychological constructs (e.g., moral values, norms, cultural orientation, mental health, religiosity, emotions, and thinking styles) from historical data. transformer-based language models @xcite are crucial for psychological text analysis because psychological constructs are often complex, and sentence-level semantics (and above) will more effectively capture psychological meanings than isolated words @xcite or non-contextual word embedding models @xcite . here, we create a pipeline called contextualized construct representation (ccr) for historicalpsychological text analysis in classical chinese. although similar method has recently been developed for contemporary psychological text analysis @xcite , it can be adapted for historical nlp. taking advantage of contextual language models, ccr does not require selecting a priori lists of words to represent a psychological construct (e.g., the popular linguistic inquiry and word @xcite ; instead, making use of thousands of existing questionnaires (which typically include face-valid declarative sentences with which participants agree or disagree) that have been developed and validated in psychology over the last century. the pipeline of ccr for classical chinese proceeds in five steps: (1) selecting a questionnaire for the psychological construct of interest; (2) converting the questionnaire, usually in english, into classical chinese; (3) representing questionnaire items as embeddings using a contextual language model; (4) generating the embedding of the target text using a contextual language model; (5) computing the cosine similarity between the item and text embeddings. this straightforward pipeline is particularly useful for social science, wherein researchers are interested in interpretability and hypothesis testing. there are two main challenges of using the ccr pipeline in analyzing chinese historical texts: (1) popular self-report questionnaires, widely accepted by psychologists, are often in english, making it difficult to align them with classical chinese texts; (2) there is a lack of psychology-specific transformer-based models for classical chinese, making it difficult to obtain high-quality representations of chinese historical texts. to address the first challenge, we propose a pipeline that uses a multilingual quotation recommendation model @xcite to convert contemporary english questionnaires into contextually meaningful classical chinese sentences ( §3.1). to tackle the second challenge, we build the first chinese historical psychology corpus (c-hi-psy) and introduce an approach based on indirect supervision @xcite @xcite and contrastive learning @xcite @xcite to fine-tune pre-trained models ( §3.2). ## evaluation and results we set up different levels of tasks to evaluate the ccr method (using sentence transformer models), and compare it with the ddr method (using word embedding models) and the prompting method (using generative llms). the results are shown in ## benchmarking: traditionalism, authority, and attitude toward reform to address the lack of benchmark datasets related to psychological measurement in classical chinese, we further validate the effectiveness of the ccr method using externally annotated data. officials' attitudes toward reforms moral values and political orientations are closely intertwined @xcite . for example, the attitude of individuals toward reforms, policy changes, and new legislation often reflects traditionalism, conservatism, and respect for authority @xcite . those with stronger traditionalist views are more likely to identify with the existing social order and resist changes to the status quo @xcite . throughout chinese history, there have been numerous instances of significant reforms, one of the most notable of which being the wang anshi's new policies @xcite in the 11th century, which faced mixed reactions from officials. we draw upon a dataset manually compiled by @xcite , who annotated the attitudes of 137 major officials toward the reform. unlike data easily influenced by annotators' subjectivity @xcite , officials' attitudes toward reforms are objective and less controversial, making them highly suitable for benchmarking our pipeline. ## discussion and conclusion historical-psychological text analysis is a new line of research focused on extracting different aspects of psychology from historical corpora using stateof-the-art computational methods @xcite . here, we create a new pipeline, ccr, as a helpful tool for historical-psychological text analysis. evaluating our model against word embedding models (e.g., ddr) and more recent llms (e.g., gpt-4), we demonstrate that ccr performs better than these alternatives while keeping its high level of interpretability and flexibility. classical chinese is of great historical significance, and the proposed approach can be particularly helpful in testing new insights about the "dead minds" who lived centuries or even millennia prior. we hope our tool motivates future work at the intersections of psychology, quantitative history, and nlp. importantly, benchmarking historical-psychological tools, especially in ancient languages, is difficult because obtaining ground truth is challenging and dependent upon the quality of historical data. that said, we validate ccr against a historically verified knowledge base about attitudes toward reform and traditionalism.