The Multilingual Corpus of Survey Questionnaires Query Interface (2021)

Abstract: The dawn of the digital age led to increasing demands for digital research resources, which shall be quickly processed and handled by computers. Due to the amount of data created by this digitization process, the design of tools that enable the analysis and management of data and metadata has become a relevant topic. In this context, the Multilingual Corpus of Survey Questionnaires (MCSQ) contributes to the creation and distribution of data for the Social Sciences and Humanities (SSH) following FAIR (Findable, Accessible, Interoperable and Reusable) principles, and provides functionalities for end-users that are not acquainted with programming through an easy-to-use interface. By simply applying the desired filters in the graphic interface, users can build linguistic resources for the survey research and translation areas, such as translation memories, thus facilitating data access and usage.

## introduction in the last decade, research in the social sciences and humanities (ssh) field has consolidated a paradigmatic transformation from theoreticallyoriented approaches to data-driven methodologies @xcite . digitizing ssh data otherwise distributed as non-machine readable formats to convenient electronic ones is a decisive and necessary step in the ssh research life-cycle @xcite . however, the digitization process of artifacts, documentation, and other research objects produces enormous amounts of data, which must be systematically stored and managed. this presents both a necessity and an opportunity to design and implement tools to manage and use such data in a fair (findable, accessible, interoperable, and reusable) way, allowing not only the preservation of ssh data but facilitating scientifically sound and transparent research processes. focusing on strengthening this paradigmatic transformation in the ssh field, the multilingual corpus of survey questionnaires (mcsq) provides easy access to textual survey data in a machine-readable format and related services streamlined to the needs of the ssh community. in this context, we present the mcsq interface: a tool that offers easy-to-use functionalities for end-users unacquainted with programming or data manipulation libraries. other than common search functionalities offered by off-the-shelf corpus management tools such as ims open corpus workbench @xcite and sketch engine @xcite , the mcsq interface provides features that are convenient for translators and survey practitioners, such as allowing users to build their own translation memories and facilitating the retrieval and comparison of multilingual data. this paper is organized as follows. in section 2, we briefly describe the mcsq, and discuss how it follows the fair principles. subsequently, in section 3, we describe the interface infrastructure and its functionalities. finally, in section 4, we present our conclusions and future work. ## the multilingual corpus of survey questionnaires the multilingual corpus of survey questionnaires (mcsq) is the first publicly available corpus of survey questionnaires. the mcsq is an opensource and open-access resource that was designed and implemented following the fair principles. the current version 3 of the corpus (entitled rosalind franklin) contains approximately 4 million tokens and includes 306 distinct questionnaires designed in the source (british) english language and their translations into catalan, czech, french, german, norwegian, portuguese, spanish, and russian, adding to 30 country-language combinations (e.g., french from switzerland). questionnaires in the mcsq comprise more than 40 years of survey research from large-scale comparative survey projects that provide cross-national and crosscultural data to the ssh, namely, the european social survey (ess), the european values study (evs), the survey of health ageing and retirement in europe (share), and the wageindicator survey (wis). within a questionnaire, a survey item @xcite is a unit for data collection. it includes the question, the response options, and other textual elements presented to the participants in the study. the questionnaires in the mcsq were designed for in-person interviews. except in the case of the wis, the translation process was implemented according to the trapd (translation, review, adjudication, pretesting and, documentation) method @xcite , a team approach for the translation of survey questionnaires. in the trapd method, two translators ('t' in the trapd acronym) should produce independent translations into the respective target language. optionally, they could split a source questionnaire and work independently on one of its parts. in a team meeting with the translators, a reviewer ('r') assesses the two translations, and an adjudicator 'a' takes the final decisions on the different translation options. teams can opt for one of the translation options or create a new one by combining the strengths of each translation. the translated questionnaire is pretested before fieldwork ('p'), and the whole process is documented ('d'). ideally, the team is composed of survey experts and language experts (e.g. professional translators) capable of assessing not only translatability aspects, but also necessary adaptations for the culture where it will be administered and to achieve the measurement objectives. questionnaires included in the mcsq were obtained from the survey projects' archives in distinct formats such as spreadsheets, extensible markup language (xml), and portable document format (pdf) files. the pdf files had to undergo an additional step of conversion to plain texts before going through the preprocessing pipeline. then, the texts were extracted from the input files and preprocessed, sentence aligned with respect to the english source and annotated with part-of-speech (pos) and named entity recognition (ner) tags. during the preprocessing steps, in addition to text cleaning and normalization, we attributed metadata to the text segments to identify and navigate among the items in the mcsq: to align the survey items, we developed our own sentence alignment algorithm, which leverages metadata available in mcsq to minimize the search space for candidates of a given alignment. namely we use the module, item name, and item type metadata to this end. additionally, we take into account the sentence length and information from domain specific bilingual dictionaries built with word2word(choe et al., 2020) as alignment heuristics. for the pos tagging task, we used flair (akbik et al., 2019) pre-trained models for the czech, english, german, french, norwegian, and spanish languages, whereas for catalan, portuguese and russian languages we trained our own models, also using the flair framework. the named entity recognition (ner) annotation was executed using pre-trained models from different sources. namely, flair for english, german, french, and spanish, spacy @xcite for catalan, norwegian and portuguese, and slavic bert @xcite for czech and russian. due to the domain specificity and nature of the texts, some of the models (e.g., catalan) performed worse than others, especially in instruction segments. we developed the mcsq aiming to be as fair (findable, accessible, interoperable, and reusable) as possible. to be findable, we attributed the rich metadata described above. the code and data used to compile the corpus are publicly available and it has attached a persistent identifier foot_1 . furthermore, the mcsq will be submitted for permanent preservation to a clarin eric foot_2 repository in 2021, where the data will receive a persistent identifier. in order to be accessible, we made the mcsq data available in open format (csv with tab separators) and safeguarded accessibly, i.e. through user registration, via the interface hereby described foot_3 . the interoperability aspect of the mcsq is a work in progress. currently, our metadata is a simplified subset of the ddi codebook foot_4 , adapted for csv files instead of xml ones, and including linguistic metadata such as part-of-speech tags. we use the universal pos tags foot_5 for part-of-speech metadata tags. furthermore, the data model that describes and structures the mcsq metadata will be submitted to fairsharing foot_6 . to ensure re-usability aspects, we are currently acquiring a license for the mcsq that will allow users to remix, adapt, and build upon the work done in mcsq. moreover, we upload publicly available documentation and materials about the corpus with persistent identifiers 8910 in the zenodo repository 11 . making data fair is a process. although we consider that findability, accessibility, and reusability facets are well underway, interoperability is yet a work in progress and needs enhancement, such as producing mappings of the mcsq meta- ## the interface of the multilingual corpus of survey questionnaires aiming at combining searches in the mcsq contents and functionalities that facilitate the usage of the mcsq data for end-users, we developed the mcsq interface foot_11 . to have complete control over the technology stack and freedom to include innovative functionalities, instead of using off-the-shelf corpus management tools, we developed our infrastructure from scratch. users must register to access all functionalities in the interface foot_12 . the corpus and all its metadata are available for visualization and download through the interface. ## conclusion the mcsq is a valuable contribution to opensource, open-access, and high-quality ssh data that follows fair principles, whilst its interface provides useful functionalities for researchers across different disciplines, such as linguists, survey practitioners, computational linguists, and translators. aside from traditional corpus search functionalities, we developed features to support end-users that are not acquainted with programming, thus facilitating the accessibility of the data and related resources to all audiences. through this work, we hope to stimulate the creation of fair and user-friendly infrastructures for secondary research. as future work, we aim to explore the following three aspects: i) the possibility of allowing users to add new data to the corpus through the interface; ii) functionalities for the construction of domain-specific bilingual dictionaries and, iii) the comparison of similar questions across rounds and studies. additionally, we aim at producing handson tutorials with a comprehensive overview of the interface functionalities and guidance on how to use them.