C -Test Collector: A Proficiency Testing Application to Collect Training Data for C -Tests (2021)

Abstract: We present the C-Test Collector, a web-based tool that allows language learners to test their proficiency level using c-tests. Our tool collects anonymized data on test performance, which allows teachers to gain insights into common error patterns. At the same time, it allows NLP researchers to collect training data for being able to generate c-test variants at the desired difficulty level.

## introduction the c-test @xcite ) is a reduced redundancy exercise frequently used for language proficiency testing. in a c-test, the second half of every second word is replaced by a gap, and the task of the test-taker is to reconstruct the text. for example, the word redundancy would be replaced with redun___. c-tests are known to correlate well with general language proficiency @xcite and are fast to take and to evaluate. thus, they are even used in commercial systems such as ondaf @xcite . despite these advantages, the creation of a wellworking c-test is a time-consuming task even for language experts. one reason is that the gap scheme is influenced e.g. by named entities or other words that would be hard to guess for learners. improved tool support for curating c-tests has mitigated this problem to some extent @xcite . even with technological support, it is still hard (even for experts) to judge the difficulty of a given test without running a pilot study @xcite . thus, there has been research on predicting @xcite or manipulating @xcite the difficulty of c-tests. these approaches heavily rely on the availability of training data, i.e. c-tests which have been taken by enough learners so that the solution probability for individual gaps can be reliably estimated. influencing factors on the part of the student, such as the native language, make the difficulty prediction even more dependent on the availability of the right kind of training data. in this paper, we present a web-application that makes it easy to collect such training data, by providing an easy-to-use platform for taking a c-test online. in the remainder of this paper, we describe the two ways to interact with the system: (i) as a user taking c-tests and receiving a proficiency assessment, and (ii) as an administrator uploading new tests and accessing the aggregated test results of the users. ## user perspective figure 1 illustrates the interaction steps of a user with the application. ## admin perspective the administrator is responsible for adding new tests with pre-set initial difficulty levels and can access the test results of the users. figure 3 shows the administration view. adding tests a c-test can be added in two ways: first, we support the export format of the c-test builder @xcite , an authoring tool offering technical support for creating a c-test from raw text including automatic detection of named entities, numeric expression and other words that should be excluded from the gap scheme. 1 the tool also provides an automatic difficulty assessment as described in @xcite , the result of which can be used as the initial value for the c-test collector. second, text can be entered manually where those parts of a word that should appear as gaps are manually indicated using a bracket notation, e.g. redun{dancy}. for each newly imported or created test, the admin has to provide the language, a title and an initial difficulty (i.e. a cefr level). as we have already discussed that it is hard for teachers to correctly assign the difficulty level, this is only used at the beginning and the difficulty is adjusted as soon as more users have taken the test. test overview the administrator is provided with a complete overview of all c-tests currently available in the application, as well as a graphical representation of the language distribution of tests (see figure 3 ). ## scoring tests and users adjusting proficiency levels of users and difficulty levels of tests is a cold start problem: in the beginning, each user only has a self-assigned language proficiency and each test has only a rough difficulty level assigned by the administrator. this section describes how tests receive adjusted difficulty ratings and how users' proficiency assignments are adapted throughout their interaction with the system in order to present them with tests of a suitable difficulty. whenever a user completes a c-test, their current language level is adjusted. similarly, the difficulty prediction of the c-test is refined with every new user taking the test, providing a larger basis for the difficulty estimation. we currently use two rating strategies (i) mean error rate and, (ii) the elo rating system @xcite . ## implementation details our tool is implemented through a rest backend server and a separated frontend application. the source code is publicly available. 2 the frontend of the application is provided through a vuejs application that communicates with the provided rest server. the interface provides the functionality to interact with the application and is more or less independent from the backend. the provided rest server connects and mediates corresponding requests for the creation and evaluation of c-tests. the server was implemented using the spring boot framework. ## https://gitlab.com/ctest-evaluator 6 conclusion and outlook we have presented an open-source implementation of a data collection tool that provides c-tests for language learners and collects training data for difficulty prediction tasks. the collected data, especially the individual incorrect solution attempts can also be used to study misspellings and cognates for learners of different l1s and l2s. so far, our prototype has only been tested with a small number of users. next steps thus involve the deployment with a larger user base to start the actual data collection process for c-tests. for many languages, this will provide the first publicly available training data.