Venue: NAACL 2025

# Grammar Control in Dialogue Response Generation

for Language Learning Chatbots

Dominik Glandorf1,2, Peng Cui3, Detmar Meurers4, Mrinmaya Sachan3

###### Abstract

Chatbots based on large language models offer cheap conversation practice opportunities for language learners. However, they are hard to control for linguistic forms that correspond to learners' current needs, such as grammar. We control grammar in chatbot conversation practice by grounding a dialogue response generation model in a pedagogical repository of grammar skills. We also explore how this control helps learners to produce specific grammar. We comprehensively evaluate prompting, fine-tuning, and decoding strategies for grammar-controlled dialogue response generation. Strategically decoding LLama3 outperforms GPT-3.5 when tolerating minor response quality losses. Our simulation predicts grammar-controlled responses to support grammar acquisition adapted to learner proficiency. Existing language learning chatbots and research on second language acquisition benefit from these affordances. Code available on GitHub1.

Footnote 1: [https://github.com/domnikglandorf/gramarctg](https://github.com/domnikglandorf/gramarctg)

+EPFL, \({}^{2}\)University of Tubingen, \({}^{3}\)ETH Zurich, \({}^{4}\)Leibniz-Institut fur Wissensmedien

**Correspondence: dominik.glandorf@epfl.ch**

## 1 Introduction

The language input that learners receive, their production of language output, and the opportunity to interact using language are paramount to second language acquisition [6]. Chatbots based on large language models (LLMs) can simulate a partner for free-flowing conversations, offering these three components. However, general-purpose chatbotis, such as ChatGPT, currently do not implement the pedagogical strategy of providing developmentally proximal input as suggested by popular learning theories [7, 8]. They do not support the _noticing_ of grammatical forms by increasing their frequency in dialogues, an effective setting for language practice by reinforcing the learner's input processing when prompting them to produce a meaningful reaction [10, 9].

Ideally, a chatbot should purposely demonstrate the usage of linguistic forms that a learner is ready to acquire, as a teacher or tutor would. The positive effects of role models on language development have been demonstrated in different conversational contexts [1, 11]. Although adapted chatbots have started to produce dialog utterances aimed at developing learners' language proficiency [1, 12], they lack grounding in a comprehensive framework of language proficiency development.

The Common European Framework of References for Languages (CEFR) [13] is such a description of _functional_ language development, complemented by the English Gramar Profile (EGP) [14], which provides a corresponding empirical description of the language _forms_ produced by learners at different developmental stages based on an extensive learner corpus linked to the Cambridge certificate testing. The EGP induces a curricular progression of linguistic forms that can define the underlying logic for a pedagogically optimized chatbot that acts as a conversation partner.

Figure 1: Example conversation of the pedagogically motivated chatbot. Based on the current learner level, the chatbot includes grammar skills in its response, and the learner replicates the advanced grammar structure in their response.



In this paper, we develop and evaluate strategies to control an LLM to use developmentally beneficial grammar forms when generating dialogue responses, i.e., learner's inputs, as exemplified in Figure 1. The set of grammar forms originates from the EGP; these are either explicitly specified by a teacher (task 1) or a learner profile exploiting the EGP-induced progression along different categories of grammar skills (task 2). We test GPT-3.5 and Llama3 using a prompting baseline for these tasks posed for everyday conversation datasets. Then, we create a dataset of constrained dialogue responses and fine-tune Llama3 on it. We also adopt a decoding strategy that modifies Llama's output toward the specified grammar. To evaluate these strategies, we measure the presence of grammar forms, response quality, and efficiency. To show how this grammar-controlled input can elicit desired grammar in learner output, we simulate the impact of controlled grammar input on learners at different proficiency levels. To this end, following Ellis and Collins (2009), we present an experiment in which we identify common input-output pairs of grammar forms in a dialogue corpus and exploit the relationships in a simulated conversation.

Our results show that guided decoding is a promising model adaptation strategy to generate EGP-controlled responses with the best balance between grammar control and response quality, successfully including 59.3% of requested grammar forms and being as grammatically correct as GPT-3.5. The learner simulation shows that 25 out of 47 tested grammatical input-output pairs can significantly increase grammar usage in learners' output. Overall, this paper contributes methods for training pedagogically controllable conversation chatbots for language learning by adaptively fostering learners' grammatical proficiency in a functional dialogue setting, a unique combination impossible to realize without the advancements of current LLMs and their grounding in a comprehensive pedagogical repository of language development.

## 2 Related Work

### Chatbots for Grammar Acquisition

The idea of intentionally shaping language learners' input can be traced back to Krashen (1985), who emphasized that input should be developmentally proximal, ie, containing linguistic forms that learners are ready to learn but not yet able to use. This beneficial input can stem from teachers acting as role models to learners (Heppt et al., 2022) or other social contexts (Pillinger and Vardy, 2022). As a negative example where this input is missing for advanced structures, heritage language learners often lack late-developing forms when their input is limited to the parent-child interaction at home (Bayram et al., 2019).

Driven by the advancement of LLMs, educational applications have recently focused on aligning chatbots with such linguistic demands. Tyen et al. (2022) restricted vocabulary above certain proficiency levels while decoding a chatbot model and reranked response candidates by their CEFR level. Similarly, Qian et al. (2023) identified unknown vocabulary with a pre-test, increased its frequency in dialogues with a similar method, and showed that learners included the target vocabulary more often in their turns. So far, only Okano et al. (2023) have explicitly controlled _grammar_ in dialogue response generation by adapting models with reinforcement learning for three broad grammar categories, which did not stem from a language development framework like the CEFR.

Little work exists on the effects of adapted grammar-controlled LLMs for conversation practice. The framework of Ellis and Collins's (2009) defines hypothetical statistical relationships between input and output, which can be potentially exploited by a controllable language input generation. We, therefore, simulate the effect on learners' output in Section 6.

### Pedagogical Grounding

In contrast to previous approaches, we base our grammar control on the CEFR, specifically the EGP. The CEFR defines language competencies in broad rubrics, such as written production on six proficiency levels, ranging from A1 / A2 (beginner) to B1 / B2 (intermediate) to C1 / C2 (advanced). For example, the rubric _Overall oral interaction_ states about B1 proficiency: "Can communicate with some confidence on familiar routine and nonroutine matters related to their interests and professional field." Descriptions in the CEFR are _function_-oriented and cater to the ultimate goal of language learning. However, language teaching needs to make learners acquire the _forms_ that are necessary to fulfill these functions. The EGP is a comprehensive repository of required language forms to communicate on CEFR proficiency levels, identified by an analysis of the Cambridge Learner Corpus with more than 50M words from English 

learners with various mother tongues. An example EGP statement on level B1 is: "Can use 'would not have' + '-ed' or 'wouldn't have' + '-ed'. (see another example in Table 1). Here, we call these statements _grammar skills_. Grounding a chatbot in this skill framework can ensure that it demonstrates the linguistic forms required to impart CEFR competencies to the learner.

### Controlling Dialogue Generation

The field of controlled text generation (CTG) distinguishes syntactical, numerical, semantic, and prefix controls Sun et al. (2023). In our case, the control is a set of grammar skills that imposes not only syntactical but also semantic constraints. Text generation must also be conditional on a previous dialog, restricting the possible space of responses. This combination differentiates the problem from more established tasks, such as paraphrase generation, that fixes the semantics a priori and imposes purely syntactic constraints.

Typical approaches to CTG include prompting and adopting pre-trained language models with decoding, instruction fine-tuning, or reinforcement learning Zhang et al. (2023). InstructCTG Zhou et al. (2023) has shown that verbalizing control conditions in natural language and fine-tuning sequence-to-sequence models with large numbers of demonstrations of control conditions and corresponding generated text successfully increases control. They synthesized training data because the control conditions are easily determined. The approach has not been tested on our control condition, whose evaluation is nontrivial.

Decoding strategies such as _Future Discriminators for Generation_FUDGE by Yang and Klein (2021) aim to fulfill control conditions while sampling from a model's output. FUDGE uses a lightweight predictor at each generation step on the current sequence combined with each of the most likely next tokens. The predictor returns the probability that the desired attribute will be present in the _completed_ sequence if the current sequence continues with the respective token. In this work, we propose a different aggregation mechanism of such prediction to align with specific characteristics of grammar skills as control conditions.

## 3 Problem Statement

Given a sequence of \(n\) previous dialogue turns \(D=[d_{1},d_{2},\ldots,d_{n}]\), and a repository of grammar skills \(\mathcal{G}\), our problem is to generate a chatbot response \(r\). To ensure that \(r\) is useful for language practice, we formulate two pedagogically motivated control tasks:

* **Explicit Grammar Constraints** require \(r\) to include a subset \(G\subset\mathcal{G}\) of grammar skills in the generated dialogue response.
* **Categorical Grammar Constraints** require \(r\) to include grammar skills on a specified CEFR level per EGP category.

In task 1, teachers can allow students to practice multiple recently taught grammar skills simultaneously. They select the skills that should appear in the conversations. The chatbot then demonstrates to the students how to use them in conversations about topics of their interest. Task 2 is motivated by a hypothetical learner model that describes grammar proficiency differentiated by categories. For example, a learner may need more input of "superlatives" skills on level A2 while they already know "would" skills on level B1 and hence should be exposed to more sophisticated ones.

### Datasets of Grammar and Dialogue

Due to the importance of English as a second language, in this work we focus on controlling for English grammar in chatbot responses. The English Grammar Profile2 (EGP) is our set \(\mathcal{G}\) of 1,222 grammar skills organized into 86 categories. For

\begin{table}
\begin{tabular}{p{45.5pt} p{279.1pt}} \hline
**Property** & **Example** \\ \hline Super Category Subcategory Guideword & Adjectives Superditives Guideword & FORM/USE: WITH ‘IN’ + NOUN Can-do Statement Learner Example CEFR Level & Can use prepositional phrases with ’in’ + singular name of a place after a superlative adjective. It’s the biggest room in the house. \\ \hline \end{tabular}
\end{table}
Table 1: An example from the 1,222 grammar skills in the English Grammar Profile, which lists the skills used in the two grammar control tasks.

 

each skill, it indicates the CEFR proficiency level ranging from A1 (beginner) to C2 (advanced), and whether it concerns syntax (type _FORM_), semantics (type _USE_), or both (type _FORM/USE_). It provided one to five example sentences per skill. Table 1 shows an example skill.

As dialogue datasets, we use _DailyDiago_Li et al. (2017), _DialogSum_Chen et al. (2021), the _Document Grounded Dataset_ by Zhou et al. (2018), _Topical-Chat_Gopalakrishnan et al. (2019), and _Wizard of Wikipedia_Dinan et al. (2019). All these unlabeled datasets reflect typical daily and topical conversations that a learner could have with a chatbot and meet our problem motivation. Statistics, examples, and licenses for the dialogue datasets are in Appendix A.

### Evaluation of Controlled Responses

The following subsections describe how we calculate metrics to evaluate \(r\), namely grammar constraint satisfaction, response quality, and efficiency.

#### 3.2.1 Grammar Constraint Satisfaction

Our response generation strategies do not guarantee fulfilling the entire control condition. To measure this primary success indicator, the presence of grammar, we automatically detect grammar skills in dialogue responses using bidirectional sentence embeddings. For task 1, we compute the ratio of explicitly specified grammar skills present in \(r\). For task 2, we compute the number of grammar categories with at least one skill present on the specified CEFR level. Section 4.1 describes our method to detect grammar and ensure robustness of the detectors.

#### 3.2.2 Dialogue Response Quality

A high quality response should fit the given dialogue context and should be grammatically and factually correct. A recent dialogue evaluation challenge proposed and validated four dimensions, _Appropriateness, Relevance, Content Richness_ and _Grarmmatical Correctness_, to measure dialogue turn quality Rodriguez-Camelar et al. (2023). The winning team showed that the assessment via only prompting an autoregressive LLM is nearly as accurate as combining it with smaller fine-trained bidirectional models Mendonca et al. (2023). Therefore, we prompt gpt-4o-2024-08-06 for quality evaluation in this work. The prompt asks GPT-4o to rate the response, given a preceding dialog, in terms of one of the four dimensions with a score from 1 to 5. As a non-LLM measure, we use the distinct-N metric, which indicates the diversity of responses for the same constraint set in different dialogue contexts. It is defined as the number of unique n-grams divided by the total number of n-grams. Here, we calculate _distinct-2_as Okano et al. (2023)) and average it over all grammar constraint sets.

#### 3.2.3 Response Generation Efficiency

The chatbot model should be useful in a real-time dialogue setting and generate text roughly at reading speed, which is between 70 and 90 words per minute for foreign language learners Iwahori (2008). Thus, we also measure the models' inference speed from the beginning of the generation until the model generates the end-of-sequence token or reaches maximum length. The local models are tested on the same GPU configuration for a fair comparison.

## 4 Methods

We first explain how we detect grammar skills in dialogues (Section 4.1) and then how we adapt models to control for grammar (Section 4.2).

### Grammar Detection

#### 4.1.1 Binary Classification Model

Evaluating and generating responses require detecting the presence of a grammar skill in an utterance. For this problem, we rely on sentence representations from a frozen BERT model with 110M parameters Devlin et al. (2019) and a binary classification head \(A\) in the form of a two-layer feedforward network of 320K parameters with labeled example sentences. We classify each token \(s\) in \(r\) with \(A\) and apply maximum pooling to detect skill \(i\):

\[G_{i}(r)=\mathbf{I}(\max_{s\in r}A(s)>0.5) \tag{1}\]

We apply \(A\) before the pooling because we assume many grammar skills to have a "central" token that indicates the presence of a skill if it appears in a certain context, which is also encoded in the token embedding of the BERT output layer.

#### 4.1.2 Training Grammar Detectors

We use three datasets of example sentences to train the grammar detectors and compare them concerning the resulting model's predictive performance on test data. Appendix B explains our strategies to 

create a synthetic, manually assembled, and automatized dataset.

For annotating a grammar skill in a sentence in training and test data, we follow this procedure: One of the authors reads the can-do statement of the grammar skill and the corresponding examples in the EGP, e.g. the statement "Can use prepositional phrases with 'in' + singular name of a place after a superlative adjective." and the examples, e.g. "1. It's the biggest room in the house. 2. I bought them because they are the cheapest clothes in the shop". Then, they judge single sentences to determine whether they contain the skill. For ambiguous skills descriptions in the EGP, we decide to stick to one interpretation for consistency. The judgments are available in the associated GitHub repository.

#### 4.1.3 Grammar Detection Evaluation

We select three subcategories of the EGP to evaluate the robustness of grammar detection, "superlatives", "would", and "negation". This selection is motivated by challenging the detectors with similar skills from the same category. These categories contain skills from various CEFR levels, both FORM and USE types, and are usually not mutually exclusive. We estimate _validation_ precision by 5-fold cross-validation. The _test_ precision is human-labeled for 20 examples per skill detected in the Document Grounded Dataset and Topical-Chat. We focus on precision because we lack ground truth labels for the entire testsets and cannot calculate recall. We try to ensure reasonable decision sensitivity of the classifier by adding a limited amount of negative examples in the training data.

Figure 2 compares the performance of the model specified in Section 4.1.1 in detecting the selected skills when training it with one of the three datasets described in Section 4.1.2. The validation precision of detectors trained on the automatized dataset is so low that we omit human evaluation. It becomes evident that the synthetic dataset is the easiest to classify, but the resulting detectors have 62 %-points lower precision on the test datasets. In contrast, the manual dataset yields a much smaller generalization gap of only 17%.

We resort to the 28 out of 53 detectors trained on this dataset, which have a test precision of at least 80% in subsequent experiments. This is respectively 50% of "negation" and "superlatives" skills, and 57% of "would" of evaluated skills. Skills on level A1 are the easiest to detect (86% precision), while no detector for a skill on level C2 has sufficient test precision. We did not observe a significant difference in the detection precision between the two test datasets (\(\chi^{2}=0.189,p=0.66\)).

### Strategies for Controlling Grammar

We explore three strategies, namely prompting, guided decoding, and instruction fine-tuning, to accomplish the two tasks of generating dialogue responses with grammar skill constraints.

- FORM: NEGATIVE"). We prompt the closed but relatively affordable gpt-3.5-turbo-01253 and the open Llama3-8B-Instruct4.

Footnote 3: [https://openai.com/index/chatgtpt/](https://openai.com/index/chatgtpt/)

Footnote 4: [https://ai.meta.com/blog/meta-llama-3/](https://ai.meta.com/blog/meta-llama-3/)

**Guided decoding** modifies the generation model outputs, i.e., the probability distribution over the vocabulary before decoding them to text. We adapt Llama3-8B-Instruct using FUDGE Yang and Klein (2021), which factors in the probability that the completed sequence will fulfill the constraints given the next token. To predict this satisfaction of constraints in the final sequence during generation, one classifier per skill with the same architecture as our grammar detectors, called _future discriminator_, is trained on subsequences of the training examples of all possible lengths. On top of FUDGE's top

Figure 2: Precision distribution on the same (validation) and an unseen (test) corpus of 53 grammar skill detectors trained on three different datasets.

 

k pruning, we ignore tokens below a probability threshold (as in epsilon sampling) to further remove unreasonable next tokens. Another modification is the transformation of the discriminator outputs. In our work, the logit \(l_{s}\) for token \(s\) is calculated as

\[\begin{array}{ll}(1-\alpha)l_{s}+\alpha\max_{g_{i}\in F(G)}(g_{i}(s)-\bar{g}),& \text{if $\sigma(l_{s})\geq\eta$}\\ -\infty,&\text{if $\sigma(l_{s})<\eta$}\\ -\infty,&\text{if $\operatorname*{rk}(l_{s})<k$}\end{array} \tag{2}\]

, where \(\sigma\) is the softmax function, \(rk\) the rank in the probability distribution, and \(g_{i}(s)\in(0,1)\) is the score of a candidate sequence with the next token \(s\) from the set of future discriminators \(F\) for grammar constraints \(G\) (opposed to the final sequence detector \(G_{i}\)). For categorical grammar constraints (task 2), the set \(G\) consists of the category members at the specified CEFR level, potentially creating larger sets of constraints, but yielding the same computation as for explicit grammar constraints (task 1). Taking the maximum across multiple skills is motivated by focusing on a single skill at a time due to the limited span of many skills within sentences. This implies that as soon as one skill is present, the focus shifts to fulfilling the remainder. \(\alpha\) balances the influence of future discriminators and is tuned as a hyperparameter (see Appendix E).

**Instruction fine-tuning** an LLM to our task is inspired by InstructCTG Zhou et al. (2023), which fine-tuned a pre-trained language model on synthesized training data transformed by prompt templates as in the prompting approach. We also adapt Llama3-8B-Instruct for comparison. The required training data for this procedure consists of pairs of prompts and completions in Llama's chat format. We apply all grammar detectors to DailyDialog, DialogSum, and Wizard of Wikipedia to gather pairs of context \(D\) and next turn \(r\) that satisfies a constraint. The resulting labeled data are transformed into the prompt and completion part using the template from the prompting approach for explicit grammar skills. To avoid overweighting a high-frequency grammar skill, we use a maximum of 500 demonstrations per skill. Skills with fewer occurrences are likely less relevant to language practice. A side effect of instruction fine-tuning is that the model also learns to respond like the speakers in the datasets.

Our **hybrid approach** combines the decoding and fine-tuning approach because they respectively operate at training and inference time. Their effects on task performance may be partially independent, which creates the possibility of a synergy.

## 5 Experimental Results

To assess our strategies to solve the two tasks, we draw random dialog snippets of four turns as the context \(D\) to generate \(r\). We use Topical-Chat and the Document Grounded Dataset as testsets; the remainder is used in fine-tuning. Computational details of the experiments are in Appendix F.

### Task 1: Explicit Skill Constraints

To assess the strategies for controlling responses for explicitly stated grammar skills, we randomly draw 100 dialogue snippets. For each, we randomly select one to three categories and one to two skills per category to assemble a set of preferred skills (1, 2, 3, 4, or 6 skill constraints). We also add dialogue snippets with ground truth answers containing the preferred skills to the test set, in total 1322 pairs of dialogue snippets and constraints.

Table 2 summarizes the evaluation metrics as defined in Section 3.2. The hybrid approach leads to the highest constraint satisfaction, but the lowest response quality. The decoding approach ranks second, performing on par with the hybrid approach for four and six skill constraints. Its response quality is better than that of the fine-tuning approach, which shows a worse constraint satisfaction. Prompting Llama3 satisfies the constraints more often than GPT-3.5, but yields a slightly worse response quality. GPT-3.5 has the highest response quality and the most distinct responses. It generates the fastest, more than twice as fast as the local Llama model. However, all models are efficient enough to generate responses at reading speed. Overall, if one can tolerate a small loss in response quality, with responses that are still as grammatical as GPT-3.5's, the decoding approach is preferred due to its higher constraint satisfaction coming at a slight quality loss versus prompting the unadapted Llama model. In an ANOVA, we found neither a significant interaction between the test dataset and the generation strategy nor a main effect of the test dataset (\(F<1.59,p>.05\)).

There is a considerable variation in the satisfaction between grammar skills, which implies that certain constraints are easier than others. A visualization of constraint satisfaction per skill is provided in Appendix H. For example, using reported speech with "would" is met in less than 7.9% of 

the cases for any model, whereas negations with "can" are fulfilled in up to 92.4%. For the categories _superlatives_ and _would_, there is a trend that grammar skills from advanced CEFR levels are more difficult to include. In contrast, this pattern is not visible among the _negations_. To ensure that high constraint satisfaction is not due to reduced response quality (as observed between models), we estimated the correlation between response quality metrics and constraint satisfaction rate within guided decoding, which was not significant for any of the five quality indicators (\(r<.37\), \(p>0.05\)).

We include a qualitative example with annotated responses from all approaches in Appendix G.

### Task 2: Categorical Skill Constraints

To assess the strategies to generate responses including skills on proficiency levels specified per category, one to three EGP categories are paired with random CEFR levels for 500 dialogue snippets, yielding 1500 test cases. Figure 3 compares the strategies in this task. The picture is comparable to the first task, with the decoding-based approaches yielding the best grammar constraint satisfaction. The Llama model fine-tuned on task 1 leads to slight gains over the Llama prompting baselines in this task. Interestingly, the gains are more pronounced when specifying the desired level for more than one category. The response quality of the decoding approach is, this time, slightly worse than the quality of the fine-tuning approach in terms of appropriateness and relevance. The content richness and grammatical correctness are comparable. The amount of skills included in the responses on a too difficult level increases among the two best-performing approaches but is much less pronounced compared to the gains on the positive side (worst case: 7.7% in fine-tuning versus best case: 4.7% in GPT-3.5).

## 6 Impact on Conversation Practice

In this application of our work, we simulate the effect of grammar-controlled chatbot responses on learner responses at different proficiency levels. While traditional grammar practice, such as fill-in-the-gap exercises, forces learners to produce target grammar (i.e., grammar to learn), practice in free-flowing dialogues requires other approaches to make learners use this grammar. According to language acquisition theories, learners sometimes immediately reproduce grammar structures that they encounter in their input and can thereby

Figure 3: Strategy performance comparison for generating responses that include grammar skills from a category and on a specified level (N=1500, single run).

 

learn to communicate with these structures (Ellis and Collins, 2009; Loewen and Sato, 2017). This idea of alignment between the _same_ grammar skills in the input and output can be extended to relationships between _different_ grammar skills in the input and subsequent output. For example, a question by the chatbot using "would" may allow the learner to use "would" in the affirmative form in their response. In Section 6.1, we analyze a corpus for both types of these patterns in dialogs. In Section 6.2, we try to exploit the identified patterns in a learning simulation using our grammar-controlled response generator.

### Grammar Co-Occurrence Analysis

We analyze DailyDialog, DialogSum, and Wizard of Wikipedia for grammar skill co-occurrence patterns in adjacent speaker turns. In a pre-processing step, each turn in the dialog datasets is labeled with the included skills using the grammar detectors of Section 4.1. For each possible tuple of grammar skills \((g_{pre}\), \(g_{post})\), we count the number of \(g_{post}\) in the next two responses of one speaker after the other speaker used \(g_{pre}\). The number is tested with Fisher's exact test for a difference to cases without \(g_{pre}\). Due to the multiple testing, a significance level of \(\frac{0.05}{n_{pairs}}\) is used (Bonferroni correction).

Table 3 shows that 24.4% of the tuples have \(g_{post}\) increased significantly. Among these, the number of \(g_{post}\) increases by up to 19.5%, and the odds ratio is between 1.16 and 76. Among the 28 alignment tuples, where \(g_{pre}=g_{post}\), three have \(g_{post}\) increased by at least 5 percent. The maximum increase occurs after the occurrence of the skill _would - QUESTIONS WITH 'LIKE'_ preceding _would - AFFIRIMATIVE_. In general, skills of the subcategory _would_ seem to precede skills of the same category in the responses.

### Co-Occurrence Intervention Simulation

In the case of significant co-occurrence relations of grammar skills, it remains questionable whether \(g_{pre}\)_causes_\(g_{post}\). Due to the temporal unfolding of the conversion, \(g_{post}\) is unlikely to cause \(g_{pre}\). However, the opposite is not necessarily true. To establish causal evidence, we simulate an intervention: First, we generate 100 grammar-controlled responses to _random_ dialogs for each statistically significant tuple in the previous analysis, assured to include \(g_{pre}\) by discarding unsuccessful generations. Afterward, a learner response is simulated and tested for \(g_{post}\). To ensure that the tuples are not only exploitable in a conversation between advanced or native speakers, we generate learner responses on a given proficiency level. The details on validating this proficiency simulation are in Appendix I.

Figure 4 indicates significantly increased \(g_{post}\) when simulating learners of different proficiencies (detailed figure in Appendix I). The rightmost column represents an unconditional learner simulation (no proficiency specified), yielding the desired effect in 25 out of 47 cases. However, the effect generally reduces when simulating a certain proficiency level. The effect appears only in four cases on the lowest proficiency level, where one would expect \(g_{post}\) to be in the learner's competency range, given the associated CEFR level. The results indicate that the co-occurrences can indeed causally serve - in some cases - to allow learners to practice the target grammar skills in their responses more often.

Figure 4: Grammar skill tuples with a significantly increased target skill are indicated in red for simulated learners on the specified proficiency level (x-axis). Grey boxes reflect pairs not expected to be significant due to too difficult target grammar skills. Diagram with y-axis labels and odds ratios in Appendix I.

\begin{table}
\begin{tabular}{l r} \hline \hline Statistic & Value \\ \hline Number of tests & 784 \\ Significant tests & 191 \\ Average frequency difference & 0.02 \\ Average odds ratio & 2.98 \\ Frequency difference > 0.05 & 11.5\% \\ \hline \hline \end{tabular}
\end{table}
Table 3: Test summary for co-occurrences of EGP skills in adjacent dialogue turns.

 

Discussion

We explored automatic grammar detection to measure and improve grammar control in dialog responses. Our preliminary work in Section 4.1 has shown that only manually curated training data yields robust grammar detectors. This implies a dependency on human annotation to improve grammar control via fine-tuning and decoding.

The improved grammar control with our approaches can readily support the pedagogical use case. Not only can we generate grammar-enriched language input to increase the chances of the learner noticing developmentally proximal grammar structures from the EGP but we can also generate them fast enough for the dialogue setting, which supports deeper input processing by the learner via the mechanisms of output and interaction Pannell et al. (2017); Bibauw et al. (2019). This reduces the need for isolated grammar drills and affords more authentic language practice, closer to the function-oriented pedagogy underlying the CEFR.

The response evaluation revealed an unexpected property of the overall winning decoding approach. It outperformed the instruction fine-tuning approach in response quality in the first task, which is surprising due to the local optimization in guided decoding, usually reducing text quality. Our finding might point to low-quality dialogue data because the model learns during fine-tuning to copy the style of dialogue responses. This hypothesis is supported by the similar response quality in the second task where the prompt different from the fine-tuning demonstrations might make the model resort to its original style. We believe that the drastically increased constraint satisfaction in the decoding approach compensates for the slightly reduced response relevance given that the answers are as grammatically correct. This effect is likely also observable in teachers or tutors who try to use a linguistic form in a given context and have to adapt the meaning of their example to apply the form.

The generated responses largely varied in grammar constraint satisfaction between skills across all approaches. Contrary to expectation, using different adaptation strategies depending on the types of skills (such as more syntactic versus more semantic) seems not helpful as the constraint satisfaction rates of all approaches are highly correlated. This can be caused by the random combination of skills with dialogue contexts. For example, the barely included skill of using "would" in reported speech may have not been appropriate in most contexts and may even have an overall lower saliency than other skills. Therefore, grammar-controlled conversation practice should preferably be used for the skills that are already possible to control, which we make transparent in Appendix H. We are in favor of a mixed-methods approach in language instruction and do not think that one approach can satisfy all needs in grammar instruction.

We mitigated this issue in the final application of the response generator in the simulation study by ensuring that the controlled grammar was in the response we used to simulate the learner. However, grammatical cooccurrences were less exploitable at lower levels of language proficiency despite previous research showing that alignment is more pronounced among them Loewen and Sato (2017). This observation questions the validity of the learner simulation, where the in-depth analysis shows that the lower proficiency is exaggerated compared to baselines from a learner corpus.

## 8 Conclusion and Future Work

This work explored the problem of generating pedagogically controllable dialogue responses with specified grammar and tested common prompting and model adaptation approaches. Specifically, we defined two tasks based on grammar skills from the English Grammar Profile and publicly available topical and daily dialogues, and conducted a comprehensive automatic evaluation by including different response quality measures and inference speed. Our simulation study confirmed that the approaches could potentially help learners of different levels of proficiency in conversation practice produce the desired grammar skills more often.

In the future, the generative models and predictions in the simulation study should be tested with real teachers and learners. Such a study would ensure utility as perceived by teachers, analyze whether learners notice reduced response quality, and quantify learning gains from grammar-controlled conversation practice. The approach will realize its full potential only if all skills from the English Grammar Profile are detectable. Moreover, it should be explored how other types of teacher-specified attributes can be automatically controlled in conversation practice 



### Limitations

We elaborate on potential shortcomings in detecting grammar across dialogue topics and learner levels, the approach's scalability, and the simplification of the pedagogical framework.

We trained our grammar detectors on sentences from native or advanced English speakers that occur in everyday and topical conversations. In the optimal case of a sufficient robustness check in Section 4.1, grammar can be detected independently of the conversation topic. However, we cannot guarantee that this holds for every possible topic a learner likes to discuss since the datasets' range of topics is limited. Our best-performing grammar control strategy, guided decoding, depends on these grammar detectors and may therefore only work as estimated in the range of conversation topics that occurred in our datasets. It is also important whether these grammar detectors can detect grammar usage in learner dialogue turns that may show orthographical and grammatical errors. Robustness would be especially crucial when automatically evaluating a learning gain study for grammar acquisition. On the other end of the proficiency spectrum, we had difficulties detecting skills at level C2. One could argue that these high-level skills require authentic language exposure and that learners on the preceding level C1 are capable of thriving in such contexts, and hence do not need a chatbot practice partner anymore.

Regarding all strategies' performance, generating dialogue responses by sampling from logit distributions may improve grammar constraint satisfaction. Performance may be limited since we only employ greedy decoding to focus on comparing the different approaches.

#### Scalability

With limited resources, we achieved sufficient detection precision for at least half of the skills in three selected grammar categories. However, it would cost more than 1,200 annotator hours to create detectors for the entire English Grammar Profile. Therefore, improving automatic methods for training robust grammar detectors for dialogue data would be helpful. The availability of more grammar detectors would render the evaluation of generated responses more comprehensive.

In the case of categorical grammar constraints, it is theoretically possible to control up to 86 categories. This amount will make the prompt longer, potentially reaching the limits of the context window. In this case, guided decoding may be the only suitable strategy because it depends not only on the skill description in the prompt but also directly on the grammar detectors. However, this hypothesis should be verified with more available skill detectors from more categories.

#### Pedagogical Framework

The CEFR framework distinguishes in its proficiency descriptions between the communicative contexts in which the language occurs. For example, it differentiates text-based and oral interaction. The proficiency levels in the EGP are determined from written production. Ignoring this modality of language use may be problematic because grammar in written essays may be less relevant to oral conversations about everyday topics. However, for this work, it was a pragmatic choice not to be overly accurate with these distinctions to focus on the potential of LLMs in language learning and not overcomplicate the task specification and evaluation.

Apart from this simplification, our repository of grammar skills, the EGP, is loaded with ambiguities. It often implicitly refers to named elements from English grammar, for example, skill #616 ("Can use 'would have' + '-ed') is linked to the past conditional perfect tense even though it avoids involving the term "past participle" explicitly. This imprecision leaves some skills underspecified, especially skills that refer to a lexical range, e.g., skill # 636 ("Can use an increasing range of adverbs with 'would', including 'strongly', 'easily', 'especially', 'actually', 'absolutely', 'gladly' \(\rightarrow\) adverbs"). In these cases, the correctness judgments of the grammar detectors were up to the researchers, potentially lowering interrater reliability.

## Acknowledgments

The computations for this study were performed on the Euler cluster operated by the High Performance Computing group at ETH Zurich. We gladly acknowledge the use of the EGP using the officially requested statement: This publication has made use of the English Grammar Profile. This resource is based on extensive research using the Cambridge Learner Corpus and is part of the English Profile program, which aims to provide evidence on language use that helps produce better language teaching materials. See [http://www.englishprofile](http://www.englishprofile)

org for more information. We thankfully used an OpenMoji by Mariella Steeb in Figure 1 under CC BY-SA 4.0.

## References

* B. Bayram, J. Rothman, M. Iverson, T. Kupisch, D. Miller, E. Puig-Mayenco, and A. Westergaard (2019)Differences in use without deficiencies in competence: passives in the Turkish and German of Turkish heritage speakers in Germany. International Journal of Bilingual Education and Bilingualism228, pp. 919-939. Cited by: SS5.
* S. Bibauw, T. Francois, and P. Desmet (2019)Discussing with a computer to practice a foreign language: research synthesis and conceptual framework of dialogue-based call. Computer Assisted Language Learning32 (8), pp. 827-877. External Links: ISSN 0026-45XX, Document, Link Cited by: SS1, SS2.
* Y. Chen, Y. Liu, L. Chen, and Y. Zhang (2021)DialogSum: A Real-Life Scenario Dialogue Summarization Dataset. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Online, pp. 5062-5074. External Links: ISBN 978-3-319-3319-3, Link, Document Cited by: SS1, SS2.
* companion volume. External Links: Link Cited by: SS1, SS2.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019)BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171-4186. External Links: ISBN 978-3-319-3319-3, Link, Document Cited by: SS1, SS2.
* Emily Dinan, S. Roller, K. Shuster, A. Fan, M. Auli, and J. Weston (2019)Wizard of Wikipedia: Knowledge-powered Conversational agents. In 7th International Conference on Learning Representations, New Orleans, LA, USA, Cited by: SS1, SS2.
* N. Ellis and L. Collins (2009)Input and Second Language Acquisition: The Roles of Frequency, Form, and Function Introduction to the Special Issue. The Modern Language Journal93 (3), pp. 329-335. External Links: ISSN 0026-45X, Document, Link Cited by: SS1, SS2.
* S. M. Gass and A. Mackey (2014)Input, interaction, and output in second language acquisition. In Theories in second language acquisition, R. Banerjee, R. Banerjee, and R. Bhattacharya (Eds.), Routledge, Cited by: SS1, SS2.
* D. Glandorf and D. Meurers (2024)Towards fine-grained pedagogical control over English grammar complexity in educational text generation. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), Mexico City, Mexico, pp. 299-308. External Links: ISBN 978-3-642-465X, Link Cited by: SS1, SS2.
* K. Gopalakrishnan, B. Hedayatnia, Q. Chen, A. Gottardi, S. Kwatra, A. Venkatesh, R. Gabriel, and D. Hakkani-Tur (2019)Topical-chat: Towards knowledge-grounded open-domain conversations. In Proc. Interspeech 2019, pp. 1891-1895. External Links: ISBN 978-3-642-465X, Link Cited by: SS1, SS2.
* B. Heppt, S. Henschel, I. Hardy, R. Hettmannsperger-Lippolt, K. Gabler, C. Sontag, S. Mannel, and P. Stanet (2022)Professional development for language support in science classrooms: evaluating effects for elementary school teachers. Teaching and Teacher Education109, pp. 103518. External Links: ISSN 0026-45X, Link Cited by: SS1, SS2.
* J. Marvin Imperial, G. Forey, and H. Tayyer Madabushi (2024)Standardize: Aligning Language Models with Expert-defined Standards for Content Generation. arXiv preprint. Cited by: SS1, SS2.
* Y. Iwahori (2008)Developing reading fluency: A study of extensive reading in erl. Reading in a Foreign Language20 (1), pp. 70-91. External Links: ISSN 0026-45X, Link Cited by: SS1, SS2.
* Stephen D. Krashen (1985)Second language acquisition and second language learning. Note: reprinted edition, Language teaching methodology series, Pergamon Pr, Oxford Cited by: SS1, SS2.
* Y. Li, H. Su, X. Shen, W. Li, Z. Cao, and S. Niu (2017)DailyDialog: A Manually labelled Multi-turn Dialogue Dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Taipei, Taiwan, Asian Federation of Natural Language Processing, Cited by: SS1, SS2.
* P. Lightbown (1992)Getting quality input in the second/foreign language classroom. Text and context: Crossdisciplinary perspectives on language study, pp. 198-197. External Links: ISBN 978-3-642-465X, Link Cited by: SS1, SS2.
* Chin-Yew Lin and Eduard Hovy (2003)Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pp. 150-157. External Links: ISBN 978-3-642-465X, Link Cited by: SS1, SS2.
* Shawn Loewen and Masatoshi Sato (2017)The Routledge handbook of instructed second language acquisition. Taylor & Francis. Cited by: SS1, SS2.
* John Mendonca, P. Pereira, H. Moniz, J. Paulo Carvalho, A. Lavie, and I. Trancoso (2023)Simple LLM Prompting is State-of-the-Art for Robust and Multilingual Dialogue Evaluation. In Proceedings of The Eleventh Dialog System Technology Challenge, Prague, Czech Republic, pp. 133-143. External Links: ISBN 978-3-642-465X, Link Cited by: SS1, SS2.
* Y. Okano, K. Funakoshi, R. Nagata, and M. Okumura (2023)Generating dialog responses with specified grammatical items for second language learning. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational 

Applications (BEA 2023)_, pages 184-194, Toronto, Canada. Association for Computational Linguistics.
* O'Keeffe and Mark [2017] Anne O'Keeffe and Geraldine Mark. 2017. The English Grammar Profile of learner competence: Methodology and key findings. _International Journal of Corpus Linguistics_, pages 457-489.
* Pannell et al. [2017] Justin Pannell, Friederike Partsch, and Nicholas Fuller. 2017. The Output Hypothesis: From Theory to Practice. _TESOL Working Paper Series_, 15.
* Pollinger and Vardy [2022] Claire Pollinger and Emma J. Vardy. 2022. The story so far: A systematic review of the dialogic reading literature. _Journal of Research in Reading_, 45(4):533-548.
* Qian et al. [2023] Kun Qian, Ryan Shea, Yu Li, Luke Kutszik Fryer, and Zhou Yu. 2023. User Adaptive Language Learning Chatbots with a Curriculum. In Ning Wang, Genaro Rebolledo-Mendez, Vania Dimitrova, Noboru Matsuda, and Olga C. Santos, editors, _Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky_, volume 1831, pages 308-313. Springer Nature Switzerland, Cham.
* Rodriguez-Cantelar et al. [2023] Mario Rodriguez-Cantelar, Chen Zhang, Chengguang Tang, Ke Shi, Sarik Ghazarian, Joao Sedoc, Luis Fernando D'Haro, and Alexander I. Rudnicky. 2023. Overview of Robust and Multilingual Automatic Evaluation Metricsfor Open-Domain Dialogue Systems at DSTC 11 Track 4. In _Proceedings of The Eleventh Dialog System Technology Challenge_, pages 260-273, Prague, Czech Republic. Association for Computational Linguistics.
* Sun et al. [2023] Jiao Sun, Yufei Tian, Wangchunshui Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Wieting, Nanyun Peng, and Xuezhe Ma. 2023. Evaluating Large Language Models on Controlled Generation Tasks. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 3155-3168, Singapore. Association for Computational Linguistics.
* Swain [1993] Merrill Swain. 1993. The Output Hypothesis: Just Speaking and Writing aren't Enough. _The Canadian Modern Language Review_, 50(1):158-164.
* Tyen et al. [2022] Gladys Tyen, Mark Brenchley, Andrew Caines, and Paula Buttery. 2022. Towards an open-domain chatbot for language practice. In _Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022)_, pages 234-249, Seattle, Washington. Association for Computational Linguistics.
* Vygotsky [1978] L. S. Vygotsky. 1978. _ Mind in Society: Development of Higher Psychological Processes_. Harvard University Press.
* Yang and Klein [2021] Kevin Yang and Dan Klein. 2021. FUDGE: Controlled Text Generation With Future Discriminators. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 3511-3535.
* Zhang et al. [2023] Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023. A Survey of Controllable Text Generation using Transformer-based Pretrained Language Models. _ACM Computing Surveys_, 56(3):1-37.
* Zhou et al. [2018] Kangyan Zhou, Shrimai Prabhumoye, and Alan W Black. 2018. A Dataset for Document Grounded Conversations. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 708-713, Brussels, Belgium. Association for Computational Linguistics.
* Zhou et al. [2023] Wangchunshui Zhou, Yuchen Eleanor Jiang, Ethan Wilcox, Ryan Coterell, and Mrinmaya Sachan. 2023. Controlled text generation with natural language instructions. In _Proceedings of the 40th International Conference on Machine Learning_, volume 202 of _ICML'23_, pages 42602-42613, Honolulu, Hawaii, USA.

## Appendix A Datasets

Table 4 contains a quantitative description of the datasets we used. An example of DailySum reflecting daily dialogs is in Figure 5. An example of the Document-Grounded Dataset that includes topical dialogs is in Figure 6. We use the datasets for their intended use in their respective licenses. DailyDialog and DialogSum are licensed under CC BYNC-SA 4.0. The Document grounded Dataset was made publicly available by the authors and used in several publications5. TopicalChat is licensed under the Community Data License Agreement - Sharing - Version 1.0 by Amazon.com6. Wizard of Wikipedia is licensed under MIT.

Footnote 5: [https://paperswithcode.com/dataset/cmu-dog](https://paperswithcode.com/dataset/cmu-dog)

\begin{table}
\begin{tabular}{l r r r} \hline \hline  & \#dialogs & \#turns & \#words \\ Dataset & & & \\ \hline Daily Dialog & 13,118 & 7.9 & 13.6 \\ DialogSum & 12,460 & 9.5 & 16.6 \\ WoW & 18,430 & 9.0 & 18.8 \\ CMU DoG & 4,221 & 31.8 & 12.6 \\ Topical-Chat & 8,628 & 21.8 & 22.6 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Descriptive statistics on the dialogue datasets used in the experiments. WoW: Wizard of Wikipedia. CMU DoG: Carnegie Mellon University’s Document-Grounded Dataset.

 



## Appendix B Grammar Skill Datasets

We follow three strategies to curate datasets for training binary detectors of a grammar skill.

The **synthetic dataset** is the only public large-scale EGP dataset of 750 labeled examples per skill generated by few-shot prompting GPT-3.5 (Glandorf and Meurers, 2024). The dataset has a high label quality but is potentially lacking diversity and hard negative examples. Following the authors' suggestion, the negative examples are augmented with random positive examples from other items during training.

We curate a **manual dataset** by matching candidate sentences and difficult negative examples with handcrafted regular expressions in a corpus of DailyDialog, DailySum, and Wizard of Wikipedia and labeling the matches. In the case of insufficient results for the expressions, we add manually validated examples from the synthetic dataset. As soon as 50 positive examples have been identified, a preliminary grammar detector is trained on the partial dataset and evaluated on the remainder of the corpus to find novel candidates. The process is repeated until the precision of the found examples is above the arbitrary threshold of 80% or does not increase after five iterations. The process is carried out for three representative grammar categories, namely "negations", "superlatives", and "would" to limit the workload.

The **automatized dataset** is the result of automatizing the manual procedure with OpenAI's GPT-4o by few-shot prompting it for regular expressions that are evaluated on the same corpus. The iterative procedure and criterion remain the same. The labeling of candidate matches is based on generated questions from the EGP description and examples that the model answers one by one for the candidates.

## Appendix C Grammar Skill Detection

The validation and test precision for each grammar skill tested in the manual dataset approach are listed in Table 5.

## Appendix D Prompts

For the first task of **Explicit Grammar Constraints** the prompt template includes the instruction, a description of each skill, and the dialogue to continue:

_Given the dialog, write a possible next turn of [Next Speaker] that includes all of these grammatical items: - {Subcategory} - {Guideword}: {Can-do statement} (CEFR [Level]) - ..._

_[Dialog]_

For the second task of **Categorical Grammar Constraints**, the prompt template instructs the model to prefer grammar patterns from specified subcategories, described only by their guidewords to keep prompts short:

_Given the dialog, write a possible next turn of {Next Speaker} that preferably uses the following grammar patterns in the response: - {Subcategory} on CEFR level [Level]): {Guidewords} - ..._

Figure 5: Example dialogue of DailySum.





\begin{table}
\begin{tabular}{l l c r r}
**Subcategory** & **Skill** & & **Level** & **Validation Set** & **Test Sets** \\ \hline superleaves & ’MY BEST FRIEND’ & A1 & 0.905 & 0.960 \\ \hline superlattices & COMPLEX NOUN PHRASES & A2 & 0.830 & 0.800 \\ \hline superleaves & WITH ‘IN’ + NOUN & A2 & 0.838 & 0.800 \\ \hline superleaves & WITH ‘OF’ + NOUN & A2 & 0.856 & 0.750 \\ \hline superleaves & COMPLEX NOUN PHRASES & B1 & 0.757 & 1.000 \\ \hline superleaves & ’THE BEST’ WITH NOUN AND PRESENT PERFECT & B1 & 0.926 & 0.500 \\ \hline superleaves & ’ONE OF THE’ & B1 & 0.851 & 0.200 \\ \hline superleaves & WITH ‘BY FAR’ & B2 & 0.935 & 0.000 \\ \hline superleaves & ELLIPSIS, WITH ‘THE’ & B2 & 0.912 & 0.684 \\ \hline superleaves & WITH NOUN AND “TO-” INFINITIVE & B2 & 0.848 & 0.800 \\ \hline superleaves & WITH NOUN AND POSTMODFIER & C1 & 0.860 & 0.947 \\ \hline superlattices & ’SLIGHTTEST’, ’FAINTEST’ & C2 & 0.940 & 0.200 \\ \hline would & AFFIRMATATIVE WITH ‘LIKE’ & A1 & 0.835 & 0.950 \\ \hline would & INVITATIONS WITH ‘LIKE’ & A1 & 0.862 & 0.000 \\ \hline would & WISHES AND PREFERENCES WITH ‘LIKE’ & A1 & 0.829 & 1.000 \\ \hline would & AFFIRMATIVE & A2 & 0.811 & 0.950 \\ \hline would & NEGATIVE & A2 & 0.813 & 0.750 \\ \hline would & QUESTIONS WITH ‘LIKE’ & A2 & 0.598 & 0.818 \\ \hline would & IMAGINED STUATIONS & A2 & 0.502 & 0.350 \\ \hline would & SUGGESITIONS WITH ‘TI WOULD BE’ & A2 & 0.067 & 0.333 \\ \hline would & WISHES AND PAEFERENCES & A2 & 0.909 & 0.950 \\ \hline would & AFTER ‘IF CLAUSES & B1 & 0.769 & 0.900 \\ \hline would & PAST AFFIRMATIVE & B1 & 0.866 & 0.650 \\ \hline would & PAST NEGATIVE & B1 & 0.816 & 0.615 \\ \hline would & QUESTIONS & B1 & 0.942 & 0.895 \\ \hline would & WITH ADVERBS & B1 & 0.423 & 0.800 \\ \hline would & FUTURE IN THE PAST & B1 & 0.954 & 0.900 \\ \hline would & IMAGINED STUATIONS IN THE PAST & B1 & 0.908 & 1.000 \\ \hline would & INDIRECTNESS & B1 & 0.932 & 0.600 \\ \hline would & POLITE REQUSTS & B1 & 0.694 & 0.059 \\ \hline would & REPORTED SPEECH & B1 & 0.661 & 0.900 \\ \hline would & WILLINGNESS IN THE PAST & B1 & 0.855 & 0.938 \\ \hline would & HABITUAL PAST & B2 & 0.952 & 0.944 \\ \hline would & WITH ADVERBS & C1 & 0.840 & 0.368 \\ \hline would & WITH ADVERBS & C2 & 0.067 & 0.238 \\ \hline negation & MAIN VERB ’BE' & A1 & 0.841 & 0.850 \\ \hline negation & AXXLLIARY YERB ’DO’, PRESENT & A1 & 0.797 & 0.950 \\ \hline negation & MODAL VERG ’CAN’ & A1 & 0.918 & 1.000 \\ \hline negation & AUXILIARY VERBS ’BE’, ’HAVE’, PRESENT & A2 & 0.839 & 0.800 \\ \hline negation & AUXILIARY VERB ’DO’, PAST & A2 & 0.944 & 0.950 \\ \hline negation & 'DO’, IMPERAIVES & A2 & 0.968 & 0.688 \\ \hline negation & AUCILIAJY VERBS ’BE’, ’HAVE’, PAST & B1 & 0.885 & 0.300 \\ \hline negation & MENTAL PROCESS VERBS + CLAUSE & B1 & 0.844 & 0.750 \\ \hline negation & QUESTIONS & B1 & 0.975 & 0.900 \\ \hline negation & 'NOT; EMPHASIS & B2 & 0.699 & 1.000 \\ \hline negation & ‘NEVER’, INverLED FRONT POSITION, FOCUS & B2 & 0.844 & 0.250 \\ \hline negation & ‘NEITHER ... NOR’ & B2 & 0.982 & 0.200 \\ \hline negation & ‘NONE’, SUBSTITUTION & C1 & 0.653 & 0.050 \\ \hline negation & ‘NOT ALL’, ’NOT EVERY’ & C1 & 0.947 & 0.842 \\ \hline negation & ‘NOT ONLY ... (BUT) ALSO’ WITH INVERSION & C1 & 0.991 & 0.875 \\ \hline negation & ‘NOT A’ + NOUN, EMPHASIS & C2 & 0.943 & 0.200 \\ \hline negation & ‘DON’T YOU ...’, WARNING & C2 & 0.912 & 0.125 \\ \hline negation & ‘NEITHER’ & C2 & 0.978 & 0.625 \\ \hline \end{tabular}
\end{table}
Table 5: The validation and test set performance for grammar skill detectors trained on the manually curated dataset.



[Dialog]

The dialogue is formatted as:

_Dialog: A: [First turn] B: [Second turn] A: [Third turn] B: [Fourth turn]_

Both prompt templates prepend the following system message to ensure the response format: _Only output [Next Speaker]'s response._

## Appendix E Hyperparameter Tuning for Decoding

This experiment on the decoding approach is designed to find the optimal balance between the original logits and the logits provided by the partial grammar classifiers. Increasing the hyperparameter \(\alpha\) increases the weight of the grammar score, with \(\alpha=0\) resulting in the original logits and \(\alpha=1\) ignoring the original logits. Figure 7 shows the trade-off between constraint satisfaction and response quality for different values of \(\alpha\). One can see that the satisfaction of the constraints increases up to a value of \(\alpha=0.99\) and then stagnates. The response quality consistently decreases across all four measures with a larger \(\alpha\). With being almost as grammatically correct as the unchanged model, \(\alpha=0.99\) is a compromise that shows around 65% of the required grammar skills while being a bit less appropriate, relevant and content-rich.

## Appendix F Computational Details

The experiments were conducted on the university's compute cluster on multiple NVIDIA GPUs, such as the RTX3090 with 24GB RAM. The scripts are written in Python 3.11 and use standard libraries such as numpy and Pandas, as well as the Huggingface Transformer library to load and configure LLMs and PyTorch for training the neural networks. The versions are referenced in the requirements.txt in the code repository7. The grammar detector training took around 1 GPU hour, instruction fine-tuning took around 48 GPU hours, and the evaluation took around 24 GPU hours. The budget spent on the OpenAI API to create training data for grammar detectors and evaluation was 75 USD.

Footnote 7: [https://github.com/dominikglandorf/grammarctg](https://github.com/dominikglandorf/grammarctg)

The license of the BERT model Devlin et al. (2019) is apache-2.0. The use of Llama3 is granted by the Meta Llama 3 Community License, Copyright @ Meta Platforms, Inc. All Rights Reserved8. The decoding approach was implemented using \(\alpha=0.95\), \(\eta=10^{-3}\), and \(k=200\). For finetuning, we used the SFTTrainer class from the TRL library9 for the training routine with a learning rate of \(5\times 10^{-4}\). For memory efficiency, the fine-tuning procedure used a low-rank adaptation of the full precision model as implemented in the PEFT library10 with \(r=64\), a dropout of 0.1 and \(\alpha=16\) (67M parameters). Every 200 steps, the model was assessed with respect to constraint satisfaction. The model with the highest validation satisfaction rate was saved at these checkpoints.

Footnote 8: [https://www.llama.com/llama3/license/FOOTNOTE:9https://huggingface.co/docs/trl/index](https://www.llama.com/llama3/license/FOOTNOTE:9https://huggingface.co/docs/trl/index)

Footnote 10: [https://huggingface.co/docs/peft/index](https://huggingface.co/docs/peft/index)

### Response Examples

Figure 8 shows the outputs of all the approaches for an example test case from task 1.

## Appendix H Grammar Constraint Difficulty

Figure 9 shows the satisfaction by skill constraint of the guided decoding strategy and the maximum among the three best-performing strategies.

## Appendix I Learner Proficiency Simulation

To ensure that the grammatical adjacency pairs are not only exploitable in a conversation between advanced and native speakers, but also work for language learners, the learner responses should not be generated by vanilla prompting for a possible response. Instead, a language model is prompted to generate the next dialogue response on a certain learner proficiency level. This technique is inspired by Imperial et al. (2024), who also used descriptions of proficiency standards in prompting. Using this method, the effect of the intervention can be estimated for learners at different levels.

A validation of prompting Llama3 to generate the next dialogue turn on a given proficiency level helps to interpret the results of the intervention simulation. Therefore, we generate dialogue responses to 500 random contexts for each of the six CEFR levels with the following prompt: _Given the dialog, write a possible next turn of [Next Speaker] that an English learner on CEFR level [Level] could produce: Dialog_

The system message was: _Only output [Next 

_Speaker)'s response using language on CEFR level [Level]. This level is described as: [Description]_. The proficiency level description is taken directly from the CEFR, specifically the rubric "Overall oral interaction" and listed in Table 6. The dialogue was formatted in the same way as for the task prompts.

Text complexity measures were calculated using the textcomplexity library11. The library comprises lexical, surface-based, sentence-based, part-of-speech-based, dependency-based, and constituency-based complexity measures, thereby representing a wide range of possible metrics. All metrics were 0-1-normalized and recoded in case increasing complexity was indicated by a decreasing measure. There are two reference values. The first is a random subset of 500 EFCamDat writings per CEFR level. The second is a transformation of this corpus into a dialogue setting. We created it by prompting Llama3-8B-Instruct to create a dialogue with exact phrases from the learner essays, including mistakes. The resulting dialogue turns are ranked by their Rouge-L recall (Lin and Hovy, 2003) of the original essay to not include turns that were added to create a dialogue, but were not taken from the essay. The Rouge-L score refers to the largest n-gram overlap between the reference and the "translation", in this case, the created dialogue utterance. The recall measures how much of the translation can be found in the reference. The second reference corpus is created for all levels except C1 and C2 due to the low number of available essays on these two levels.

Footnote 11: [https://github.com/tsproisl/textcomplexity](https://github.com/tsproisl/textcomplexity)

Figure 10 shows the results of the complexity comparison. It becomes evident that Llama3's generated responses show a larger variation in complexity compared to both baselines. The complexity of the reference essays does not increase much from B1 to C2 whereas the generated text complexity increases up to proficiency level C1. In the lowest proficiency levels, A1 and A2, we observe much lower text complexity of the generated text. The generated responses on level A2 align better with the reference corpus on level A1 than A2.

Figure 7: Performance of the decoding approach with different values for balancing factor \(\alpha\).

 

* [15] M. C. Bader, J. S. H. Lee, and A. K. Jain, "A simple approach to the problem of a single-layer neural network for image recognition," in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_, pages 2378-2390, 2014.
* [16] R. D. Manning, L. P. McAllester, and N. Ramanan, "An introduction to artificial intelligence: The role of deep learning," _Journal of Machine Learning Research_**10**, no. 1, pp. 103-118, 2014.
* [17] G. Hinton, I. Goodfellow, Y. Bengio, and Z. Courville, "Deep residual networks with applications to machine learning," _arXiv preprint arXiv:1412.6090_**, 2014.
* [18] T. Darrell, F. Donahue, and E. Rabinovich, "The lager's guide: A survey on machine learning," _IEEE Transactions on Neural Networks_**10**, no. 1, pp. 1-10, Jan. 2014.
* [19] V. Balasubramanian, A. Krizhevsky, and A. Zisserman, "Very good representations for large scale visual recognition," _Advances in neural information processing systems_**17**, no. 1, pp. 1-10, Jan. 2015.
* [20] W. Dong, X. Li, and Q. Weinberger, "Learning generative adversarial nets for large scale visual recognition," _CoRR_**abs/1412.1110**, 2014.
* [21] J. Deng, L. Fei-Fei, and G. LeCun, "ImageNet: A review of deep convolutional models," _ACM Trans. Mach. Learn. Res. Lett._**10**, no. 1, pp. 1-10, Jan. 2015.

[MISSING_PAGE_POST]

 

Figure 10: Comparison of text complexity of responses generated by Llama on different proficiency levels.

Figure 9: Satisfaction by constraint for the decoding approach. The red bars indicate the maximum satisfaction among GPT-3.5, Llama3, and Llama3 Decoding. Llama3 Decoding approach achieves the highest satisfaction among the top 3 approaches for almost all constraints. Note: Within each category, the CEFR level is ascending from left to right.

 



\begin{table}
\begin{tabular}{|p{34.1pt}|p{280.5pt}|} \hline
**Level** & **Description** \\ \hline A1 & Can interact in a simple way but communication is totally dependent on repetition at a slower rate, rephrasing and repair. Can ask and answer simple questions, initiate and respond to simple statements in areas of immediate need or on very familiar topics. \\ \hline A2 & Can interact with reasonable ease in structured situations and short conversations, provided the other person helps if necessary. Can manage simple, routine exchanges without undue effort; can ask and answer questions and exchange ideas and information on familiar topics in predictable everyday situations. \\ \hline B1 & Can communicate with some confidence on familiar routine and non-routine matters related to their interests and professional field. Can exchange, check and confirm information, deal with less routine situations and explain why something is a problem. Can express thoughts on more abstract, cultural topics such as films, books, music, etc. \\ \hline B2 & Can interact with a degree of fluency and spontaneity that makes regular interaction, and sustained relationships with users of the target language, quite possible without imposing strain on either party. Can highlight the personal significance of events and experiences, and account for and sustain views clearly by providing relevant explanations and arguments. \\ \hline C1 & Can express themselves fluently and spontaneously, almost effortlessly. Has a good command of a broad lexical repertoire allowing gaps to be readily overcome with circumlocutions. There is little obvious searching for expressions or avoidance strategies; only a conceptually difficult subject can hinder a natural, smooth flow of language. \\ \hline C2 & Has a good command of idiomatic expressions and colloquialisms with awareness of connotative levels of meaning. Can convey finer shades of meaning precisely by using, with reasonable accuracy, a wide range of modification devices. Can backtrack and restructure around a difficulty so smoothly that the interlocutor is hardly aware of it. \\ \hline \end{tabular}
\end{table}
Table 6: Overall oral interaction proficiency of CEFR Levels from Council of Europe (2020)

Figure 11: Detailed results of learner response simulation with. Significant increases of a grammar skill \(g_{post}\) in the learner response after grammar skill \(g_{pre}\) are indicated in red for simulated learners on the CEFR proficiency levels. The number indicates the odds ratio between the skill frequencies. Grey boxes reflect unexpected effects due to too difficult \(g_{post}\) grammar items.