Text Simplification of College Admissions Instructions: A Professionally Simplified and Verified Corpus (2022)

Abstract: Access to higher education is critical for minority populations and emergent bilingual students. However, the language used by higher education institutions to communicate with prospective students is often too complex; concretely, many institutions in the US publish admissions application instructions far above the average reading level of a typical high school graduate, often near the 13th or 14th grade level. This leads to an unnecessary barrier between students and access to higher education. This work aims to tackle this challenge via text simplification. We present PSAT (Professionally Simplified Admissions Texts), a dataset with 112 admissions instructions randomly selected from higher education institutions across the US. These texts are then professionally simplified, and verified and accepted by subject-matter experts who are full-time employees in admissions offices at various institutions. Additionally, PSAT comes with manual alignments of 1,883 original-simplified sentence pairs. The result is a first-of-its-kind corpus for the evaluation and fine-tuning of text simplification systems in a high-stakes genre distinct from existing simplification resources.

## introduction access to the higher education system for minoritized populations, especially low-income students, first-generation in college students, students of color, and students whose first spoken language is not english, is an important social challenge @xcite @xcite @xcite @xcite . however, researchers have consistently explained that much of higher education's communication with prospective students is too complex, too lengthy, and requires a wealth of prior knowledge of the higher education system to successfully navigate @xcite ; concretely many institutions in the us publish admissions application instructions at or above the 14th grade english reading comprehension level, far too high for the average prospective student or adult to read and comprehend @xcite @xcite . as a result, the verbose, difficult communication places an unnecessary barrier between students and access to higher education @xcite @xcite @xcite . with modern conditional text generation models, one plausible way to lower such barriers is automatic text simplification @xcite , i.e., simplify text such that it is more readable and accessible, while adhering to the texts' original semantic content. text simplification is known to benefit a range of readers, including children @xcite and l2 learners of english @xcite . yet, no studies have explored how to simplify this information, without losing important details, to render the admissions process more accessible for prospective students and their families. crucially, existing work in text simplification largely rely on two established datasets in the news and wiki domains @xcite @xcite ; unlike these two domains, college admission texts certainly falls under the category of being more specialized and difficult to generalize, where both the jargon and concepts encapsulated within these texts are not ones that one encounters every day. consequently we lack understanding both in terms of the nature of (original) all conditionally accepted applicants must consent to, submit to and successfully complete a criminal background check through certiphi screening, inc. failure to do so will constitute failure to meet the prematriculation requirements established by suny optometry and will result in the withdrawal of a conditionally accepted offer. (simplified) if you are conditionally accepted, you must consent to, submit to, and complete a criminal background check through certiphi screening, inc. if you do not, we will withdraw your conditionally accepted offer. this work introduces psat (professionally simplified admissions texts), the first manually simplified and verified corpus of admissions instructions from 112 randomly sampled us postsecondary institutions. our professional simplification process are guided by extensive existing literature on manual simplification, including lowering syntactic complexity, improving lexical cohesion, and elaborating jargon @xcite @xcite . given that the admissions process normally requires application fees, completion of an application, writing and submitting of essays, sending of transcripts, and possibly more processes, it is crucial to understand how to compose admissions application instructions to clearly but accurately explain these processes, especially to first-generation in college students who may not have support from their secondary school, family, or household @xcite . to this end, every document in psat was manually verified by 2 subject-matter experts (among a total of 10 experts) who are employed as full-time admissions professionals in us institutons of higher education. in addition to full document simplification, we also create manual alignments (by the author who originally simplified all documents), aligning each sentence in the original text to its simplified version. this entailed 1,883 sentence pairs total. our analyses showed that the simplified documents reduced the reading level of these texts from grade 13.3 to grade 9.8, making it much more accessible for minority and emergent bilingual students. for experiments on automatic simplification, in this paper we focus on sentence simplification utilizing the aligned sentence pairs. we start from zeroshot transfer from existing simplification models pre-trained on both wikilarge @xcite and newsela @xcite , as well as fine-tuned models on psat. we showed that this is a challenging domain for simplification. additionally, our professional manual inspection of model outputs points out domain-specific errors that impact the accuracy of the simplifications. ## data collection and simplification the main work of psat included performing manual text simplification, followed by acceptability judgements made by subject-matter experts with domain-specific professional work experience in either admissions or financial aid. below, we detail the corpus development process, including the gathering of college admissions instructions (section 3.1), professional simplification and its principles (section 3.2) and verification (section 3.3). ## analysis we also study lexical items that are most associated with the original texts vs. the simplified version. concretely we calculate the logodds ratio of each word w i , comparing the conditional probability of w i in the original instruc- @xcite . excluding punctuation and numbers, the words with the highest and lowest log-odds ratios with respect to simplified texts are shown in finally, we measure the abstractiveness of the simplified texts; namely, how much paraphrasing was used? for this we use the aligned portion of psat; the percentage of unique uni-, bi-, and tri-grams in the simplified texts are 34.4%,52.4%,58.6%, respectively. thus the amount of new jargon introduced through simplification is enough to be significant; this indicates that simpler vocabulary is being introduced, and significant paraphrasing happened during simplification. ## sentence simplification experiments we establish baselines for the sentence simplification task in psat, using the aligned sentences. with these experiments we hope to evaluate existing models-which are trained in other domainsin a zero-shot manner, gauge the utility of finetuning on psat, and discuss the challenges of simplification in this domain. ## manual inspection finally, because psat is a very new domain that comes with its own characteristics and requirements for accurate simplification, we perform a small scale manual inspection with the professional who authored the simplified documents; this professional has not seen model-generated outputs in nlp before. because of time commitment required to inspect outputs from all 5 models, we were only able to study 25 examples. nonetheless we hope the insights here will guide future work on domainspecific evaluation metrics. concretely, we randomly sample 25 sentences from the test set. we ask the professional to inspect outputs based on three metrics: fluency, simplicity, and accuracy. under each metric, we ask the professional to choose the highest ranked simplification; an option of none is available if no model provided satisfactory outputs for any criteria. results in first, several simplifications required adding words to the simplified version that did not appear in the original, however the model did not learn to do so. the most frequent example was simplifications adding "official" to the phrase "official transcript(s)". here, subject-matter experts felt that adding "official" was necessary to make it clearer and more simple for students to understand that they needed to submit official transcripts instead of copies or screenshots of their transcripts. similarly, many simplifications added the subject and modal verb "you must" to phrases such as "submit your official transcript(s)", "complete an application", and "apply by the priority date", as discussed in section 3.2. the subject-matter experts felt it was necessary to clarify that a student "must" complete these tasks, otherwise they would not be considered for admission. however, the model could not generate such phrasing. finally, many models removed keywords from the original versions. this most often happened when an institution accepted multiple application types, such as the coalition for college application and the common application. for instance, one sentence original sentence read, "we accept the coalition for college application and the common application". however, several models attempted to simplify the sentence and remove either the coalition for college application or common application, as they both contain the word "application", and the model may have removed these "redundant" phrases. however, both application types needed to appear in the simplified version to gain subject-matter expert approval, as an accurate admissions instructions contain all of the different applications that a student may use to apply for admission, not just the simplest application. as a result, several models did not accurately keep keywords from original to simplified versions because the models did not have this domain-specific information that subject-matter experts had. ## discussions and conclusions this work presents psat, a text simplification corpus consisting of admission instructions texts from 112 us higher education institutions and their simplified versions. psat texts are professionally simplified and verified, rendering it the first-of-its-kind and most accurate dataset in this high-stake domain. we showed that this dataset is challenging for existing simplification models, especially due to domain mismatch, and domain-specific requirements for the accuracy of information. we recognize the limitations of this study. while there are over 6,000 institutions of higher education in the us, this dataset sampled a small number of these institutions given time constraints and the substantial work necessary to gather and simplify text and work with subject-matter experts to approve simplified texts. ultimately, we will perform future research-using this dataset-to specifically identify which simplifications were acceptable and unacceptable (at the lexical item-, sentenceand paragraph-level), thus informing admissions practitioners as to how text can or cannot be simplified for students and prospective students, broadly. we hope psat is a first step towards automatic simplification systems for fairer access to higher education.