NLP in Engineering Education - Demonstrating the use of Natural Language Processing Techniques for Use in Engineering Education Classrooms and Research

s and excluded those that seemed irrelevant to the scope of this review. For example, the paper by Zhu, Yan, and Song (2016) had both the terms “Natural Language Processing” as well as “education” in the abstract, however, this was because the paper described the evolving academic landscape through faculty hiring data, and these were specializations of the faculty. This article, and similar work irrelevant to the scope of this review were excluded in this step. Articles with incomplete or ambiguous abstracts, however, made it to the next step. Finally, in the fifth step, I conducted Appraisal through a full-text analysis of the articles that had passed step four. For each article I assigned a quality score based on whether the article was relevant to my research questions (Relevant = 1, Not =0) and whether or not the authors followed methodological transparency in describing their study (Methodologically transparent = 1, Not = 0). Thus, articles could attain scores of 0 through 2. Articles with scores less than 2 were discarded (i.e., were either not relevant, or were not methodologically transparent). After the full text appraisal, only studies that met all inclusion criteria were included as part of the review. In terms of relevance, for example, only studies that described how Natural Language Processing techniques were 28 implemented in specific education contexts were included. This led to exclusion of studies like the one by Azam (2011) which addressed the scope of Natural Language Processing in education, but was a literature review on the issues and challenges related to speech generation by intelligent systems. Similarly, in terms of methodological transparency, it was important that the articles had methodological transparency while describing the Natural Language Processing techniques. If the articles did not elaborate on how or what techniques were used, but rather provided an example of a black-box software, it was excluded from the review. An example is the article by Zeng-Treitler et al. (2014) in which the authors describe the use of a computer application called Glyph to convert text to set of illustrations to provide pictures for health communication. While the context was described adequately, as were the evaluation of the results, because there were no details provided for the Natural Language Processing techniques themselves, this study was excluded from my review. After full text appraisal, I found that I was left with 19 articles for inclusion in my review. I then conducted snowball referencing (Borrego et al., 2014) on the 19 articles in my dataset, and included 15 more articles which were referenced by the original 19 articles found through the database Search-Screen-Appraise process. All of these 15 additional articles passed all my inclusion criteria through full text appraisal. Thus, a total of 34 articles were finalized for inclusion in my review. 2.2. 4 Demographics of Included Publications The articles included in the syntheses were coded for demographic information related to the year of publication, the source of the article (i.e., conference or journal), and 29 whether or not it was used in an Engineering Education context. An understanding of these demographic details for the articles included in the review can help us think about the interest among researchers on this topic (as evidenced through a consistency in the number of publications per year) as well as provide those who may be interested in exploring similar techniques with an idea about the venues these are published in (i.e., predominantly conferences as opposed to journals). 2.2. 4.1 Year of publications. As mentioned in an earlier section, I had enforced a date restriction to only include studies published since 2010 for this synthesis. Figure 4 shows the graph of the number of publications for each year in the eight years included in this review. Note that since all the publications from 2017 will probably not be indexed until early 2018, the numbers for 2017 are incomplete, and hence not insightful. We can see from the data that the number of publications were more or less consistent across the years, which can be interpreted as a continued interest among Manuscript One. Fig. 4: Depicting number of articles per year considered