Natural Language Processing for Computer Scientists and Data Scientists at a Large State University (2021)

Abstract: The field of Natural Language Processing (NLP) changes rapidly, requiring course offerings to adjust with those changes, and NLP is not just for computer scientists; it’s a field that should be accessible to anyone who has a sufficient background. In this paper, I explain how students with Computer Science and Data Science backgrounds can be well-prepared for an upper-division NLP course at a large state university. The course covers probability and information theory, elementary linguistics, machine and deep learning, with an attempt to balance theoretical ideas and concepts with practical applications. I explain the course objectives, topics and assignments, reflect on adjustments to the course over the last four years, as well as feedback from students.

## introduction thanks in part to a access to large datasets, increases in compute power, and easy-to-use programming programming libraries that leverage neural architectures, the field of natural language processing (nlp) has become more popular and has seen more widespread adoption in research and in commercial products. on the research side, the association for computational linguistics (acl) conference-the flagship nlp conference-and related annual conferences have seen dramatic increases in paper submissions. for example, in 2020 acl had 3,429 paper submissions, whereas 2019 had 2,905, and this upward trend has been happening for several years. certainly, lowering barriers to access of nlp methods and tools for researchers and practitioners is a welcome direction for the field, enabling researchers and practitioners from many disciplines to make use of nlp. it is therefore becoming more important to better equip students with an understanding of nlp to prepare them for careers either directly related to nlp, or which leverage nlp skills. in this paper, i reflect on my experience setting up and maintaining a class in nlp at boise state university, a large state university, how to prepare students for research and industry careers, and how the class has changed over four years to fit the needs of students. the next section explains the course objectives. i then explain challenges that are likely common to many university student populations, how nlp is designed for students with data science and computer science backgrounds, then i explain course content including lecture topics and assignments that are designed to fulfill the course objectives. i then offer a reflection on the three times i taught this course over the past four years, and future plans for the course. ## course objectives most students who take an nlp course will not pursue a career in nlp proper; rather, they take the course to learn skills that will help them find employment or do research in areas that make use of language, largely focusing on the medium of text (e.g., anthropology, information retrieval, artificial intelligence, data mining, social media network analysis, provided they have sufficient data science training). my goal for the students is that they can identify aspects of natural language (phonetics, syntax, semantics, etc.) and how each can be processed by a computer, explain the difference between classification models and approaches, be able to map from (basic) formalisms to functional code, and use existing tools, libraries, and data sets for learning while attempting to strike a balance between theory and practice. in my view, there are several aspects of nlp that anyone needs to grasp, and how to apply nlp techniques in novel circumstances. those aspects are illustrated in figure 1 . no single nlp course can possibly account for a level of depth in all of the aspects in figure 1 , but a student who has taken courses or has experience in at least two of the areas (e.g., they have taken a statistics course and have experience with python, or they have taken linguistics courses and have used some data science or machine learning libraries) will find success in the course more easily than those with no experience in any aspect. this introduces a challenge that has been explored in prior work on teaching nlp @xcite : the diversity of the student population. nlp is a discipline that is not just for computer science students, but it is challenging to prepare students for the technical skills required in a nlp course. moreover, similar to the student population in @xcite , there should be course offerings for both graduate and undergraduate students. in my case, which is fairly common in academia, as the sole nlp researcher at the university i can only offer one course once every four semesters for both graduate and undergraduate students, but also students with varied backgroundsnot only computer science. as a result, this is not a research methods course; rather, it is more geared towards learning the important concepts and technical skills surrounding recent advances in nlp. others have attempted to gear the course content and delivery towards research @xcite giving the students the opportunity to have open-ended assignments. i may consider this for future offerings, but for now the final project acts as an open-ended assignment, though i don't require students to read and understand recent research papers. in the following section, i explain how we prepare students of diverse backgrounds to succeed in an nlp course for upper-division undergraduate and graduate students. ## preparing students with diverse academic and technical backgrounds boise state university is the largest university in idaho, situated in the capital of the state of idaho. the university has a high number of non-traditional students (e.g., students outside the traditional student age range, or second-degree seeking students). moreover, the university has a high acceptance rate (over 80%) for incoming first-year students. as is the case with many universities and organizations, a greater need for "computational thinking" among students of many disciplines has been an important driver of recent changes in course offerings across many departments. moreover, certain departments have responded to the need and student interest in machine learning course offerings. in this section, we discuss how we altered the data science and computer science curricula to meet these needs and the implications these changes have had on the nlp course. foot_0 data science the data science offerings begin with a foundational course (based on berkeley's data8 content) that has only a very basic math prerequisite. 2 it introduces and allows students to practice python, jupyter notebooks, data analysis and visualization, and basic statistics (including the bootstrap method of statistical significance). several courses follow this course that are more domain specific, giving the students options for gaining practical experience in data science skills relative to their abilities and career goals. one path more geared towards students of stem-related majors (though not targeting computer science majors) as well as some majors in the humanities, is a certificate program that includes the foundational course, a follow-on course that gives students experience with more data analysis as well as probability and information theory, an introductory machine learning course, and a course on databases. the courses largely use python as the programming language of choice. computer science in parallel to the changes in data science-related courses, the department of computer science has seen increased enrollment and increased request for machine learningrelated courses. the department offers several courses, though they focus on upper-division students (e.g., artificial intelligence, applied deep learning, information retrieval and recommender systems). this is a challenge because the main computer science curriculum focuses on procedural languages such as java with little or no exposure to python (similar to the student population reported in freedman (2008)), forcing the upperdivision machine learning-related courses to spend time teaching python to the students. this cannot be underestimated-programming languages may not be natural languages, but they are languages in that it takes time to learn them, particularly relevant python libraries. to help meet this challenge, the department has recently introduced a machine learning emphasis that makes use of the data science courses mentioned above, along with additional requirements for math and upper-division computer science courses. preqrequisites unlike agarwal (2013) which attempted and introductory course for all levels of students, this course is designed for upper-division students or graduate students. students of either discipline, computer science or data science, are prepared for this nlp course, albeit in different ways. computer scientists can think computationally, have experience with a syntactic formalism, and have experience with statistics. data science students likewise have experience with statistics as well as python, including data analysis skills. though the backgrounds can be quite diverse, my nlp course allows two prerequisite paths: all students must take a statistics course, but computer science students must take a programming languages course (which covers context free grammars for parsing computer languages and now covers some python programming), and the data science students must have taken the introductory machine learning course. figure 2 depicts the two course paths visually. ## nlp course content in this section, i discuss course content including topics and assignments that are designed to meet the course objectives listed above. woven into the topics and assignments are the themes of ambiguity and limitations, explained below. ## conclusions & open questions with each offering, the nlp course at boise state university is better suited pedagogically for students with some data science or computer science training, and the content reflects ongoing changes in the field of nlp to ensure their preparation. the topics and assignments cover a wide range, but as students have become better prepared with python (by the introduction of new prerequisite courses that cover python as well as changing some courses to include assignments in python), more focus is spent on topics that are more directly related to nlp. though i feel it important to stay abreast of the ongoing changes in nlp and help students gain the knowledge and skills needed to be successful in nlp, an open question is what changes need to be made, and a related question is how soon. for example, i think at this point it is clear that neural networks are essential for nlp, though it isn't always clear what architectures should be taught (e.g., should we still cover recurrant neural networks or jump directly to transformers?). it seems important to cover even new topics sooner than later, though a course that is focused on research methods might be more concerned with staying upto-date with the field, whereas a course that is more focused on general concepts and skills should wait for accessible implementations (e.g., huggingface for transformers) before covering those topics. with recordings and updated content, i hope to flip the classroom in the future by assigning readings and watching lectures before class, then use class time for working on assignments. 11 11 this worked well for the foundations of data science course that i introduced to the university; the second time i much of my course materials including notebooks, slides, topics, and assignments can be found on a public trello board. 12