Naive B ayes versus BERT : J upyter notebook assignments for an introductory NLP course (2021)

Abstract: We describe two Jupyter notebooks that form the basis of two assignments in an introductory Natural Language Processing (NLP) module taught to final year undergraduate students at Dublin City University. The notebooks show the students how to train a bag-of-words polarity classifier using multinomial Naive Bayes, and how to fine-tune a polarity classifier using BERT. The students take the code as a starting point for their own experiments.

## introduction we describe two jupyter 1 notebooks that form the basis of two assignments in a new introductory natural language processing (nlp) module taught to final year students on the b.sc. in data science programme at dublin city university. as part of a prior module on this programme, the students have some experience with the nlp problem of quality estimation for machine translation. they have also studied machine learning and are competent python programmers. since this is the first data science cohort, there are only seven students. four graduate students are also taking the module. the course textbook is the draft 3 rd edition of @xcite . 2 it is impossible to teach the entire book in a twelve week module and so we concentrate on the first ten chapters. the following topics are covered: the course is fully online for the 2020/2021 academic year. lectures are pre-recorded and there are weekly live sessions where students anonymously answer comprehension questions via zoom polls and spend 20-30 minutes in breakout rooms working on exercises. these involve working out toy examples, or using online tools such as the allennlp online demo foot_0 @xcite to examine the behaviour of neural nlp systems. assessment takes the form of an online end-ofsemester open-book exam worth 60% and three assignments worth 40%. the first assignment is worth 10% and involves coding a bigram language model from scratch. the second and third assignments are worth 15% each and involve experimentation, using google colab foot_1 as a platform. for both assignments, a jupyter notebook is provided to the students which they are invited to use as a basis for their experiments. we describe both of these in turn. ## notebooks we describe the assignment objectives, the notebooks we provide to the students foot_2 and the experiments they carried out. ## future improvements we surveyed the students to see what they thought of the assignments. on the positive side, they found them challenging and interesting, and they appreciated the flexibility provided in the third assignment. on the negative side, they felt that they involved more effort than the marks warranted, and they found the code in the notebooks to be unnecessarily complicated. the object-oriented nature of the code was also highlighted as a negative by some. for next year, we plan to 1) streamline the code, hiding some of the messy details, 2) reduce the scope of the assignments, and 3) provide more bert fine-tuning example notebooks.