The Flipped Classroom model for teaching Conditional Random Fields in an NLP course (2021)

Abstract: In this article, we show and discuss our experience in applying the flipped classroom method for teaching Conditional Random Fields in a Natural Language Processing course. We present the activities that we developed together with their relationship to a cognitive complexity model (Bloom’s taxonomy). After this, we provide our own reflections and expectations of the model itself. Based on the evaluation got from students, it seems that students learn about the topic and also that the method is rewarding for some students. Additionally, we discuss some shortcomings and we propose possible solutions to them. We conclude the paper with some possible future work.

## introduction in this article we would like to provide experience on developing a flipped classroom environment in a natural language processing course. the most common approach of teaching involves a teacher "pouring" knowledge to its students, as if students were plants and knowledge was water. there is a tendency, though, to make classes more active and interactive, in which communication does not only happen from the teacher to the students, but also from student to student. bloom's taxonomy 1 (bloom et al., 1956; anderson et al., 2001) is a model that describes the cognitive load of different types of work or activities. on the one hand, some activities, such as remembering or understanding specific characteristics of, for instance, a model of any kind, would be considered to require a low cognitive load. on the other hand, evaluating which model is best by considering the characteristics of each, could be considered to be in a higher level of complexity in bloom's taxonomy. if we want our students to be more knowledgeable and reflective, we need to go beyond the lower levels in bloom's taxonomy. the traditional approach of introducing topics in a lecture would make students remember and understand the covered topics. we believe, though, that with these teaching practices there is a time limitation to go beyond the first two levels of bloom's taxonomy. a more efficient approach could be to ask students to work on a set of topics beforehand, get prepared, and then, work on activities that involve a higher cognitive load. these activities could involve applying and analyzing the acquired knowledge to other aspects. in this article, we present our experience in teaching conditional random fields as a flipped classroom. we teach this in a course at the master level about natural language processing (nlp). the article is structured as follows. first we introduce the flipped classroom method and some challenges. after that, we present the program, course and lecture in which the new teaching method will be used. then, we discuss the characteristics of the implied student and also the evaluation method that we use. later, we describe the lectures structure and the different activities and we classify them according to bloom's taxonomy. we then include some discussion, based on the experience. finally, we conclude the work and suggest some possible future directions. flipped classroom @xcite is a teaching approach in which students get first exposure to the material of a lecture outside of class, and the in-class activities involve applying the learned content. provision of lecture material can be done in a number of ways, such as reading material, video lectures as slideshows, podcasts, and so on. students are expected to do the homework, and the majority of the in-class activities fully depend on that. because of that, the teacher should make sure that students do their homework, because even though we name it in various ways, watching lectures or reading articles is still homework @xcite , and the challenge of making students accomplish with that is still there. a possible idea for making sure that students do the reading homework could be to ask them to do a quiz or reward them somehow. apart from challenges regarding students, we may not forget that all the activities, homework, readings, and so on have to be retrieved, selected or produced. furthermore, as the content covered in class is more complex, the teacher will have to be more prepared. all this results in a larger working load in the preparation of the class and its activities. ## background about program, course and lecture in this section, we briefly introduce the m. sc. program, the course and the specific lecture in which we will be focusing on. ## evaluation method in recent years, the usual teaching practice in the language processing series has been lectures. as the goal of this experiment is to check whether the flipped classroom can support students in their learning or not, we will implement this teaching method for the section about conditional random fields, and analyze how students feel about it. we evaluate this teaching practice by asking students to fill in a survey. in the survey we ask students about their general knowledge about some topics (memms, crfs, label bias problem) but also about whether they would be able to use crfs for their own work. please find below the questions that we asked in the questionnaire: the response to these questions could be either "yes", "roughly" or "no". finally, there are two questions related to the specific teaching method, in which we ask students whether the teaching practice is more demanding (mentally) and also whether it is more rewarding, compared to the lecturing approach. the survey was made one week before the lecture day and after the lecture was done. ## activities in this section we describe the activities that we made for students before the lectures, during the lectures and after the lectures. the activities will be made public, hoping that they will be useful for other nlp teachers and/or researchers. ## discussion and evaluation our expectation is that students will learn more, in a better way with a similar effort (from their side). we think that the reward (for students) of this teaching method is high, on the one hand. on the other hand, the required time for preparation (for teachers) is higher. the preparation time increase is directly related with the fact that the topics that are covered are expected to be more complex. as students already got lecturing as homework, then the in-class activities become more complicated, and it is because of this that the teacher must have a deeper understanding of the topic in question. besides, in our specific case the video lectures were made by us. we could save time by using the available resources in online learning platforms, but we decided to make them ourselves, making the preparation more timeconsuming. the evaluation, based on a survey that students had to answer, seems positive, although there are issues. considering how much students learned, we can say that the learning goals were satisfied, as it can be seen in figure 1 . each column in the plots represents the answers of our students to a question regarding knowledge on different concepts. they had to answer either yes, roughly or no. as we can see, students did not have much expertise on the topic before the lecture and preparation. afterwards, the responses show that students got the required understanding. besides, we also asked whether this teaching method is more rewarding. 7 out of 15 answered yes. there was one that answered no. finally, there were other 7 out of 15 people that did not take any stance, as they responded "i don't know". from these results we cannot strongly confirm that the method, in the way that we implemented it, is rewarding. it seems quite rewarding, though. together with these answers, we gave the students the option of writing further comments. there were some positive comments, and some others were possible issues with suggestions for improvements. one mentions that it is hard not to be able to ask questions when you watch the lecture, and that it is hard to remember the context of the question in class. a possible solution to this issue could be to include a discussion forum for each video lecture, so that students could post the question immediately in the forum. another student pointed out that homework distribution was far from being optimal, as all homework was given in the first week and in the second week there was nothing. this should definitely be thought in a way that the homework load is more balanced. on the bright side, it seemed positive to have the chance of watching again the lecture videos. also, some felt that group work was better and that it was nice to have more time to understand code and exercises. ## conclusion and future directions in this paper, we presented a possible class structure for teaching conditional random fields in almost two lectures. this class is formulated as a flipped classroom, in which there is a strong workload in the students' preparation and this allows students to get a further understanding of the topic, compared to traditional lecturing. we contemplate the flipped classroom as a relevant teaching method for teaching complex topics. we believe that by asking students to do part of the work beforehand has allowed us to go one step beyond in the understanding of crfs. furthermore, the exercise types that we covered were in the highest orders in bloom's taxonomy, showing the efficiency of the method. the feedback that we got from students seems to show that, in general, they learn about the topic in question. it further seems that the method is rewarding for some students. we believe that the methodology has advantages, e.g. students get a deeper understanding of the topic, and disadvantages, for instance a higher preparation time. considering both aspects, a possibility could be to find a balance between flipping only a portion of the whole lecture and having the other portion as a more traditional lecture. in our prepared lectures, we decided to emphasize on a problem that conditional random fields fix, the label bias problem. in the future, it would be interesting to include a discussion about the observation bias @xcite , which happens when the prediction is made by totally ignoring the labels. as students have a very varied background, we could already observe differences in the time of execution of the exercises. a possible solution to this is to apply differentiated teaching @xcite , where the teaching content is adjusted to some groups of students, making the activities more tailored to those student clusters.