Category: education

## **LEARNING SIMULATION THROUGH TEAM PROJECTS** 

Department of Industrial and Operations Engineering 1205 Beal Avenue The University of Michigan Ann Arbor, MI 48109, U.S.A. 

## **ABSTRACT** 

For several years, team projects have been an integral part of the simulation course at the department of Industrial and Operations Engineering at the University of Michigan.  We believe that team projects are an effective tool for learning how to perform simulation.  In this paper, we present a brief summary of research on cooperative learning from the field of Education Research.  Based on findings from that research, we present the procedure we follow in assigning, running, and evaluating team projects during an academic semester.  We analyze students‘ responses to a survey on their preference and perceived value of the team project as conducted in this course. 

Two student papers, published in this conference, provide for examples of completed team projects. 

## **1 INTRODUCTION** 

“Students who learn in small groups generally demonstrate greater academic achievement, express more favorable attitudes toward learning, and persist through SMET [Science, Mathematics, Engineering and Technology] courses or programs to a greater extent than their more traditionally taught counterparts” (Springer, Stanne, and Donovan, 1998).  Cooperative learning, a type of active learning, is an instructional paradigm in which teams of students work collaboratively to accomplish a common goal.  Several conditions for a successful experience have been identified by Johnson, Johnson and Smith, 1991, 1998) (Table 1). 

Based on this instructional paradigm, Felder and Brent (1994) described general procedures for conducting successful cooperative learning in technical courses.  For example, they suggested that positive interdependence between team members could be promoted by requiring a single final product from each team, assigning rotating group roles, and evaluating the team’s performance based on a single (or the weakest) team member’s performance. This method insures that students have an incentive to help 

Omer Tsimhoni Changxu Wu 

each other within teams.  The complementary element of promoting individual accountability may be achieved by collecting ratings from students on the efforts made by their teammates and by adjusting individual grades accordingly.  For practical material to be used in creating a curriculum along with an excellent discussion of effective team work, see Oakley, Felder, Brent, and Elhajj, 2004. 

Table 1:  Conditions for successful cooperative learning (Johnson, Johnson, and Smith, 1991) 



<!-- Start of picture text -->
||||||
|---|---|---|---|---|
|1.|Positive interdependence|
|Team members rely on each other to achieve the final goal|
|2.|Individual accountability|
|Each team member is held accountable for the entire pro-|
|ject|
|3.|Face-to-face promotive interaction|
|At least some of the team work has to be done by interac-|
|tion during|a face-to-face|(not|virtual) meeting|
|4.|Appropriate use of collaborative skills|
|Students are encouraged and helped to develop collabora-|
|tive|skills|
|5.|Group processing|
|Team members assess how well they are doing and iden-|
|tify|changes|they|can make|to improve|

<!-- End of picture text -->

There is a large and rapidly growing body of research on the effectiveness of cooperative learning, which can be readily applied to higher education in general and the teaching of simulation in specific.  Although team projects have been used in simulation courses for many years (e.g., Banks, Carson, Nelson, and Nicol, 2001 p. 324), to the best of our knowledge there are no published papers on applying the theory of cooperative learning to simulation team projects. 

The goal of the present paper is to highlight aspects of cooperative learning that can be applied to the learning and teaching of simulation.  We provide an example from a simulation course we teach at the Department of Industrial and Operations Engineering at the University of Michigan. 

We use this example to illustrate some of the steps we believe are helpful in conducting team projects during an academic semester.  We also provide an analysis of a survey of students’ preference and perceived value of the team project as taught in this course. 

Two student papers, (Curin et al, 2005 and Roggenkamp et al., 2005), published in this conference, provide for examples of completed team projects. 

## **2 SETTING UP A TEAM PROJECT** 

The minimal set of required steps in setting up a team project includes the selection of teams, assigning a clear project description, and collecting a final report.  There are, however, several other steps that can provide additional guidance to the students and further their learning experience. 

In setting the team project, the five conditions for successful cooperative learning (Table 1) were considered. 

_Positive interdependence._ The task assignment, the interaction of instructors with students, and the grading scheme are set such that team members must rely on each other to achieve the final goal. 

_Individual accountability._ Project presentations and progress updates are made by each team member so that all individuals feel accountable for the outcome. 

_Face-to-face promotive interaction._ A few sessions are held during laboratory time to encourage face-to-face interaction between students on their projects. 

_Appropriate use of collaborative skills._ Collaborative skills and approaches are discussed in class to encourage students to reflect on their roles in their teams and to collaborate better. 

_Group processing._ Written and oral feedback is given by the students as an assessment of how they are doing and in attempt to promote positive criticism. 

Several additional criteria were considered in structuring the steps for our team project, as shown below: 

_Inform the students about other simulation projects_ . By reading about simulation projects in proceedings and by listening to presentations of other teams, students get ideas on how to perform their simulation project.  By evaluating the work of other teams, students learn to actively apply their knowledge. 

_Share reports and evaluations online with all the class._ Evaluations and reports are posted online close to the time of submission and are available to all enrolled students.  This method tends to promote a serious attitude to submissions because of individual accountability. 

_Give the students free choice as much as possible_ . Students are allowed to select their teammates and to select the topic of their simulation.  Students roles within the team are self selected.  It is noted that this criteria contradicts some of the advice given by some education experts (i.e., there are some educational advantages to team selec- 

tion by the Professor and to the assignment of rotating roles in the team).  We are nevertheless convinced that students are generally very motivated because they are the ones that make the choice and it is this motivation that justifies the departure from the guidelines mentioned above. 

_Set frequent checkpoints for feedback about the progress of the project_ .  These checkpoints serve as targets for the completion of certain parts of the project.  They also help the instructors keep in touch with the progress of individual teams and provide guidance to teams that need it. 

## **3 STRUCTURE OF OUR TEAM PROJECT** 

Table 2 lists the steps that are currently followed in our simulation class and the week at which they are typically performed.  This list of steps is a result of an evolution through several years in which the course has been taught by several Professors.  The list continues to change from year to year and from Professor to Professor.  What is shown is a snapshot from the latest semester. 

Table 2:  Components of Team Project 

||Table 2:  Components of Team Project|
|---|---|
|Week|<br>Component|
|2|Analysis of a simulation paper (from WSC<br>proceedings)|
|2|Eachstudentsubmits a one-page bio|
|2-4|Demonstration of simulation projects from<br>previous terms|
|4|Projectassigned|
|4|Selfselectionof teams|
|6|Pre-proposaldue|
|7|Feedback on draft proposal provided in writ-<br>ing|
|8|Proposaldue|
|10|In-class discussion of project goals, concep-<br>tual model, and data collection|
|10|Students’ feedback for each project is posted<br>online|
|11|Effort report 1ofselfand other team members|
|10-13|Meeting with teams to answer questions and<br>provide guidance|
|13|Project presentation and evaluation in lab with<br>focus on input analysis, conceptual model,<br>andmodel implementation|
|16|Students’ evaluations of all projects are posted<br>online|
|16|Draft final report due|
|17|Final report due in WSCproceedings format|
|17|Effort report 2 of self and other team members|



## **3.1 Analysis of a Simulation Paper** 

The team project begins implicitly with the first homework assignment of the semester.  Students are asked to find in the WSC proceedings an article about an applied simulation pro- 

ject.  They then prepare a report on how it accomplished the standard steps in a simulation study (Banks et al., 2000).  The primary purpose of the assignment is to learn the steps of a simulation study.  A secondary purpose is to familiarize the students with the format of a conference proceedings and to get them thinking about possible topics of interest to them. 

## **3.2 Biographical Form** 

Each student submits a one-page biographical form which includes their picture, a brief description of their interests, a description of their possible contribution to team work, and their availability for team meetings.  A compilation of all the forms is then posted on the class website (available only for enrolled students).  Students get to know each other in preparation for forming teams. 

## **3.3 Demonstration of Simulation Projects from Previous Terms** 

Simulation projects from previous terms are demonstrated and briefly discussed as part of several lectures.  Students learn the scope of the project by example.  Seeing some of the animations is very appealing and motivating for students.  As a downside, there is a tendency of students to be disappointed if a topic that is of interest to them “has already been taken” by teams from previous terms. 

## **3.4 Project Assigned** 

The project is assigned about a month into the semester.  A detailed description of the goals, deliverables, and grading of the project is discussed in class.  Suggested guidelines for team selection are discussed as well. 

opinion between team members.  Third, it is difficult to think through all of the limitations of a project at such an early stage in the course.  Fourth, there is a tendency to avoid simulating the same topic as other teams.  The following criteria for a good topic are suggested: 

1. the topic and project goal are interesting academically, or practically useful, or both 

2. the system of choice can be modeled by discrete event simulation and cannot be solved analytically.  Follow a checklist “when simulation is not appropriate” on Bank, et al. 2001 p. 5 

3. the system venue is publicly accessible for data collection or the data already exist and are available (to conform with IRB requirements) 

4. team members are enthusiastic about this topic 

Feedback and ideas for improvement are given in writing and in person based on the submitted pre-proposal. 

## **3.7 Proposal** 

In the full proposal, students describe the situation they plan to simulate, the data they plan to collect and how, the questions they plan to answer, the conceptual model and the assumptions that underlie it. 

The submission of the proposal is accompanied by a submission of a skeleton of the final report in the format of the WSC conference.  The minimum required for submission is the title and authors names but some students will add the beginning of an introduction and some headers for the final report. 

## **3.8 In-Class Discussion of Project Goals, Conceptual Model, and Data Collection** 

## **3.5 Self Selection of Teams** 

Students can form teams of three or four students based on their familiarity with other students from some activities conducted in the first few classes, as well as from the Bio forms submitted earlier.  Teams are not to be formally declared until the submission of the first proposal draft, but most are formed within the first few days after the project is assigned. 

## **3.6 Topic Selection and Pre-Proposal** 

The pre-proposal consists of a one-page description of the selected topic along with the questions to be addressed.  Students are asked to provide specifics on the venue of the simulation and possible contacts they might involve in the project. 

Selecting a topic for simulation is probably the most demanding and crucial decision in conducting the project. First, it is difficult to find a perfect topic among the many existing options.  Second, there are usually differences of 

During one of the lectures, each student presents to a group of several students from other teams his or her team’s proposal and their initial progress.  This task promotes individual accountability because each student has to present their team’s progress by themselves. 

During the presentations, each student provides feedback to the presenter.  Some feedback and evaluation in writing is submitted at the end of the lecture and immediately posted online as a scanned pdf document.  The evaluations have worked fine, but they have not been very useful to most students.  The feedback that students can provide is limited by their knowledge of simulation at that point.  Nevertheless, there is still a lot to learn from writing and reading these evaluations. 

## **3.9 Effort reports** 

Effort reports are very useful to the instructors as they provide information about the effort and performance of each 

team member and thus allow differentiation between grades if necessary.  Two effort reports are submitted by students following a procedure suggested by Kaufman, Felder, Fuller (2000).  Students report their teammates’ effort in preparing the project.  An adjustment of up to 10% of the grade is possible if there is agreement by the team members that one of the students has not been helpful, or that one of the students has been making extra effort. 

## **4 POST EVALUATION OF TEAM PROJECTS BY STUDENTS** 

To learn of students’ preference and opinion,a survey was posted to students in the class of Winter 2005.  Questions A and B were answered on a 5-point scale and questions C, D, and E were answered in free text.  The questions were: 

- A.  Overall, I liked having a team project in this course 

## **3.10 Meeting with Teams to Answer Questions and Provide Guidance** 

- B.  The team project helped me learn how to perform a simulation study 

- C. Why did you like the team project 

During the team project students meet with the instructors to discuss their projects.  The meetings are currently arranged per need.  We are considering to require at least one thirty-minute meeting with each team. 

## **3.11 Project Presentation and Evaluation in Lab** 

Each student presents the project to three students from other teams.  The presentation is made in a computer laboratory so that simulation models can be demonstrated and discussed during the presentation.  The focus of the presentation is on input analysis, the conceptual model, and the initial model implementation 

After the presentations, each student writes an evaluation for one of the presentations they heard.  The evaluation is graded for level of detail and thoroughness regardless of the quality of the evaluated project.  The evaluator has an incentive to submit a thorough analysis. 

All evaluations are posted on the class website close to their time of submission.  This serves several purposes. First, immediate feedback is available for all team members on how their project was presented by their teammates and received by other students.  Second, students can view the evaluations of other projects and learn about them.  Third, students can compare their submission to those of other students and learn how to improve future submissions. 

## **3.12 Final Report** 

A draft of the final report is submitted a week before the final report is due so that feedback can be provided by the instructors and so that sufficient time is allotted to writing.  The format of the report follows the format suggested in the WSC author kit.  The idea of an academic report rather than a lengthy technical report is appealing because it is shorter and therefore requires more analysis and synthesis.  In addition, we hope it may be useful for some of the students in future publications at conferences through academia or industry. The final report is accompanied by a running package of the simulation model and optionally a movie file representing a typical model run (for later demonstration purposes).  Data files and statistical analysis files are submitted as well. 

- D.  Why did you not like the team project 

- E.  If you had to change one or two things about the team project what would they be? 

As shown in Figures 1, a vast majority of the students (30 of 36) agreed or almost agreed with the statement: “Overall, I liked having a team project in this course.” 

| Response | Percent of students |
| --- | --- |
| 1 | ~5% |
| 2 | ~10% |
| 3 | ~5% |
| 4 | ~55% |
| 5 | ~25% |

Figure 1: Question A.  Overall, I liked having a team project in this course. 

As shown in Figure 2, most students (32 of 36) agreed or almost agreed with the statement: “The team project helped me learn how to perform a simulation study.” 

| Response | Percent of students |
| --- | --- |
| Disagree | 10% |
|  |  |
|  |  |
| Agree | 35% |
|  | 55% | 
|  |  |

Figure 2: Question B.  The team project helped me learn how to perform a simulation study. 

Free text answers for question C, D, and E were grouped by similarity.  Tables 3, 4, and 5 show a ranking of the rea- 

sons for liking the team project, not liking the team project, and suggestions for changes, respectively. 

|Table 3:Reasonsfor Liking theTeam Project|Table 3:Reasonsfor Liking theTeam Project|Table 3:Reasonsfor Liking theTeam Project|
|---|---|---|
|Rank|Responses|WhyI liked the teamproject|
|1|15|The team projects allows me to apply<br>the knowledge learned in the class to<br>the real world|
|2|12|I Enjoy the collaboration and learn-<br>ingfrom others|
|3|10|Theproject helped me learn better|
|4|3|Project beneficial for my personal<br>career andjob interviews|



|Table4:ReasonsforNOT Liking theTeam Project|Table4:ReasonsforNOT Liking theTeam Project|Table4:ReasonsforNOT Liking theTeam Project|
|---|---|---|
|Rank|Responses|Why I did not like the team pro-<br>ject|
|1|18|Team work was too time con-<br>suming|
|2|5|Difficult to coordinate the work<br>and be efficient|
|3|2|It was difficult to select an ap-<br>propriate system to simulate|



|Table 5:ThingsIWould ChangeAbout theTeam Project|Table 5:ThingsIWould ChangeAbout theTeam Project|Table 5:ThingsIWould ChangeAbout theTeam Project|
|---|---|---|
|Rank|Responses|WhatIwould change|
|1|3|Iprefera smaller teamsize|
|2|2|I would equalize the level of diffi-<br>cultyacross teams|
|3|1|I prefer frontal presentations rather<br>thanpresentation in smallgroups|



## **5 DISCUSSION OF STUDENT SURVEYS** 

The post evaluation of team project by students suggests an overwhelming agreement with the educational purpose of team projects and a similar tendency to like the team project.  Among the primary reasons for liking the team project are the ability to apply knowledge to real world problems, the ability to collaborate and meet other students, and the different way of learning. 

It is not surprising that not all students liked the team project. Team projects are time consuming and usually conflict with other time demands in this course and others. Furthermore, several students conveyed their preference for small teams (smaller than 4).  Large teams are hard to coordinate and may lead to specialization of roles that may result in discontent. 

## **6 CONCLUSION** 

If implemented properly, and on the basis of research on cooperative learning, team projects can be an excellent tool for learning simulation.  The ability to apply knowledge to real-world applications and to solve real problems is very motivating and results in a good learning experience. 

Working in teams allows for tackling complex problems that would be too difficult for an individual to address.  By the same token, it is possible to perform a simulation project from start to end in a relatively short amount of time. 

Working in teams has several educational and social benefits.  It promotes learning from each other and brings out the individual strengths of different students.  Students are motivated and supported by their peers and get to learn by doing rather than by listening. 

As a secondary benefit, a team project of this scale teaches students about working collaboratively and coordinating their work.  This is a priceless lesson for their future as engineers working on real world problems under tight deadlines. 

Despite the advantages of the team project as a learning tool, there are several downsides that cannot be ignored.  First, team projects are time consuming and require a lot of overhead in coordination.  Second, they do not work well for everyone.  Third, they require a level of maturity from students, that does not always exist. 

As demonstrated by the student surveys, there is an overall agreement that team projects are useful but there are still many valid complaints and suggestions for improvement. 

We are always learning from our students (through formal and informal methods) how to improve the structure of our team projects so that they can learn simulation even better. 

## **ACKNOWLEDGEMENTS** 

We would like to thank the Professors who taught this course in the past few years for planting the seed for many of the ideas presented in this paper. 

## **REFERENCES** 

Banks, J., Carson, J. S. II, Nelson, B. L., Nicol, D. M. 2001.  Discrete Event Simulation, Third Edition. Prentice Hall, Inc.  Upper Saddle River, New Jersey. Curin, S. A., Vosko, J. S., Chan, E. W. and Tsimhoni, O. 2005.  Reducing service time at a busy fast food restaurant on campus.  In _Proceedings of the 2005 Winter Simulation Conference,_ eds. M. E. Kuhl, N. M. Steiger, F. B. Armstrong, and J. A. Joines. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. 

- Felder, R.M., and Brent, R., 1994.  Cooperative Learning in Technical Courses: Procedures, Pitfalls, and Payoffs. ERIC Document Reproduction Service, ED 377038 

Johnson, D.W., Johnson, R.T., and Smith, K.A. 1991.  Cooperative Learning: Increasing College Faculty Instructional Productivity, ASHE-ERIC Higher Education Report No. 4, George Washington University 

- Johnson, D.W., Johnson, R.T., and Smith, K.A. 1998. _Active learning: Cooperation in the College Classroom_ . Edina, MN: Interaction Book Co. 

- Kaufman, D.B., Felder, R.M., and Fuller, H. 2000 Accounting for Individual Effort in Cooperative Learning Teams. _J. Engr. Education, 89_ (2), 133-140 (2000). 

- Oakley, B., Felder, R.M., Brent, R., and Elhajj, I. 2004. Turning Student Groups into Effective Teams." _J. Student Centered Learning, 2_ (1), 9–34 

- Roggenkamp, D., Park, D., and Tsimhoni, O. 2005.  A simulation model for facilitators of Tony Rizzo’s bead game.  In _Proceedings of the 2005 Winter Simulation Conference,_ eds. M. E. Kuhl, N. M. Steiger, F. B. Armstrong, and J. A. Joines. Piscataway, New Jersey: Institute of Electrical and Electronics Engineers. 

- Springer, L., Stanne, M. E., and Donovan, S. 1998. Effects of cooperative learning on undergraduates in science, mathematics, engineering, and technology: A metaanalysis. (Research Monograph No. 11). Madison: University of Wisconsin-Madison, National Institute for Science Education, _Review of Educational Research_ 

## **AUTHOR BIOGRAPHIES** 

**OMER TSIMHONI** is an Adjunct Assistant Professor at the Department of Industrial Engineering of the University of Michigan.  He received his Ph.D. from the University of Michigan in 2004.  He is also a Research Fellow at the University of Michigan Transportation Research Institute (UMTRI).  His primary research interest is in the application of simulation to cognitive modeling and human factors.  His e-mail address is <omert@umich.edu> 

**CHANGXU WU** is a Ph.D. student at the Department of Industrial Engineering of the University of Michigan. He received his M.S. from the University of Michigan in 2004. His primary research interest is in the area of modeling of human cognition and performance with simulation and mathematics. His email address is <changxuw@umich.edu> 



Question: In the University of Michigan Industrial and Operations Engineering simulation course team-project study, what conditions from cooperative-learning research did the authors emphasize, and how did Winter 2005 students rate liking the team project?
Answer: The authors structure projects around Johnson, Johnson, and Smith’s five conditions for successful cooperative learning: positive interdependence, individual accountability, face-to-face promotive interaction, appropriate use of collaborative skills, and group processing. In a Winter 2005 survey, a vast majority of students—30 of 36—agreed or almost agreed that overall they liked having a team project in the course.