The Use Of Quantitative And Qualitative Measures To Evaluate A Summer Camp For Elementary School Girls

In June 2000, Mercer University hosted Mercer MESSAGE, a summer camp for elementary school girls. Mercer MESSAGE is part of the Central South Summer Camp Program. The program consists of four essential elements: 1) Mercer MESSAGE, an all-girl summer math and science day camp, 2) Mercer TECH, a co-ed summer engineering day camp, 3) Central South Summer Camp reunions during the school year, and 4) continued communication with campers through e-mail or phone. The program targets 5, 6 and 7 grade girls and features a tiered-mentoring process that is designed to match campers with high school students, college students, and college faculty who excel in science, mathematics and/or engineering. The mentors work in both camps and continue their mentoring activities throughout the following school year. The Central South Summer Camp evaluation plan, which includes qualitative and quantitative measures, will be used to assess program effectiveness and to help determine if girls who participate in this program have developed a sustained interest in math, science or engineering. Introduction It is commonly accepted that we need to attract more women and minorities to the fields of math, science, and engineering. Although women and minorities are increasing their presence in some areas of science and math, their overall representation is inadequate. The problem is especially evident in the field of engineering. In spite of the fact that women make up approximately half of the workforce, less than 10% of practicing engineers are women. Clearly, we need to attract, and retain, more women in the scientific workforce. There is some reason to believe that part of the problem can be traced to our school system. Even in the early school years, many teachers believe that boys are better at math and science than girls, even though the girls earn better grades in these subjects. 3 In fact, girls outperform boys on national tests in nearly every subject at the elementary school level. However, in middle school and high school years, the test scores of girls decline dramatically relative to test scores of boys, especially in math and science. 3,4 Targeting girls before the middle school decline may be an effective way to prevent this trend. 5 A number of authors suggest that mentoring can help retain women in math, science and engineering. Successful mentoring has been demonstrated at the high school or college level. 6,7,8 Others have found mentoring to be an important factor in keeping women in the workforce. P ge 6.046. 1 Proceedings of the 2001 American Society for Engineering Education Annual Conference & Exposition Copyright  2001, American Society for Engineering Education With the increasing availability of the Internet, mentoring through the use of e-mail is becoming more prevalent. Another recognized way to enhance and reinforce math and science skills is through outreach programs or summer camps. Programs that have been implemented may be relatively short, perhaps one or two days; other programs are one or two weeks long. 13,14 Some programs, especially those that involve undergraduate research, last for six weeks or more. Many of these programs report anecdotal success stories, but few have reported the use of quantitative measures to evaluate program effectiveness. Notable exceptions include Arizona State University for outreach programs and MentorNet for e-mentoring . The Central South Summer Camp Program Mercer University has received funding to deliver a program titled, Central South Summer Math, Science, and Engineering Camps for Rising 5th, 6th, and 7th Graders. The Central South community is adjacent to the campus of Mercer University, and the project directors, Linda Hensel (biology), Hope McIlwain (math) and Joan Burtner (engineering) are all faculty members at Mercer University. The 2000-2001 program sponsors are: the American Honda Foundation, the American Association of University Women, the Mercer Center for Community Development, and the Bibb County Public School System. The program has the following goals: • establish annual summer math, science, and engineering day camps for girls and boys from the Central South neighborhood • develop curricula that can be tested in the camp by teachers and then be integrated into the public school curriculum • create tiered mentoring networks among the camp attendees, high school students, and college students that can be maintained throughout the school year • expose college math, science, and engineering students with a possible interest in teaching to the camp format and teachers from the public school system • expose the high school mentors to college math, science, and engineering students as a mechanism for informal mentoring Beginning in June 2000, the project directors piloted two academically focused summer day camps for talented, minority children from the Central South neighborhood. One camp (Mercer MESSAGE) focuses on science and math skills and is offered to rising fifth and sixth grade girls. MESSAGE is an acronym for Math, Engineering, and Science Summer All Girl Experience. Linda Hensel and Hope McIlwain co-directed Mercer MESSAGE. The other camp (Mercer TECH) focuses on engineering and technology skills and is offered to rising sixth and seventh grade boys and girls as well as some of their teachers. TECH is an acronym for Teachers Educating with Computers and Hands-on Lessons. Joan Burtner directed Mercer TECH. The two camps include a number of common features: P ge 6.046. 2 Proceedings of the 2001 American Society for Engineering Education Annual Conference & Exposition Copyright  2001, American Society for Engineering Education • Emphasis on science, math, and technology • Planned and directed by three women college faculty members (one each from science, math, and engineering) • Focus on academically qualified public school students from underrepresented minorities • Tiered mentoring through the use of talented minority high school and college students • Goal to reinforce the idea that studying science, math, and engineering is fun and interesting • Use of active pedagogy that includes field trips, oral presentations, and the use of computers in the classroom • Curriculum planning that involves college faculty and public school teachers. Program Features Tiered Mentoring The entire program depends on the active involvement of a tieredmentoring team. One high school student and one college student are assigned to mentor a group of four campers. The high school mentors are scientifically talented female students at Southeast and Southwest High Schools. The college students are male or female science, math, and engineering majors from Mercer University. The college mentors were selected using the criteria that Mercer University uses to select orientation assistants for entering freshman. High school mentors were selected by similar criteria. Mentors received training prior to the first camp offering. In addition, mentors are expected to maintain periodic contact with camp participants (and each other) throughout the school year. Camp Reunions To facilitate this contact, three one-day-long camp reunions were scheduled. At the time of this writing, two of the three reunions have been held. The first reunion was held on the Mercer campus on September 30, 2000. The camp directors, campers, mentors and public school teachers all attended. The second reunion was held on November 18, 2000. For this reunion, the camp directors, campers, mentors, and public school teachers traveled to Chattanooga, Tennessee, to visit the Aquarium and the Children's Museum. The third reunion is scheduled for March 2001. E-mail Contact The use of e-mail as a means of communication between mentors and campers is encouraged. With funds provided by the Georgia Lottery System, each of the participating public schools is equipped with computers and e-mail capability. Campers, mentors, and public school teachers involved in the project receive training in the use of e-mail on an as-needed basis. Parents sign a release allowing their children to set up email accounts. Of course, if the parents do not want to authorize the e-mail account, the Page 6.046. 3 Proceedings of the 2001 American Society for Engineering Education Annual Conference & Exposition Copyright  2001, American Society for Engineering Education camper can still participate in the camp program; the only consequence is that the camper is excluded from the e-mail activities. Evaluation Team Most of the evaluation components are conducted by a three-person evaluation team. The evaluation team consists of Joan Burtner (Director of Mercer TECH), Theresa May (elementary school teacher) and Marcus Lee (engineering student). The program developers believe that the use of a three-person team enhances the potential for program improvement on a multi-perspective level. Program Evaluation Plan The program evaluation has been designed using guidelines developed especially for science, math, and engineering education. 15,16 Since the program is so new, much of the evaluation team’s efforts during the first three years of the plan are focused on program improvement. The evaluation team is gathering information regarding the value of the instructional components of the program as well as the development of the mentoring program. The formative evaluation can be divided into two separate areas. The implementation evaluation will answer the question, “Is the program being delivered as planned? ” The progress evaluation will answer the question, “What is the impact of the activities? ” Throughout the formative evaluation phase, data will be analyzed and discussed and appropriate changes will be made to the Mercer MESSAGE and Mercer TECH programs. The plan may be characterized as a mixed-mode evaluation. Qualitative measures such as focus groups and journals comprise an essential part of the evaluation program. Although qualitative measures are m