Design and Evaluation of “Portfolio Practice I,”
a Course for Online Graduate Students
Junko NEMOTO
nemoto.junko.nu@ehiem-u.ac.jp
Ehime University
Matsuyama, Japan
Akiko TAKAHASHI
Tokushima University
Tokushima, Japan
Atsue TAKEOKA
Kochi University
Kochi, Japan
Hiroshi NAKANO
Kumamoto University
Kumamoto, Japan
Katsuaki SUZUKI
Kumamoto University
Kumamoto, Japan
Abstract: This paper describes our implementation of a new course entitled “Portfolio
Practice I” for online graduate students. After introducing the course design and presenting
our results, we examine, from system, reflective, and cognitive viewpoints, how the learners
responded to the course experience as part of the curriculum.
Key words: portfolio, showcase, learner perception, reflection

INTRODUCTION“Portfolio” is becoming a common term in higher education. The purpose and method of portfolio

use is varied. It can be formative (learning) and/or summative (assessment) (Trevitt, Macduff, & Steed,2014; Van Tartwijk & Driessen, 2009). Capable of being adapted to the specific purpose of the target

program or organization, portfolios provide a flexible and influential tool. While there have been anumber of teacher education papers dealing with [e]portfolios, the majority of these focus on thedevelopment process (Balaban, Mu, & Divjak, 2013). Recent efforts have been more focused on learnerperspectives, with researchers exploring successful factors in the development and use of EPSPI(Electronic Portfolio Students Perspective Instrument) (Ritzhaupt, Ndoye, & Parker, 2010), investigating

factors that influence student use of personal, academic, and professional development portfolios (Ahmed& Ward, 2016), and developing an instrument for assessing an e-portfolio (Balaban et al., 2013). Given

that the aims and goals of portfolio stakeholders vary, satisfying the needs of all is difficult (Parker, 2012).
Indeed, this is one of the main challenges to bringing portfolio usage into successful operation.

This paper describes our implementation of “Portfolio Practice I,” a new course for online graduate
students. After a trial year, we introduced the course as a mandatory part of our curriculum. Our focus

was on both the learning process and the summative assessment provided by the students themselves.
After describing the design of the course, we review how the learners perceived course implementation
and report on their overall experiences.

PORTFOLIO COURSE DESIGN
The target program is for master’s to train practitioners in education general. Most are adultstudents and all took online courses with Moodle. The program has 12 core competencies that represent

the basic knowledge and skills to be fulfilled by completing the required course as well as seven optional

competencies obtained through elective courses. Portfolio Practice I is designed for first year online
graduate students as a required course. The yearly course opened in 2015. Its design is based on a prioryear’s test operation in which we offered the students at the target graduate school several reflection
activities throughout the trial period. The current course design is based on what we learned from this trial
year. The main purpose of the course as it is currently designed is to provide opportunities for students torevisit what they have been required to do in the program and reflect comprehensively on the learning
activities involved. It is intended as a vehicle for students to reflect on all activities in the graduate schoolin an integrated manner. This critical reflections are introduced on a regular basis throughout the term;
showcase development and a summative reflection—essentially a rehearsal of the final exam—occur atthe end of each semester. All full-time master’s students are registered for this course automatically;

however, to complete the course, students need to complete other required courses, since all the activitiesof this portfolio course are premised on those courses.

In conducting the course, we use Moodle as our LMS and Mahara as our e-portfolio system.

Moodle is used for course activities such as submitting the reflection report; implementation of the final

exam practice and developing the showcases is done with Mahara. All students are required to take anorientation course soon after the announcement of student application results. Admitted students aretaught the basic ideas of portfolio development and learn how to use an e-portfolio system by using it tointroduce themselves.

Course activities focus on students revisiting activities from their other courses, both separately andin an integrated manner. The course consists of two parts: (1) reflection activities related to specificmandatory courses, and (2) showcase development and a competency-based reflection (a rehearsal of thefinal exam) which provide a means of summarizing the learning result by showing the student’s work,

together with appropriate supporting evidence. Combining both types of activities was one of our primarypoints of emphasis in designing course.

As shown in Figure 1, the course is separated into two blocks: first semester and second semester.

Students take the course concurrently with other courses. In fact, the first part of each block is a reflection

on each student’s other required courses. These course reflections are performed on a regular basis.
Activities such as rehearsal for the final exam and showcase development are conducted at the end of thesemester after students have completed their other required courses. These activities are intended tointegrate what the students have learned throughout their program of study. Rehearsal of the final exam is

a competency-based reflection activity; submission of competency-based reflection report is a condition

for completing the program and the rehearsal is included as preparation for the final exam. Showcase
activity is for the students to organize or summarize what they learned to the target readers. Students

develop showcase under the assumption that they apply for the job. Between the first and second
semester, we offer an optional activity for students to introduce and exchange views on their showcaseswith peers, seniors, and instructors at a study camp. Study planning for the second semester is also included
as an activity of the course.

RESULTS FROM THE LEARNER’S PERSPECTIVE
Among 22 registered students, 12 learners completed the course in 2015. Ten students failed tocomplete the course because they had not completed the other required courses that form the foundationof the portfolio course. These ten students did not engage in any activities in the portfolio course; alltwelve participants who completed the first course activity completed all the course activities.

We administered questionnaire surveys at the end of each semester. The surveys consisted of Likert

scale-type questions and written questions asking students to explain the reason for their selectedresponse; both question types were included in each section. Students answered the Likert scale questions

using a 5-point scale (5: applicable, 1: not applicable). Table 1 shows questionnaire results reflecting thestudents’ experience and perceptions of the two semester courses. Overall, the results indicate that bothsemester courses were generally found to be quite acceptable. By way of comparison, scores for the firstA.

The usability of Mahara as a course tool

1. I'm glad to use Mahara as a learning tool for a half a year.
2. I enjoyed portfolio development with Mahara.
3. Portfolio development with Mahara was difficult.
4. I think activities with Mahara will be of help when becoming a portfolio designer
in the future.
B.

Effect of participating of "Portfolio Practice I"

1. In this course, "Portfolio Practice I", there are opportunities, situations, andexperience for continuous learning reflection.
2. Through this course, "Portfolio Practice I”, I recognized the importance ofopportunities, situations, and experience for continuous learning reflection.
3. I think my reflection skill was improved by opportunities, situations, andexperience for continuous learning reflection in this course.
4. Continuous learning opportunities, situation, and experience in this course wasuseful to improve learning reflection.
5. Continuous learning opportunities, situation, and experience in this course will be
useful to improve future learning reflection.
C.

Effect of linking other courses in the course

1. In this course, "Portfolio Practice I”, there was a situation in which I could
consider the meaning of attained knowledge, skills and values, then link those.
2. Through this course, "Portfolio Practice I”, I found the importance of considerationand linking of the meaning of attained knowledge, skills and values.
3. Experience in this course such as considering and linking of the meaning oflearned knowledge, skills and values helped to improve learning activities
4. Experience in this course such as considering and linking of the meaning oflearned knowledge, skills and values will help to improve future learning activities
D.

Effect of interaction with peers in the course

1. In this course, "Portfolio Practice I”, there are opportunities to recognize one’s
own strength and weakness, and one's standing through interaction with others.
2. I found the importance to recognize one’s own strength and weakness, and one's
standing through the interaction with others.
3. I improved skills to recognize my own strength and weakness, and my standing
through interaction with others.
4. Recognizing my own strength and weakness, and my standing through interaction
with others in this course helped to improve learning skills
5. Recognizing own strength and weakness, and my standing through interaction withothers helps in this course to improve learning skills
E.

Reflection/connection/feeling/motivation

1. [Reflection] Rehearsal of final exam helped me to reflect on own learning.
2. [Reflection] Showcase development helped me to reflect on own learning.

1st Sem. Av.
SD

2nd Sem. Av
SD

4.5 0.7

4.3 0.9

4.3 0.7
3.2 0.9
4.2 0.7

1st Sem. Av.
SD

2nd Sem.
Av SD

4.6 0.7

4.3 0.8

4.0 0.7

3.9 0.9

4.3 0.7

4.2 0.6

4.7 0.5

4.2 0.6

1st Sem. Av.
SD

2nd Sem. Av
SD

4.3 0.8

4.3 0.9

4.3 0.8

4.0 0.9

4.6 0.74.3 0.8

1st Sem. Av.
SD

2nd Sem. Av
SD

4.3 0.7

4.2 0.8

4.1 0.7

3.8 1.0

4.0 0.7

3.8 0.9

4.2 0.7

4.1 0.7

1st Sem. Av.
SD

2nd Sem. Av
SD

4.3 (1.0)

4.3 0.8

4.1 0.9

4.1 0.8

4.6 0.5

4.4 0.7

4.3 0.8

4.4 (0.7)

3. [Connection] Course reflection enables rehearsal of final exams smoothly.

4.2 0.6

5. [Feeling] Course reflection helped to know which part of my ability was enhanced.

4.1

4. [Connection] rehearsal of final exam enables showcase development smoothly.
6. [Feeling] Final exam helped to know which part of my ability was enhanced.
7. [Feeling] Showcase helped to know which part of my ability was enhanced.

8. [Plan] Rehearsal of the final exam helped to consider future learning and career.

9. [Plan] Showcase development helped to consider future learning and career.
10 [Motivation] Sharing outcome of showcases and competency checks motivate me
to improve on my own.

4.0 0.7
3.8 0.6
3.5 0.9

0.7

4.2 0.7
4.0 1.0
4.1 0.8
4.0 1.04.2 0.8

4.3 0.7

4.3 0.8

4.1 0.8

4.3 (0.8)

3.4 0.8
3.3

0.9

4.3 0.8
4.3 0.9
4.0 0.6
4.3

0.8

4.1 0.9

semester were higher than those for second semester, although the scores for four of the questions () were higher in the second semester. “I enjoyed portfolio developmentwith Mahara” showed a higher agreement score, while “Portfolio development with Mahara was difficult”
in Section A of the questionnaire had a lower score in the second semester. In the written comments, somestudents mentioned that the second semester activities were easier than the first because they were moreused to the Mahara e-portfolio system, as the iterative activities had acclimated them to Mahara. Since thee-portfolio software will control, restrict or enhance portfolio development (Barrett, 2000), ensuring theusability of the e-portfolio system is essential for success. Written comments show that many students
experienced difficulty getting used to the system, indicating that continuous improvement in thefunctionality of Mahara is one of our future challenges.

Results in Section B of the questionnaire indicate that students felt that course activities were a good

opportunity for learning, but when compared to other questions, for question B-3, “I think my reflection
skill was improved by opportunities, situations, and experience for continuous learning reflection in thiscourse,” the scores were relatively low. The reason was not clear from the written responses. Althoughmost students commented positively on the reflection activity and its effect, some touched on the time

limitation and workload. Section C results indicate that learners believed that the course helped them

review other courses in an integrative way. In the comments section, one student mentioned that

“understanding of competency was one of this activity’s outcomes.” We believe that linking courses
would also help when considering curriculum design. Regarding peers in Section D, while students
believed that peers were useful resources to improve their course activities, some felt that they had notreceived enough feedback from their peers. Some students also felt that commenting to others wasdifficult, and some noted the limited time for discussion. In their written comments, many students
mentioned the usefulness of showcase development. The reasons why showcase development was

positively received varied. Many found that summarizing and supporting their ideas in discussions withothers helped them recognize their own strengths and weaknesses. Some students were encouraged by thechance to review their thoughts with others and receive constructive feedback. In this way, they appeared
to gain insights and were able to reconfirm and/or reevaluate the importance of self-assessment. On theother hand, students generally felt that the rehearsal of the final exam was more of a formality, since themain activity of the rehearsal was following instructions rather than presenting their own feelings orexercising their own creativity. There were exceptions. One student expressed her feeling that “therehearsal of the final exam helped to integrate learning outcomes and competency.” Overall, thecombination of a flexible approach such as showcase development and a formal approach such as therehearsal of the final exam was positively received.

DISCUSSION

For self-refection to confirm achievement, both showcase and competency based-reflection areeffective, although learners tend to feel that showcase development is more worthwhile. For use inassessment, competency-based reflection is essential for accumulating learning outcomes and reviewing

individual lessons and assignments. The design of our course included both participant and practitioner

views, which was important to achieving our intended course goal. If the observation of Ritzhaupt et al.
(2008) is correct—that understanding the purpose of a portfolio is the key to successful e-portfolio

development—then we have achieved a measure of success. Although satisfying all stakeholders’ needs
or expectations is not easy, our approach shows a way of overcoming the difficulty by integrating

summative and formative activities (Ritzhaupt et al., 2008). Embedding activities into the learning
context is also important (Ritzhaupt et al., 2010). In our approach, the activities were incorporated into

the leaning context in a specific course. In a survey conducted by Parker et al. (2012) of students in ateacher preparation program who experienced the creation of a portfolio, approximately 34% of therespondents suggested integrating e-portfolio development throughout their coursework rather than in thefinal stage of their program. Among the 12 students in our course, 11 (92%) in the first semester and 10(83%) in the second semester indicated that portfolio development is useful as a required course. It wasapparent that students tend to be more positive when portfolio activities are done in a regular pattern. For

further analysis of student perspectives on the matter, a detailed analysis of the contents of student
portfolios will be needed.

REFERENCES
Ahmed, E., & Ward, R. (2016). Analysis of factors influencing acceptance of personal, academic andprofessional development e-portfolios. Computers in Human Behavior, 63, 152-161.
doi:http://dx.doi.org/10.1016/j.chb.2016.05.043
Balaban, I., Mu, E., & Divjak, B. (2013). Development of an electronic Portfolio system success model:
An information systems approach. Computers & Education, 60(1), 396-411.
doi:http://dx.doi.org/10.1016/j.compedu.2012.06.013
Barrett, H. (2000). Electronic Portfolios = Multimedia Development + Portfolio Development The
Electronic Portfolio Development Process.
Retrieved fromhttp://electronicportfolios.com/portfolios/EPDevProcess.html
Parker, M. N., Abdou; Ritzhaupt, Albert D. (2012). Qualitative Analysis of Student Perceptions of EPortfolios in a Teacher Education Program. Journal of Digital Learning in Teacher Education,
28(3), 99-107.
Ritzhaupt, A. D., Singh, O., Seyferth, T., & Dedrick, R. F. (2008). Development of the ElectronicPortfolio Student Perspective Instrument: An ePortfolio integration initiative. Journal ofComputing in Higher Education, 19(2), 47-71. doi:10.1007/bf03033426
Trevitt, C., Macduff, A., & Steed, A. (2014). [e]portfolios for learning and as evidence of achievement:
Scoping the academic practice development agenda ahead. The Internet and Higher Education,
20, 69-78. doi:http://dx.doi.org/10.1016/j.iheduc.2013.06.001
Van Tartwijk, J., & Driessen, E. W. (2009). Portfolios for assessment and learning: AMEE Guide no. 45.
Med Teach, 31(9), 790-801.