The Evaluation on the Practice by using “RepoReco” that
Supports Undergraduate Students’ Reflection in Their Writing Processes

Yoshikazu TATENOThe University of Tokyo, Japan
yoshikazu@tate-lab.net
Takaaki Ohkawauchi
Rikkyo University, Japanohkawauchi@rikkyo.ac.jp
Tomoki HIRANO
Uchidayoko Educational Resources Information Center, Japanhirano.tomoki@uchida.co.jp
Jun NAKAHARAThe University of Tokyo, Japan
jun@nakahara-lab.net
Abstract: It is widely known that undergraduate students have serious difficulties in writing
academic reports. In particular, novice writers lack a strong ability to reflect on the writing process.
Thus, this study focuses on the problems in students’ reflection on their writing process.
This study seeks to evaluate the effectiveness of writing sessions in which tutors provide
one-on-one instruction to students using a computer application named “RepoReco.” Theexperiment simulates an academic writing course as an extracurricular study.
RepoReco visualizes the writing processes to support students’ reflection. The system can be used
on a web browser. It records the writing processes of undergraduate students’ verbatim and thendisplays them visually. It records (1) the time of their writing; (2) position of the cursor; (3) cut,copy, and paste actions; and (4) changing numbers of characters.
To evaluate students’ report writing using this system, we conducted an experiment in alaboratory setting. Twelve undergraduate students were asked to evaluate the effectiveness of an
instruction session in a questionnaire and interview. We focused on whether the students
consciously reflected on their writing process as they wrote. The experiment results indicated that
RepoReco indeed helped the students to reflect on their writing process.

Keywords: Academic Writing, Reflection, Writing Process, Writing Center1. INTRODUCTION
1.1. Social backgroundIn recent years, Japanese universities have been required to support instruction on students’ academic
writing in extracurricular courses. Although academic writing instruction in curricular courses was previously
required, the situation has changed. Because the number of study counselors at university libraries and writing
centers is increasing, extracurricular courses have become mandatory. However, there are fewer studies on
instruction in extracurricular courses as compared to those on curricular courses.
In this study, the authors will focus on the instruction at writing centers, which are institutes that support
students’ writing activity in extracurricular courses. Generally, writing centers have graduate student tutors who aretrained on how to give instruction (Sadoshima 2008). Students bring their own reports to the writing center to receive
one-on-one help from a tutor.
1.2. Problems in previous researchNorth (1984) is well known for two philosophies of instructions for writing centers. One is to “raise an
independent writer.” This means that tutors at the writing center should concentrate not only on improving student
reports, but also on helping students learn to revise and polish their work on their own. An “independent writer” is
one who can find problems in his/her own report and rewrite it appropriately.
Thus, tutors should avoid just touching up students’ reports. Instead, they should teach in a way thatguides students to reflect consciously on their writing processes and to notice mistakes on their own.
North’s other philosophy is “to support the writing processes of students.” In writing centers, students canlearn not only how to improve their written reports, but also about readiness for writing. These two philosophies aretaught to writing center tutors.
Under the philosophies above, tutors should avoid simply giving advice on problems or touching upreports. Instead, they should promote students’ reflection on the writing process. In the other words, tutors should not
only focus on the written report itself, but also understand and encourage the student’s writing processes.
Tateno et al. (in press) invented a web browser application system named “RepoReco” as a tool for tutors
to understand students’ writing processes. The system was developed using the Process Display method, which canvisualize a writer’s writing processes (Lin et al. 1999). Tateno et al.’s (in press) experiment demonstrated that thesystem aids tutors’ understanding of students’ writing processes.
However, although Tateno et al. (in press) evaluated the system’s effects on tutors’ instruction, they did
not investigate whether the students themselves became more reflective in relation to their writing processes. Thus,in this study, the authors conducted an experiment to assess whether undergraduate students could become morereflective on their writing processes through instruction sessions given by a tutor using RepoReco.

2. SYSTEM
2.1. System design
The web application RepoReco is designed to support users in writing academic reports. The system canbe used on web browsers. The developers assume that students do not use the system on their own but are supported
by tutors in face-to-face instruction. The system was developed on the web server Apache 2.2 and database server
MySQL5.1. The developers programmed it using PHP 5.2.9 and Action Script 3.0.
The system is equipped with special features: recording data and visualizing the writing process during
the actual report writing. The system records the following data: time interval of writing; position of the cursor;
timing of copy, cut, and paste actions; and increasing/ decreasing numbers of characters (see Table 1). It thenvisualizes the data in graphs. The tutors can look at the graphs as the users write.
2.2. Interface
We will describe the main two interfaces of the systems, the “writing pane” and “reflection pane.”
2.2.1. Write a new report – Writing pane
To write a new report, users open the writing pane. This pane has a “recording” button in the upper right.
Before the users begin writing, they click the recording button so that as they write, the data are automaticallyrecorded. Then, they simply write their report as they would when filling out a web form. When finished, they clickthe “stop button,” after which they automatically move to the reflection pane.
2.2.2. Review and reflection - Reflection pane
The reflection pane shows users the visualized data that the system records as they write (see Figure 1).
Looking at the reflection pane allows a tutor to understand how a student is writing a report and to offer appropriate
advice based on the process.
The special features of the reflection pane will be introduced below. The reflection pane includes two
smaller panes: the “playing (record viewing) pane” and “graph pane.”-Playing (record viewing) pane
Users can review the recorded data from their writing processes on the playing (record viewing) pane.
When they click the play button, a representation of how they wrote the report is played like a movie. If they want tocheck certain parts of the report, they can jump to a different paragraph by controlling the progress bar. The users
can also change the play speed from two to eight times faster.

Figure 1: Reflection pane
-Graph pane
The graph pane shows the visualized data that were automatically recorded as the user wrote. These data
are recorded without interrupting the users’ writing process and may help tutors to understand the students’ writing
process and to offer advice based on their observations.
When a user clicks the “play” button, the graph pane appears at the same time that the play pane starts toshow the review movie.
Next, we will represent the data of what the system records during the students’ writing. The system
records “Stop”; “Reverse”; “Copy, cut, and paste”; and “Numbers of characters” in graph. We will describe eachfeature and what types of data are visualized in the system. First, to visualize “Stop” timing, the system records thetime intervals of typing. When a writer stops writing, it is supposed that s/he is thinking about what or how to write.
Second, the system records the data of cursor position to grasp “Reverse” actions. “Reverse” refers to instances when
the writer moves the cursor to a previous part of the report. When the writer rewrites part of the report, it may beassumed that s/he has chosen to do so after reviewing it. Thirdly, the system records and displays the timing of“Copy, cut, and paste” actions. Lastly, it displays the increasing and/or decreasing numbers of characters.
Additionally, the users can scale the graphs on a pane to any size. Meanwhile, by viewing the data described above,tutors can learn about the writer’s individual writing habits.
Display
Stop
ReverseCopy and paste
Number of characters in graph

Table 1: Recorded data
Recorded data
Time interval of typing
Position of cursor
Timing of copy, cut and paste
Increasing and decreasing numbers of characters

3. EXPERIMENT
3.1. Purpose
The purpose of the experiment is to investigate whether students who are given writing instruction using
the RepoReco system learn how to reflect on their writing processes. The instruction session paired each writer witha tutor. The authors hypothesized that, through the instruction with a tutor using RepoReco, the students would
become more aware of their writing process.
3.2. Participants
The participants were 12 freshman and sophomore students at a Japanese university. Each student was
paired with a tutor, forming 12 pairs. All the tutors had experience teaching extracurricular academic writing courses.
The criteria for their participation were (1) teaching experience as a teaching counselor at a university library or (2)teaching experience at an academic writing center. The tutors’ length of teaching experience ranged from threemonths to six years.
3.3. Method
3.3.1. Lecture
Before the experiment began, all the participants were given a lecture. The tutors were given one lecture
and the students another. The lecture given to the tutors included (1) an explanation of how to use the RepoReco
system and (2) an explanation of how to advance practice. Each lecture took about 15minutes.
3.3.2. Report task
The students were asked to read assigned texts and write a report on the theme “Express your opinion
about communication with a mobile phone.” This theme is at the same level as those used in university entrance
exams. When writing about their opinions, the students could reference the material they had read. The time limit for
writing was 60 minutes, and the maximum report length was about 1000 characters (700 words).
3.3.3. Instruction session with the tutor
The instruction session with the tutor was held after the students finished writing their reports. In thissession, the tutor gave advice to improve the student’s report. The tutor session was structured based on Sadoshima’s(2008) program, summarized in (1) to (3) below.
1. The tutor reads a report written by a student and inquires about what is written (about 5 min.).
2. Together, the tutor and student watch the visualized data on RepoReco. As they watch, the tutor asks
questions such as “What did you think when you wrote this paragraph?” or “Do you usually write a report
this way?”
3. The tutor gives advice on how the student can improve the report.
3.3.4. Rewrite and follow-up
After the instruction session, the student rewrites the report for 30 to 60 minutes.
When the program is over, the student completes the questionnaire and semi-structured interview with the firstauthor.

4. ANALYSIS
4.1. Purpose
The purpose of the analysis is to assess whether the instruction session led students to consciously reflect
on their writing process.
The questionnaire asked students to evaluate whether they had consciously reflected on their writing
during the experiment. In addition to this main evaluation topic, the authors asked questions to deepen the analysis of
the instruction. Questions were related to 1) whether the students improved the contents of their report after the
instruction session with the tutor and 2) their evaluation of the instruction session using the system.
4.2. Method
The students answered the questionnaire and participated in semi-constructed interviews. The authors usedboth sets of data to evaluate the advantages of the instruction with RepoReco. Table 2 lists the questionnaire items.
The students rated each item on a scale of 1 to 4 (1 = “do not agree” and 4 = “totally agree”). In the semi-constructedinterviews, the author asked questions related to the students’ questionnaire responses.
Table 2: Questionnaire
Question items
1

2Frequency
34[Questions about reflection on the writing processes]
1

I reflected on my process of report writing through the
instruction by the tutor using the system.

0

0

1

11

**

2

I became more conscious of my writing habits through the
instruction.
I learned how to improve my report writing.

0

1

5

6

**

0

1

6

5

**

0

0

4

8

**

0

2

4

6

*

0

0

4

8

**

[Questions about evaluation of the instruction session using the system]00
I enjoyed the session with the tutor using the system.
6
6
I that there was a heavy workload in the session.

2

10

**

0

0

**

3

9

**

3

[Questions about improving the report contents]
1

2
3

1
2
3

Compared to when I wrote the previous report, I could
make my opinion clear as a result of the instruction by thetutor using the system.
Compared to when I wrote the previous report, I could
relate my opinion and supporting evidence.
I could identify problems in my report.

I would like to do the same session again with a tutor
using the system.
**(p < .01) *(p < .05)

004.3. Results and discussion
4.3.1. Evaluation and analysis of the reflection on the writing process
Table 2 presents the questionnaire results. The questionnaire asked whether the students increased theirawareness of their writing processes. The student responses were examined using a binomial test. The answers weredivided into two groups: a 1/2 group and 3/4 group.
The results showed a significant difference in the student responses to questions about reflection on thewriting process.
Next, the authors analyzed the students’ free descriptions in the questionnaire. Student A wrote thefollowing.
“I became aware of my unconscious habits after the tutor gave me advice using the system and improved my report. I
learned to be conscious of my writing process when I write academic reports. When I talked with the tutor, I often
noticed some new issues to improve that I could not notice by myself. I could instantaneously see my writing habitsfrom a different perspective through the instruction provided by the tutor.”
Based on Student A’s comment, the author considered that Student A was able to reflect on his/her writing
process. Moreover, Student A mentioned that through the dialogue with the tutor, she became aware of problems she
had not noticed before and gained a different perspective on her writing strategy and the report contents. Thus, theauthors concluded that the instruction using RepoReco helped students increase their consciousness of the writing
process.
4.3.2. Analysis of the improvement in the report contents
To further deepen the discussion, the authors analyzed the student responses to questionnaire items about
improving the report contents. The results show a significant difference in “making opinion clear,” “relating opinion
and the evidence,” and “becoming aware of the problems.” These findings suggest that the instruction session withthe tutor using RepoReco improved the quality of the student reports.
4.3.3. Analysis of the instruction session
The authors also analyzed the student responses to questions about evaluation of the instruction session.
The responses included a range of opinions, from “I enjoyed the session” and “I would like to do this session again”
to “I felt there was a heavy workload in the session (inverted scale).” The results show that, overall, the students
were satisfied with the instruction provided by the tutors using RepoReco. In the free response section of thequestionnaire, Student B wrote the following:“It was interesting to review the process of my writing, watching the recorded data on the system. The useful point of
the tutor’s session with the system was that I could become reflective about my writing process.”
Student B described the instruction as “interesting” and “useful,” supporting the finding that it was helpful forstudents’ report writing. To conclude this analysis, it can be said that the students seemed satisfied with the
instruction sessions.

5. CONCLUSION
This paper found that instruction with a tutor using the web application system RepoReco can help
students learn to reflect on their writing processes. The experiment simulated an extracurricular, academic writing
course. The results suggested that the students were able to reflect on their writing processes following instruction by
a tutor using RepoReco. Moreover, the results indicated that 1) the instruction might improve the quality of student
reports and 2) the students seemed satisfied with receiving writing instruction from a tutor using the RepoReco
system.
Additionally, in future research, the authors would like to analyze specifically 1) how the students’ writing
habits improved and 2) how the improvements in their writing habits influenced the quality of the report contents.

Literature Reference
North, S.M (1984). The idea of a writing center. College English, 46（5）：433-446
Lin, X., Hmelo, C., Kinzer, C. K., & Secules, T. J. (1999). Designing technology to support reflection. Educational Technology
Research and Development, 47(3), 43–62.
Sadoshima,S.(2009)Fostering Self-Directed Writers : Analyses from Writing Center Tutorial in Japan. Japanese Teaching Societyof Japan, Vol.66, pp.11-18. (in Japanese)
Tateno,Y., Ohkawauchi,T., Hirano,T., Nakahara,J.(in press).Development of "Reporeco", a Visualization System for Students'
Writing Processes: Focusing on a Case of an Academic Writing Course as an Extracurricular Study. Japan Journal of
Educational Technology, Vol.37, No.3 (in Japanese)