Feature Article

The Effects of Metacognition on

Reading Comprehension Tests in an
Intermediate Reading and Writing
CourseJames D. Dunn
Tokai University
Reading comprehension skills are paramount to the future success of Japanese students.
Contextual clues and key word strategies help students take tests such as the TOEIC andEIKEN, as well as develop critical thinking skills for use in their lives. The introduction ofreading comprehension strategies, in and of itself, helps students improve their scores, but
the question if students are fully utilizing these skills during a reading comprehension test
still remains. To determine whether students are better off with a “directed implementation”
versus a “non-directed implementation” of these skills is examined with sixty-four university
students of predominately Japanese ethnicity. The data clearly shows that students who are
involved in a metacognitive process while utilizing these skills score higher on a reading
comprehension test versus students that do not. These results suggest that having students
actively utilize these skills, and showing examples of their metacognitive processes, positively
influences test outcome.
英文読解力を高める事は日本の学生が、将来成功するために最も大切である。文
脈のヒントやキーワードを見極める力は、TOEICや英検といったテストにおいて学生を助ける学習だけでなく、実生活において批判的思考力を伸ばすのに大いに役立つ。また、テストの点数を上げるためには、それらの批判的思考力をふまえた英語読解能力が必要である。しかしながら、学生のテストの点数が、手段の導入の有無によってより有効かどうかを見極める必要がある。本研究結果より、英文読解テストでメタ認知的アプローチをとった学生は、アプローチを行わなかった学生と比べ、明らかに高い点数を記録したことが判明した。よって、これらのメタ認知的英文読解プロセスを多く活用した方がテスト結果に良い影響があることが分かった。

329

Dunn

The landscape of learning English in Japan puts a great deal of weight on thevalue of being able to score well on tests like the Test of English for International
Communication (TOEIC) and other, similar, standardized testing systems. Allof these tests have many types of assessments, but one common style of evaluation
among them is reading comprehension sections. These sections often represent a
substantial amount of the point total. One role of an educator, teaching reading
and writing in Japan, is to give students skills that will enable them to successfully
navigate these types of reading comprehension tests. Skills that help students
perform better on standardized tests are also applicable to other situations that
may arise in their future.

BackgroundReading Strategies and Their Place in JapanDavid Moore (2012) identifies seven reading comprehension strategies that
effective educators encourage in their students. The seven strategies stated areplanning and monitoring, determining importance, asking questions, making
inferences, making connections, synthesizing, and visualizing. These strategies
help students to improve their mastery of the reading process and afford gains
in reading comprehension and vocabulary acquisition (Short & Fitzsimmons,
2007).
It is widely agreed that expert reading is an interaction of four main areas: thereader, the text, the fluency of the reading, and the strategic aptitude of the reader(Nuttal, 1996; Anderson, 1999, 2002; Rasinski, 2010). One major necessityof reading comprehension is the realization that successful comprehension ofreading does not occur without concerted effort on the reader’s part (Weinstein& Mayer, 1985; Weinstein & Underwood, 1985; Adams & Hamm, 1994;
Gettinger & Seibert, 2002; Baker, 2008). Furthermore, students must invoke thestrategies or risk losing the benefits of the strategies taught to them (Alexander& Jetton, 2000). This implies that simply teaching students the strategies to
employ when reading is not enough. Students must be made to employ strategies
in a clear and accountable way. One way to do this is to have the students show
their thought processes on the test paper. This practice is akin to a math teacher
330

The Effects of Metacognition, OnCUE Journal, 9(4), pages 329-343

who would require students to show their work on a math problem in order toassist the formulas and algorithms becoming cemented in the student’s working
memory (Schoenfeld, 1992). Requiring English students to show their work
seems to be a logical extension of this teaching method.
In Japan, the Test of English for International Communication (TOEIC)
is one of the most popular exams for the evaluation of English communication
ability (Childs, 1995). The TOEIC tests English ability in grammar, reading,and listening. Many Japanese companies use the TOEIC as a gauge for linguisticcompetence for their employees—requiring their score either during thecompany hiring process, or during their employment at a company for purposesof procuring a position with the company overseas (Chapman, 2003; Powers,
Kim, & Weng, 2008). In addition to English reading skills being important tofinding employment, they are also essential for Japanese employees who deal
with correspondence from other countries. Higher TOEIC test scores caninfluence both the employability and future opportunities of students, as higherscores can translate into better jobs and, according to a student of the author, insome companies, bonuses for scoring over a certain threshold. All of these factors
have the possibility of influencing pedagogy in the language learning classroomthat employs reading comprehension.
Within the TOEIC, there are three reading sections—incomplete sentences,
reading comprehension passages, and double reading questions—which require
answering questions about email correspondence (http://www.toeic.or.jp).With
much of the TOEIC focused on reading comprehension, reading comprehension
skills could prove useful for students.
This need for reading comprehension skills, and the fact that the private
university at which this author teaches utilizes a TOEIC-style unified reading
comprehension test for both first- and second-year students, led to the action
research described in this paper. Would skills and strategies be enough to help
students improve their reading comprehension scores, and would the students
need assistance in their metacognitive processes to fully utilize the strategies
introduced in class?
331

Dunn

Metacognition in Education
Metacognition, or “thinking about thinking” (Alexander & Jetton, 2000),
includes actively knowing about when, where, and how to implement strategies
for learning or problem solving, as well as engaging in the use of said strategies.
The “thinking about thinking” definition of metacognition is something of a grossoversimplification. Metacognition can be thought of as two overlapping areas inthis regard: metacognitive knowledge and metacognitive processes (Dunlosky& Metcalf, 2009; Jacobs & Paris, 1987; Nelson & Narens, 1990; Schwartz &Bacon, 2008). Metacognitive knowledge can be broken down into three aspects( Jacobs & Paris, 1987). The declarative aspect is what students know about theirability to learn. The procedural aspect is what the students know about how
to implement strategies. The conditional aspect regulates the when and wherestrategies are used. Metacognitive processes can be broken down into monitoring
and control (Nelson & Narens, 1990). The metacognitive process of monitoringis represented by a student understanding where they are in a task or learning
situation and judging if the task is able to be completed. The metacognitive processof control is in charge of making choices as to which strategy to implement andwhen to terminate a task if it is deemed, by the monitoring process, as unable tobe completed.
These five factors are all in play in a student’s brain at the same time (Carroll,2008) and are relevant to the student’s educational life. This action research set
out to determine if requiring students to perform metacognitive processes andutilizing metacognitive knowledge during a test would result in better reading
comprehension scores. Since students need to consciously employ strategies inorder for them to be effective (Alexander & Jetton, 2000), metacognition on
the reading comprehension strategies taught before a test is required during thetaking of a test. While there have been studies on reading comprehension skills
and metacognition (Carrell, Pharis, & Liberto, 1989; Cubukcu, 2008; Takallou,
2011;Tregaskes & Daines, 1989), none of them, save for Cubukcu (2008) in adifferent manner, have required proof of a student’s metacognitive processes on areading comprehension test.
While there are studies that suggest metacognition has a positive effect on
332

The Effects of Metacognition, OnCUE Journal, 9(4), pages 329-343

reading comprehension (Cubukcu, 2008; Takallou, 2011;Tregaskes & Daines,
1989), further studies needed to be conducted to gauge if explicit teaching of
metacognitive strategies influences the use of reading comprehension strategies.
The underlying hypothesis of this paper is that students who are required toshow proof of their metacognitive processes, during the employment of reading
comprehension strategies will achieve higher test scores than students who are
not required to show their processes.
This study compared the reading comprehension test scores of students who
were required to show their metacognitive processes and those who were noton a reading comprehension test. For the purposes of this study, metacognitive
processes refer to participants consciously thinking about the implementationof the previously introduced reading strategies during a reading comprehension
test. The current pilot study was conducted in order to find an answer tothe following question regarding the effect of students illustrating their
metacognitive processes on a reading comprehension test.
1. Is there any significant difference in reading comprehension test scores
between learners who are required to engage in metacognition by showing
their reading comprehension strategies and those who are not?

Methodology
Subjects
The participants in this pilot study consisted of 64 college students split between
a control group (n = 31) and a test group (n = 33) including 62 Japanese students,
one Chinese student, and one Korean student. All students were sophomores
at a private university in Japan, enrolled in the required Reading and Writing
Two (RW2) classes which had been streamed into the high-beginner to lowintermediate levels of English as defined by the Common European Frameworkof Reference for Languages (CFER). Each participant was a member of either thescience or engineering department, and all participants had completed six weeks
of reading comprehension strategy instruction focusing on contextual clues tofind meaning, key word identification, making connections, and synthesizing
information from multiple areas of a reading passage on intermediate (CEFR
333

Dunn

A2 and B1 level) reading comprehension activities. The reading subjects thatthe students were exposed to ranged from humorous stories and work emails togeneral science topics and mechanical engineering. Each group attended class for
90 minutes twice a week, where they studied both reading comprehension andprocess writing methods. Reading comprehension strategy instruction occurred
during one of their two classes a week, for the six-week period. As a result of reading
comprehension strategy instruction, all students were familiar with the reading
strategies prior to the taking of the test. The control group originally consisted of
33 members, but two participants did not take the reading comprehension test,
so the null scores were dropped for this pilot study.
Instructional BackgroundOf the seven reading comprehension strategies given by Moore (2012), thisstudy focuses on three—planning and monitoring, making connections, andsynthesizing—which were explicitly taught to the students in both the control
and test groups. These three were chosen so as to give the students a broad rangeof skills in a short amount of time that can assist in this aspect of a test like the
TOEIC. The planning and monitoring strategy was chosen to give students a skill
to manage time during the test. Making connections and synthesizing strategy
was added to give students higher order thinking skills to assist in choosing
correct answers. Making connections as a reading comprehension skill is very
similar to the metacognitive strategy Cucukcu (2008) examined, i.e., searchingaccording to goals.
In both the control and test groups, reading comprehension test scores atthe start of the course were lower than expected. The control and test groupboth scored an average of just over ten out of twenty-five points on their firstreading comprehension test, where the hope was that they would have an averagescore that was in the passing range. This prompted the author to introduce
reading comprehension strategies into the classes to help bring students’ skills
up. From week four of the course, all participants were instructed in a variety ofreading comprehension strategies including, but not limited to the following:developing a plan to read, identifying key words, identifying meaning from334

The Effects of Metacognition, OnCUE Journal, 9(4), pages 329-343

context, making connections between the questions and reading passage, andputting information from two or more locations in the text together to synthesis
new information. Making connections inside reading passages as well as between
the test questions and the reading passage is a skill that students can make use ofin their future professional lives. Similarly, being able to synthesize informationfrom multiple sections of a reading is an invaluable skill that translates beyond
the classroom into the students’ possible future employment areas. Planning andmonitoring was chosen to give students the ability to look at a text critically and
immediately start breaking down the questions and text to make sense of theinformation presented.
Both groups were instructed on showing their metacognitive processes byeither drawing lines between the questions and reading passage or labeling thearea where the student felt the answer was with the numeral of the question
number. Since all of the participants were members of either the science orengineering department, a small number of the reading exercises focused on
different aspects of science and engineering.
Reading Comprehension Test
The test of reading comprehension was taken from a reading comprehension
practice from the department of philosophy in the University of Venezia(Gebhardt, 2012). This test was chosen for two primary reasons: one, it is freely
available and two, it contains reading passages of various topics and a variety ofquestion types. The test consisted of four reading passages of 100 to 150 words
each. The content of the four tests ranged widely with topics such as movies andmedicine. While the test was not designed for science and engineering students,
it did in a limited sense cover one science topic about medicine. The first three
reading passages were paired with five multiple-choice reading comprehension
questions each, and a fourth reading of 10 true/false questions. The maximum
score on the test was 25 points, and the time given to each group to complete
the test was 60 minutes. The students were not allowed to leave early if theyfinished the test within the 60-minute window. It should be noted that this test
underwent no reliability testing and was used primarily for the two reasons stated
335

Dunn

above. Future research should employ standardized reading comprehension tests
that are more commonly recognized in the educational community.
ProcedureThis study was conducted in week 10 of a 15-week course in a private Japanese
university. The control group (n = 31) and test group (n = 33) were both given
the same reading comprehension test. For the control group, the instruction was
to take the test normally. For the test group, in addition to taking the test andchecking their answers, they were required to show their thought processes byconnecting key words in the questions to the area in the reading passage where
they thought the answers were located. Out of four sets of reading comprehension
and questions, participants in the test group were required to show their thoughtprocess on at least one. Students were not directed in how to show their thoughtprocesses, but were encouraged to draw lines to utilize the making connections
strategy (Figure 1), as was practiced in the training sessions. Some participants inthe test group only showed where they found the answers by indicating the pointin the passage with a number that corresponded to the number of the question,
but this did demonstrate that they were making connections between thequestions and the locations of the answers in the text. All of the participants inthis study were required to answer all the questions to avoid artificially lowering
the group’s mean score.

Figure 1. Example of drawing lines to utilize the making connections strategy.

336

The Effects of Metacognition, OnCUE Journal, 9(4), pages 329-343

Results
Data Analysis
An unpaired t-test was employed to analyze the two sets of test scores. A
statistical representation of the analyzed data is given in Table 1. Table 1
comprises of data that addresses the question— Is there any significant difference
in reading comprehension test scores between learners who are required to engage inmetacognition by showing their reading comprehension strategies and those who arenot?
As the data shows (Table 1), the test group displays a mean of 13.39 points
with a standard deviation of 1.97, and the control group displays a mean of
11.48 points with a standard deviation of 1.86. This data suggests that the twogroups differed significantly in their average and mean test scores (t = 3.9855; p= 0.0002).

Discussion

In previous reading quizzes, the difference between two groups’ scores was notstatistically significant (t = 1.7059; p = .2301) (Table 2), which is particularly
Table 1
Final Test Data Analysis
Mean

SD

n

Control Group

11.48

1.86

31

Test Group

13.39

1.97

33

p = 0.0002 d = 1.0
Table 2Control vs. Test Score Comparison
Control Mean

Test Mean

Quiz 1

10.40

10.64

Quiz 2

11.04

11.10

Study Test

11.48

13.39

337

Dunn

noteworthy when comparing them to the scores in this study. It bears noting thatthe students in the test group were only required to show their metacognitive
processes on one of the four test items. Only one reading passage was required
due to time restraints of the reading comprehension test. Future experiments
could investigate the effect of requiring the students to show their metacognitive
processes on every test item.
As the results indicate (Table 1), the required metacognitive processes while
participants actively employing reading comprehension strategies resulted inbetter overall scores. The t-test value of 3.9855 and standard error of differenceof 0.479 are very promising for future studies. The p-value of 0.0002 is also
very exciting in that it shows a strong statistical significance, even with such asmall sample size (n=64). The scores reflect the direct and positive influenceof engaging in metacognitive processes during reading comprehension strategy
implementation. In order to gauge the effect size of this study, a Cohen’s d wasdetermined (Table 1). The d value of 1.0 shows a very large effect size. The
combination of the large effect size and the high statistical significance is excitingin that it suggests this study could easily be scaled up to larger numbers withsimilar results as those found in this action research.
The ability to increase the likelihood of finding correct answers on a reading
comprehension test by requiring metacognitive activity during the test makes
this study interesting. By requiring proof of reading strategies during the takingof the test, scores increased significantly. This also supports previous work on
metacognition and education where a positive impact was observed whenintroducing metacognition to a class (Cubukcku, 2008). The hypothesis put
forth is supported but will need future studies to replicate the findings. The
correlation between showing the participants’ strategy application on one partof a four-part test also begs the question of what would happen if they wererequired to show this implementation on all parts of a test. It is the belief of
this author that the difference between the control group and test group would
widen even further.

338

The Effects of Metacognition, OnCUE Journal, 9(4), pages 329-343

Pedagogical Implications
As the data suggests, having students show their work on a test resulted in betterscores on reading comprehension tests. This pilot study suggests that requiring
the students to actively participate in thinking about implementation of reading
strategies, as well as showing proof of these thought processes, may be effective in
helping to improve test scores. It is interesting to note that in subsequent lessons
this author observed students in the test group continue to show their work,
albeit at a reduced rate, after the study.
Metacognition has the potential to offer a wide range of benefits for students.
One outcome an educator might expect would be better self-monitoring oflearning behavior in students and ease of strategy implementation when neededon tests such as TOEIC. Metacognition also influences other learning skills anddispositions by aiding in planning, monitoring, and regulation of a student’s
cognitive processes which are relevant to both deductive and inductive thinking,as well as critical thinking (Dimmitt & McCormick, 2012; Pintrich, 2002). In
regards to this author’s students, metacognition can provide benefits past theclassroom. Science and engineering fields require a great deal of critical thinking.
Metacognition and the associated skills and dispositions not only help to pass areading comprehension test, but those same cognitive processes will help themin their future endeavors. It has been shown in this action research that students
can benefit from top-down instruction of skills and strategies, but it is perhaps
more important that the students are expected to use metacognitive processes tofully utilize what they have learned.
Future ResearchThis study must be replicated using a larger pool of participants from a greater
range of skills. This author hopes to include other teachers and classrooms
in similar studies in the near future. Larger sample sizes and expanding therequirement of showing the implementation of strategies to the entire test should
result in more definitive data that this style of metacognition has a positive
impact on scores. Standardization of the reading comprehension test and using
more sophisticated methods of data analysis would also benefit the collection
339

Dunn

of data. As previously mentioned, students in the test group continued to show
their work in later quizzes, which is another avenue of study that needs to be
explored. The question of lasting benefits of any method is incredibly important
for teachers and students alike. Further research into metacognition with otherstyles of cognitive processes, such as Cubukcu (2008) examined, would beinteresting.

Conclusions

Ultimately, the best strategies for students are the ones that are actually employedwhen needed. Giving students every possible way to do better in their lives, be
it on a TOEIC test or in their future jobs, is essential. By giving students theopportunity to engage in metacognitive processes of “thinking about theirthinking” or actively thinking about what they know, teachers help them to build
skills that will stay with them beyond the walls of the classroom.

Acknowledgements

I would like to thank Simeon Flowers for his assistance in determining thestatistics for data contained in this paper.

References

Adams, D., & Hamm, M. (1994). New designs for teaching and learning:
Promoting active learning in tomorrow’s schools. San Francisco, CA: JosseyBass.
Alexander, P. A., & Jetton, T. L. (2000). Learning from text: A
multidimensional and developmental perspective. In M. L. Kamil, P. B.
Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research:
Vol. III (pp. 285-310). Mahwah, NJ: Lawrence Erlbaum Associates.
Allington, R. L. (2006). What really matters for struggling readers: Designingresearch-based programs (2nd ed.). Boston, MA: Pearson.
Anderson, N. (1999). Exploring second language reading: Issues and strategies.
Boston, MA: Heinle & Heinle.
Anderson, N. J. (2002). The role of metacognition in second/foreign
340

The Effects of Metacognition, OnCUE Journal, 9(4), pages 329-343

language teaching and learning. ERIC Digest. Washington, DC: ERIC
Clearinghouse on Language and Linguistics. Retrieved from http://files.
eric.ed.gov/fulltext/ED463659.pdf
Baker, L. (2008). Metacognitive development in reading: Contributors andconsequences. In K. Mokhtari & R. Sheorey (Eds.), Reading strategies offirst and second-language learners: See how they read (pp. 25-42). Norwood,
MA: Christopher-Gordon.
Carrell, P. L., Pharis, B. G., & Liberto, J. C. (1989). Metacognitive strategy
training for ESL reading. TESOL Quarterly, 23(4), 647-678.
Carroll, M. (2008). Metacognition in the classroom. In J. Dunlosky & R.A.
Bjork (Eds.), Handbook of metamemory and memory (pp. 411-427). New
York, NY: Psychology Press.
Chapman, M. (2003). The role of the TOEIC in a major Japanese company.
Proceedings of the 2nd Annual JALT Pan-SIG Conference. Kyoto, Japan.
Retrieved from https://jalt.org/pansig/2003/HTML/Chapman.htm
Childs, M. (1995). Good and bad uses of TOEIC by Japanese companies. In
J. D. Brown & S. Yamashita (Eds.), Language testing in Japan (pp. 66-75).
Tokyo, Japan: JALT.
Cubukcu, F. (2008). Enhancing vocabulary development and reading
comprehension through metacognitive strategies. Issues in Education
Research, 18(1), 1-11.
Dimmitt, C., & McCormick, C. B. (2012). Metacognition in education. In
K. R. Harris, S. Graham, and T. Urdan (Eds.), APA educational psychology
handbook: Vol. 1. Theories, constructs, and critical issues (pp. 157–187). doi:
10.1037/13273-007
Dunlosky, J., & Metcalf, M. (2009). Metacognition. Thousand Oaks, CA: Sage.
Gebhardt, F. (2012). Reading Comprehension Practice. (B1 Idoneita).
Department of Philosophy. University of Venezia, Italy. Retrieved
from: http://lettere2.unive.it/lingue/lingua_ING/B1%20reading%20
comprehension.pdf
Gettinger, M., & Seibert, J. K. (2002). Contribution of study skills to academic
competence. School Psychology Review, 31(3), 350-365.
341

Dunn

Jacobs, J. E., & Paris, S. G. (1987). Children’s metacognition about reading:Issues in definition, measurement, and instruction. Educational Psychologist,
22, 255-278. doi: 10.1207/s15326985ep2203&4_4
Moore, D. (2012). Reading comprehension strategies. Cengage Publishing.
Retrieved from http://ngl.cengage.com/assets/downloads/edge_
pro0000000030/am_moore_read_comp_strat.pdf
Nelson, T. O., & Narens, L. (1990). A theoretical framework and new findings.
Psychology of Learning and Motivation, 26, 125-173. doi: 10.1016/S00797421(08)60053-5
Nuttal, C. (1996). Teaching reading skills in a foreign language (2nd ed.).
Oxford, England: Heinemann English Language Teaching.
Pintrich, P. R. (2002). The role of metacognitive knowledge in learning,teaching, and assessing. Theory Into Practice, 41(4), 219-225. doi: 10.1207/
s15430421tip4104_3
Powers, D., Kim, H. J., & Weng, V. (2008). The redesigned TOEIC (Listening
and Reading) Test: Relations to test-taker perceptions of proficiency inEnglish. ETS Research Report, No. RR-08-56. Princeton, NJ: ETS
Rasinski, T. V. (2010). The fluent reader (2nd ed.). Boston, MA: Pearson.
Schoenfeld, A. H. (1992). Learning to think mathematically: Problem solving,
metacognition, and sense making in mathematics. Handbook for researchon mathematics teaching and learning (pp. 334-370). New York, NY:
Macmillan.
Short, D. J., & Fitzsimmons, S. (2007). Double the work: Challenges andsolutions to acquiring language and academic literacy for adolescent
English language learners: A report to Carnegie Corporation of New York.
Washington, DC: Alliance for Excellent Education.
Schwartz, B. L., & Bacon, E. (2008). Metacognitive neuroscience. In J.
Dunlosky & R. A. Bjork (Eds.), Handbook of metamemory and memory (pp.
355-371). New York, NY: Psychology Press.
Takallou, F. (2011). The effect of metacognitive strategy instruction on
EFL learners’ reading comprehension performance and metacognitiveawareness. Asian EFL Journal, 13(1), 275-320. Retrieved from http://
342

The Effects of Metacognition, OnCUE Journal, 9(4), pages 329-343

www.ibrarian.net/navon/paper/The_Asian_EFL_Journal_Quarterly.
pdf ?paperid=19213415#page=275
Tregaskes, M. R., & Daines, D. (1989). Effects of metacognitive strategies onreading comprehension. Literacy Research and Instruction, 29(1), 52-60.
Weinstein, C. E., & Mayer, R. F. (1985). The teaching of learning strategies. In
M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 315329). New York, NY: Macmillan.
Weinstein, C. E., & Underwood, V. (1985). Learning strategies. In J. Segal &S. Chipman (Eds.), Thinking and learning skills (pp. 241-258). London,
England: Lawrence Erlbaum Associates.

Author bio

James Dunn is a full time teacher at Tokai University. His research interests are in
Cognitive Linguistics and Critical Thinking skills. His educational goal is to help
students understand that they are capable of more than they might expect fromthemselves. james.d.dunn@outlook.com
Received: February 4, 2015
Accepted: December 18, 2015343