Student Perceptions of Critical Thinking PracticeSalvador Murguia, Debra Occhi, Janette Ryan & Peter Verbeek1
1
Corresponding author, Email: pverbeek@miyazaki-mic.ac.jp

Abstract
The practices of articulating, defining, and studying critical thinking as an objective of
any liberal arts discipline are difficult at best. Experienced teachers may have a good
sense of how critical thinking can be encouraged or taught, but may have difficulties
in finding valid and reliable ways of assessing critical thinking outcomes. Moreover,few measures exists that track how students perceive and understand critical thinking
practice both in and outside the classroom. This study compared student perceptionsof critical thinking practice in four types of courses offered at an English immersion
liberal arts university in Japan. Students were provided with an on-line survey
containing 80 items describing critical thinking practices in interpretation, analysis,
evaluation, inference, explanation, and self-regulation. Upper-class students (N=62)
identified third and fourth year content courses taught in English by a single instructor
as the type of course in which critical thinking practice was significantly moreprevalent compared to both English and Japanese language courses taught by a single
instructor, as well as to 1st/2nd year team-taught content courses taught in English.
First year students (N=48) identified single instructor English language courses as thetype of course for which they perceived critical thinking practice to be most prevalent.
These results are discussed in the context of future assessment of critical thinking
practice by type of course as well as by individual instructor. Key words: criticalthinking assessment, liberal arts, English immersion, student perceptions, on-line
survey.

Acknowledgements

We wish to thank our MIC faculty colleagues who contributed ideas for items for the
Critical Thinking Survey (CTS). Special thanks go to Gregory Dunne for his
dedicated work on our committee and his thoughtful editing of the manuscript. We
also thank Seiko Hara, Katsuhiko Kai, Futoshi Kobayashi, Jun Maeda, and Lloyd
Walker for their expert help with the Japanese translation of the survey items.
Anderson Passos and Lloyd Walker deserve our thanks for setting up the on-line
administration of the CTS. Research is often a team effort and this work would nothave been possible without the help of these trusted colleagues. Finally, we wish to
thank the MIC students who volunteered their time to participate in the survey. We
hope that our joint faculty efforts will further benefit the development of their criticalthinking skills.

The mission statements of colleges and universities often explicitly underscore theimportance of critical thinking. Indeed, disciplines across the spectrum of higher
education reinforce the necessity of critical thinking throughout the trajectories of
their own array of courses (Goldsmid &Wilson, 1980; McPeck, 1990; Grauerholz &Bouma-Holtrop, 2003). Miyazaki International College (MIC), the English-based
liberal arts college in southern Kyushu, Japan, at which we teach, has at the core of its
academic program a “philosophy of critical thinking” (MIC, 2011a:4). “Thisphilosophy asserts that academic capability is not acquired through passive reading oftext or listening to lectures, but is achieved through explorative activities that require
students to be actively engaged in reading, writing and discussion as part of the processof problem solving. Through this kind of “active learning” (initiative based learning)
students engage in the dynamic development of higher order thinking skills that enable
them to analyze, synthesize, evaluate and create” (Ibid.).
The practices of articulating, defining and studying (Baker, 1981; Geerston,2003) critical thinking as an objective of any discipline are difficult at best
(Grauerholz & Bouma-Holtrop, 2003). In a rather brief but telling explanation,
Grauerholz and Bouma-Holtrop (2003) note the peculiarities critical thinking presents
for researchers:
Critical thinking seems to be much like good art: we know it
when we see it, we have some sense of how we might
encourage or even teach it, but we are not sure how to assessor measure it. (p.485)
Rather than pursuing what definitively constitutes critical thinking or trying todefine1 sharply the boundary between this concept and other similar interpretive
ventures (i.e. logic, reason, hermeneutics, etc.) this article focuses on how student
For a review of definitions of critical thinking as understood in the social sciences, across the liberal arts curriculum,and in the natural and medical sciences, see Grauerholz & Bouma-Holtrop (2003); Verbeek (2006).

perceptions of critical thinking practice may be tracked across a four-year liberal arts
curriculum.
At MIC student development is assessed throughout the four years both in andoutside the classroom. English skills are regularly tested via level exams and TOEIC.
However, critical thinking is difficult to assess via objective measures. This situation
has created problems in explaining what, beyond English skills and overseas
experience, MIC graduates have to offer the job market; critical thinking skills per seare often couched as problem-solving or communicative skills. Nonetheless, theattributes MIC students display in internships and interviews have helped the college
maintain a high percentage of job placement for its graduates relative to that of othertertiary institutions in Japan (MIC 2011b:38).
One venue for potential assessment of engagement in critical thinking has
been the course evaluation survey conducted in each class near the end of eachsemester (Appendix I). In early 2010 an ad hoc MIC faculty committee was formed toreview and possibly revise the course evaluation survey. One of the first actions of thecommittee was to conduct a detailed comparison of data by type of course generated
by the course evaluation instrument over eight semesters (2005-2008) [‘MIC Student
Feedback on Teaching. Some Questions and Preliminary Analyses.’ Committee reportavailable upon request]. The committee quickly realized that the critical thinking
section of the survey deserved improvement as the sole item referring to criticalthinking, “[the instructor] encouraged critical thinking,” showed either poor or nocorrelation with the other items on the form and was rated inconsistently across thedifferent types of courses [i.e. language; integrated (team-taught); specialized] thatwere evaluated.

As committee members noted that meanings associated with “criticalthinking” had their obvious limitations and may have, in effect, contributed to thevariance in the results, critical thinking became one point of departure for further
investigation. Most of the students at MIC have spent K-12 in the Japanese school
system. The three years of Japanese middle school and high school tend to be
conducted in a teacher / text-centered transmission style with little requirement for
active learner participation. For example, commenting on a lack of active learned
participation within Japanese education, Kawashima and Petrini (2004 cited in
Verbeek, 2006) state:

Learning skills that require students to formulate their own
questions in academia or social events are not encouraged,
and neither are autonomy and independent learning, all ofwhich have been associated with the cultivation of criticalthinking skills and dispositions.

Similarly, Nishibata (2010) noted that “until this situation is remedied education willcontinue to be limited and stagnant” (p. 229).
In addition to an educational background that may not be conducive to thedevelopment of critical thinking skills, the committee also focused on the Japanesetranslation of the term “critical thinking” as there was some concern that term itself
may be problematic. On the current evaluation form, the term “critical thinking” istranslated as
mondaiwhich, broken into its component parts means“problem / question” andishiki“consciousness.” The lack of aclear and corresponding translation suggests that students may either misinterpret or
experience confusion when answering a single question about critical thinking.

Outcomes assessments of instruction are commonly measured through student
evaluation of teaching (SET) instruments. In a general survey of overall methods, agreat deal of variation can result from such issues as administering SETs consisting ofdifferent class sizes (Bedard & Kuhn, 2008; Balam & Shannon, 2010); among in-classand online instruments (Sorenson & Reiner, 2003; Dommeyer, Baum, Hanna &Chapman, 2004); between qualitative and quantitative data gathering (Nasser-Abu
Alhija, & Fresko, 2009); and, of course, throughout different geographical regionsthat place different values upon the meaning of outcomes (Burden 2008; Davies,
Hirschberg, Lye, & Johnston, 2010). When designed properly, the results of these SET
instruments may serve to verify instructor performance (Mason, Steagall, & Fabritius,
1995) and increase students’ sense of participation in the educational experience.
Conversely, a number of these instruments have, at times, proven problematicto the careers of instructors (Newton, 1988; Sproule 2002; Stark-Wroblewski,
Ahlering, & Brill 2007), as well as depicting students’ educational experience
inaccurately.2If, for example, a survey instrument combines several items that bear nological relation to each other and then somehow summarizes these items bygenerating an overall average, it would follow that such an average or summarized
score would be arbitrary at best. Furthermore, if such aggregated items have thistenuous relationship, there would be no real basis for interpreting what is actually
being measured. With this line of reasoning, some scholars have emphasized amultidimensional approach geared toward capturing a larger breadth of items, andthus illustrating a more thorough and detailed evaluation of all educational

Indeed, it has been noted by several scholars that due to the limitations of some instruments, sole dependence uponthese evaluations for the retention, promotion and tenure of faculty may be equivalent to fostering a form ofpseudoscience, rather than anything resembling the scientific rigor fostered by higher education. See Hills, Naegle &Bartkus (2009) and Sproule (2002).

performances (Marsh, 1982; March and Hocevar, 1991).
The intersection of these SET instruments and our own focus on themeasurement of critical thinking is lacking in thorough research across the social
sciences, and has thus created several limitations for researchers. Among theselimitations are the arguments that the ambiguity of the term critical thinking, as wellas the lack of definitive research about it engenders substantial speculation as towhether or not critical thinking can even be taught (McPeck, 1985; 1990). It may
follow then, that measuring critical thinking is equally challenging. To complicate
these and other matters associated with research on critical thinking, located at theintersection of these limitations are difficulties in interpreting evaluative data as ameasure of student outcomes.
In spite of conceptual and practical difficulties such as discussed above,
incorporating student data generated through self-reporting methods (Shepelak et al,
1992; Tam, Pak, Hui, Kwan, & Goh, 2010) as well as quantitative and qualitative
surveys (Stoecker, Schmidbauer, Mullin, & Young, 1993), has certainly given way topromising departures in research on critical thinking in higher education (cf. Verbeek,2006). Here we report on our committee’s development, administration, and analysis
of a multi-scale on-line survey designed to track student perceptions of the teaching of
critical thinking skills across the four-year curriculum at MIC.

MethodThe ad hoc course evaluation committee comprised of the authors and Gregory Dunne,
acted as both a collegial advisory group as well as a research team interested in
exploring the possibilities of creating a useful course evaluation instrument. Meetingbi-weekly for roughly a nine-month period, we were able to review literature on
course assessments; discuss the advantages and disadvantages that evaluations present
for faculty, students and staff; assess the strengths and weakness of our own current
evaluation instrument; devise a novel critical thinking practice evaluation instrument,
the Critical Thinking Survey (CTS); run this instrument as a pilot; and finally, analyze
the data that this new instrument generated. The CTS research was reviewed andapproved by the Testing, Research and Assessment Committee (TRAC) at MIC.

Critical Thinking Survey (CTS)
In order to provide students with an understandable and accessible format forcomprehending the concept of critical thinking, the committee identified specificexamples of critical thinking in subject areas within the two main learning divisionsof the humanities and the social sciences at MIC. After consulting with various
faculty members from both of these divisions about how critical thinking is assessed
and recognized in their respective disciplines, the committee then generated a list ofitems that captured the larger breadth of their suggestions.
Survey scales and subscales. The decision was made to incorporate the skills
and sub skills of Facione [1990 adapted by Verbeek (2006)] into the item descriptorsof critical thinking. An initial list of 110 items was created but was reduced to 80
items (Appendix II) in the interests of keeping the survey to a manageable length. The
expertise and cooperation of bilingual faculty and staff was sought to translate these
items into Japanese to ensure that students had the option to read items in bothlanguages.

Table 1. Scales and sub-scales of the Critical Thinking Survey (CTS).
_____________________________________________________________________
Main scale
Sub-scales
Number of items_____________________________________________________________________
Interpretation
Decoding significance5Categorization
5
Clarifying meaning
5
AnalysisExamining ideas
5
Identifying arguments
5
Analyzing arguments
5
Evaluation
Assessing claims
5
Assessing arguments
5
Inference
Querying evidence
5
Conjecturing alternatives
5
Drawing conclusions
5
Explanation
Stating results
5
Presenting arguments
5
Self-regulation
Self-examination
10
Self-correction
5
Total

80

Table 1 presents the grouping of the final 80 survey items by 6 main scales andtheir corresponding sub-scales (N=15) that reflect the Facione (1990) critical thinking
skills and sub-skills. The 80 items were listed on the survey in random order. Student
participants were asked to check a box next to each of the following course typelabels: Japanese language courses, English language course, Other language courses3,
Team-taught courses, and 3rd/4th year courses, for the aspect of critical thinking
represented by the respective item that in their opinion was practiced in that particular
type of course. For each item students were asked to check all boxes that applied.
Through this method the students were asked for their memory-based perceptions ofcritical thinking practice by type of course. The data generated in this fashion thus
provide insight into how students perceived similarities and differences in theclassroom practice of critical thinking across the MIC curriculum. These data do not,however, provide insight into how students evaluated individual courses or instructors
on critical thinking practice.
On-line administration. In order to improve the survey-taking process, thecommittee decided to trial putting the new survey on line. The regular paper versionof the MIC course evaluation is given in-class at the end of each semester and thispractice takes up valuable classroom time and needs to be supervised by faculty not
being evaluated. An on-line survey would allow for a more sensible use of student andfaculty time. The other important benefit of an on-line mode would be ease of datacollection, manipulation, transmission to stakeholders and the creation of a databankof responses. With technical expertise and cooperation from the system administrator,
an on-line version of the CTS was developed and administered to a sample of 48 firstyear students and 62 third and fourth year students during the beginning of the 2011
fall semester.
Student sampling. All students present on campus during the planned time ofadministration (i.e. excluding 2nd year students on student abroad) were formallyasked to voluntarily participate in the CTS. However, initially only a small percentageof them actually did so. Faced with this situation, the authors encouraged the students
in their own courses to take the survey and in some cases escorted them to a computerlab to take the survey. Teachers of other courses, in particular those teaching Japanese
language courses, did the same. These joint faculty efforts resulted in a total
convenience sample of 110 students on which the analyses that follow are based.
Students took approximately 10 to 15 minutes to complete the CTS and their
responses were stored by student ID. Prior to the analyses the student IDs werereplaced by randomly assigned serial numbers thus ensuring the confidential nature of
their participation.
Results
1.

Statistical Analysis
Student responses were downloaded from the server and entered in a single

data file. For each checked box in a student record a value of 1 was recorded and foreach blank box a value of 0 (zero) and these selection data were used for all analyses
reported below. Analysis of Variance (ANOVA) and t-tests were used to test fordifferences among type of course and critical thinking scales and sub-scales. We used
two-tailed tests throughout. Controls for unequal sample size and variances were used
when necessary. We first analyzed the responses of the first year students followed bythose of the third and fourth year students. Finally, we compared the responses of thefirst year students to those of the third and fourth year students for the type of coursesthat both cohorts experienced so far, i.e. Japanese language courses, English courses
and Team-taught courses.
0.7

*

**

*

0.6

*

*

Japanese Courses
English Courses
Team-Taught Courses

0.5
0.4
0.3
0.2
0.1
0.0

Interpretation

Analysis

Evaluation

Inference

Explanation Self-regulation

Critical Thinking Category*p <0.05; **p <0.01

Figure 1. Main categories of critical thinking skills items selected by first year
students for Japanese-, English-, and Team-taught courses. Mean (range: 0-1) + SEM
are shown.

First Year Students (N=48)
Analysis by main critical thinking skills category. Two-way ANOVA withcritical thinking skill (interpretation, analysis, evaluation, inference, explanation,
self-regulation) and type of course (Japanese courses, English courses, Team-taught
courses) as main factors was used for the initial analysis. There was a significant main
effect for type of course, F 2,846 = 21.14. p < 0.0001, but not for critical thinking skill,
F 5,846 = 0.75, p = 0.59, nor for the interaction, F 10,846 = 0.60, p = 0.82.
Post-hoc Bonferroni tests showed that interpretation items were selected more
often for the English courses compared to both the Japanese courses (p < 0.05) andTeam-Taught Courses (p < 0.01) [Fig. 1; Table 3]. In addition, evaluation items wereselected more often for the English courses compared to the Team-taught courses (p <0.05), and the same pattern was found for explanation (p < 0.05) and self-regulation(p < 0.05) [Fig. 1; Table 3]. No significant differences were found between Japanese
Courses and Team Taught Courses. Taken together these initial results suggest that thefirst year students differentiated between the three types of courses and preferentiallylinked the teaching of interpretation, evaluation, explanation, and self-regulation
skills to the English courses.
Analysis by critical thinking sub skills. Separate two-way ANOVAS wereconducted to explore which critical thinking sub skills contributed significantly to thecourse differences in interpretation, evaluation, explanation, and self-regulation.
Each follow-up two-way ANOVA showed a significant main effect for type of course(p < 0.001; p < 0.05; p < 0.001, p < 0.001, respectively), but not for critical thinking
sub skill nor for the interaction.

Table 2. Post hoc comparison of critical thinking sub skill items selections by 1st year
students by type of course. Mean (range 0-1) + SEM are shown.
_____________________________________________________________________
Type of Course_____________________________________________
Main skill/sub skill
Japanese
English Team-taught_____________________________________________________________________
Interpretation
Decoding significance0.31 (0.05)

0.55 (0.06)

p < 0. 01

0.31(0.05)
p < 0. 01

Categorization

0.44 (0.06)

0.50 (0.06)

0.35(0.06)

Clarifying meaning

0.35 (0.05)

0.65 (0.04)

0.38(0.06)

p < 0. 001

p < 0. 01

Evaluation
Assessing claims

0.36 (0.05)

0.42 (0.06)

0.23(0.05)

p < 0. 05
Assessing arguments

0.47 (0.05)

0.45 (0.06)

0.35 (0.06)

0.50 (0.05)

0.50 (0.06)

0.28 (0.05)

Explanation
Stating results1

p < 0. 01
Presenting arguments

0.48 (0.05)

0.52 (0.05)

0.32 (0.05)
p < 0. 05

Self-regulation
Self-examination

0.36 (0.05)

0.55 (0.05)

0.30 (0.05)
p < 0. 01

Self-correction

0.36 (0.05)

0.48 (0.06)

0.30 (0.05)
p < 0.05_____________________________________________________________________
1
Japanese courses > Team-taught courses, p < 0.01.

The results of Bonferroni posttests are summarized in Table 2. As shown, there
were no significant differences among the courses neither for the interpretation sub
skill categorization nor for the evaluation sub skill assessing arguments.
The course differences on sub skills revealed by the posttests mirrored thosefound for the main skills; noticeably, compared to Team-taught courses, students
preferentially associated the teaching of all but two of the selected critical thinking
sub skills with the English courses. Items belonging to the interpretation sub skill
categories decoding significance and clarifying meaning were also selectedsignificantly more often for the English courses compared to for the Japanese courses.
There were no significant differences between Japanese courses and Team-taught
courses with the sole exception of the explanation sub skill category stating results forwhich students favored the Japanese courses.

0.6

***

***

***

***

***

***
Japanese Courses
English Courses
Team-Taught Courses
3rd/4th Year Courses

0.5
0.4
0.3
0.2
0.1
0.0
Interpretation

Analysis

Evaluation

Inference

Explanation Self-regulation

CRITICAL THINKING CATEGORY*** p < 0.001

Figure 2. Main categories of critical thinking skills items selected by third and fourthyear students for Japanese-, English-, Team-taught-, and 3rd/4th year courses. Mean
(range:0-1) + SEM are shown.

Third and Fourth Year Students (N=62)
Two-way ANOVA revealed a significant main effect for type of course(Japanese-, English-, Team-taught-, and 3rd/4th year courses), F 3,1464 = 95.99, p <0.0001, but neither for main critical thinking category, F 5,1464 = 1.32, p = 0.25, nor for
the interaction, F 15,1464 = 0.55, p = 0.92.
Bonferroni posttests indicated that for each of the six main critical thinking
skill categories the third and fourth year students selected 3rd/4th courses significantly
more often compared to English courses (p < 0.001), Japanese Courses (p < 0.001), as
well as the Team-taught courses (p < 0.001) [Fig. 2]. No significant differences were
found among the Japanese courses, English courses and Team-taught courses for anyof the main critical thinking categories.

Comparison of the Critical Thinking Selections of First Year Students (N=48) and
Third and Fourth Year students (N=62).
The critical thinking selections of the second semester first year students andthe third and fourth year students were compared for the type of courses that bothcohorts experienced: Japanese language courses, English courses and Team-taught
courses. Unpaired t-tests with Welch’s correction to control for unequal sample size
and unequal variance were used for this comparison. The results of this analysis are
summarized in Table 3. The comparison showed that the selections of the first year
students (shown in bold face) significantly exceeded those of the third and fourth year
students for both the Japanese language courses and the English courses, but not for
the Team-taught courses.

Table 3. Comparison of critical thinking skill item selections by 1st year students(N=48) and 3rd/4th year students (N=62) for Japanese-, English-, and team-taught
courses. Mean (range: 0-1) + SEM are shown.
_____________________________________________________________________

Critical thinking skill

1st year

3rd/4th year

students

studentsp_____________________________________________________________________
Japanese courses
Interpretation

0.37

0.05

0.21

0.03

0.009

Analysis

0.38

0.05

0.23

0.03

0.009

Evaluation

0.42

0.05

0.25

0.04

0.009

Inference

0.38

0.05

0.21

0.04

0.005

Explanation

0.49

0.05

0.28

0.04

0.001

Self-regulation

0.37

0.05

0.21

0.03

0.009

Interpretation

0.57

0.03

0.33

0.03

0.0001Analysis

0.47

0.05

0.23

0.03

0.0003

Evaluation

0.43

0.06

0.20

0.03

0.0006

Inference

0.46

0.05

0.20

0.03

0.0001

Explanation

0.51

0.05

0.27

0.04

0.0003Self-regulation

0.52

0.05

0.26

0.03

0.0001

Interpretation

0.33

0.05

0.23

0.03

n.s.

Analysis

0.31

0.05

0.22

0.03

n.s.

Evaluation

0.31

0.05

0.20

0.03

n.s.

Inference

0.26

0.05

0.20

0.03

n.s.

Explanation

0.30

0.05

0.23

0.04

n.s.

Self-regulation

0.30

0.05

0.21

0.03

n.s.

English courses

Team-taught courses_____________________________________________________________________

Discussion

Our study showed that when given the appropriate tools students clearly differentiate
between courses in terms of their perception and recollection of the degree and kindof critical thinking practice that different types of courses tend to offer. Third andfourth year students identified single-taught third and fourth year courses as the typeof courses in which critical thinking practice was offered to a significantly greater
degree than in any of the other types of courses in the MIC liberal arts curriculum. In
fact, this was true for each of the six main critical thinking skills (interpretation;
analysis; evaluation; inference; explanation; self-regulation) measured by the 80 item
Critical Thinking Survey (CTS) that we designed for this study.
First year students identified English courses as the type of course in which
several aspects of critical thinking practice were significantly more often part of theclassroom proceedings compared to team-taught courses and Japanese courses. For
example, the responses of the first year students suggest that critical thinking practicein interpretation, and, specifically, in decoding significance and clarifying meaning,occurred significantly more often in the English courses compared to both Japanese
language courses and team-taught courses. Practice in evaluation (assessing claims),
and explanation (stating results and presenting arguments), was rated by the first year
students as being significantly more prevalent in English courses than in team-taught
courses. Moreover, the responses of the first year students suggest that compared toteam-taught courses English courses were significantly more likely to inspire students
to engage in critical self-examination and self-correction in the course of their studies.
Finally, when we compared the responses of the first year students with thoseof the third and fourth year students for the courses that both cohorts experienced, wefound that first year students consistently rated the incidence of critical thinking
practice in both Japanese language and English courses significantly higher than thethird and fourth year students did. There was no difference between the two cohorts intheir perception and recollection of critical thinking practice in team-taught courses,
which they both identified as being infrequent.
What do these results tell us? Can we assume that the student perceptions ofdifferences in critical thinking practice among the types of courses more or lessaccurately reflect what goes on in terms of critical thinking practice in MIC
classrooms? When we reflect on these questions we need to consider both thestrengths and the weaknesses of our study. Starting with the latter, the conveniencesample that we were able to work with (N=110), although sizeable in terms of thecurrent total student enrollment (N=260), was limited in terms of its make-up, as it did
not include second year students who were on study abroad at the time of the study.
Moreover, the sample was a convenience sample, which does not rule out biased
responding due to a particular motivation (or lack thereof) to participate in the CTS.
Future studies should either use equal random samples taken from first- throughfourth year student cohorts, or preferably, plan on surveying the entire student body.
Another limitation of our study is that the CTS administration procedurerequired student participants to recall their perceptions of critical thinking practicefrom memory. While recall from memory is a common procedure for most kinds ofstudent evaluations of teaching, in this particular case the time frame differed between
the two cohorts. We asked third and fourth year students to reflect on team-taught
courses, which is a type of course they had taken two or three years ago. In contrast,
first-year students were either enrolled in a team-taught course at the time they tookthe CTS, or had been enrolled in this type of course during the previous semester.
Interestingly, the perceptions of critical thinking practice in team-taught courses did
not differ significantly between the two cohorts.
The main accomplishment of our study is that for the first time in the historyof the institution it provides a detailed look at student perceptions of critical thinking
practice in the different types of courses offered at MIC. As such it constitutes a major
departure from the limited way critical thinking practice has traditionally been
evaluated by MIC students, namely through a single item at the end-of-the-semester
course evaluation form [“(The instructor) encouraged critical thinking”]. We hope, as
we suggest in more detail below, that the CTS in its current -or in a modified form
will became a standard assessment tool at MIC.
One of the strengths of our study is that we polled students on multiple aspectsof critical thinking practice without ever mentioning the term critical thinking.
Instead we asked students to select from a set of descriptions of critical thinking
classroom practices that we developed based on previous research and the ideas andsuggestions of faculty colleagues who teach the type of courses listed on the CTS. In
this sense we believe that the CTS is a valid measurement tool of student perceptionsof critical thinking classroom practice at MIC.
Another strength of our study is that the CTS is designed to generate data thatallow for detailed comparisons among the various types of courses taught at MIC.
Traditional course evaluations have been designed to provide data on individual
courses and instructors, and such individualized data do not lend themselves well tovalid comparisons among course types.
In conclusion, we clearly see a future for the CTS as the instrument of choice

to track trends in critical thinking practice at MIC. Keeping a finger on the pulse ofcritical thinking practice is important for faculty and administration alike as fostering
critical thinking is central to the mission of the institution. We would like to suggest
administering the CTS each semester, perhaps at the time of course registration. CTS
data could be stored by individual student, and multiple assessments during the 4-year
curriculum would allow for longitudinal developmental trend tracking, bothindividualized by student, as well as aggregated by cohort or gender, or other salient
student characteristics (e.g. TOEIC scores, etc.).
As for the future assessment of critical thinking practice by individual course
and instructor we suggest to revise the current student evaluation form byincorporating a selection of the most salient items from the CTS. To that end ourcommittee will conduct a detailed item analysis using the current dataset and makerecommendations to the Faculty Council based on the outcome of this analysis.

References

Baker, P.J. (1981). Learning Sociology and Assessing Critical Thinking. Teaching
Sociology, 8, 325-63.
Balam, E. M., & Shannon, D. (2010). Student Ratings of College teaching: A
Comparison of Faculty and Their Students. Assessment & Evaluation inHigher Education, 35, 209-221.
Bedard, K. & Kuhn, P. (2008). Where Class Size Really Matters: Class Size andStudent Ratings of Instructor Effectiveness. Economics of Education Review,
27, 253-265.
Burden, P. (2008). ELT Teacher Views on the Appropriateness for Teacher
Development of End of Semester Student Evaluation of Teaching in a
Japanese Context. System, 36, 478-491.

Davies, M. Hirschberg, J., Lye, J. & Johnston, C. (2010). A Systematic Analysis of
Quality of Teaching Surveys. Assessment & Evaluation in Higher Education,
35, 87-100.
Dommeyer, C.J., Baum, P., Hanna, R.W., & Chapman K. (2004). Gathering Faculty
Teaching Evaluation by in-class and Online Surveys: Their Effects on
Response Rates

and Evaluations. Assessment & Evaluation in Higher

Education, 29, 611-623.
Geertsen, R. (2003). Rethinking Thinking about Higher-Level Thinking. Teaching
Sociology, 31,1-19.
Goldsmid, C.A. & Wilson, E.K. (1980). Passing on Sociology: The Teaching of a
Discipline. Washington D.C: ASA Teaching Resource Center.
Grauerholz, L. & Bouma-Holtrop S. (2003). Exploring Critical SociologicalThinking Teaching Sociology, 31, 485-496.
Hills, S.B., Naegle, N., & Bartkus, K. (2009). How Important are Items on a Student
Evaluation? : A Study of Item Salience. Journal of Education for Business,
May/June, 297-303.
Jang, S., Guan, S., & Hsieh, H. (2009). Developing an Instrument for Assessing
College Students’ Perceptions of Teachers’ Pedagogical Content Knowledge.
Procedia Social and Behavioral Sciences, 1, 596-606.
Kawashima, A., & Petrini, M. A. (2004). Study of critical thinking skills in nursing
students and nurses in Japan. Nurse Education Today, 24, 286-292.
Marsh, H. W. (1982). SEEQ: A Reliable, Valid, and Useful Instrument forCollecting Students’ Evaluations of University Teaching. British Journal of
Educational Psychology, 52, 77-95.
Marsh, H. W. & Hocevar, D. (1991). The Multidimensionality of Students’Evaluations of Teaching Effectiveness: The Generality of Factor Structures
across Academic Discipline, Instructor Level, and Course Level. Teaching &Teacher Education, 7, 9-18.
Mason, P. M., Steagall, J.W., & Fabritius, M. M. (1995). Student Evaluations ofFaculty: A New Procedure for Using Aggregate Measures of Performance.
Economics of Education Review, 14, 403-416.
McPeck, J.E. (1985). Critical Thinking and the 'Trivial Pursuit' Theory ofKnowledge. Teaching Philosophy, 8, 295-308.
McPeck, J.E. (1990). Critical Thinking and Subject Specificity: A Reply to Ennis.
Educational Researcher 19(4),10-12.
Miyazaki International College (2011a). 2011 Bulletin. Kiyotake, Japan.
Miyazaki International College (2011b). 2011 Daigaku Annai. Kiyotake, Japan.
Newton, J.D., (1988). Using Student Evaluation for Teaching in Administrative
Control: The Validity Problem. Journal of Accounting Education, 6, 1-14.
Nasser-Abu Alhija, F. & Fresko, B. (2009). Student Evaluation of Instruction:
What can be learned from students’ written comments? Studies in Educational
Evaluation, 35, 37-44.
Nishibata, J.C. (2010). The Current State of Affairs in Japanese Education.
In D. K. Sharpes (Ed.), Handbook on International Studies in Education (pp.
219-232). Charlotte, NC: Information Age Publishing.
Richardson, J.T.E. (2005). Instruments for Obtaining Student Feedback: A Reviewof the Literature. Assessment & Evaluation in Higher Education, 30, 387-415.
Shepelak, N.J., Curry-Jackson, A. & Moore, V.L. (1992). Critical Thinking in
Introductory Sociology Classes: A Program of Implementation and Evaluation.
Teaching Sociology, 20, 18-27.
Sproule, R. (2002). The Underdetermination of Instructor Performance by Datfrom
the Student Evaluation of Teaching. Economics of Education Review, 21,287-294.
Sorenson, D.L. & Reiner, C. (2003). Charting the Uncharted Seas of Student
Ratings of instructions. New Directions for Teaching and Learning, 96, 1-24.
Stark-Wroblewski, K., Ahlering, R.F., & Brill, F.M. (2007). Toward a More
Comprehensive

ApproachtoEvaluating

Teaching

Effectiveness:

Supplementing Student Evaluations of Teaching with Pre-Post Learning
Measures. Assessment & Evaluation in Higher Education, 32, 403-415.
Stoecker, R., Schmidbauer, M., Mullin, J. & Young, M. (1993). Integrating Writing
and the Teaching Assistant to Enhance Critical Pedagogy. Teaching Sociology,
21, 332-40.
Tam, K., Pak, S. T., Hui, C.H., Kwan, S., & Goh, M. K. H. (2010). Implicit Person

Theories and Change in Teacher Evaluation: A Longitudinal Field Study.
Journal of Applied Social Psychology, 40, 273-286.
Verbeek, P. (2006). Critical Thinking in Higher Education. Comparative Culture,
12, 95-101.

APPENDIX I
In this course we learned new concepts to help our thinking.
In this course we learned to tell the difference between reasonable and unreasonable arguments.
In this course we were challenged to solve problems.
In this class we looked at arguments from various perspectives.
In this course we learned to focus on the key points when we present an argument.
In this course we learned to explain how we arrived at a specific conclusion.
In this course we learned to clearly organize our thoughts.
In this course we checked whether evidence is plausible.
In this course we learned to decide whether or not an argument is valid.
In this course we learned to present the good and bad points of an idea.
In this course we looked for the logic in arguments.
In this course we learned why some ideas are important and others are not.
In this course we made connections between our learning and the world.
In this course we applied our knowledge to new situations.

In this course we were taught to ask questions that help us understand.
In this course we were encouraged to question ideas.
In this course we examined our own ideas and feelings.
Because of this course I am motivated to stay well informed.

In this course we learned to draw conclusions that are consistent with one another.
In this course we learned to look for connections between issues.
In this course we learned to recognize good evidence.
In this course we read texts and interpreted meaning.
In this course we learned to develop an informed opinion about things.
In this course we learned the difference between a wild guess and an educated guess.
In this course we looked for all possible explanations for an event.
In this course we learned why some popular beliefs are false.
In this course we learned to ask the right questions to help us learn.
In this course we learned to recognize and correct our own biases and prejudices.
In this course we learned to challenge our own ideas.
In this course we learned to identify biased opinions.
In this course we learned about the relationships between issues.

Because of this course I am more willing to reconsider and change my views.
In this course we were challenged to change our opinions based on new evidence.
In this course we learned about different ways of testing a premise.
In this course we learned the difference between a personal opinion and an informed opinion.
In this course we learned to clearly express our thoughts.
In this course we learned not to blindly accept some conclusions or statements.
In this course we looked for explanations that are relevant and can be tested.
In this course we learned to support our opinions with reasons.
In this course we learned to summarize and paraphrase reading passages.
In this course we studied the cause and effect of things.
In this course we learned why some things belong together and others do not.
In this course we investigated why people can have different ideas about an issue.
In this course we learned the importance of being fair in our criticism.
In this course we looked for the reasons why people hold certain opinions.
In this course we learned how to prepare a convincing presentation step-by-step.
In this course we learned to keep an open mind for new ideas.
In this course we learned to organize information systematically.
In this course we learned not to draw hasty conclusions.

In this course we compared and contrasted ideas.
In this course we learned to present results based on evidence.
In this course we tested the advantages and disadvantages of competing ideas.
In this course we learned to make a strong case based on clear premises and conclusions.
In this course we tested hypotheses.
In this course we learned to draw conclusions about characters in stories from their actions and dialogue.
In this course we learned to question premises as reasons for accepting a conclusion.
In this course we learned to look for premises likely to lead to the conclusion.
In this course we learned to focus on the most important parts of a problem.
In this course we looked for similarities and differences between issues.
In this course we learned the difference between inductive and deductive reasoning.
In this course we monitored our own progress and sought help when we fell behind.
In this course we learned to develop new ways of solving an old problem.
In this course we learned how to draw logical conclusions from evidence.
In this course we learned to identify the main issues of a problem.
In this course we learned to recognize weaknesses in our arguments.
In this course we tested the evidence for popular beliefs.

In this course we analyzed language in order to learn.
In this course we had the opportunity to write a defensible thesis and develop an argument.
In this course we learned to understand the reasons behind opinions.
In this course we learned to form explanations.
In this course we learned to explain the reasons for my opinions.
In this course we learned to check whether evidence is credible.
In this course we studied the pros and cons of an argument.
In this course we learned to evaluate the merits of the literature we read.
In this course we learned to look for the evidence behind an argument.
In this course we learned to tell the difference between fact and opinion.
In this course we checked evidence for a reliable source.
Because of this course I am more likely to persist until I find the correct answer to a problem.
In this course we learned to be logical when we present our arguments.
In this course we learned to look for other valid ways of obtaining evidence.