IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Measurement and Analysis of Student (Dis)engagement in
Higher Education: A Preliminary Study

Akihiro Saito
Hakuoh University, Japan
Michael E. Smith
Aomori University of Health and Welfare, Japan

29

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Abstract
Higher education is attracting more participation from an increasingly diverse student body.
This diversity invites concerns on effective instructional delivery as the extent of students’
engagement in learning now varies widely. Anecdotes on students’ “undesirable” dispositions
in course participation are not uncommon in higher education settings. This project set out todevelop a questionnaire, developed for higher education in the Japanese context, on a range ofstudent dispositions. The scale was a five-point Likert instrument designed to interpret learners’
disengagement as an attitudinal disposition. The paper discusses the conceptual contours ofdisengagement as a student disposition that provided the basis for the context-specific scale
items. It reports the procedures taken to obtain the factor structures of the dataset. The
questionnaire was administered to 145 engineering students in Japan. An exploratory factor
analysis revealed a five-factor solution – lack of commitment, distractedness, lack of
preparedness, anti-social orientation, and lack of focus. Avenues for further research aresuggested, and implications for practice are discussed.
Keywords: higher education; learner engagement; student attitudes; rating scales.

30

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Introduction
Participation in higher education is increasing worldwide, and the demands placed on postsecondary education are changing (Laurillard, 2012). A much larger proportion of the
population take on a degree program at universities, and higher education has become muchless elitist. In Japan, this trend was observed earlier than in other parts of the world. The
relaxation of regulations governing the organization of universities and their education
provision took place in the 1990s (Amano, 2014). This liberalization of higher education led
to a rapid increase of tertiary institutions. This phenomenon was accompanied by a steady risein the numbers of university goers. Higher education enrolment in Japan is now as high asnearly 75% (Kariya, 2011) with 56.7 % for enrollment in 4-year institutions in 2014 (seeHarada, 2015). However, enrollments are decreasing quickly as the number of 18-year-olds
continues to drop in an increasingly aging population.
In the mid-2000s, following an extended period of economic stagnation, Japan seemed to entera new phase of economic recovery. The role that public reforms played was important in laying
the foundation for growth during this period. From the late 1990s onward, at the center of thegovernment’s reform measures for an economic turn-around was the tertiary education system.
Particularly notable was the Koizumi administration’s partial privatization of national
universities (Yamamoto, 2004). These higher education institutions now operate ascorporations with considerable autonomy from centralized control. Education is a politicalissue as it is framed and funded by the state to a great measure, and this invites competingpressures from market and business industry as well as from academia and education lobbyists.
There is now a general and widespread recognition that tertiary education is a driving force inboosting economic competitiveness in a knowledge-driven globalized economy (Newby, Weko,
Breneman, Johanneson, & Maassen, 2009). This entails that higher education provides
conditions for learners to develop a reasonable level of academic achievement and practicalskills across disciplines, while simultaneously, balancing these skills with professional
knowledge. This trend has been gaining traction in higher education both abroad and in Japan.
It is now the role of tertiary education institutions to provide the conditions and opportunities
for shaping the student mindset toward lifelong learning and personal development.
Increasing participation in higher education worldwide has brought greater diversity to thestudent body in terms of academic bent (Altbach, Reisberg, & Rumbley, 2009). Increasing
enrollments suggest that the degree of academic orientation and commitment of students is
accordingly varied. Within a university, the range of dispositions and ability within courses isdiverse. In the sociological literature, it is axiomatic that a household’s standard of living is
implicated in access to higher education, and books in the home are a proxy for cultural capitalin the family (Kariya, 2011). The economic and cultural capital and students’ dispositions
nurtured by these resources increase both the chance of enrolling in higher education andstudents’ academic success at university (e.g., Kariya, 2012). These socio-economic indicatorsand cultural background variables translate into observable differences, such as whether
students have relevant background knowledge and a curiosity about a particular subject.
Namely, there are now a considerable number of students who would not have considered or
been able to go to university a mere generation or two ago.
Thus, it is not just that enrolment rates are higher than at any other time in history, the emphasis
is now on concerns about effective teaching and outcome. Conceiving ways to actively engage
students in their learning now poses a real challenge. In the Japanese higher education context
as well as in other parts of the world, the challenge that educators face is how to empower

31

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

students so that they take greater autonomy for their learning. Addressing this concern requires
learner-centered teaching methods such as an active learning approach, which more effectively
stimulates students’ use of higher cognitive activities that a successful student would use
spontaneously (American Psychological Association [APA] Work Group of the Board of
Educational Affairs, 1997; Cavanagh, 2011; Prince, 2004). Concord between the current
instructional models, new pedagogical trends, and students’ needs are necessary as Japanesesociety continues to evolve as a post-industrial knowledge-based economy. This paper
addresses one roadblock to this endeavor common in Japan: teachers sometimes find thatstudents do not engage in a way that is expected or desired for learning and success in higher
education settings. For instance, Escandon (2004) refers to lesson disruption in a Japanesetertiary setting, “The class objectives cannot be achieved because most of the students avoid
learning … or keep other fellow students from engaging [in] learning practices, [and] teachers
have to commit themselves to disciplining students instead” (p. 3). While acknowledging thatthese unnervingly “undesirable” dispositions ought not to be interpreted from the instructor’s
perspective alone, as we discuss subsequently, the paper reports on a project in which a scale
was developed to measure students’ disengagement as a disposition in higher education.
Engagement is a key construct linked with learning and academic success, such as higher
grades, completion rates, and achievement test scores: Evidence suggests that engagement is
responsive to changes in pedagogic practices and holds great potential as a key target ofintervention and improvement efforts (Fredricks, Filsecker, & Lawson, 2016). The scale
reported in this study was designed to measure the dispositions of students displayed in relation
to current university learning, with the scale being the interpretative continuum of thedialectical attitude of compliance with and resistance to the institutional assumptions whichshape “desirable” forms of engagement in tertiary education settings in Japan.
Literature Review
The literature on engagement abounds with a variety of theoretical inclinations and pedagogicalfocuses (see Christenson, Reschly, & Wylie, 2012). While there is much variation regarding
how engagement has been defined and researched, there is some agreement that it is amultifaceted construct (Fredricks, Filsecker, & Lawson, 2016). In the sociological literature,some undesirable forms of engagement are conceived as resistance, which represents students’reactions to a school’s attempts at defining them as a person of less worth than others in a waythat often results in their exclusion from the path to academic achievement (Alpert, 1991).
From this perspective, failure to learn emerges as a result of political resistance, not of an innate
disability. The literature states that resistance could take shape in a subtler form. Students canminimize participation in school practices while displaying exterior conformity to theideological assumptions of schooling as a mode of quiet subversion (Giroux, 1983). In more
psychologically-oriented studies, undesirable forms of participation are understood as
misbehavior which indicates a behavior that is perceived by the instructor to interfere with theprimary vector of the learning activity. Other scholars have addressed the issue using school
connection and life course theories to explain the role engagement plays in whether or not a
student completes his or her schooling (Fredricks, 2014). Along with this line of inquiry, somestudies report students’ perspectives, which suggest that they view schooling as standing in theway of them socializing with their peers, leading them to invest minimal effort in their school
work.
Some past studies operated within only the teacher and school’s perspective and tended to
misconstrue the meaning of students’ oppositional dispositions (Alpert, 1991). Alpert argues

32

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017that both conceptions of students’ dispositions as resistance and misbehavior are limiting as
some oppositional dispositions could be construed as legitimate modes of participation in a
democratic society of which the schools are a part. Students’ limited engagement in classroom
discussion and criticism of the instructor over instructional delivery, for example, can becommonly observed phenomena. Notably, these modes of rejection and challenge are revealed
without resorting to violence or leading to class disruption. Hence, while investigating
disengagement as an oppositional disposition is important, labeling these as mere misbehaviors
can be inaccurate as they do not involve formal rule violation. In the Japanese higher education
context, McVeigh (2002) and Escandon (2004) reported their observations of disengagementin the form of students’ “disruptive practices” that are manifested as students’ resistance –apparent apathy, neglect, and lack of interest, among others. The theoretical position adopted
by McVeigh regarding resistance was unlike that which had been described in the sociologicalliterature. Rather, he states that resistance “designates actions and attitudes that do not directly
challenge but scorn the system. This form of subtle resistance ignores … and is a type ofdiversion … from … the dominant structures” (McVeigh, 2002, pp. 185–186). Drawing upon
McVeigh’s (2002) conceptions based on anecdotes and his personal observations in theJapanese context, Escandon (2004) enumerated possible forms of resistance in the following
categories 1 to 7.
1. Bodily dispositions: These describe physical conditions in which students are positioned.
These conditions are thought to exercise impact over whether desirable or undesirable
learning comes to be shaped.
2. Absence: Expressed in students’ attitude and behavior that denigrate the importance ofattending and participating in class activities.
3. Unresponsiveness: Displayed in students not responding and pretending not to know.
4. Neglect and forgetfulness: These behaviors manifest as willful inattention and a learned
neglect of in-class activities, translating into forgetfulness as regards learning materials andassignment due and exam dates.
5. Indifference: This form of resistant attitude manifests itself in a range of behaviors such as
sleeping in class, daydreaming, and not taking notes.
6. Inaccuracy: This is observable in end-products, such as disregarding lecture points and task
instructions, such as essays.
7. Rudeness: This is manifested in behaviors which are disrespectful to the teacher and peers,such as chattering with friends and ignoring requests to be quiet (pp. 6–8).
We stress that these are not empirically established categories and only informed theconstruction of the questionnaire items that we have modified and added to as reported
subsequently (see Appendix). We therefore avoided making any presumption about whether
the items in our questionnaire would cluster together in the same manner as the asserted
categories as above may imply. We note that there is a difference between asserting thesecategories from the instructor’s personal observation and demonstrating these as well as latent
explanatory factors in an empirically based manner. McVeigh (2002) and Escandon (2004) areamong the few scholars who have investigated disengagement in higher education in Japan.
However, neither of them examined the extent of Japanese tertiary students’ dispositions per
se. Escandon attempted to ascertain whether students would assess a list of simulated student
behaviors as being disruptive or not disruptive as well as their perception of the extent of thesebehaviors being observed within the research site (see Escandon, 2004). Therefore, in order tobuttress our understanding of current students’ dispositions in the Japanese context, we sought
to measure their dispositions as reported by students themselves.

33

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Methodology
Participants
A convenience sample of 145 engineering students at a regional university in the northern partof Japan participated voluntarily. The authors, Saito and Smith, teach English as a Foreign
Language to first- and second-year students. In some of his classes at the university, Saito
distributed information about the project which included details such as the purpose of the study,
the research policy assuring participants’ anonymity and privacy, and an estimate of the timerequired to complete the survey online. Participants’ privacy and anonymity throughout the
research process included explaining the use of quotations, and informed consent that they
could withdraw from the project at any time without incurring any penalty. Written consent
was obtained from those who agreed to take part in the survey. Initially, 146 responses werereturned, but one incomplete survey was excluded from analysis. The sample was dominantly
male (n = 123, 84.8%), and the proportion of females was 15.2 percent (n = 22). Their ages
ranged from 18 to 21. The participants majored in six different areas of study including system
and information engineering (n = 48), mechanical engineering (n = 26), electronic-electricalengineering (n = 25), civil engineering and architecture (n = 23), design engineering (n = 13),and bioenvironmental science/engineering (n = 10).
Apparatus
The current study sought to empirically explore the structure of students’ dispositions throughthe questionnaire items (Appendix). The current study, like other studies of learner engagement,
did not draw upon any single definition or conceptualization of the multifaceted construct ofdisengagement. We sought to examine a wide range of undesirable dispositions such as
disengagement to determine whether context-sensitive learner dispositions could be measured,
what perceptions and behaviors they displayed, and how their responses related to one another.
The following priorities guided the development of the instrument: (a) the need for a contextspecific questionnaire, which taps into dispositions relevant to a higher education setting inJapan; (b) the instrument to which respondents can respond in their first language, Japanese;and (c) the context-sensitivity informed by local professionals’ observations and experience aswell as the literature.
The item pool of the scale was drawn from the conceptions of student dispositions as observedin Japanese higher education contexts (Escandon, 2004; McVeigh, 2002). We added a group of
items to this list of dispositions so that we would be able to see the degree to which students
were adaptable to more dialogic, active learning approaches, as noted earlier, toward whichcurrent higher education pedagogical practices are increasingly inclined. This ad hoc constructreflects student attitudinal dispositions toward a classroom pedagogic style, which draws upon
the more active learning mode of education (Cavanagh, 2011; Prince, 2004). A total of 34 items
were selected from the item pool initially. These items constituted the self-report questionnaire
consisting of two parts. In Part 1, 34 Likert-scale items elicited participants’ self-reports of theirattitudes including behavioral dispositions in relation to university learning. Part 2 asked
participants’ demographic information such as gender and major. The Likert-scale items werecreated and refined in accordance with guidelines for the development of a valid and reliable
inventory (Dörnyei, 2003).
In Part 1, a five-point Likert scale where 1 represents “never” and 5 represents “frequently”
was used for all the items (never = 1, rarely = 2, sometimes = 3, occasionally = 4, frequently =34

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 20175). These 34 five-point Likert scale items were randomized in order in an online format. The
questionnaire items were drafted in Japanese and reviewed by two English-Japanese teacherswith seven years of experience for translation accuracy and readability for Japanese students.
The questionnaire was pilot-tested with three engineering students (aged 19, 20 and 21). Basedon the feedback obtained from these informants, the wording in the questionnaire items wasrevised for accuracy and validity.
Procedure
A cover letter was distributed among potential participants, which included an explanation of
the aim of the study and assured potential participants’ anonymity and right to withdraw in the
midst of participation. This was done through some regular classes at Saito’s workplace. In thisway, informed consent was obtained from all the potential participants. Participants were asked
to respond to items on the final Japanese version of the online questionnaire. They entered their
responses manually from chosen computers or mobile devices online outside of class time. The
data entry period lasted for two months. The completed questionnaires were computer-coded.
The data were screened and analyzed using Statistical Package for Social Sciences 22.0. We
used exploratory factor analysis, which was designed as a means to explore a dataset as wellas to develop an instrument which could be utilized and refined through other inferential
options such as confirmatory factor analysis, among others, to test hypothetical constructs (seeCostello & Osborne, 2005, p. 8).
The data were subjected to tests of normality and skewness. A graphical inspection of the data
indicated that the distributions for a large majority of the items were bell-shaped, except items
6 (skewness = 2.26, kurtosis = 5.89) and 28 (skewness = 2.89, kurtosis = 9.68). The literaturesuggests that factor analytic procedures that employ maximum likelihood extraction methods
are not adversely affected if skewness is smaller than 2.00 and kurtosis not larger than 7.00
(Fabrigar, Wegener, MacCallum, & Strahan, 1999). Because they exceeded these thresholds,
items 6 and 28 were excluded from further analysis (see Appendix). The values of skewness
and kurtosis of the remaining 32 items came in the ranges between +.037 and +1.95, andbetween -.89 and +5.89, respectively. For these items, maximum likelihood exploratory
analysis was used with a threshold of .45 for factor loadings as a result of the study’s sample
size of 145 (Hair, Black, Babin, & Anderson, 2009).
Results
Factor Structure
A factor analysis on the 32 items was performed using maximum-likelihood extraction withpromax rotation. The Kaiser-Meyer-Olkin values exceeded the recommended value of 0.6(Kaiser, 1974), and the Bartlett’s Test of Sphericity reached statistical significance (p = .000),
supporting the factorability of the correlation matrix. An initial analysis was run to see
eigenvalues for each factor in the data. Based on the minimum eigenvalue of 1 and a thresholdof .45 for factor loadings (Hair et al., 2009), nine cross-loaded items (1, 9, 12, 14, 17, 19, 24,
33, 34) were culled. The same procedure was iterated three more times, and further seven items(2, 3, 4, 10, 21, 25, 32) were dropped. The remaining 16 items yielded a five-factor solution
accounting for 65.8% of the total variance, with KMO = .819, and the Bartlett’s Test of
Sphericity reached statistical significance (p = .000), indicating a reasonable factor analysis(Bartlett, 1954).

35

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Table 1 shows the factor loadings after rotation. The four items that clustered on Factor 1
accounted for 34.0% of the common variance. These items reflected students’ behavioral
tendency such as absence (I repeatedly cut class) and failure (I fail exams). We named thisfactor lack of commitment because these items indicate students’ inability to act while they are
responsible for attendance and preparation as necessary requirements. Factor 2, which
accounted for 12.3% of the common variance, included five items that related to nonparticipation and diverting elements, such as chatting and socializing with peers while
attending class. Therefore, we labeled this factor distractedness. Factor 3, which explained
7.9% of the common variance, included three items. These items indicated the degree ofstudents’ engagement with course requirements, such as homework and assignment preparation,and thus, it was named lack of preparedness. Factor 4 included two items, accounting for 7.5%of the total variance, with the factor loadings of .70 and .90. This factor was labeled anti-social
orientation because the items were related to students’ willingness to collaborate with peers
who they do not know. Factor 5, the proportion of variance explained by this factor was 6.6%,
with factor loadings of .64 and .84. Factors 4 and 5 were measured by only two items each, but
we decided to retain these as independent components in terms of interpretability; both pairsof items loaded on each component in a conceptually meaningful way. Furthermore, inspectionof scree plots revealed a clear break after the Factor 5, and these five factors met the Kaiser
greater than 1 criterion (Kaiser, 1960). Table 1 displays the estimate of reliability for eachextracted factor. The extracted factors demonstrated good internal consistency, ranging
from .74 to .80. In brief, five factors were extracted from the 16 items. These components
represent lack of commitment, distractedness, lack of preparedness, anti-social orientation, andlack of focus factors, which together are hypothesized to constitute disengagement among thisparticular sample of students. The mean scores of the five factors were lower than the midpoint(3) of the 5-point Likert scale: lack of commitment (M = 1.88), distractedness (M = 2.12), lackof preparedness (M = 2.36), anti-social orientation (M = 1.85), and lack of focus (M = 2.43)(Figure 1). The mean values indicate that instances of resistance and misbehavior are notwidespread but less than moderate as far as our results suggest.
Table 1: Factor-loadings for exploratory factor analysis with maximum likelihood rotation ofdisengagement items (N = 145).
Statement
F1 Lack of commitment (α = .79)
5 I repeatedly cut class.
7 I oversleep and cut class.
8 I cut class and do something of priority outside school (e.g.,socializing with friends, part-time job, family, etc.).
23 I fail exams.
F2 Distractedness (α = .77)
26 I chat about things unrelated to lecture contents.
13 I pretend as if I were paying attention, but I am actually
doing something else such as texting on my phone and doing
assignments for another course.
29 I disregard requests to be quiet and soon get back tochatting with friends.
11 When called on I discuss the question being asked of and/or
the response with fellow students before giving an answer.
27 I use digital devices such as smartphones for non-class36

Loading

M

SD

1.02
.618
.563

1.99 1.014
1.97 1.076
1.56 .686.505

1.99 1.024

.711
.681

2.18 .887
2.15 .974.654

1.48 .718.596

2.37 .912.484

2.43 1.026

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

purposes during class.
F3 Lack of preparedness (α = .80)
15 I do not remember that I had assignments to do until I come .787
to class.
20 I attend class without completing homework.
.718
16 I forget assignment deadlines and exam dates.
.509
F4 Antisocial orientation (α = .77)
30 In group or pair work, I will not talk to the other student(s) .907
if they are people I don’t know.
31 I will not do pair work if the other student is a person I .700
don’t know.
F5 Lack of focus (α = .74)
18 I daydream during class with my mind focused on nothing .842
in particular.
22 I cannot concentrate in class.
.643

2.71 .971
2.28 .924
2.10 .963
1.79 .843
1.90 .915

2.59 .902
2.27 .966

Note. Percentage variance explained: total variance, 68.30; F1, 33.97; F2, 12.33; F3, 7.91; F4,
7.47; F5, 6.63.
3

2

2.43

2.36

2.5

2.12
1.88

1.85

1.5
1
0.5
0
Lack ofcommitment

Distractedness

Lack ofpreparedness

Anti-social
orientation

Lack of focus

Figure 1: Factors influencing student disengagement with mean values.
Participants chose one from a set of response options on a 5-point scale as regards frequency
of a particular behavior stated in each question: “never = 1”, “rarely = 2”, “sometimes = 3”,“occasionally = 4” to “frequently = 5”.
Means of Observed Variables
The mean scores of variables for lack of commitment come in the range between 1.56 and 1.99,
with the mean for the four variables being 1.88 (Figure 2). This result indicates these behaviors
as indicated in the item statements are only marginally observed among the study’s participants.
A slightly smaller mean for Item 8 might be construed that only a handful of the students cut

37

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017class for miscellaneous reasons outside school.
Lack of commitment
2.5
2

1.99

1.99

1.97
1.56

1.5
1
0.5
0
5 I repeatedly cut class. 7 I oversleep and cutclass.

8 I cut class and dosomething of priority
outside school.

23 I fail exams.

Figure 2: Mean values of measured variables influenced by lack of commitment.
Participants chose one from a set of response options on a 5-point scale as regards frequency
of a particular behavior stated in each question: “never = 1”, “rarely = 2”, “sometimes = 3”,“occasionally = 4” to “frequently = 5”.
Distractedness
3
2.5
2
1.5

2.18

2.37

2.15

2.43

1.48

1
0.5
0
26 I chat about 13 I pretend as if I 29 I disregard 11 When called on 27 I use digitalthings unrelated towere paying
requests to be
I discuss thedevices such as
lecture contents. attention, but I am quiet and soon get question being
smartphones for
actually doingback to chatting
asked of or the
non-classsomething else,with friends.
response withpurposes during
such as texting on
fellow students
class.
my phone etc.
before giving ananswer.

Figure 3: Mean values of measured variables influenced by distractedness.
Participants chose one from a set of response options on a 5-point scale as regards frequency
of a particular behavior stated in each question: “never = 1”, “rarely = 2”, “sometimes = 3”,“occasionally = 4” to “frequently = 5”.

38

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Lack of preparedness
3

2.71
2.28

2.5

2.1

2
1.5
1
0.5
0
15 I do not remember that I had
assignments to do until I cometo class.

20 I attend class withoutcompleting homework.

16 I forget assignment
deadlines and exam dates.

Figure 4: Mean values of measured variables influenced by lack of preparedness.
Participants chose one from a set of response options on a 5-point scale as regards frequency
of a particular behavior stated in each question: “never = 1”, “rarely = 2”, “sometimes = 3”,“occasionally = 4” to “frequently = 5”.
Responses on items for distractedness indicate that several distractive behaviors, such as using
digital devices and chatting, can be observed (Figure 3). The means for these items come in therange between 1.48 and 2.43, with the mean for the five items being 2.12. The mean for Item
29 was as small as 1.48, indicating most participants’ responses came between “never = 1” and“rarely = 2”. This result might mean many participants are inclined to respect instructors’request and comply with the behavioral expectations of the class. This characteristic resonateswith the discourse of “the well-disciplined behavior of Japanese people”, with “conduct in daily
life, social solidarity, and education at home” being “the major components of the moral
upbringing of Japanese people” (Iwasa, 2017, p. 1). Meanwhile, responses to the items of lackof preparedness (Figure 4) show a certain degree of carelessness in participants preparing forstudies, such as routine homework assignments. However, the level of carelessness becomesmoderate when their failure to act entails adverse consequences, such as exam and assignment
due dates (Item 16).

39

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Anti-social orientation
1.92

1.9

1.9
1.88
1.86
1.84
1.82
1.79

1.8
1.78
1.76
1.74
1.72

30 In group or pair work, I will not talk to the 31 I will not do pair work if the other student isother student(s) if they are people I don’t know.
a person I don’t know.

Figure 5: Mean values of measured variables influenced by anti-social orientation.
Participants chose one from a set of response options on a 5-point scale as regards frequency
of a particular behavior stated in each question: “never = 1”, “rarely = 2”, “sometimes = 3”,“occasionally = 4” to “frequently = 5”.
The graphs in Figure 5 show that the mean values for anti-social orientation is well below the
mid-point of 2.5. This result can be interpreted as a general willingness among participants tocooperate with peers in their studies. In the meantime, the graphs for the statements about lackof focus (Figure 6) show some degree of lack of concentration; although a follow-up survey isnecessary to identify the reasons for this scattering of attention.
Lack of focus
2.7
2.6

2.59

2.5
2.4
2.27

2.3
2.2
2.1
18 I daydream during class with my mind
focused on nothing in particular.

22 I cannot concentrate in class.

Figure 6: Mean values of measured variables influenced by lack of focus.
Participants chose one from a set of response options on a 5-point scale as regards frequency
of a particular behavior stated in each question: “never = 1”, “rarely = 2”, “sometimes = 3”,“occasionally = 4” to “frequently = 5”.

40

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Discussion
The project sets out to measure and interpret students’ disengagement as attitudinal andbehavioral dispositions in a higher education setting in Japan, drawing on the scholarly
observations of local tertiary students (Escandon, 2004; McVeigh, 2002). Overall, the results
provided good support for the metric properties of the instrument and the five-factor structureof student disengagement as an attitudinal/behavioral disposition with this sample. The study
has three implications. Methodologically, we have tested a new measurement scale to assess
students’ disengagement as an attitudinal disposition informed by the context-specific concerns
as well as the literature. Theoretically, the latent structure of student disengagement has been
identified with empirical evidence. Practically, the findings open the path to the developmentof potentially important educational interventions. For example, the means for lack of
preparedness (M = 2.36) and lack of focus (M = 2.43) factors are slightly higher than those of
the other latent factors. These factors play a greater role than other factors in shaping students’course participation or lack thereof. We argue that structures ought to be installed which help
predispose students to study preparation whilst focusing on the goals to be achieved inuniversity learning. In the introduction section, we identified some challenges we encountered,such as diversity in students’ knowledge base and repertoire of skills, practical teaching design,and learner-centeredness in active learning. The curricular structure and interventions could beredesigned so as to help dispose students to developing positive behavioral patterns forparticipating in learning. In order to achieve this, a mechanism ought to be built in such thatthe gap between declarative, “university” knowledge and professional, functional knowledge
can be decreased. Bridging this gap may be sought, to raise one example, by installing learnercentered, constructive alignment in the curriculum (Biggs, 2014), which is rarely found in
higher education settings in Japan, whereby “what the student does is actually more importantin determining what is learned than what the teacher does” (Shuell 1986 as cited in Biggs &Tang, 2011, p. 97).
On a more micro level, we argue, a mechanism that helps generate positive behavioral patterns,
such as preparing for class participation as well as investing time in outside class assignments,
might be of worth. Although Japan’s university dropout rate was 10% as at 2005 – the lowest
among the 27 developed countries (Matsutani, 2012) – interventions might be possible tofurther improve this figure. Theories of achievement and motivation state that students’ desires
to expand their knowledge, understanding, and skills are major contributors to their level ofengagement in academic tasks; particularly important is consideration of the extent to which
students hold valued long-term goals and the extent to which they perceive their current
learning experiences as leading to the attainment of those goals (Miller, Greene, Montalvo,
Ravindran, & Nichols, 1996). Miller et al. indicates “the need to emphasize the coordinationof proximal goals with distal valued outcomes (future consequences) since the latter may prove
to be important for sustaining effort in academic areas of low interest to students” (p. 416). Ina similar vein, Jang (2008) argues that providing a rationale that helps students identify alecture’s otherwise hidden value, understand the worth of their effort, discover the usefulnessof the lecture, and discover the personal meaning in the lesson. Jang states that when this
instructional strategy is successful, it “can help create an opportunity for students to perceive,
accept, and personally endorse – hence internalize the self-system – the value of the learning
activity” (pp. 798-799). At any rate, there needs to be a system which sustains our practice andthat helps students envision the link between their learning experiences and their career paths
beyond university. At this micro level, as the identified latent factor anti-social tendency
indicates, we contend that creating an inclusive environment and atmosphere which nurturesmutual respect and support among students would be an asset in helping shape more

41

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

interpersonal, peer interactions and discussion participation that could potentially lead tocooperative, active engagement. Palloff and Pratt (2005) identifies a learning community as avehicle through which a course is efficiently conducted: “[b]y learning together in a learning
community, students have the opportunity to extend and deepen their learning experience, testout new ideas by sharing them with a supportive group, and receive critical and constructive
feedback” (p. 8). Likewise, Elboj and Niemela (2010) have observed an increase in dynamic
and supportive interactions along with the solidarity among students by providing them with
dialogic and collaborative learning environments. In a series of these efforts to enhance thequality of education, the instrument as we propose here could be used to assess the usefulnessof pedagogical interventions by monitoring possible flux in student attitudinal and behavioral
patterns and identifying the structure which potentially helps shape positive patterns.
Conclusion
In this paper, we have proposed a scale to measure student disengagement in a tertiary settingin Japan. We have identified the five-factor structure, and the metric properties of the scale
were generally well supported. This instrument might be put to use for a diagnostic purpose inhigher education. The mean value for each item and factor was below the mid-point of 2.5 ona five-point scale, except Items 15 and 18. This can be construed that most students at theuniversity are well-behaved and diligent, and receptive to the active, collaborative learning
style that we facilitate in our English as a foreign language classrooms. This result is in linewith our observation of student behavior and performance in our classes at the university.
Meanwhile, some limitations warrant caution. First, the measures were self-reports, whichmight have invited bias in terms of social desirability (Oppenheim, 1992) and self-deception(Hopkins, Stanley, & Hopkins, 1990). Under certain circumstances, people might intentionallyclaim that they are better than they are, while unconsciously minimizing faults and maximizing
virtues in order to maintain a sense of self-worth. It ought to be noted, however, that self-reports
offer a range of advantages and may constitute the method of choice for exploring a constructwhich involves both intra- and extra-psychic processes, such as resistance and misbehavior(Alpert, 1991). Nevertheless, the reliability of the questionnaire should be further reinforced
through other means, such as reports via students’ self-observation notes, and interviews andfocus-groups. Second, because participation in the study was voluntary, it is possible that theparticipants were the “well-behaved” of the students who were approached. Third, the validityof the instrument was explored among the convenience sample of engineering students only.
The scale needs to be cross-validated with larger student cohorts across disciplines andinstitutions: Factor structures should be further corroborated by conducting confirmatory factor
analysis with a larger dataset.
Acknowledgement
This work was supported by the Japan Society for Promotion of Science JSPS KAKENHI(Grant-in-Aid for Young Scientists - B) [Grant Number 26870521].

42

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

References
Alpert, B. (1991). Students’ resistance in the classroom. Anthropology & Education Quarterly,
22(4), 350–366. https://doi.org/10.1525/aeq.1991.22.4.05x1193w
Altbach, P. G., Reisberg, L., & Rumbley, L. E. (2009). Trends in global higher education:
Tracking an academic revolution. Paris, France: UNESCO.
Amano, I. (2014). Globalization and higher education reforms in Japan: The obstacles togreater international competitiveness. In-depth. Retrieved July 27, 2017 fromhttp://www.nippon.com/en/in-depth/a02801
APA Work Group of the Board of Educational Affairs. (1997). Learner-centered psychologicalprinciples: A framework for school reform and redesign. Washington, DC: APA WorkGroup of the Board of Educational Affairs.
Bartlett, M. S. (1954). A note on the multiplying factors for various chi square approximations.
Journal of Royal Statistical Society, 16(Series B), 296–298.
Biggs, J. (2014). Constructive alignment in university teaching. HERDSA Review of Higher
Education, 1, 5–22.
Biggs, J., & Tang, C. (2011). Teaching for quality learning at university (4th ed.). Maidenhead,
UK: Open University Press.
Cavanagh, M. (2011). Students’ experiences of active engagement through cooperative
learning activities in lectures. Active Learning in Higher Education, 12(1), 23–33.
https://doi.org/10.1177/1469787410387724
Christenson, S. L., Reschly, A. L., & Wylie, C. (2012). Handbook of research on student
engagement. New York, NY: Springer. https://doi.org/10.1007/978-1-4614-2018-7
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four
recommendations for getting the most from your analysis. Practical Assessment,
Research and Evaluation, 10(7).
Dörnyei, Z. (2003). Questionnaires in second language research: Construction, administration,and processing. Mahwah, NJ: Lawrence Erlbaum.
Elboj, C., & Niemela, R. (2010). Sub-communities of mutual learners in the classroom: Thecase of Interactive Groups. Journal of Psychodidactics, 15(2), 177–189.
Escandon, A. (2004). Education/learning resistance in the foreign language classroom: A casestudy. AIS St Helens Centre for Research in International Education, Research Paper
Series, Working Paper(5).
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the useof exploratory factor analysis in psychological research. Psychological Methods, 4(3),272-299. https://doi.org/10.1037/1082-989X.4.3.272
Fredricks, J. A. (2014). Eight myths of student disengagement: Creating classrooms of deeplearning. Thousand Oaks, CA: Corwin Press. https://doi.org/10.4135/9781483394534

43

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Fredricks, J. A., Filsecker, M., & Lawson, M. A. (2016). Student engagement, context, andadjustment: Addressing definitional, measurement, and methodological issues. Learning
and Instruction, 43, 1–4. https://doi.org/10.1016/j.learninstruc.2016.02.002
Giroux, H. A. (1983). Theories of reproduction and resistance in the new sociology ofeducation: A critical analysis. Harvard Educational Review, 53(3), 257–293.
https://doi.org/10.17763/haer.53.3.a67x4u33g7682734
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2009). Multivariate data analysis (7thed.). London: Prentice-Hall.
Harada, K. (2015). Universities struggle to cope with shrinking population and globalization.
features. Retrieved July 13, 2017 from http://www.nippon.com/en/features/h00095
Hopkins, K. D., Stanley, J. C., & Hopkins, B. R. (1990). Educational and psychologicalmeasurement and evaluation. Englewood Cliffs, NJ: Prentice-Hall.
Iwasa, N. (2017). Children’s everyday experience as a focus of moral education. Journal of
Moral Education, 1–11. https://doi.org/10.1080/03057240.2016.1268112
Jang, H. (2008). Supporting students’ motivation, engagement, and learning during anuninteresting activity. Journal of Educational Psychology, 100(4), 798–811.
https://doi.org/10.1037/a0012841
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educationaland Psychological Measurement, 20, 141–151.
https://doi.org/10.1177/001316446002000116
Kaiser, H. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36.
https://doi.org/10.1007/BF02291575
Kariya, T. (2011). Japanese solutions to the equity and efficiency dilemma? Secondary schools,
inequity and the arrival of ‘universal’ higher education. Oxford Review of Education,
37(2), 241–266. https://doi.org/10.1080/03054985.2011.559388
Kariya, T. (2012). Gakuryoku to kaiso [“Academic achievement and social strata”]. Tokyo:
Asahi Shimbun.
Laurillard, D. (2012). Teaching as a design science: Building pedagogical patterns for learningand technology. New York: Routledge.
Matsutani, M. (2012, January 10). Student count, knowledge sliding, The Japan Times.
Retrieved from http://www.japantimes.co.jp/news/2012/01/10/reference/student-countknowledge-sliding/#.V1KTYCFkiUk
McVeigh, B. J. (2002). Japanese higher education as myth. London: M. E. Sharpe.
Miller, R. B., Greene, B. A., Montalvo, G. P., Ravindran, B., & Nichols, J. D. (1996).
Engagement in academic work: The role of learning goals, future consequences, pleasingothers, and perceived ability. Contemporary Educational Psychology, 21(4), 388–422.
https://doi.org/10.1006/ceps.1996.0028
Newby, H., Weko, T., Breneman, D., Johanneson, T., & Maassen, P. (2009). OECD reviews oftertiary education: Japan. Paris: OECD.

44

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Oppenheim, A. N. (1992). Questionnaire design, interviewing and attitude measurement (Newed.). London: Pinter.
Palloff, R. M., & Pratt, K. (2005). Collaborating online: Learning together in community. San
Francisco, CA: Jossey-Bass.
Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering
Education, 93(3), 223–231. https://doi.org/10.1002/j.2168-9830.2004.tb00809.x
Yamamoto, K. (2004). Corporatization of national universities in Japan: Revolution forgovernance or rhetoric for downsizing? Financial Accountability & Management, 20(2),153–181. https://doi.org/10.1111/j.1468-0408.2004.00191.x
Corresponding author: Akihiro Saito
Email: akihiro.saito@gmail.com

45

IAFOR Journal of Education

Volume 5 – Issue 2 – Summer 2017

Appendix
English Version of the Original Questionnaire Items
Item N
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

How much do you agree with these statements regarding your behavior? 1 =Never; 2 = Seldom; 3 = Sometimes; 4 = Often; 5 = Very often
I sit near the back of the classroom.
I sit alone away from other students in class.
I attend lectures without sufficient sleep or with fatigue from part-time work or
club activity.
I avoid making eye contact with the teacher when he/she is going to call on
students.
I repeatedly cut class.
I do not attend class on key dates such as quizzes and exams.
I oversleep and cut class.
I cut class and do something of priority outside school (e.g., socializing withfriends, part-time job, family, etc.).
I do not respond when the teacher calls on me.
I pretend not to be aware of being called upon by the teacher.
When called on I discuss the question being asked of and/or the response withfellow students before giving an answer.
I pretend as if I have thought out the teacher's question.
I pretend as if I were paying attention, but I am actually doing something elsesuch as texting on my phone and doing assignments for another course.
I forget to bring necessary materials to class.
I do not remember that I had assignments to do until I come to class.
I forget assignment deadlines and exam dates.
I nap in class.
I daydream during class with my mind focused on nothing in particular.
I do not take notes of important points in class.
I attend class without completing homework.
I resort to copying fellow students’ assignments or work.
I cannot concentrate in class.
I fail exams.
I get the exam details from friends.
I fail to meet assignment requirements.
I chat about things unrelated to lecture contents.
I use digital devices such as smartphones for non-class purposes during class.
I groom myself in class (e.g., do makeup, look in the mirror).
I disregard requests to be quiet and soon get back to chatting with friends.
In group or pair work, I will not talk to the other student(s) if they are people I
don’t know.
I will not do pair work if the other student is a person I don’t know.
I do not take a course if I have to do a presentation in class.
I do not take a course if it has group- or pair-work activities in class.
I will not take a course if it has a teacher hovering over in class to talk to individual
students.

46