EVALUATING  THE  SUCCESS  OF  A  COURSE  BASED  ON  A  MULTILITERACIES-­‐LED  PEDAGOGY    James  E.  Owens  English  Language  Institute,  Kanda  University  of  International  Studies      ABSTRACT    A   Multiliteracies   pedagogy   suggests   a   new   way   of   perceiving   the   idea   of  ‘literacies’   that   is   more   compatible   with   the   modern   world,   rather   than   the  traditional  one-­‐dimensional  view  of  literacy.  It  involves  providing  access  to  what  learners   really   need   in   order   for   them   to   empower   themselves   by   becoming  functional,   effective   and   successful  members  of   modern  society,   where   older,  simpler  models  of  teaching  literacy  are  arguably  less  relevant.  With  this  in  mind,  a  new  curriculum  for  a  Foundational  Literacies  course  has  been  developed  at  a  Japanese  university.  In  this  paper  the  process  of  design  is  described,  and  student  responses  to  the  new  course,  in  the  form  of  answers  to  a  survey,  are  examined.  
The  results  suggest  the  course  has  been  successful  to  a  large  degree.  The  course  is   undergoing   constant   re-­‐evaluation.   Reading   about   this   process   may   prove  useful  for  other  teachers  and  course  designers  who  currently  find  themselves  in  similar  situations.  
  INTRODUCTION    At  a  university  in  Japan,  Freshman  students  in  the  English  Department  were  until  recently  obliged  to  take  Basic  Reading  and  Basic  Writing  courses.  The  shortcomings  of  these  courses  are   outlined   in   Owens   (2012).   A   decision   was   subsequently   made   to   change   academic  direction.  This  was  also  influenced  by  Kramsch  (2006),  who  suggests  that  language  courses  should   focus   more   on   symbolic   competence   rather   than   the   hitherto   emphasised  communicative  proficiency,  typically  viewed  as  a  set  of  distinguishable  cognitive  skills.  The  institution  thus  decided  that  all  curricula  within  the  English  Language  Institute  (ELI)  were  to  follow  a  teaching  pedagogy  inspired  by  the  Multiliteracies  approach,  a  term  first  coined  and  advocated  by  the  New  London  Group  (Cazden,  Cope,  Fairclough,  Gee,  Kalantzis,  Kress,  Luke,  Luke,  Michaels,  &  Nakata,  1996).        In  this  paper,  this  pedagogy  is  explained,  and  one  newly  created  course  that  was  trialled  using  this  approach  with  high  tier  students  is  described.  Finally,  the  course  is  evaluated  in  terms  of  how  successful  it  has  been  in  meeting  its  aims,  by  appraising  student  responses  to  surveys.  
The  feedback  is  broadly  very  positive;  however,  it  also  offers  some  suggestions  for  improving  the  course  that  are  worth  consideration.  It  is  hoped  that  educators  working  in  all  contexts  will  find  this  process  constructive  to  their  own  efforts  in  course  design  and  evaluation.  
  

  

45  

LITERATURE  REVIEW    Multiliteracies  A  Multiliteracies  pedagogy  suggests  a  pluralistic,  “broader  view  of  literacy”  that  accounts  for  the   "...multiplicity   of   communications   channels   and   increasing   cultural   and   linguistic  diversity  in  the  world  today"  (Cazden  et  al.,  1996,  p.  60).  In  short,  functioning  well  in  modern  society  requires  a  larger  range  of  competencies  than  the  traditional  four  skills  approach;  it  necessitates  a  capacity  to  interact  in  many  different  ways  with  many  different  people.    
  In  this  context,  'multi-­‐'  is  ambiguous.  On  one  level,  it  suggests  a  rejection  of  "the  rules  of  a  single,   standard   form   of   the   national   language"   (Cope   &   Kalantzis,   2012,   p.   1).   More  pertinently,  'multi-­‐'  also  denotes  "multimodality"  (p.  2).  The  world  no  longer  operates,  and  arguably  never  has,  in  simple  written  text:  "…the  textual  is  also  related  to  the  visual,  the  audio,  the  spatial,  the  behavioural,  and  so  on"  (Cazden  et  al.,  1996,  p.  64).  Older,  simpler  models  of  teaching  literacy  that  focus  solely  on  the  ability  to  read  and  write  traditional  text  formats  are  losing   relevancy.   Additionally,   a  plethora  of  literature   advocates   against  teaching   reading  and  writing  separately  (e.g.  Grabe,  2001).  Thus,  the  aforementioned  Basic  Writing  and  Basic  Reading  courses  were  replaced  by  an  integrated  Foundational  Literacies  (FL)  course,  which  was  first  trialled  with  only  advanced  stream  students.  
  Course  Objectives  The   Japanese   Ministry   of   Education   (MEXT)   recently   introduced   a   policy   of   awarding  generous  grants  to  institutions  able  to  send  students  away  on  scholarships  to  universities  in  English-­‐speaking   countries.   In   order   to   be  eligible  for  such   scholarships,   students   need   to  achieve  high  scores  in  certain  proficiency  tests,  the  most  widely  accepted  example  being  the  TOEFL  test.    
  There  were  therefore  a  series  of  outcomes  that  the  newly-­‐created  FL  course  set  out  to  reach:    1.  To  replace  an  outdated,  flawed  Basic  Skills  course,  2.  To  make  the  new  integrated  course  compatible  with  the  aims  of  a  Multiliteracies-­‐based  pedagogical  approach  to  learning,  3.  To  ensure  the  course  improves  learners'  TOEFL  scores,  without  reducing  the  classes  to  a  'teaching  the  test'  formula,  4.  To  make  the  content  more  relevant  to  learners'  actual  lives  and  needs  both  in  a  Japanese  context  and  also  for  study  abroad.  
  A   decision   was   reached   to   make   the   course  genre-­‐based   in   terms   of   organisation.   In   this  regard,   the   committee   was   inspired   by   curriculum   development   taking   place   in   other  institutions  around  the  world,  for  example  Georgetown  University  (Byrnes,  Crane,  Maxim,  &  Sprang,   2006)   in   the   USA,   and   the   successful   manipulation   of   genre-­‐based   courses   to  improve   students’   writing   ability   in   South   East   Asia   (Chaisiri,   2010).   Using   genre   as   an  organising   principle   is   particularly   relevant   to   the   goal   of   making   the   content   relevant   to  learners’  lives,  as  “genre  pedagogies”  utilise  “…texts  that  students  will  have  to  write  in  their  target  contexts,  thereby  supporting  learners  to  participate  effectively  in  the  world  outside  the  ESL  classroom”  (Hyland,  2007,  p.  48).  
  In  relation  to  the  principle  of  learner  empowerment,  genre-­‐based  approaches  are  therefore  
useful   because   genre   directly   relates   “the   social   purpose   of   a   text   to   language   structure”  
  

46  

(Cope  &  Kalantzis,  2012,  p.  126).  Thus,  such  approaches  help  learners  to  acquire  “…the  power  of  choice  and  the  linguistic  potential  to  join  new  realms  of  social  activity  and  social  power”  (p.  129)  by  enabling  them  to  achieve  “genre  awareness”  (Johns,  2008).  
  The  desirable  outcome  would  be  for  learners  on  this  new  course  to  become  familiar  with  a  variety  of  different  genres  and  the  uses  of  language  located  in  a  social  context.  As  part  of  this,  learners  should  be  able  to  critically  analyse  texts.    
  In  terms  of  course  objectives,  we  wanted  students  to  reach  an  improved  level  of  competency  in  the  following  areas  by  the  end  of  the  academic  year:    •   understand   reading   and   writing   as   a   form   of   communication   and   understand   how  language  is  organized  in  different  texts  and  why  •   develop  their  own  voice  in  writing  for  different  audiences  •   understand  how  to  read  and  write  texts  that  exist  around  us  in  the  (digital  and  non-­‐digital)  world  •   enjoy  reading  and  writing  in  English  •   participate  in  online  contexts  •   be  able  to  critically  analyse  texts  •   develop  skills  and  vocabulary  for  the  TOEFL  exam    The  following  section  briefly  describes  the  new  FL  course  that  has  been  designed  with  the  above  objectives  in  mind.  
  COURSE  OUTLINE    The  FL  course  consists  of  two  90-­‐minute  lessons  a  week,  taking  place  over  two  fifteen-­‐week  semesters  per  year.  In  the  trial  course,  the  first  week  focuses  on  an  Introduction  to  the  course,  including  the  use  of  a  Standard  Reading  Exercise  (SRE)  to  be  used  as  a  weekly  homework  activity   (Owens,   2014).   The   first   unit   focuses   on   the   genre   of   Email,   chosen   for   its   clear  relevancy  in  the  lives  of  students.  This  unit  runs  for  3-­‐4  weeks,  followed  by  a  Narrative  unit  (see  Owens  &  Johnson,  2016).  The  final  unit  of  the  semester  deals  with  the  Product  Review  genre.    
  The   second   semester   begins   with   a   review   of   the   Email   unit,   followed   by   a   brief   unit  explaining  ethos,  pathos  and  logos  as  modes  of  persuasion.  Neither  of  these  two  units  are  assessed.  Following  this  is  an  Information  Report  unit,  which  includes  an  overview  and  some  activities  where  students  practise  reading  skills  effective  in  TOEFL  exams,  in  addition  to  key  lessons   dealing   with   plagiarism   and   citation.   Finally,   there   is   an   Essay   unit.   Essay   is   not  granted  the  high  status  it  is  often  afforded  in  more  traditional  courses.  The  focus  instead  is  on  general  mastery  and  understanding  of  a  range  of  text  and  genre  types.  In  the  freshman  year  the  course  is  designed  to  introduce  some  foundational  principles  of  text  organization,  style   and   rhetoric,   as   related   to   different   rhetorical   situations   and   contexts.   Nonetheless,  within  this  context  Essay  is  still  considered  an  important  genre  for  students  to  master,  given  the  continued  emphasis  on  academic  skills  that  comprise  their  sophomore  classes.    
  The   curriculum   intentionally   moves   from   familiar   genre   types   to   more   literary   and,  eventually,   academic   materials,   covering   what   Byrnes   et   al.   (2006)   refer   to   as   “primary,”  

  

47  

“blurred”  and  “secondary”  discourses.  This  has  the  added  benefit  of  gradually  familiarising  learners  with  more  difficult  or  unfamiliar  texts,  similar  to  those  they  are  likely  to  encounter  in  the  TOEFL  test  at  the  end  of  the  academic  year.  At  the  end  of  each  unit,  students  produce  their  own   original   examples   of   the   relevant   text   type,   to   show   they   are   aware  of   the   key  features   (in   terms   of   organisation,   lexicogrammar,   format,   layout,   appropriacy,  register…etc)  of  each  genre.  These  examples  are  assessed  and  form  the  bulk  of  their  final  grade.   Other  assessed   aspects   of   the   course  include   the   aforementioned   SRE,   vocabulary  components  (Owens  &  Reed,  2017),  and  occasional  class  blogs.  
  A  decision  was  reached  to  make  the  course  digital-­‐based,  and  to  make  it  a  requirement  for  all   students   taking   this   trial   course   to   purchase   iPads   (with   a   view   to   broadening   this  requirement  for  all  students  and  courses  in  the  future).  Most  materials  for  the  course  were  made   available   online   or   in   the   form   of   iBooks.   What   follows   is   an   evaluation   of   student  responses  to  a  survey  asking  for  feedback  on  the  new  course.  
  COURSE  EVALUATION    Each   year   a  digital   survey   is   given   to   students   at   the  end  of   the   course  in   which   they  are  invited  to  offer  feedback.  The  extent  to  which  students  can  be  objective  in  knowing  what  is  best   for   their   own   learning   needs   is,   of   course,   questionable.   Nonetheless,   anonymous  surveys  allow  a  platform  for  honest  student  reaction,  as  their  opinions  may  be  more  reticent  in  the   classroom   for   a   range   of   cultural   reasons.  This   paper  evaluates   responses  from   the  2014-­‐5   academic   year,   as   this   was   the   first   year   the   course   in   its   current   form   was   fully  expanded  to  all  6  top-­‐tier  classes.  
  The  answers  are  overwhelmingly  positive.  One  limitation  is  that  only  those  teachers  working  on  the  committee  administered  the  survey  to  their  students,  meaning  only  three  of  the  six  classes  responded  to  the  survey,  amounting  to  58  respondents  in  total.  The  implications  of  this  will  be  discussed  later.  
  Course  Aims    In   Question   1,   students   were   asked   to   evaluate,   on   a   scale   of   1   (strongly   disagree)   to   5  (strongly  agree)  how  much  they  agreed  with  each  statement  (the  objectives  of  the  course).  
An   average   score   of   above   3   would   therefore   suggest   an   overall   positive   appraisal.   Ideal  marks  would  score  between  4  and  5.        The  responses  are  summarised  and  weighted  in  Table  1  (below).  As  can  be  seen,  all  sections  scored  an  average  mark  of  above  4  (i.e.  somewhere  between  agree  and  strongly  agree)  with  one  exception.  This  was  especially  the  case  with  Course  Objectives  1  and  3,  which  scored  4.62  and   4.56  respectively.   Objective   7   was   the  only  statement  that  did  not   obtain  a   weighted  average  score  of  above  4,  with  as  many  as  12%  of  respondents  feeling  that  they  disagreed  with  the  idea  that  the  course  had  helped  improve  their  TOEFL  skills  and  vocabulary.  Even  with  regard  to  this  objective,  however,  the  most  popular  answers  were  agree  and  strongly  agree.  
  The  course,  then,  seems  to  have  been  especially  useful  for  helping  students  “understand”  the  way  different  texts  and  genres  operate  (hence  strong  answers  for  1  and  3),  but  less  successful  in  helping  achieve  high  TOEFL  scores.    
  

48  

  
  
Table  1:  Responses  to  Question  1  -­‐  Evaluation  of  course  objectives      strongly  disagree  disagree  no  strong  opinion  agree  

strongly  agree  Total  

Weighted  Average  The  course  has  helped  me  to  understand  reading  and  writing  as  a  form  of  communication  and  understand  how  language  is  organized  in  different  texts  and  why.  

1.72%  

0.00%  

0.00%  

31.03%  67.24%  

  

  

1  

0  

0  

18  

39  

58  

4.62  

The  course  has  helped  me  to  develop  my  own  voice  in  writing  for  different  audiences.  

0.00%  

1.72%  

6.90%  

34.48%  56.90%  

  

  

0  

1  

4  

20  

33  

58  

4.47  

The  course  has  helped  me  to  understand  how  to  read  and  write  texts  that  exist  around  us  in  the  (digital  and  non-­‐digital)  world.  

1.75%  

0.00%  

0.00%  

36.84%  61.40%  

  

  

1  

0  

0  

21  

35  

57  

4.56  

The  course  has  helped  me  to  enjoy  reading  and  writing  in  English.  

0.00%  

7.02%  

7.02%  

33.33%  52.63%  

  

  

0  

4  

4  

19  

30  

57  

4.32  

The  course  has  helped  me  to  participate  in  online  contexts.  

1.72%  

1.72%  

13.79%  44.83%  37.93%  

  

  

1  

1  

8  

26  

22  

58  

4.16  

The  course  has  helped  me  to  be  able  to  critically  analyse  texts.  

3.45%  

3.45%  

6.90%  

32.76%  53.45%  

  

  

2  

2  

4  

19  

31  

58  

4.29  

The  course  has  helped  me  to  develop  skills  and  vocabulary  for  the  TOEFL  exam.  

1.72%  

10.34%  13.79%  37.93%  36.21%  

  

  

1  

6  

8  

22  

21  

58  

3.97      Usefulness  The   next  question   asked   students   to  rate  the   usefulness   of   each   genre/unit   (see  Table  2).  
Again,  this  produced  very  positive  results  overall  with  all  units/genres  rated  very  useful.  This  was  especially  true  of  Email  (4.88)  and  Essay  (4.77).  In  order  from  most  useful  to  least  useful,  students  rated  the  units  as:    Email,  Essay,  Information  Report,  Product  Review,  Narrative.    
  The  first  three  of  the  above  all  scored  a  weighted  average  of  above  4.  The  latter  two  scored  slightly  under.  As  many  as  19.2%  of  respondents  found  the  Narrative  unit  not  at  all/not  very  
useful.  This  is  a  relative  complaint,  however,  as  the  most  frequent  answers  were  quite  (32.7%)  or  very  (30.8%)  useful.  More  specifically,  a  couple  of  students  stated,  in  the  free  comment  

  

49  

section,   that   they   had   a   “hard   time   finding   purpose   (sic)   of   doing   narrative   and   product  review.”  This  suggests  that  the  objectives  of  these  units  need  to  be  made  more  explicit.  
  Table  2:  Question  2  -­‐  Usefulness    not  at  all  useful–  not  very  
useful–  no  strong  opinion–  Unit  1:  Email  0.00%  0  

0.00%  0  

0.00%  0  

11.54%  6  88.46%  
46  

  
52  

  
4.88  

Unit  2:  Narrative  1.92%  1  17.31%  9  17.31%  9  32.69%  17  30.77%  
16  

  
52  

  
3.73  

Unit  3:  Product  Review  0.00%  0  11.76%  6  15.69%  8  41.18%  21  31.37%  
16  

  
51  

  
3.92  

Unit  4:  I nformation  Report  0.00%  0  

1.96%  1  

5.88%  3  33.33%  17  58.82%  
30  

  
51  

  
4.49  

Unit  5:  Essay  0.00%  0  

0.00%  0  

0.00%  0  23.08%  12  76.92%  
40  

  
52  

  
4.77  

–  quite  useful–   very  useful–   Total–    Weighted  Average–    Interest  The  results  for  level  of  interest  in  each  genre/unit  (Table  3)  were  also  very  encouraging.  Three  units  scored  a  weighted  average  rating  of  higher  than  4  and  the  other  two  came  close  (3.98  and  3.94).  In  order  from  highest  level  of  interest  to  lowest,  the  students  rated  the  genres  as:  Narrative,  Email,  Product  Review,  Essay,  Information  Report.    
  This  is  almost  the  exact  opposite  of  the  order  in  which  students  rated  usefulness.  While  there  are  many  feasible  explanations  for  this  inverse  correlation,  one  could  be  cultural:  there  may  be  a  tendency,  perhaps,  reinforced  by  the  Japanese  education  system,  to  deem  anything  that  seems   arduous   and   less   fun   to   be   intrinsically   more   useful   than   anything   they   do   not  associate   with   hard   work.   Regardless   of   the   reason,   however,   all   units   received   overall  positive  ratings.    
  Table  3:  Question  3  -­‐  Interest      not  at  all  interesting–  –  not  very   no  strong  opinion–  quite  very  
Total–    Weighted  interesting–  interesting–   interesting–  Average–  Unit  1:  Email  0.00%  0  

5.77%  3  

7.69%  4  34.62%  18  51.92%  
27  

  
52  

  
4.33  

Unit  2:  Narrative  0.00%  0  

5.77%  3  

5.77%  3  36.54%  19  51.92%  
27  

  
52  

  
4.35  

Unit  3:  Product  Review  0.00%  0  

7.84%  4  

11.76%  6  45.10%  23  35.29%  
18  

  
51  

  
4.08  

Unit  4:  Information  Report  0.00%  0  

5.88%  3  17.65%  9  52.94%  27  23.53%  
12  

  
51  

  
3.94  

Unit  5:  Essay  1.92%  1  

11.54%  6  11.54%  6  36.54%  19  38.46%  
20  

  
52  

  
3.98  

  
  

  

50  

  
  
Difficulty  In   terms   of   difficulty   (Table   4)   the   ideal   weighted   score   would   be   between   3   and   4   (i.e.  
between  just  right  and  quite  difficult)  with  students  feeling  challenged  but  not  overwhelmed,  which   a   score   of   5   (very   difficult)   might   indicate.   Anything   lower   than   3   is   not   ideal.   No  student  awarded  a  score  of  1  to  any  unit,  and  every  unit  scored  on  average  between  3  and  4.  
Additionally,  the  predominant  answer  for  each  unit  was  either  ‘just  right’  or  ‘quite  difficult’.  
Overall,  this  is  a  very  pleasing  set  of  results.  
  Table  4:  Question  4  -­‐  Difficulty  level  quite  too  easy–   quite  easy–   difficulty  level  was  very  difficult–   just  right–   difficult–  Total–   Weighted  Average–  Unit  1:  Email  0.00%  0  19.23%  10  51.92%  27  25.00%  13  3.85%  
2  

  
52  

  
3.13  

Unit  2:  Narrative  0.00%  0  23.08%  12  32.69%  17  32.69%  17  11.54%  
6  

  
52  

  
3.33  

Unit  3:  Product  Review  0.00%  0  15.69%  8  37.25%  19  37.25%  19  9.80%  
5  

  
51  

  
3.41  

Unit  4:  I nformation  Report  0.00%  0  

6.00%  3  34.00%  17  44.00%  22  16.00%  
8  

  
50  

  
3.70  

Unit  5:  Essay  0.00%  0  

3.85%  2  19.23%  10  50.00%  26  26.92%  
14  

  
52  

  
4.00    –    From  easiest  to  most  difficult,  the  students  ranked  the  units  as:  Email,  Narrative,  Product  Review,   Information   Report,   Essay.   This   negatively   correlates   with   the   order   in   which  students  rated  interest,  and  positively  with  the  order  in  which  they  ranked  usefulness,  with  Narrative  considered  easiest,  most  interesting,  and  least  useful,  and  Essay  most  difficult  and  most  useful,  but  (with  the  exception  of  Information  Report)  the  least  interesting.  This  lends  some   weight   to   the   aforesaid   cultural   explanation   that   Japanese   students   might   find  difficulty  inherently  valuable.  Countering  this  is  the  opinion  of  one  student,  who  stated  in  the  comments   section   that   he/she   values   “interesting”   activities   in   class   that   are   “…useful   to  make  students  be  interested  in  reading  and  writing  (sic)”.    
  Additional  Comments  In  a  non-­‐compulsory  section  of  the  survey,  41  respondents  voluntarily  commented  on  what  they  enjoyed  most.  Of  these,  20  responses  pointed  to  how  useful  they  found  the  Email  unit.  
Eight  similarly  praised  the  Essay  unit,  three  the  Narrative  unit,  and  another  three  said  they  liked  everything.  The  remaining  comments  focused  on  other  details  of  the  course.    
  In  response  to  a  question  inviting  them  to  discuss  aspects  of  the  course  they  did  not  like,  only  28  people  replied,  and  of  these  11  responded  to  say  that  there  was  nothing  they  wished  to  see  improved  at  all,  for  example:  “I  want  you  to  continue  your  unique  and  wonderful  class.”  Of  the  remaining  17  replies,  seven  commented  on  the  Narrative  unit.  One  of  these  stated,  “Narrative  unit  was  interesting.  However,  I  was  not  pretty  sure  that  it  was  useful  for  studying  English.”  Similarly,  another  wrote,  “Narrative  was  interesting  for  me,  but  I  cannot  find  the  clear  goal  or  purpose  of  writing  my  own  experience.”  As  stated  above,  perhaps  these  issues  

  

51  

can  be  solved  by  more  effective  communication  of  the  course  objectives,  and/or  by  making  the  materials  themselves  more  directly  relevant  to  students’  lives.  
  Four   students   offered   mild   criticism   of   Product   Review,   and   three   focused   on   Essay  (specifically  its  difficulty).  Other  issues  that  were  raised  by  single  complainants  were  dislike  of  the  Information  Report,  aversion  towards  having  their  errors  corrected,  and  needing  more  help  in  distinguishing  the  difference  between  formal  and  informal  vocabulary.  The  numbers  here  are  low  in  comparison  with  the  positive  comments  above,  but  they  can  still  be  addressed.  
  Components  of  the  course  Students  were  asked  to  rate  the  different  elements  of  the  course  (Table  5),  and  again  most  of  the  answers  were  very  positive.  In  order  from  most  useful  to  least  useful,  they  featured  as  follows:      Assignments  (4.58),    Classroom  work  (4.51),    Standard  reading  exercise  (4.38),  Vocabulary  lists/tests  (4.38),    Class  blogs  (3.74).    
  In  the  case  of  the  first  three,  a  majority  of  respondents  rated  them  very  useful,  and  the  first  four  all  scored  a  weighted  average  higher  than  4  in  terms  of  usefulness.  Most  satisfyingly,  none  of  the  respondents  considered  assignments  either  not  at  all  useful  or  not  very  useful.    
  However,  class  blogs  scored  a  slightly  disappointing  3.74,  and  3.9%  of  students  considered  the  vocabulary  lists  not  at  all  useful,  which  is  some  cause  for  concern.  The  use  of  blogs  itself  might  not  be  a  problem,  however,  as  two  students  commented  on  how  “interesting”  they  were  and  how  they  are  a  “good  tool,  because  we  could  share  other  opinions  and  lifestyle”.  
Rather,   the   problem   seems   to   be   a   failure   to   use   the   blogs  well   or   often   enough.   As  one  student  expressed  it,  “I  think  if  we  had  used  more  often  the  class  blog,  it  would  have  been  more   useful.”   Perhaps   it   needs   to   be   used   more   regularly   and   consistently,   and   more  connected  with  classroom  activities.  
  Table  5:  Question  7  -­‐  How  would  you  rate  the  usefulness  of  each  component  of  the  course?    
not  at  all  useful–  not  very  useful–  no  strong  opinion–  Classroom  work  (the  iBooks,  Moodle...etc)  0.00%  0  

1.96%  1  

7.84%  4  27.45%  14  62.75%  
32  

  
51  

  
4.51  Class  Blogs  0.00%  0  12.00%  6  26.00%  13  38.00%  19  24.00%  
12  

  
50  

  
3.74  

Vocabulary  lists  /  tests  

3.92%  2  

3.92%  2  9.80%  5  47.06%  24  35.29%  
18  

  
51  

  
4.06  

The  standard  reading  exercise  0.00%  0  

2.00%  1  

8.00%  4  40.00%  20  50.00%  
25  

  
50  

  
4.38  

Assignments  0.00%  0  

0.00%  0  

3.85%  2  34.62%  18  61.54%  
32  

  
52  

  
4.58  

  

–  

  

quite  useful–   very  useful–   Total–    Weighted  Average–  

52  

  
  
Overall  opinions  of  the  course  Question  8  asked  respondents  to  rate  the  overall  course,  and  most  students  provided  a  very  positive  evaluation  (Table  6).  A  huge  majority  (75%)  rated  the  course  as  Excellent,  with  the  remainder  mostly  answering  Good  (23.1%).  When  asked  for  suggestions  for  improving  the  course,  there  were  19  responses.  Eleven  of  these  said  “nothing”  could  be  improved,  or  simply  pointed  out  positives.  The  remaining  eight  all  had  different  suggestions.    
  Table  6:  Question  8  -­‐  Overall  opinion  of  the  course  Generally   speaking,   how   useful   do   you   feel   the   course   has    been  overall?  
  Answer  Choices–  Responses–  –  Very  poor  0.00%  0  –  Poor  

1.92%  1  –  Okay  

0.00%  0  –  Good  

23.08%  12  –  Excellent  75.00%  39  Total  

52  

  
  
One  student  suggested  the  assignments  be  better  timed  so  as  not  to  coincide  with  tests  and  heavy   workloads   they   were   receiving  from   other  courses.   Other   suggestions  included   the  request   that,   “If   you   tell   us   the   purpose   of   each   homework   or   assignment   beforehand,  perhaps  we  can  understand  better  why  those  are  useful  for  us,”  and  wanting  the  teacher  to  pick  up  on  classmates’  use  of  L1  in  class.  All  these  points  refer  to  how  teachers  could  become  more  vigilant  rather  than  suggesting  a  change  in  course  materials  or  direction.  
  Two  students  wanted  more  assignments,  with  one  requesting  an  additional  Letter  unit,  and  another  wanting  blogs  and  the  SRE  to  have  their  own  assignments.  These  last  two  elements  are  already  taken  into  account  in  the  homework  portion  of  grading,  and  an  additional  Letter  unit  would  not  be   feasible,  in   terms   of   time  available   or  workload.  However,  one  possible  solution  here  is  the  inclusion  of  some  basic  writing  tips,  or  lessons  contrasting  letter-­‐writing  with  email,  into  the  Email  unit.  
  DISCUSSION  &  ANALYSIS    The  survey  highlights  some  areas  for  possible  improvement  in  developing  the  course  for  the  future.  One  such  example  is  whether  there  should  be  more  focus  on  increasing  TOEFL  scores,  

  

53  

or  if  this  will  be  covered  in  other  courses.  Most  importantly,  the  objectives  of  the  course  need  to  be  made  more  explicit,  especially  to  learners  who  may  be  expecting  a  more  traditional  reading  and  writing  course.  It  could  be  argued  that  they  need  to  be  weaned  off  an  engrained  obsession  with  hard  work  for  its  own  sake  that  comes  from  years  of  rote-­‐learning  and  cram  schools.  Treating  students  as  adults  and  explaining  the  purpose  and  rationale  for  the  work  they  are  doing  can  only  help  motivate  students  to  invest  in  their  work  more.  Indeed,  overall  better  communication  with  students  would  help  in  many  regards.  This  is  especially  true  when  considering  assignment  deadlines,  explaining  the  purpose  of  activities,  and  discouraging  the  use  of  Japanese  in  class.  
  There  are  also  some  clear  limitations  to  the  nature  of  this  type  of  survey  as  an  effective  gauge  of  the  quality  of  a  course.  However,  this  was  an  anonymous  survey  and  it  can  be  assumed  that   students   answered   honestly.   Completion   was   not   compulsory,   and   about   half   of   all  students  responded.  As  with  any  voluntary  survey,  this  affects  the  reliability  of  the  results,  as  it  can  be  presumed  that  responses  come  principally  from  the  most  motivated  students,  and  therefore  those  with  the  most  positive  attitudes  towards  the  course.  More  pertinently  in  this  situation,  only   teachers   working   on   the   course   development   committee   administered   the  survey   to   their   students.   These   teachers   were   likely   more   invested   in   the   materials   and  therefore  more  motivated  in  guaranteeing  their  success.  This  would  clearly  have  an  effect  on  results,  too,  albeit  more  indirectly.  Additionally,  given  that  Chen,  Lee,  and  Stevenson  (1995)  suggest  respondents  from  certain  Asian  countries  (especially  Japan)  have  a  cultural  tendency  to   choose   a  ‘no   opinion’   option   if   given  the   opportunity,  future   surveys  should   only  allow  students  four  potential  responses  to  the  multiple-­‐choice  questions.  
  On  the  whole,  the  course  seems  to  be  popular  and  challenging,  with  the  results  of  the  survey  proving  it  to  be  well-­‐balanced  in  terms  of  usefulness,  level  of  interest  and  difficulty.    Most  of  the  materials  at  this  stage  need  little  alteration,  but  the  course  should  be  subject  to  ongoing  evaluation   and   potential   change.   These   surveys,   administered   annually,   offer   one   such  effective   way   of   doing   so,   and   are   thus   recommended   to   all   curriculum   designers,   in   any  country  or  educational  context.  They  are  easy  to  design  and  administer,  in  paper  or  digital  form,  and  are  relevant  to  all  teaching  contexts,  regardless  of  pedagogy  or  available  tools.  
  CONCLUSION    This  paper  hopefully  gives  some  indication  as  to  how  a  Multiliteracies-­‐inspired  pedagogical  approach  has  been  followed  to  create  a  course  that  sufficiently  challenges  and  empowers  our  learners,  in  response  to  recent  ideas  in  the  field  of  applied  linguistics  research.  This  is  an  exciting  period  of  change  at  our  institution,  as  we  go  about  updating  all  our  ELI  curricula.  
There  will  no  doubt  be  issues  and  further  changes,  as  the  course  undergoes  a  continual  re-­‐evaluation   and   design   process,   which   will   hopefully   ensure   its   long-­‐term   success.   Future  projects   could   entail   looking   in   depth   at   the   annual   collection   of   student   feedback   and  responding   by   making   additional   improvements   to   the   course.   Specifically,   projects   are  underway   to   make   adjustments   to   the   Standard   Reading   Exercise   and   Vocabulary  components  of  the  course.        There   are   many   institutions   currently   redesigning   their   courses   to   better   reflect   newer  attitudes   in   the   world   of   language   teaching.   In   this   context,   our   attempts   are   one   such  

  

54  

example,  and  this  paper  will  hopefully  be  of  help  to  any  teachers  or  course  designers  who  are  attempting  similar  processes.  
    REFERENCES    Byrnes,  H.,  Crane,  C.,  Maxim,  H.,  &  Sprang,  K.  (2006).  Taking  text  to  task:  Issues  and  choices  in  curriculum  construction.  ITL:  International  Journal  of  Applied  Linguistics,  152(1),  85–110.  doi:  10.2143/ITL.152.0.2017864    Cazden,  C.,  Cope,  B.,  Fairclough,  N.,  Gee,  J.,  Kalantzis,  M.,  Kress,  G.,  Luke,  A.,  Luke,  C.,    Michaels,  S.,  &  Nakata,  M.  (1996).  A  pedagogy  of  multiliteracies:  designing  social  futures.  
Harvard  Educational  Review,  66(1),  60-­‐92.  doi:  10.17763/haer.66.1.17370n67v22j160u    Chaisiri,  T.  (2010).  Implementing  a  genre  pedagogy  to  the  teaching  of  writing  in  a  university  context  in  Thailand.  Language  Education  in  Asia,  1,  181-­‐199.  doi:  10.5746/LEiA/10/V1/A16/Chaisiri    Chen,  C.,  Lee,  S.,  &  Stevenson,  H.W.  (1995).  Response  style  and  cross-­‐cultural  comparisons  of  rating  scales  among  East  Asian  and  North  American  students.  Psychology  Science  6,  170–
175.  doi:  10.1111/j.1467-­‐9280.1995.tb00327.x    Cope,  B.  &  Kalantzis,  M.  (2012).  Literacies.  Melbourne:  Cambridge  University  Press.    
  Grabe,  W.  (2001).  Reading  &  writing  relations:  L2  perspectives  on  research  and  practice.  In  B.  Kroll  (Ed.),  Exploring  the  dynamics  of  second  language  writing.  New  York,  NY:  Cambridge  University  Press,  242-­‐262.  
  Hyland,  K.  (2007).  Genre  pedagogy:  language,  literacy  and  L2  writing  instruction.  Journal  of  Second  Language  Writing,  16(3),  148-­‐164.  doi:  10.1016/j.jslw.2007.07.005    Kramsch,  C.  (2006).  From  communicative  competence  to  symbolic  competence.  The  Modern  Language  Journal  90(2),  249-­‐252.  doi:  10.1111/j.1540-­‐4781.2006.00395_3.x    Johns,  A.  M.  (2008).  Genre  awareness  for  the  novice  academic  student.  Language  Teaching  41(2),  237-­‐252.  doi:  10.1017/S0261444807004892    Owens,  J.  (2012).  Designing  a  curriculum:  How  are  we  doing  it?  In:  The  Proceedings  of  The  2012  International  Conference  and  Workshop  on  TEFL  and  Applied  Linguistics,  Department  of  Applied  English,  Ming  Chuan  University,  Crane  Publishing,  208-­‐216.  
  Owens,  J.  (2014).  Using  a  standard  reading  exercise  in  a  foundational  literacies  course.  
Language  Education  in  Asia  (LEiA),  5(1),  81-­‐92.  doi:10.5746/LEiA/14/V5/I1/A07/Owens    Owens,  J.  &  Johnson,  N.  (2016).  Connecting  reading,  writing  and  culture  through  a  literacies-­‐based  approach  to  narrative.  The  PanSIG  Journal,  2015,  133-­‐141.  
  

  

55  

Owens,  J.  &  Reed,  J.  (2017).  So  many  words,  so  little  time:  Implementing  a  new  method  for  teaching  vocabulary  in  a  literacies  course.  The  PanSIG  Journal,  2016,  242-­‐251.  

  56