Reflection Paper

Quantitative Methods: BasicStatistics, Their Significance and theSoftware
Simon Park
Asia University
This reflection paper describes and compares two different presentations
concerning quantitative methods and analyses. The first is a poster presentation
by Chris Pirotto and Robert Dykes that describes common errors in quantitativeanalysis. The paper details both the poster and the discussion that the writer had
with the presenters. The paper then describes Caroline Handley’s presentation on
the usage of JASP, a free software for conducting quantitative analysis. The paper
describes the presentation as well as the writer’s impressions from it. The paper
ends with a conclusion describing the writer’s reflections of the two presentations.
I went to the JALT College and University Educators Special Interest Group(CUE SIG) 2018 Conference to develop my knowledge in order to conduct
academic studies and write papers. Specifically, I was interested in learning about
quantitative analysis and how I could apply it to my own research and study
design for the upcoming semester. I wanted to use the social media application
Instagram in my classes to see if it had any effect on student performance andmotivation. To do this, I intended to collect large amounts of data and analyze
them, two things I had very little experience with. Chris Pirotto and Robert
Dykes’ poster presentation, “Quantitative Methods: Learn From Our Mistakes”,
and Caroline Handley’s workshop, “Quantitative Data Analysis With JASP”,
were both excellent in their own ways and taken together provided me with auseful introduction on statistics and quantitative analysis, the exact kind of205

Park

experience I had been hoping for heading into the conference.
After my first semester conducting research at a university, I spent thesummer vacation designing a follow-up study that would allow me to collect
much more data for the upcoming semester. However, I was a novice concerning
statistics and came to the CUE SIG conference wanting to learn more about
them. Pirotto and Dykes’ cleverly named poster presentation directly addressed
the problems I had been struggling with. Both were knowledgeable on thesubject of statistics and had conducted research using quantitative methods.
Their poster detailed common mistakes and misconceptions about statisticalanalysis. Many useful topics were presented: the significance of non-significantfindings, frequently omitted but vital statistical data, and mistakes with variabletypes. One point of emphasis for both Pirotto and Dykes was effect size, anunderreported statistical category that both presenters helped to clarify due inpart to their passion for the topic.
A p-value helps you determine the significance of your results when you
perform a hypothesis test in statistics (Rumsey, n.d.). As such, it is an importantfactor of statistical information that can seemingly validate a researcher’s hardwork. Traditionally, a small p-value (typically ≤ 0.05) has indicated evidence
against the null hypothesis (Rumsey, n.d.), or more simply, that your study yieldeda statistically significant result. However, Pirotto and Dykes’ poster demonstrated
how increasing the population size increases the chance of finding any difference
and that a low p-value does not tell you how big the difference is between
groups in your study. Effect size does this as it quantifies the size (or magnitude)of the difference between groups. It is especially valuable for quantifying theeffectiveness of a given treatment, relative to some comparison (Coe, 2002).
Effect size is the kind of statistical item that I came to the conference to learn
about. How to measure and evaluate effect size was a point of emphasis from
Pirotto and Dykes’ poster presentation. They informed me of developments
concerning Cohen’s d value and applied linguistics. Plonsky and Oswald (2014)
urged L2 researchers to “adopt the new field-specific benchmarks of small (d =.40), medium (d = .70), and large (d = 1.00) in order to interpret the practicalsignificance of L2 research effects more precisely” (p. 12).This information
206

Quantitative Methods, OCJSI 1, pages 205-209

would prove useful to me in a later workshop at the conference.
Perhaps the most beneficial part of the poster presentation was simply being
able to discuss statistics with two researchers who had more experience withstatistics than myself. It was informative and enjoyable to verbalize and discusswith others the concepts and problems I had been studying on my own. Bothpresenters were patient and clear in their explanations. Simply going over theirposter together was a learning experience for my statistical education and a niceprimer for properly employing quantitative methods going forward. They alsobrought several books and showed me relevant passages to emphasize theirpoints. The poster itself contained QR codes linked to useful statistical websitesthat novice researchers can visit.
Their poster presentation served as both a useful review session and an
informative lesson on the importance of effect size. As Plonsky and Oswald(2014) stated, “effect sizes have much to offer in terms of more precisely
informing theory and practice in L2 research” (p. 27). This is certainly something
an aspiring researcher should keep in mind.
I work with Caroline Handley at Asia University and in her role as theProfessional Development Chair, she has helped me conduct my research. I haveasked her questions about data analysis, and she has informed me about JASP,
an open source software package for data analysis. JASP is easily downloadable
at https://jasp-stats.org. It is also easy to use, and I recommend those interested
in quantitative analysis to experiment with it. I attended Handley’s session
immediately after Pirotto and Dykes’ poster session. The timing was fortuitousas the poster presentation was a statistics lesson for me. As a result, I was able tounderstand and enjoy Handley’s presentation more comprehensively because ofthe experience.
In her workshop, Handley explained and demonstrated how JASP worked.
The presentation was a practical demonstration of how to analyze data withthis software. Handley detailed three common statistical tests used in SLA—ttests, ANOVA, and repeated measures—and demonstrated how to performthem using JASP. Although Handley had intended for the audience to replicate
what she was doing on their own devices, I simply observed and took note of207

Park

the various processes. Handley opened up Excel spreadsheets, copied and pasted
data into the JASP software, analyzed the data, and detailed what each statistic
and number on the resultant data tables indicated. She also showed us how toinsert properly APA formatted data tables into papers. I found this to be a usefulfeature of JASP.
Handley also highlighted important statistical items like effect size andconfidence intervals, concepts I had become familiar with from the quantitative
methods poster presentation by Pirotto and Dykes. After Handley showed
the results of an example analysis, I saw that Cohen’s d to measure effect size isavailable on JASP. At the start of the day, I would have had no comprehension ofits meaning or significance. I felt a sense of achievement at seeing the effect size
item and having some understanding of it. In fact I gained even greater clarity
through the process of writing this reflection as I have since emailed the JASP
website concerning Plonsky and Cohen’s d value. I only wish the workshop
had been longer. Based on my needs and current abilities, I would have enjoyedspending more time learning about JASP. Fortunately, I have since downloaded
the software and as a result of Handley’s workshop, I can experiment with thesoftware on my own.
In my individual study of statistics, particular statistical procedures suchas repeated measures ANOVA, independent sample t-tests, and delayed posttests often seemed beyond my comprehension. Although I had started to study
quantitative analysis on my own, I had always felt that I would need extensive
training and study in addition to buying expensive software in order to progress
to more advanced statistical analysis. However, in a short 30-minute workshop,
Handley had demonstrated that these methods were not out of my reach
intellectually or economically. Seeing her operate JASP, I realized that I would infact be able to perform the type of analyses that I had struggled with in the past.
This was an empowering and encouraging development. During the workshop
itself I began re-thinking my study design and expanding my options, I and haveadapted my study for this semester. Handley’s session showed me a way to expand
my research practices and approach my research from different perspectives.
In conclusion, I attended the JALT CUE SIG 2018 Conference with the208

Quantitative Methods, OCJSI 1, pages 205-209

goal of developing myself as a researcher. Because of Pirotto and Dykes’ poster
presentation and Handley’s workshop, I developed my understanding of basic
statistics and their significance, learned more about statistical procedures, andbegan to learn how to operate a free software. This has allowed me to put what
I had learned about statistics into practice, and that has expanded my researchpossibilities. By any measure, it was a successful day. The two sessions I havedescribed worked in harmony to improve my understanding of quantitative
methods and conducting research, something that could be said about theentire conference. None of this guarantees future success of course; however,as I embark on new studies, there is no question that the CUE SIG 2018
and specifically these two quantitative methods sessions have made me more
confident as I endeavor to complete my own academic studies.

References

Coe, R. (2002, September). It’s the effect size, stupid. Paper presented at theAnnual Conference of the British Educational Research Association,
University of Exeter, UK.
Plonsky, L., & Oswald, F. (2014). How big is “big”? Interpreting effect sizes inL2 research. Language Learning, 64(4), 878-912. https://doi.org/10.1111/
lang.12079
Rumsey, D. J. (n.d). What a p-value tells you about statistical data. Retrieved
from https://www.dummies.com/education/math/statistics/what-a-pvalue-tells-you-about-statistical-data/

Author bio

Simon Park was born and raised in the United States. He graduated from Boston
University in 2003 and then spent several years teaching English. He received aMaster’s in Education at Temple University in 2014. He currently works at Asia
University in Tokyo, Japan as a visiting faculty member. parksn@asia-u.ac.jp
Received: October 15, 2018
Accepted: March 15, 2019209