英語などを勉強するには？
Science-backed tips forlanguage study
科学で裏付けられたコツ

Self-introduction: Pyry Kontio （コンティオ プル）• Joined RCO this year. From Finland.
• Has expertise on:• linguistics (言語学)• second language acquisition (第二言語習得研究)• My daily endeavours: improving ETL pipeline, web development• In my freetime:• I contribute to open source software, the Rust ecosystem inparticular.
• Lately, I started learning OpenCL, and trying to implement aneural network that runs on GPU.

Today’s agenda: Two principles that should be
more widely know & their application• About declarative (宣言的記憶) and procedural (手続き記憶)
memory• About input – and what do generative adversarial networks (GAN,敵対的生成ネットワーク) have to do with this?
• What does this all mean in practice?
• Homework for ya’llAbout declarative memory (宣言的記憶)
and procedural memory (手続き記憶)• Procedural memory: knowing how to DO things (skills, maneuvers).
• You know how to walk.
• You know how to type on your computer.
• You know how to navigate in a familiar place• Declarative memory: knowing ABOUT things (episodes, facts)• You know where you are from.
• You remember which year you graduated from high school.
• You remember that the capital of France is Paris.
• You know that Suumo, Hot Pepper and Townwork belong to the Recruit Group.

Explicit memory (顕在記憶)
vs implicit memory? (潜在記憶)• Explicit – being aware that you know; having the ability to introspect (内観) your knowledge• Some of declarative knowledge is explicit• Implicit – not being able to introspect or describe (説明) yourknowledge.
• All procedural knowledge is implicit• Of course, it‘s possible to know explicitly that ”I am able to walk”• However, this is meta-knowledge!
• That doesn’t mean that the actual knowledge HOW to walk would be
introspectable or describable• One might try to explain it, but the explanation is different from how brains
represent the memory/knowledge.

What does this this have to do with language?
• Native speaker: Much of language is stored in procedural memory• All of syntax, morphology and pronunciation (= so-called grammar, 文法)• Much of discourse knowledge (= 会話の進め方など)• Function word vocabulary (the, a, to, do, because, に、ある、けど、とき)• However: some things are not: content words (but not theirgrammatical properties) are stored in declarative memory.
• Beginner L2 learner: Almost none of language is in procedural memory!

↕
• Advanced L2 learner: A considerable amount is in procedural memory!

Why is proceduralised = fluent?
• Procedural memory learns slowly and forgets slowly• Once it has learned a skill, one is able to use that skill• Effortlessly• Without thinking about it• With a low error rate
• Not disturbed by other things• Declarative memory learns fast and forgets fast.
• You must actively concentrate and thing to be able to use the knowledge.

Much of language is supported by procedural
memory, and acquiring a language means
proceduralising it!
• Here’s some facts that might be surprising!
• You can memorise things declaratively, by just… memorising things. Andreviewing.
• However, to memorise things procedurally, you must LEARN BY DOING.
• There is no know way declarative memories would turn INTO proceduralmemories INSIDE brain! They use anatomically (解剖学的に) different networks!
It seems that the “format” they store data in are totally different.
• Instead, it seems that if there is two representations of the same data, theylearn in parallel, minding their own business. (=自分の事に専念する)

Learn by doing: what does “practice” mean?
• “Practice” in education means (unfortunately!) totally different
thing than “practice” in psychology.
• It means using language IN THE SAME WAY that you would use it in”real life”.
• That means: USING LANGUAGE TO COMMUNICATE MEANING.
• You will not learn to use language by doing a 「問題集」• That’s not communication – it’s a wrong type of exercise for that!

“The four skills””Learning by doing”
means practicing
these skills.
Understand/produce
meaningful language
to communicate incontext

画像の引用元：http://www.ctetadda.com/2016/09/english-pedagogy-language-skills.html

Part II: The role of input and GANs;
Input is the most important thing!!!

画像の引用元：http://www.ctetadda.com/2016/09/english-pedagogy-language-skills.html

Why input is important?
• Language is not something we invent by ourselves.
• We learn it from our environment.
• Language is shared – that’s why it’s possible to communicate with it!
• We gain the data for ”training the model” inside our brain fromenvironment. (However, there is arguments about innateness (生得的) of the training
procedures and the structure of the language learning architecture…)• Input is the ultimate RAW MATERIAL for language acquisition

Why a HUGE quantity of input is important;
MORE input than output might be beneficial?
• Every researcher agrees that input is necessary for acquisition (習得)• There are many examples of research, where even generation of language
improved, although only input was practiced!
• This means that more input improves more or less everything!

• However, not everybody agree on the role of output• Some say it’s beneficial because it trains the generation of language• Some say that it doesn’t make difference
Some say that it doesn’t make difference, but it encourages discussion andcommunication, which provides more input• Some even say that you could learn bad habits (= 悪習慣)

Why a HUGE quantity of input is important;
MORE input than output might be beneficial?
• Why is that? Why would brain need input rather than output tolearn?
• I’ll try to explain using an analogy to GAN.
• Note that this is just an analogy, so it’s not perfect!

What are Generative Adversarial Networks?
(敵対的生成ネットワーク)• A new (Goodfellow et al. 2014) machine learning technique that
produces impressive results in generating lifelike photos and otherdata!

画像の引用元: Karras et al. (2017) Progressive Growing Of Gans For Improved Quality, Stability, And Variation

How does GANs work?
- Generator network (生成ネットワーク)- Produces data
- Discriminator network（識別ネットワーク）- Tries to tell if thedata is fake or not- Both are trained:- Generator: with these slight changes,
you would have deceived thediscriminator!
- Discriminator: with these slightchanges, you would have disclosed
the fakes!

画像の引用元: https://medium.com/@devnag/generative-adver
sarial-networks-gans-in-50-lines-of-code-pytorch-e81b79659e3f

An analogy (類似点) to language ability!
Generator ≒ your ability to produce language = speaking and writing
Discriminator ≒ your ability to understand = listening and reading
Training generative models generally takes a lot more data!
↔ People get faster better at listening than speaking– that’s normal, not a bad thing!
The generator in GAN can’t be better than the discriminator is!
↔ You need to get better at listening
for your speaking to be able to progress!

The simple facts• Input is necessary• Output might be beneficial, but only in presence of enough input.

• Why? Because if you haven’t had enough input (= ability to distinguishbetween signal and noise) you are outputting noise.
• Is using your generator to produce lots of noise a good thing? I doubtit.

まとめ。What does this mean in practice?
• Learn by doing – by communicating• Communication is the exchange of meaningful, useful information for the sakeof that information, not for the sake of practicing the language• Communication does not necessarily mean talking! Even reading a book orlistening to a podcast is communication, because a meaningful message isconveyed• It isn’t communication if you don’t understand the message. That means you
should try and do things that are adequate (= 適切) to your level. Not too easy,
no too hard!

What does this mean in practice?
• Why communication is great• Communication includes both of the earlier points:• It engages your procedural memory, because you are not concentrating on thelanguage but the meaning• It provides you input

What does this mean in practice?
• You should get a lot of input! Possibly a lot more, than you already get.
• Listening to podcasts/radio, reading books, watching TV shows.
• Example sentences or grammar explanations without context or ameaningful don’t count as input. They don’t train your skills ofcommunication, and they don’t activate your procedural memory.

What does this mean in practice?
• Engaging an advanced speaker in conversation is good for you• Motivation and concentration: there’s a motivation to listen and try tounderstand what the other person says
• Interactivity: If you don’t understand, you can signal it, and get the samemessage rephrased, more simply said.
• This leads, at its best, to good-quality, personalised input!

What does this mean in practice?
• Knowing about grammar doesn’t mean that you know thelanguage.
• Doing grammar practice drills is mostly a waste of time.
• Trying to just speak without listening is a waste of time.
• Practicing a few hours a week isn’t often enough.

Homework• Goodhart's law: “When a measure becomes a target, it ceases to be agood measure”• Think about two things:• What do the language tests in Japanese schools measure?
• As this measure becomes a target, what are the ways people are optimising for thattarget?

• What does this tell about the effect of tests in Japanese school system?

Thanks!