The EUROCALL Review, Volume 25, No. 2, September 2017

Project
Governmental partnerships for language learning: A
commercial language platform for young workers in
Colombia
Gustavo García Botero*, Jacqueline García Botero** and Frederik Questier****Vrije Universiteit Brussel, Belgium, **Universidad del Quindío, Armenia,
Colombia, ***Vrije Universiteit Brussel, Belgium______________________________________________________________*gustavo.garcia.botero @ vub.be | **jgarciab @ uniquindio.edu.co | ***fquestie @vub.ac.be
Abstract
In June 2015, the Colombian government via the Labor Ministry announced a project foryoung workers called 40.000 Primeros Empleos. In the framework of this project, theMinistry of Labor signed an alliance with the language platform Duolingo as a strategy
to provide participants with English learning opportunities and a free language
certificate. With the help of a monitoring and evaluation perspective, this study
describes Colombian English language learning policies and their relationship with thelabour market. The discussion presented here intends to maximize the outcomes ofthese kinds of agreements and to provide insights for researchers and national
stakeholders willing to carry out similar projects in their countries. Certification is also
thoroughly analyzed as a means of estimating the possible impact of this partnership.
Keywords: Computer-assisted language learning, Duolingo, foreign language learning
policies, labor market, mobile-assisted language learning.
1. Introduction
The perspective of economics, globalization and internationalization have brought along
the creation of open trading practices that require interactions in different languages for
economic achievement. In governmental policies across the world (e.g., The European
Union, Japan, China, The Middle East) an economic interest in foreign language
education has been added to the ideas of cultural understanding and development ofcommunication skills. Furthermore, current geopolitical issues have pushed
governments to extend their language policies to the general population that is lacking
in foreign language skills. Wagner (2005) insists that equality-oriented projects should
be clear on what they are actually bridging. We see this in projects such as Odysseus, aEuropean project to introduce a second language at the workplace, matching thelanguage needs of migrant workers with the vocational workplace context (GrünhageMonetti, Halewijn & Holland, 2003). The relationship between foreign language learningand equality of opportunity is already being discussed (e.g., Byram, 2008).
In this vein, promoters of information and communication technology (ICT) in educationconsider that ICT will lead to a breakthrough in learning and contribute to social change
and economic development (Wagner et al., 2005). In particular, mobile technologies
can allow the development of policies that enable equal access for all. These
technologies maximize informal and non-formal learning opportunities, expanding
diverse educational settings (West & Vosloo 2015). Mobile learning projects such as the
English in Action project in Bangladesh (2012) and the Samsung SDS Multi-Campus
project (2011) are examples of pro-equity mobile learning initiatives. In Latin America,

42

The EUROCALL Review, Volume 25, No. 2, September 2017
these kinds of efforts are still in their infancy. In 2012, there were 17 identified mobile
learning initiatives resulting from pilot projects driven by non-profit organizations oruniversities; that same year, Colombia was the only country reporting active
government support for mobile learning initiatives through public funding (Lugo &Schurmann, 2012). Until the publication of this paper, there have not been any more
updated reports on mobile learning projects in Latin America. This implies that themobile learning projects being carried out in different Latin American countries are stillin the phases of piloting and experimentation.
2. A mobile language solution that attracts governments
Despite the need for concrete and credible results showing positive societal changes
caused by mobile technology use (Jara, Claro & Martinic, 2012), the mobile language
platform Duolingo is gradually becoming a foreign language learning solution forgovernments in Latin America. In 2014, Duolingo reached an agreement with the
Guatemalan government to facilitate English learning in 97 public schools (Orellana,2014). In 2016, Duolingo reached an additional agreement with Mexico. Moreparticularly, the Jalisco Bilingüe program intends to provide 100,000 people with English
skills in 2017 (El Universal, 2016). Similarly, Duolingo will begin training 350 primaryeducation teachers on the pedagogic use of Duolingo after having reached anagreement with the government of Costa Rica (Presidencia de la República de Costa
Rica, 2016).
In June 2015, the Colombian Ministry of Labor received a donation from Duolingo. The
direct beneficiaries are 40,000 young adults enrolled in a major program called 40.000
Primeros Empleos. The aim of this program is to facilitate the integration of young
professionals into the labor market. The Colombian government presents theparticipation of Duolingo as a strategy to provide high-quality English education for freewith the possibility of obtaining a free language certificate (Ministerio de Trabajo,
2015a).
Given that there have been different operational issues related to the 40.000 Primeros
Empleos program, the implementation of Duolingo in the framework of the project has
been delayed. Therefore, the objectives of this study are twofold: to provide a generalperspective of language learning in the Colombian labor context, and and to discuss thepotential benefits of the Duolingo-Colombian government partnership. By doing so, thispaper positions itself as a reference document for national and international educational
stakeholders who consider Duolingo a language learning solution in their educational
plans.
3. Method
Assessing governmental projects
One approach to measuring the impact of national initiatives around ICT in education isthrough Monitoring and Evaluation (M&E). M&E is a process that provides government
officials, development managers and civil society with better means for learning frompast experience (Grant et al., 2002, p.5). Without this approach, “ it would be
impossible to judge if work was going in the right direction, whether progress andsuccess could be claimed, and how future efforts might be improved” (UNDP, 2002,
p.5). Even though M&E should be an integral component of any planned ICT for
education program (James & Miller, 2005), and even though it can play a significant rolein supporting equity approaches to ICT for education (Wagner, 2005), it is only startingto gain popularity in developing countries.
A descriptive approach to M&E provides fundamental insights that help estimate thesuccess of national initiatives. When dealing with ICT for education projects, Kozma andWagner (2005) refer to four core aspects in long term M&E. One first important factor isthe national context, which determines the potential of the project. A second factor
is input, which comprises the analysis of the ICT resources (such as the software) andthe role they play in students' knowledge and skills. A third factor is the outcomes,
which are synonymous with the impact of the project regarding students' increased
knowledge. A final aspect is cost, where cost-benefit and cost-effectiveness
relationships are analyzed to measure the return on the investment.

43

The EUROCALL Review, Volume 25, No. 2, September 2017
Based on the core elements highlighted by Kozma and Wagner (2005), the DuolingoColombian government partnership in the project 40.000 Primeros Empleos is analysed.
Furthermore, the discussion considers ICT in a conceptual education framework as
proposed by Wagner, Day, James, Kozma, Miller & Unwin, (2005). The adaptation ofthis framework is presented in figure 1.

Figure 1. Monitoring and evaluation discussion framework adapted from Wagner, Day,James, Kozma, Miller & Unwin, (2005).
4. Discussion
4.1. Development context (National educational context)
Colombian foreign language policies and their relationship with the labor market
Despite speaking a language that is shared by 400 million people around the world,
Colombia has traditionally considered foreign language learning as an important aspectin the integration, understanding, and construction of ties beyond the borders of the
Spanish language (1). Not only does Colombia have the mission to preserve the 65
indigenous languages spoken in its territory, it also needs to respond to the demands ofa globalized world where economic, political and social issues are at stake (Ministerio de
Educación Nacional, 2014). The importance of having contact with a foreign language
was recognized in the General Education Act of 1994, which highlights the value of"acquiring conversation and reading skills in a foreign language" (Ley N°115 del 8 deFebrero de 1994). The act was reformed by the Ley N° 1651 del 12 de Julio del2013, which gave priority to English. While it is true that English had already taken
prevalence in bilingualism plans such as the National Bilingualism Plan of 2004, theeducational reform officially legitimized English as the priority foreign language inColombia.
The privileged position given to English is not agreed upon in its entirety. There areconcerns about the situation of indigenous languages (Guerrero Nieto, 2008) and theteaching of other foreign languages necessary for Colombia (Valencia Giraldo, 2005).
Similarly, some views question the idealization of English (Vélez-Rendón, 2003) and its
promotion as a panacea against poverty (Torres-Martínez, 2009). Authors such as
González (2007) go one step further and call into question the interests of institutions
such as the British Council or Cambridge Language Assessment for an English-oriented
bilingualism plan.
Even though these claims should not be disregarded, it is also true that bilingualism inEnglish is an asset that allows people to benefit from globalization. The English
language empowers individuals as it provides more widespread access to knowledge.
Not only is English the most common language in websites (2), English is also largely
used in the spreading of scientific knowledge, to such an extent that its use is discussed
in terms of lingua franca (e.g., Tardy, 2004). English also facilitates social mobility andis positively correlated with quality of life (Education First, 2015). These aspects add tothe general advantages of bilingualism and bilingual education (3), which in turn have apositive impact on social capital. Social capital is understood as "the links, shared values
and understandings in the society that enable individuals and groups to trust each other
and, as a consequence, work together" (Brian, 2007:102).
An additional criticism towards foreign language learning plans in Colombia is that they
convey a philosophy of employability and job training as the sole drive for education(Bonilla Carvajal & Tejada-Sánchez, 2016). Certainly, having skills in English is likely tohave a positive impact on the economy of Colombia. The 500 biggest companies inColombia require 100,000 people with English skills, while the tourism sector has an
immediate need for 110,000. There is also a potential need for 40,000 English qualified

44

The EUROCALL Review, Volume 25, No. 2, September 2017
employees in the tourism sector by 2019 as well as 300,000 employees in ICT andBusiness processing outsourcing (BPO) (Ministerio de Educación Nacional, 2014).
In Colombia, companies would be willing to pay more to employees having English
language skills (Ministerio de Educación Nacional, 2014). This is also the case at aninternational level where employees with an exceptionally high level of English, ascompared to their country’s level, earn a 30%-50% higher salary (Education First,2015). Thus, English encourages competitiveness and social development (Seargeant &Erling, 2011) and it is positively correlated with per capita income and economic growth(Education First, 2015).
The above reasons have pushed the Colombian government to encourage English
language learning beyond formal education, specifically in the labor context, and it hasdeveloped synergies between employers and employees by facilitating resources andsubsidizing English language learning related projects (Ministerio de Educación Nacional,2014). In 2012, for example, the Colombian national learning service, SENA,
implemented English for Work, a continuing education project that allowed participants
to learn English and be recruited by the companies participating in the program. As aresult of the recent agreement with Duolingo in the frame of the 40.000 Primeros
Empleos project, the beneficiaries will study in the Duolingo platform for six months andthey will be provided with a language learning certificate after receiving a passing gradeon the Duolingo test. The following sections describe Duolingo and its potential impact
on the beneficiaries of the 40.000 Primeros Empleos program.
4.2. Software
Duolingo
iscomposed
of
three
main
pedagogic
tools:
the Duolingo
platform, the Duolingo for schools dashboard and the Duolingo test center (thatprovides a language certificate). Because the 40.000 Primeros Empleos program does
not take into account the Duolingo for Schools dashboard, this feature is not widely
addressed in this study.
4.2.1. The Duolingo platform
Duolingo is a popular language learning platform, with over 100 million users registeredaround the world (4). Users can access the platform from any web browser as well as
through an app that is available for devices running with Android or iOS. Duolingo isfree of charge and users can choose from a variety of languages to learn according totheir source language. If the language of the user is Spanish, for instance, he/she cantake a Duolingo language course to learn English, French, Portuguese, Italian, andGerman, Catalan, Guaraní, Esperanto, and Russian (5).
One first remark about Duolingo comes from its origin as a platform. In 2013, Duolingo
co-founder Louis Von Ahn wrote a conference paper entitled " Duolingo: Learn alanguage for free while helping to translate the web." This article clearly states that the
primary objective of Duolingo is breaking the language barrier in the web by translating
its content: "Our goal is to encourage people, like you and me, to translate the Web into
their native languages." (Von Ahn, 2013). This is achieved by Human Computation inwhich people "can be engaged to perform meaningful tasks through some otheractivities that they are already deeply interested in" (Law, 2011). In human
computation, humans use computers to do something while computers will use that
input to achieve something else (Garcia, 2013). This concept, though very innovative,
puts into question the real educational aim of Duolingo. When asked about the main
criticism of Duolingo, Mr. Von Ahn brings up the extended idea that Duolingo's central
purpose is not to teach a language, but that users translate material for the company(Entrevista interactiva Duolingo, 2014).
While Duolingo does use crowdsourcing to translate documents on the web (including
for CNN and Buzzfeed), use of this feature is not mandatory, and it is only presented
through a tab in its laptop version. Eighty percent of Duolingo users practice languages
only through the Duolingo app, and thus they are not able to see the to-be-translated
texts (Entrevista interactiva Duolingo, 2014). Figure 2 shows how Duolingo presents thetranslation option to the students in the laptop version.

45

The EUROCALL Review, Volume 25, No. 2, September 2017

Figure 2. Duolingo translation option in the laptop version.
Having said this, a user who accesses Duolingo is likely to perceive the educational
outlook of the platform. Duolingo presents a language skill tree that is to be discovered
through gamified language exercises. It offers a considerable degree of personalized
language learning through elements that promote self-directed study and engagement
through motivational features. Furthermore, in 2015 Duolingo launched Duolingo for
Schools, a free dashboard which offers educators the possibility of integrating Duolingo
into their language instruction. Teachers can see and use the complete language
curriculum provided by Duolingo, and they can also assign homework and track theactivity of students in the app. Figures 3 and 4 show the educational outlook of
Duolingo.

Figure 3. Duolingo language tree.

46

The EUROCALL Review, Volume 25, No. 2, September 2017

Figure 4. Duolingo for schools dashboard.
One of the reasons why Duolingo has been given several awards in the industry (6) is
its gamified method of instruction. In addition to Human Computation, Duolingo's
success also relies on its Game with a Purpose method (Garcia, 2013) in which users"play" while learning a language. The visual outlook of the platform, the way to advancein the language course, the use of "lingots" as a fictional currency to help the user
attain his or her learning goals and the possibility of competing against friends are allavailable options to keep the learning experience amusing and motivating.
Duolingo, alluding to “two languages” in many romance languages (7), uses translation
at the level of words and sentences as its primary delivery method. Whenever a user
decides to learn a language with Duolingo, the platform presents him/her with a variety
of exercises. These practices include the translation of sentences from the source andtarget language of the user; the transcription of words or phrases in the target
language; the pairing of words in the two languages; the choosing of the correcttranslated phrase through a multiple choice exercise; practice with flash cards in the
two languages; and the translation of unknown words just by clicking or tapping on
them. Figure 5 shows some types of exercises proposed by Duolingo.

Figure 5. Duolingo exercise types.
4.2.2. The Duolingo test center
When advertising its language certificate, Duolingo points out what job seekers andstudents around the world have to go through to get a language certification. They have47

The EUROCALL Review, Volume 25, No. 2, September 2017
to pay a considerable amount of money to take the test, travel to an examination
facility that can be far from where the person lives, and wait for weeks to receive theresults (Duolingo Test Center, language certification for all, 2014). To tackle thoseissues, Duolingo created an app called Duolingo Test Center. This app allows people totake a language test using their mobile devices for a cost of only twenty dollars, one
tenth of other tests (Duolingo Test Center, language certification for all, 2014).
4.3. Impact
4.3.1. The effectiveness of Duolingo on the beneficiaries of the 40.000
Primeros Empleos program
As part of its marketing strategy, Duolingo claims that 34 hours of studying on theplatform equals a semester of language study at university. This claim is based on theDuolingo effectiveness study funded by Duolingo and carried out by Vesselinov andGrego (2012). In the website of Duolingo (8), it is suggested that the platform iseffective because it covers the content of a university language course in less time. By
this reasoning, the effectiveness of Duolingo is interpreted regarding the time taken tocomplete a language course and not considering actual learning outcomes. If this trainof thought is followed in the case of the beneficiaries of the Duolingo-Government
partnership, it should be noted that none of them is a real beginner in English. On
average, a beneficiary with a high school diploma would already have studied a total of
720 hours of English (9). Considering that the English course in Duolingo begins fromscratch, it is likely that the beneficiaries of the 40.000 Primeros Empleos program takeshortcuts in the application that allow them to progress rapidly on the course.
Therefore, it is expected that time engagement (and hence learning gain) with Duolingo
will be considerably reduced.
The hypothesis of minimal learning gain of the beneficiaries of the 40.000 Primeros
Empleos program is also supported by the delivery techniques Duolingo implements toprovide language learning. Duolingo shares the characteristics of the Grammar
Translation Method (10). Advocates of translation in foreign language learning argue
that translation activities build self-confidence and reduce foreign language anxiety(Cook, 2010). They also claim that translation protects students’ linguistic and cultural
identities (Hall, Graham, and Cook, 2012, p. 283) and concentrates on structures thatbuild on what learners can already do (Campbell, 2002). Conversely, other views
highlight that translation is an outdated method (Cook, 2002) that encourages a false
equivalence between two languages and impedes automatic fluent language use(Jordan, 2015). Duolingo focuses on corrective feedback and measures language
progress by assessing knowledge of the structure of the language (11). From studies
about translation in language learning (e.g., Kobayashi & Rinnert, 1992; Uzawa, 1996)
it can be derived that Duolingo could have a positive impact on the reading and writing
skills of users. Nevertheless, grammar becomes so important in Duolingo that it
frequently asks users to translate sentences that are detached from the real world, e.g.,"The birds are reading the newspaper” or "Sorry, I am an apple." Therefore, Duolingo
neglects other paramount aspects of language learning which are essential for the
beneficiaries of the Duolingo-government partnership, namely communicative andintercultural competence.
For these reasons, we argue that Duolingo’s primary learning focus is vocabularyacquisition. According to Cook (2013), knowledge of words requires more than justknowing their meaning. It requires a complex range of information about their spoken
and written form, the ways they are used in grammatical structures and diverse aspectsof their meanings. In Duolingo, many of these aspects are addressed. A Duolingo user isfaced with different kinds of exercises that make him/her conscious of the meaning,spelling, and pronunciation of a word. The platform also provides a syllabus consistingof grammatical points, vocabulary items and a list of vocabulary with the grammaticalcategories per word. Once again, these features fall into the assumptions of traditional
language learning (Cook, 2013) which focus on conscious learning that does notproduce true language competence (Krashen, 2014).
By looking deeper into the Duolingo effectiveness study by Vesselinov and Grego(2012), it becomes clear that language gain is equated to vocabulary learning. The

48

The EUROCALL Review, Volume 25, No. 2, September 2017
study used the WebCAPE test as a learning outcomes instrument. WebCAPE is amultiple choice test that is used as a placement tool for students in their first two yearsof language instruction, and that is clearly form-based (Krashen, 2014). In other words,
this test does not evaluate communicative ability as it is often assessed in a face-toface setting.
The results of the study by Vesselinov and Grego show that the average effectiveness(gain) was 8.1 WebCAPE points per one hour of study (Vesselinov and Grego, 2012).
Taking into account the WebCAPE score scale, the interpretation of the scale by higher
education institutions (12), and the false beginner level of the beneficiaries of
the 40.000 Primeros Empleos program (13), it is likely that the beneficiaries will not
experience a major improvement in their learning outcomes.
Through this partnership, the Colombian government expects participants to achieve a
B1level of English proficiency in the Common European Framework of Reference ofLanguages, or L (14) (Ministerio de Trabajo, 2015a). The CEFRL is based oncompetences in listening, reading, spoken interaction, spoken production, and writing.
Furthermore, it considers language learners as agents who have tasks – not exclusively
language-related – to accomplish (Council of Europe, 2002). Since Duolingo is
exclusively language related, it is likely that the B1 benchmark is achieved regarding
vocabulary (15). Nevertheless, it is less likely that the beneficiaries of the partnership
acquire communicative competence at level B1 by using Duolingo only. At this level, alanguage learner should possess the capacity to understand the main points on familiar
matters encountered at work or school, produce texts on familiar topics, describe
experiences and events by giving reasons or explanations, and deal with situations
happening in the country where the language is spoken (Council of Europe, 2002).
An additional issue regarding the Duolingo-Colombian government agreement is that itis planned for a beneficiary to spend six months studying with Duolingo before taking
the Duolingo language test. While Duolingo manages its own language platform as wellas its own test center, there are differences in how Duolingo provides and tests
language learning. Despite the fact that a learner completes the Duolingo language
course by translating words and sentences, translation in the test center is nonexistent. Furthermore, not only is the vocabulary learned in the platform uncertain to betested by the test center, the exercises provided by the test center (i.e., cloze
exercises) require a much higher cognitive effort than that experienced by studying in
the Duolingo platform.
Regarding the Duolingo language certificate, it is true that the Duolingo test center uses
the latest testing technology to generate a CEFRL-aligned score (16), and that it has
validity and reliability (Ye, 2014), correlating substantially with the TOEFL (Ye, 2014)
and IELTS (Bézy & Settles, 2015) total scores (17). Nonetheless, similar to the
WebCAPE test, the Duolingo language test relies heavily on language form. Focus on
language form is seen from the instructions given in the test, e.g. "Select the real
English words in the list", "fill in the missing words using the drop-down menus", "typein English the statement that you hear." According to the Duolingo test center, abeneficiary successfully completing the Duolingo language test will be able to "fulfill
most communication goals, even on unfamiliar topics, understand the main ideas ofboth concrete and abstract writing, and interact with native speakers fairly painlessly”(Duolingo test center, 2014). Nevertheless, the focus on form in the Duolingo language
test does not really assure that the person having the certificate will actually be able tocommunicate in English effectively. A final remark about the language certificate is that
Duolingo claims that it is recognized by "leading institutions around the globe." A closer
view reveals that there is still a long way to go for the certificate to be widely accepted.
The certificate is only recognized by ten institutions, including the government ofColombia and platforms like Uber and Linked In. It remains to be seen how the Duolingo
language test will gain its place in the context of language certification.
5. Conclusion
During the discussion about the impact of Duolingo in the frame of the 40.000 Primeros
Empleos program, different concerns about the Duolingo-Colombian partnership wereaddressed. The prevalent role of language structure and the possible (limited) learning

49

The EUROCALL Review, Volume 25, No. 2, September 2017
gains obtained by the beneficiaries of the partnership were emphasized. Furthermore,
the paper questions the role of the Duolingo language certificate as a means ofdemonstrating foreign language competence. Other factors that might have a negative
impact on this agreement relate to the risk of the beneficiaries dropping out of thelanguage course. Duolingo does not provide information on the percentage of students
who finish its language courses, but what we can see from the Duolingo effectivenessstudy is that about 60% of the participants drop out of the course. Studies oncommercial software similar to Duolingo, such as Auralog’s TELL ME MORE and Rosetta
Stone, show severe participant attrition (Nielson, 2011). Data from similar content
delivery methods such as MOOCs reveal that although many thousands of participants
enrol in these courses, the completion rate for most courses is below 13% (Onah,2014).
5.1. Maximizing the impact of the agreement
While there are issues that call into question the impact of the Duolingo-Colombian
government partnership, there are still actions that can be implemented to maximize
the positive impact on the beneficiaries. In the frame of the 40.000 Primeros
Empleos program, the participants should follow a 40-hour training course ontransversal and key competences to face their first job experience (Ministerio deTrabajo, 2015b). Transversal competences include critical and innovative thinking,interpersonal skills, intrapersonal skills (18), and global citizenship (UNESCO, 2015).
Given that participants are expected to self-direct their language study, we suggest that
the Ministry of Labor should allocate a few hours of training to intrapersonal skills toensure successful Duolingo use. Research indicates that, by making a specific emphasis
on self-regulation (19) and self-regulatory sub-processes (20), participants are more
likely to make decisions on how to better use stand-alone language solutions such as
Duolingo (Fischer, 2007; Garcia-Botero & Questier, 2016; Nielson, 2011; Onah &Sinclair, 2017). Therefore, some participants would likely be willing to cover all the units
in the language course, while others would prefer to do a placement test or study on
the to-be-translated documents provided by Duolingo.
We also recommend that the activity of the participants in Duolingo be followed throughthe Duolingo for Schools dashboard. Having the possibility of data tracking allows theunderstanding of the actual use of the platform and its real impact. The dashboard canbe a channel of communication between the participants and the Ministry of Labor,
which in turn could use the data obtained from the dashboard to create strategies toprevent and minimize dropping out.
Although this paper is not optimistic about the learning gain resulting from the use ofDuolingo for the beneficiaries of the agreement, we do think that it is important toconsider Duolingo at more basic levels of language knowledge, as is shown in the study
by Vesselinov and Grego (2012). We believe that when participants' knowledge ofEnglish surpasses what Duolingo offers, they should be encouraged to explore the otherlanguages provided by Duolingo that are also of interest for the Colombian labormarket. As with English, the Ministry of Labor could provide participants with acertification for their "excellence and perseverance" for the completion of any otherlanguage course in Duolingo. This certification would have a positive impact on theparticipants' resume, and it would be proof of their commitment to complete goals.
5.2. Final comments
The active participation of Duolingo is an interesting add-on in the efforts to spread
foreign language learning among the Colombian population. Duolingo encourages
lifelong learning and it can be used with devices that people already have, representinga cost-effective strategy to learn a language. However, within the conditions establishedby the Government-Duolingo partnership, we have reservations about the realcontribution of such a project for the targeted population. Governments interested indeveloping similar ICT projects should realize that simply having access to software
does not ensure success. Cooperation between different branches of the government
should be promoted (Vosloo, 2012), as well as a combination of input factors that
positively influence impact (such as human resources, assessment, and teacher andstudent training). When a partnership is reached by means of a donation, M&E should

50

The EUROCALL Review, Volume 25, No. 2, September 2017not only be carried out by the donor, but also by all implied stakeholders in order tomaximize the benefits to the beneficiaries.
Language projects, like the one analyzed in this study, should consider language interms of language competence and not only regarding language form. It should be
noted that learning a language should also encourage the development of 21st-century
skills such as critical thinking, teamwork and problem-solving, which are necessary forhuman and economic development. Tasks should be similar to situations where users
apply their knowledge in the real world (Wagner et al., 2005). We encourage theColombian government to assess the considerations stated in this paper, and we call forpolicy makers and governments to have an M&E approach prior, during and after the
implementation of ICT projects for education.
Acknowledgements
We thank the Colombian Ministry of Labor for their updates about the project.
Funding
Gustavo García Botero's research is funded by the European Commission's Erasmus
Mundus Action 2-Eureka SD project. Grant number: 2013-2591/001-001.
References
Baker, C. (2014). A parents' and teachers' guide to bilingualism (Vol. 18). Multilingual
Matters.
Bézy & Settles. (2015). The Duolingo certificate of English and East Africa: Preliminary
linking results with IELTS and CEFRL. Duolingo research report. DRR-15-01, August 4,
2015.
Bonilla Carvajal, C. & Tejada-Sánchez, I. (2016). Unanswered Questions in Colombia’s
Foreign Language Education Policy. Profile, 18(1), 185–201.
Brian, K. (2007). OECD Insights Human Capital. How what you know shapes yourlife. OECD Publishing.
Byram, M. (2008). From foreign language education to education for intercultural
citizenship: Essays and reflections. Multilingual Matters, 17.
Campbell, S. (2002). Translation in the context of EFL – The Fifth Macroskill? TEFLIN,
13, 1.
Cook, G. (2002). Breaking taboos. Readings in Methodology, 46.
Cook, G. (2010) Translation in language teaching: An argument for reassessment.
(Oxford: Oxford University Press
Cook, V. (2013). Second language learning and language teaching. Routledge.
Council of Europe (2002). Common European Framework of Reference for Languages:
learning, teaching, assessment. Cambridge University Press.
Duolingo
Test
Center,Languagehttps://englishtest.duolingo.com/en.

certification

for

all,

2014.

Education First (/2015). English Proficiency Index. Comparing English skills between
countries. EF EPI. Available from https://www.ef.edu/epi/.
El Universal. (2016). Duolingo presenta Jalisco Bilingüe. El Universal, retrieved fromhttp://www.eluniversal.com.mx/articulo/techbit/2016/03/14/duolingo-presenta-jaliscobilingue#imagen-1.
Englishin
Action.
2012.
Welcome
Author. http://www.eiabd.com/eia/

51

toEnglishin

Action.

Dhaka,

The EUROCALL Review, Volume 25, No. 2, September 2017
Entrevista
Interactiva
Duolingo:Louis
Von
from: https://www.youtube.com/watch?v=t0vpQ57bBSY.

Ahn(2014)

Retrieved

Fischer, R. (2007). How do we know what students are actually doing? Monitoring
students' behavior in CALL. Computer Assisted Language Learning, 20(5), 409-442.
García, I. (2013). Learning a language for free while translating the web. Does duolingo
work? International Journal of English Linguistics, 3(1), 19.
García Botero, G. & Questier, F. (2016). What students think and what they actually doin a mobile assisted language learning context: new insights for self-directed language
learning in higher education. In S. Papadima-Sophocleous, L. Bradley & S. Thouësny
(Eds), CALL communities and culture – short papers from EUROCALL 2016, (pp. 150154). Research-publishing.net. https://doi.org/10.14705/rpnet.2016.eurocall2016.553.
González, A. (2007). Professional development of EFL teachers in Colombia: Between
colonial and local practices. Íkala, Revista de Lenguaje y Cultura, 12(18), 309-332.
Grant, A., Roldán, P., Peñate, H., López, L., Teruel, R., Gómez, M. E. & Mora, L.
(2002). Monitoring & evaluation: some tools, methods and approaches (No. C10-61).
World Bank.
Grünhage-Monetti, M., Halewijn, E. & Holland, C. (2003). ODYSSEUS – Second
language at the workplace: Language needs of migrant workers: Organising language
learning for the vocational/workplace context. Strasbourgh: Council of Europe
Publishing.
Guerrero Nieto, C. H. (2008). Bilingual Colombia: What does it mean to be bilingual
within the framework of the National Plan of Bilingualism? PROFILE Issues in Teachers’
Professional Development, 10, 27-45.
Hall, G. & Cook, G. (2012). Own-language
learning. Language Teaching, 45(3), 271-308.

use

in

languageteaching

andJames, T. & Miller, J. (2005). Developing a monitoring and evaluation plan for ICT in
education. Monitoring and Evaluation of ICT in Education Projects, 57.
Jara, I., Claro, M. & Martinic, R. (2012). Mobile learning for teachers in Latin America:
Exploring the potential of mobile technologies to support teachers and improvepractice. United Nations, Educational, Scientific & Cultural Organization.
Jordan, G. (2015). Geoff Jordan vs. Duolingo. Retrieved from: http://eltjam.com/geoffjordan-vs-duolingo.
Kobayashi, H. & Rinnert, C. (1992). Effects of First Language on Second Language
Writing: Translation versus Direct Composition. Language Learning, 42(2), 183-209.
Kozma, R. B. & Wagner, D. A. (2005). Core indicators for monitoring and evaluation
studies in ICTs for education. Monitoring and evaluation of ICT in education
projects, 35.
Krashen,S.
(2014).
Does
Duolingo“Trump”
University-Level
Learning? International Journal of Foreign Language Teaching, 9(1), 13-15.

Language

Larsen-Freeman, D. & Anderson, M. (2013). Techniques and Principles in Language
Teaching. 3rd edition. Oxford University Press.
Law, E. (2011). Defining (human) computation. CHI 2011: Workshop on Crowdsourcing
and Human Computation.
Ley
N°
1651
del
12
de
Julio
del
2013.
(2013).
Retrieved
http://www.colomboworld.com/academico/resoluciones/Ley1651de2013LeydeBilinguismo.pdf.

fromLugo, M. T. & Schurmann, S. (2012). Turning on mobile learning in Latin America:
Illustrative initiatives and policy implications. Paris, UNESCO.
Milton, J. (2006). French as a foreign language and the Common European Frameworkof Reference for Languages. In Proceedings from the Crossing Frontiers: Languages andthe International Dimension Conference, pp. 1-6.

52

The EUROCALL Review, Volume 25, No. 2, September 2017
Ministerio de Educación Nacional, (2014). Colombia very well! Programa nacional deinglés
2015-2025
DocumentodeSocialización,
41.
Retrievedfrom
http://www.colombiaaprende.edu.co/html/micrositios/1752/articles343287_recurso_1.pdf.
Ministerio de Trabajo. (2015a). ¿Qué ofrece el programa 40 mil primeros empleos?
Retrieved from http://unidad.serviciodeempleo.gov.co/soy-joven-y-quiero-participaren-el-programa.
Ministerio de Trabajo. (2015b). Cartilla de presentación alianza Duolingo. Retrieved
from http://www.mintrabajo.gov.co/40000-primeros-empleos.html.
Nielson, K.B. (2011). Self-study with language learning software in the workplace: What
happens? Language Learning & Technology, 15, 110-129.
Onah, D.F.O., Sinclair, J. & Boyatt, R. (2014). Dropout Rates of Massive Open Online
Courses: Behavioural Patterns. EDULEARN14 Proceedings, pp. 5825-5834.
Onah, D.F.O. & Sinclair, J.E. (2017). Assessing Self-Regulation of Learning Dimensions
in a Stand-alone MOOC Platform. International Journal of Engineering Pedagogy(iJEP), 7(2), 4-21.
Orellana (2014). Estudiantes de Guatemala aprenderán inglés con Duolingo. La
prensa. Retrieved
from http://www.laprensa.hn/vivir/tecnologia/70578798/estudiantes-de-guatemala-aprender%C3%A1n-ingl%C3%A9s-con-duolingo.
Presidencia de la República de Costa Rica, (2016). 350 docentes de inglés se capacitanpara utilizar plataforma Duolingo dentro del Aula. Presidencia de la República de Costa
Rica.
Retrieved
from http://presidencia.go.cr/comunicados/2016/03/docentes-deingles-se-capacitan-para-utilizar-plataforma-duolingo.
Samsung SDS (2011). Samsung SDS Multi-Campus. http://www.multicampus.co.kr.
Seargeant, P. & Erling, E. J. (2011). The discourse of ‘English as a language forinternational development’: Policy assumptions and practical challenges. Dreams andRealities.
Tardy, C. (2004). The role of English in scientific communication: lingua franca or
Tyrannosaurus rex? Journal of English for Academic Purposes, 3(3), 247-269.
Torres-Martínez, S. (2009). Las vicisitudes de la enseñanza de lenguas en
Colombia. Diálogos
Latinoamericanos,
15.
Retrievedfrom
http://www.redalyc.org/pdf/162/16220868004.pdf.
UNDP (2002). Handbook on Monitoring and Evaluating for Results. New York: UNDP
Evaluation Office.
UNESCO (2015). 2013 Asia-Pacific Education Research Institutes Network (ERI-Net)
Regional Study on transversal competencies in education policy and practice: Phase 1.
Regional synthesis report. UNESCO Bangkok office.
Uzawa, K. (1996). Second language learners' processes of L1 writing, L2 writing, andtranslation from L1 into L2. Journal of Second Language Writing, 5(3), 271-294.
Valencia Giraldo, S. (2005). Bilingualism and English language teaching in Colombia: A
critical outlook. Presentation delivered at the Conference on English Language Teachingin Colombia, Universidad del Quindío, October, 2005.
Vélez-Rendón, G. (2003). English in Colombia: A sociolinguistic profile. World Englishes,22(2), 185-198.
Vesselinov, R. & Grego, J. (2012). Duolingo effectiveness study. City University of New
York, USA.
Von Ahn, Luis (2013). Duolingo: learn a language for free while helping to translate theweb. Proceedings of the 2013 International Conference on Intelligent User
Interfaces. ACM.
Vosloo, S. (2012). Mobile learning and policies: Key issues to consider.

53

The EUROCALL Review, Volume 25, No. 2, September 2017
Wagner, D., Day, B., James, T., Kozma, R. B., Miller, J. & Unwin, T. (2005). Monitoringand evaluation of ICT in education projects. A Handbook for Developing
Countries. Washington DC: InfoDev/World Bank.
Wagner, D. A. (2005). Pro-equity approaches to monitoring and evaluation: Gender,
marginalized groups and special needs populations. Monitoring and Evaluation of ICT in
Education Projects, 93.
West, M. & Vosloo, S. (2015). Policy guidelines for mobile learning. UNESCO Publishing.
Ye, F. (2014). Validity, reliability, and concordance of the Duolingo English
Test. Pittsburgh, PA. Retrieved from https://s3.amazonaws.com/duolingo-certificationsdata/CorrelationStudy.pdf.
Zimmerman, B. J. (1986). Becoming a self-regulated learner: Which are the key
subprocesses? Contemporary Educational Psychology, 11(4), 307-313.
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic
learning. Journal of Educational Psychology, 81(3), 329.
[1] A detailed timeline displaying the history of foreign language policies in Colombia is
included in Bonilla Carvajal & Tejada-Sanchez (2016:188).
[2] Usage of content languages for websites. W3Techs.com.
[3] Baker (2014) recapitulates the advantages of bilingualism: Creativity, raised selfesteem, wider communication, literacy in two languages, greater tolerance ofdifferences, sensitivity to communication, security in identity, as well as economic andemployment benefits.
[4] Source: 100M users strong, Duolingo raises $45M led by Google at a $470M
valuation to grow language-learning platform.
[5] As of July, 2017.
[6] iPhone app of the year 2013, Google's Best of the Best 2013 and 2014,
TechCrunch's Best Education Startup, 2014.
[7] Duolingo: Design guidelines. https://www.duolingo.com/design.
[8] https://www.duolingo.com/effectiveness-study.
[9] http://www.mineducacion.gov.co/1621/article-97498.html.
[10] A synthesis of the characteristics of the "Grammar translation" method is found inLarsen-Freeman (2013).
[11] In the Duolingo skill tree, users pass through grammar units such as present
simple, prepositions, adverbs, past simple, infinitives, present perfect, future orconditionals.
[12] Institutions such as the New York University interprets the scores of the Webcape
test as follows: Beginner level: from 0 to 379 points; intermediate level: from 380 to
624 points; advanced: from 625+.
[13] According to our calculations, the current number of words existing in the Duolingo
English course reaches 1914. Assuming that a Colombian high school graduate alreadyhas about 720 hours of English instruction, it is likely that he/she already knows aconsiderable number of the words offered by the platform. For instance, in the first unit
Duolingo users learn the following words: Man-woman-I-am-a-boy-girl-not-fromMexico-you-Spain-are-he-she-is-my-what-your-name. These words are already learnedin primary school.
[14] The Colombian government decided to adopt the CEFRL as an instrument in theassessment of foreign language programs.
[15] Research on vocabulary in the CEFRL shows that the wordlists of languages cometo an average of 1,000 words at the level of A2 and 2,000 words at the level B1(Milton,2006). The Duolingo English course has 1,914 words.

54

The EUROCALL Review, Volume 25, No. 2, September 2017[16] https://englishtest.duolingo.com.
[17] These tests are also aligned with the CEFRL.
[18] Intrapersonal skills include self-discipline,
motivation, and commitment (UNESCO, 2015).

enthusiasm,

perseverance,

self-[19] Zimmerman (1989) defines self-regulation as the degree students
metacognitively, motivationally and behaviorally active their own learning process.

are[20] Zimmerman (1986) refers to self-observation, self-judgement, and self-reaction as
the key sub processes in self-regulation. Duolingo provides tools that, if used efficiently,
encourage self-regulatory processes which are positively correlated with academic
performance.

55