Developing non-formal learning activities focused on increasing evolutionary knowledge and scientific literacy
Creators: name: de Lima, Joelyn
name: Adnađević, Tanja
name: Barstad, Johan
name: Georgiou, Martha
name: Holm, Sile
name: Kenig, Bojan
name: Martinelli, Lucia
name: Nogueira, Teresa
name: Savković, Uroš
name: Tsopoglou-Gkina, Despina
name: Milosevic, Tamara
Publisher: name: Zenodo
Year: 2023
Subjects: non-formal evolution education
scientific literacy
workshops
citizen science projects
arts and science
educational trails
info boards
games
MOOCs
public outreach
non-formal education evaluation

description: Non-formal learning activities have an important role in continuous education of the general audience, and thus provide an essential element in increasing evolutionary knowledge and scientific literacy for European citizens, beyond the formal schooling. This guide brings together explanations of how non-formal learning can be designed, with special emphasis on evaluating the activities. The guide describes common activity types and examples encountered in our scoping conducted via surveys, personal contacts, and literature and internet search. It also provides reflection points for practitioners (activity designers, educators, science communicators) to embed evaluation as a form of engaging activity participants, as well as a tool to assess the impact and improve practice.
description: {"references": ["Aanen, D. K., Allan, E., Ashby, B., Ayhan, N., Beniermann, A., Bettencourt Dias, M., Bininda- Emonds, O., Brandt, M., Cavadas, B., Cvetkovi\u0107, D., Dillon, J., Dost\u00e1l, P., Drobniak, S., Dufour, H., Fonseca, M. J., Georgiou, M., Gerhold, P., Gonzalez, J., Groeneveld, B., ... Zandveld, J. (2018). Memorandum of Understanding for the implementation of the COST Action \"Building on scientific literacy in evolution towards scientifically responsible Europeans\" (EuroScitizen) CA17127. https://doi.org/10.5281/zenodo.3601038", "Adna\u0111evi\u0107, T., Milosevic, T., &amp; Radov\u010di\u0107, D. (2020). Exploratory study of evolution-themed, non- formal education in Europe. Zenodo. https://doi.org/10.5281/zenodo.3712725", "Allen, S., Campbell, P., Dierking, L., Flagg, B., Garibay, C., Korn, R., Silverstein, G., &amp; Ucko, D. (2008). Framework for Evaluating Impacts of Informal Science Education Projects [On-line] (A. Friedman, Ed.). The National Science Foundation. http://insci.org/resources/Eval_Framework.pdf", "Almquist, A. J., &amp; Cronin, J. E. (1988). Fact, Fancy, and Myth on Human Evolution. Current Anthropology, 29(3), 520\u2013522. https://doi.org/10.1086/203672", "Alters, B. J., &amp; Nelson, C. E. (2002). Perspective: Teaching Evolution in Higher Education. Evolution, 56(10), 1891\u20131901. https://doi.org/10.1111/j.0014-3820.2002.tb00115.x", "American Society for Quality [ASQ]. (n.d.). What is the Plan-Do-Check-Act Cycle? Retrieved September 16, 2022, from https://asq.org/quality-resources/pdca-cycle", "Anderson, D., Kisiel, J., &amp; Storksdieck, M. (2006). Understanding Teachers' Perspectives on Field Trips: Discovering Common Ground in Three Countries. Curator: The Museum Journal, 49(3), 365\u2013386. https://doi.org/10.1111/j.2151-6952.2006.tb00229.x", "Anderson, D., Lucas, K. B., Ginns, I. S., &amp; Dierking, L. D. (2000). Development of knowledge about electricity and magnetism during a visit to a science museum and related post-visit activities. Science Education, 84(5), 658\u2013679. https://doi.org/10.1002/1098- 237X(200009)84:5&lt;658::AID-SCE6&gt;3.0.CO;2-A", "Annetta, L. A., Minogue, J., Holmes, S. Y., &amp; Cheng, M.-T. (2009). Investigating the impact of video games on high school students' engagement and learning about genetics. Computers &amp; Education, 53(1), 74\u201385. https://doi.org/10.1016/j.compedu.2008.12.020", "Barron, B. (2006). Interest and Self-Sustained Learning as Catalysts of Development: A Learning Ecology Perspective. Human Development, 49(4), 193\u2013224. https://doi.org/10.1159/000094368", "Beltrame, L., Martinelli, L., &amp; Ampollini, I. (2019). Beyond the Formal Mechanisms of Public Engagement. Communicating Biobanking Research with Other Means. TECNOSCIENZA: Italian Journal of Science &amp; Technology Studies, 9(2), 159\u2013180.", "Bequette, J. W., &amp; Bequette, M. B. (2012). A Place for Art and Design Education in the STEM Conversation. Art Education, 65(2), 40\u201347. https://doi.org/10.1080/00043125.2012.11519167", "Bevan, B. (2016). STEM Learning Ecologie\u2014Relevant, Responsive, and Connected. https://www.nsta.org/connected-science-learning/connected-science-learning-march- 2016/stem-learning-ecologies", "Bishop, B. A., &amp; Anderson, C. W. (1990). Student conceptions of natural selection and its role in evolution. Journal of Research in Science Teaching, 27(5), 415\u2013427. https://doi.org/10.1002/tea.3660270503", "Bonney, R., Cooper, C. B., Dickinson, J., Kelling, S., Phillips, T., Rosenberg, K. V., &amp; Shirk, J. (2009). Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. BioScience, 59(11), 977\u2013984. https://doi.org/10.1525/bio.2009.59.11.9", "Bonney, R., Phillips, T. B., Ballard, H. L., &amp; Enck, J. W. (2016). Can citizen science enhance public understanding of science? Public Understanding of Science, 25(1), 2\u201316. https://doi.org/10.1177/0963662515607406", "BouJaoude, S., Asghar, A., Wiles, J. R., Jaber, L., Sarieddine, D., &amp; Alters, B. (2011). Biology Professors' and Teachers' Positions Regarding Biological Evolution and Evolution Education in a Middle Eastern Society. International Journal of Science Education, 33(7), 979\u20131000. https://doi.org/10.1080/09500693.2010.489124", "Bowman, K. L. (2008). The evolution battles in high-school science classes: Who is teaching what? Frontiers in Ecology and the Environment, 6(2), 69\u201374. https://doi.org/10.1890/070013", "Branch, G., &amp; Meikle, W. E. (2012). Taking Students to the Museum: Interview with Warren D. Allmon, Judy Diamond, and Martin Weiss. Evolution: Education and Outreach, 5(1), 123\u2013 127. https://doi.org/10.1007/s12052-012-0408-z", "Brandt, M., Groom, Q., Magro, A., Misevic, D., Narraway, C. L., Beniermann, D. A., B\u00f8rsen, T. H., Bruckermann, T., Perez, J. G., Meeus, S., Roy, H., Pinto, X. S., Torres, R., &amp; Jenkins, T. (2021). Promoting scientific literacy in Evolution through citizen science. EcoEvoRxiv. https://doi.org/10.32942/osf.io/9atzb", "Braund, M., &amp; Reiss, M. J. (2019). The 'Great Divide': How the Arts Contribute to Science and Science Education. Canadian Journal of Science, Mathematics and Technology Education, 19(3), 219\u2013236. https://doi.org/10.1007/s42330-019-00057-7", "Brise\u00f1o-Garz\u00f3n, A., Anderson, D., &amp; Anderson, A. (2007). Entry and Emergent Agendas of Adults Visiting an Aquarium in Family Groups. Visitor Studies, 10(1), 73\u201389. https://doi.org/10.1080/10645570701263461", "British Science Association. (n.d.). The history of the Festival. British Science Association. Retrieved March 27, 2022, from https://www.britishscienceassociation.org/the-history-of- the-festival", "Brossard, D., Lewenstein, B., &amp; Bonney, R. (2005). Scientific knowledge and attitude change: The impact of a citizen science project. International Journal of Science Education, 27(9), 1099\u20131121. https://doi.org/10.1080/09500690500069483", "Bultitude, K., McDonald, D., &amp; Custead, S. (2011). The Rise and Rise of Science Festivals: An international review of organised events to celebrate science. International Journal of Science Education, Part B, 1(2), 165\u2013188. https://doi.org/10.1080/21548455.2011.588851", "Burget, M., Bardone, E., &amp; Pedaste, M. (2017). Definitions and Conceptual Dimensions of Responsible Research and Innovation: A Literature Review. Science and Engineering Ethics, 23(1), 1\u201319. https://doi.org/10.1007/s11948-016-9782-1", "Cassidy, A. (2006). Evolutionary psychology as public science and boundary work. Public Understanding of Science, 15(2), 175\u2013205. https://doi.org/10.1177/0963662506059260", "Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., &amp; Emanuel, E. (2013). The MOOC Phenomenon: Who Takes Massive Open Online Courses and Why? (SSRN Scholarly Paper ID 2350964). Social Science Research Network. https://doi.org/10.2139/ssrn.2350964", "\u00c7il, E., Maccario, N., &amp; Yanmaz, D. (2016). Design, implementation and evaluation of innovative science teaching strategies for non-formal learning in a natural history museum. Research in Science &amp; Technological Education, 34(3), 325\u2013341. https://doi.org/10.1080/02635143.2016.1222360", "Cipriani, G., Cipriani, L., Danti, S., Picchi, L., &amp; Di Fiorino, M. (2019). Links Between Painting and Neurology: The Example of Dementia. American Journal of Alzheimer's Disease &amp; Other Dementias\u00ae, 34(4), 217\u2013222. https://doi.org/10.1177/1533317519826293", "Clark, D. G. (1997). The educational value of the rural trail: A short walk in the Lancashire countryside. Journal of Geography in Higher Education, 21(3), 349\u2013362. https://doi.org/10.1080/03098269708725441", "Cohn, J. P. (2008). Citizen Science: Can Volunteers Do Real Research? BioScience, 58(3), 192\u2013197. https://doi.org/10.1641/B580303", "Council of the European Union. (2018). Council Recommendation of 22 May 2018 on key competences for lifelong learning (Text with EEA relevance.). Official Journal of the European Union. https://eur-lex.europa.eu/legal- content/EN/ALL/?uri=uriserv:OJ.C_.2018.189.01.0001.01.ENG", "Crall, A. W., Jordan, R., Holfelder, K., Newman, G. J., Graham, J., &amp; Waller, D. M. (2013). The impacts of an invasive species citizen science training program on participant attitudes, behavior, and science literacy. Public Understanding of Science, 22(6), 745\u2013764. https://doi.org/10.1177/0963662511434894", "Crivellaro, F., &amp; Sperduti, A. (2014). Accepting and understanding evolution in Italy: A case study from a selected public attending a Darwin Day celebration. Evolution: Education and Outreach, 7(1), 13. https://doi.org/10.1186/s12052-014-0013-4", "Cunningham, D. L., &amp; Wescott, D. J. (2009). Still More \"Fancy\" and \"Myth\" than \"Fact\" in Students' Conceptions of Evolution. Evolution: Education and Outreach, 2(3), 505\u2013517. https://doi.org/10.1007/s12052-009-0123-6", "Darwin, C. (1845). Journal of researches into the geology and natural history of the various countries visited by H.M.S. Beagle ... From 1832-6. (2 ed., pp. 1\u2013540). J. Murray, https://doi.org/10.5962/bhl.title.2107", "David, P. H. C., S\u00e1-Pinto, X., &amp; Nogueira, T. (2019). Using SimulATe to model the effects of antibiotic selective pressure on the dynamics of pathogenic bacterial populations. Biology Methods and Protocols, 4(1), bpz004. https://doi.org/10.1093/biomethods/bpz004", "de Mora, J. C., &amp; Kennedy, K. J. (2019). Schools and Informal Learning in a Knowledge-Based World. Routledge.", "Derrett, R. (2004). Festivals, events and the destination. In I. Yeoman, M. Robertson, J. Ali- Knight, S. Drummond, &amp; U. McMahon-Beattie (Eds.), Festival and events management: An international arts and culture perspective (pp. 32\u201350). Elsevier Butterworth-Heinemann. https://www.cabdirect.org/cabdirect/abstract/20033214337", "DeWitt, J., &amp; Osborne, J. (2007). Supporting Teachers on Science\u2010focused School Trips: Towards an integrated framework of theory and practice. International Journal of Science Education, 29(6), 685\u2013710. https://doi.org/10.1080/09500690600802254", "Diamond, J., &amp; Evans, E. M. (2007). Museums Teach Evolution. Evolution, 61(6), 1500\u20131506. https://doi.org/10.1111/j.1558-5646.2007.00121.x", "Diamond, J., &amp; Scotchmoor, J. (2006). Exhibiting Evolution. Museums &amp; Social Issues, 1(1), 21\u2013 48. https://doi.org/10.1179/msi.2006.1.1.21", "Dickinson, J. L., Zuckerberg, B., &amp; Bonter, D. N. (2010). Citizen Science as an Ecological Research Tool: Challenges and Benefits. Annual Review of Ecology, Evolution, and Systematics, 41(1), 149\u2013172. https://doi.org/10.1146/annurev-ecolsys-102209-144636", "Dobzhansky, T. (1973). Nothing in Biology Makes Sense except in the Light of Evolution. The American Biology Teacher, 35(3), 125\u2013129. https://doi.org/10.2307/4444260", "Dodick, J. (2007). Understanding evolutionary change within the framework of geological time. McGill Journal of Education / Revue Des Sciences de l'\u00e9ducation de McGill, 42(2). https://mje.mcgill.ca/article/view/2222", "Doering, Z. D., &amp; Pekarik, A. J. (1996). Questioning the Entrance Narrative. Journal of Museum Education, 21(3), 20\u201323. https://doi.org/10.1080/10598650.1996.11510333", "Duarte, M., Nogueira, T., Botelho, A., Cavaco, S., Duarte, A., Santos, T. P., Sousa de, R., &amp; Portela, S. (2021). ANIMAL AND HUMAN CORONAVIRUSES: Evolution and pathogenicity of the virus and disease prevention. Zenodo. https://doi.org/10.5281/zenodo.5710679", "European Commission. (n.d.). European Disability Expertise\u2014Employment, Social Affairs &amp; Inclusion\u2014European Commission. Retrieved July 29, 2021, from https://ec.europa.eu/social/main.jsp?catId=1532&amp;langId=en", "European Commission, Directorate-General for Education, Youth, Sport and Culture. (2017). European Researchers' Night: Science is wonderful! Publications Office of the European Union. https://data.europa.eu/doi/10.2766/997015", "European Commission, Directorate-General for Research and Innovation. (2015). Science education for responsible citizenship: Report to the European Commission of the expert group on science education. Publications Office. https://data.europa.eu/doi/10.2777/13004", "Evans, E. M., Spiegel, A. N., Gram, W., Frazier, B. N., Tare, M., Thompson, S., &amp; Diamond, J. (2010). A conceptual guide to natural history museum visitors' understanding of evolution. Journal of Research in Science Teaching, 47(3), 326\u2013353. https://doi.org/10.1002/tea.20337", "Falk, J. H., &amp; Dierking, L. D. (1997). School Field Trips: Assessing Their Long-Term Impact. Curator: The Museum Journal, 40(3), 211\u2013218. https://doi.org/10.1111/j.2151- 6952.1997.tb01304.x", "Falk, J. H., &amp; Dierking, L. D. (2002). Lessons Without Limit: How Free-choice Learning is Transforming Education. Rowman Altamira.", "Finne, H., Levin, M., &amp; Nilssen, T. (1995). Trailing Research: A Model for Useful Program Evaluation. Evaluation, 1(1), 11\u201331. https://doi.org/10.1177/135638909500100102", "Gata, D., Valakos, E. &amp; Georgiou., M. (2022, under review). Engaging and Assessing Students via a Museum Educational Program. Disciplinary and Interdisciplinary Science Education Research.", "Giusti, E. (2012). Yale Peabody Museum of Natural History's \"Travels in the Great Tree of Life.\" Evolution: Education and Outreach, 5(1), 68\u201375. https://doi.org/10.1007/s12052-012-0397- y", "Gregory, T. R. (2009). Understanding Natural Selection: Essential Concepts and Common Misconceptions. Evolution: Education and Outreach, 2(2), 156\u2013175. https://doi.org/10.1007/s12052-009-0128-1", "Gro\u00df, J., Kremer, K., &amp; Arnold, J. (2019). Evolution Learning and Creationism: Thinking in Informal Learning Environments. In U. Harms &amp; M. J. Reiss (Eds.), Evolution Education Re- considered: Understanding What Works (pp. 285\u2013305). Springer International Publishing. https://doi.org/10.1007/978-3-030-14698-6_16", "Gura, T. (2013). Citizen science: Amateur experts. Nature, 496(7444), 259\u2013261. https://doi.org/10.1038/nj7444-259a", "Haahr, M. (2004). The Art/Technology Interface: Innovation and Identity in Information-Age Ireland. The Irish Review (1986-), 31, 40\u201350. https://doi.org/10.2307/29736133", "Haeckel, E. (1877). Anthropogenie; oder, Entwickelungsgeschichte des menschen, Keimes- und stammesgeschichte (3. umgearb. aufl., pp. 1\u2013836). W. Engelmann. https://doi.org/10.5962/bhl.title.3961", "Hausamann, D. (2012). Extracurricular Science Labs for STEM Talent Support. Roeper Review, 34(3), 170\u2013182. https://doi.org/10.1080/02783193.2012.686424", "Henshall, A. (2020, June 30). How to Use The Deming Cycle for Continuous Quality Improvement. https://www.process.st/deming-cycle/", "Hew, K. F., Hu, X., Qiao, C., &amp; Tang, Y. (2020). What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach. Computers &amp; Education, 145, 103724. https://doi.org/10.1016/j.compedu.2019.103724", "Hofstein, A., &amp; Kempa, R. F. (1985). Motivating strategies in science education: Attempt at an analysis. European Journal of Science Education, 7(3), 221\u2013229. https://doi.org/10.1080/0140528850070301", "Hofstein, A., &amp; Rosenfeld, S. (1996). Bridging the Gap Between Formal and Informal Science Learning. Studies in Science Education, 28(1), 87\u2013112. https://doi.org/10.1080/03057269608560085", "Hogle, J. G. (1996). Considering Games as Cognitive Tools: In Search of Effective \"Edutainment.\" https://eric.ed.gov/?id=ED425737", "Home Page\u2014RRI Tools. (n.d.). Retrieved July 30, 2021, from https://rri-tools.eu/", "Hone, K. S., &amp; El Said, G. R. (2016). Exploring the factors affecting MOOC retention: A survey study. Computers &amp; Education, 98, 157\u2013168. https://doi.org/10.1016/j.compedu.2016.03.016", "Hwang, G.-J., Sung, H.-Y., Hung, C.-M., Huang, I., &amp; Tsai, C.-C. (2012). Development of a personalized educational computer game based on students' learning styles. Educational Technology Research and Development, 60(4), 623\u2013638. https://doi.org/10.1007/s11423- 012-9241-x", "Irwin, A. (2018). No PhDs needed: How citizen science is transforming research. Nature, 562(7728), 480\u2013482. https://doi.org/10.1038/d41586-018-07106-5", "Ivanova, E. (2003). Changes in Collective Memory: The Schematic Narrative Template of Victimhood in Kharkiv Museums. Journal of Museum Education, 28(1), 17\u201322. https://doi.org/10.1080/10598650.2003.11510471", "Jensen, E., &amp; Buckley, N. (2014). Why people attend science festivals: Interests, motivations and self-reported benefits of public engagement with research. Public Understanding of Science, 23(5), 557\u2013573. https://doi.org/10.1177/0963662512458624", "Karlstrom, K., Semken, S., Crossey, L., Perry, D., Gyllenhaal, E. D., Dodick, J., Williams, M., Hellmich-Bryan, J., Crow, R., Watts, N. B., &amp; Ault, C. (2008). Informal Geoscience Education on a Grand Scale: The Trail of Time Exhibition at Grand Canyon. Journal of Geoscience Education, 56(4), 354\u2013361. https://doi.org/10.5408/informal_geoscience_education_", "Kempa, R. F., &amp; Diaz, M. M. (1990). Motivational traits and preferences for different instructional modes in science. Part 1: Students' motivational traits. International Journal of Science Education, 12(2), 195\u2013203. https://doi.org/10.1080/0950069900120208", "Kenig, B., Adna\u0111evi\u0107, T., &amp; Jovanovic, V. (2019). Evolution on the way\u2014ESEB2019 poster. figshare. https://figshare.com/articles/poster/Evolution_on_the_way_- _ESEB2019_poster/9642647/1", "Kennedy, E. B., Jensen, E. A., &amp; Verbeke, M. (2018). Preaching to the scientifically converted: Evaluating inclusivity in science festival audiences. International Journal of Science Education, Part B, 8(1), 14\u201321. https://doi.org/10.1080/21548455.2017.1371356", "Khoon, L. C., &amp; Ramaiah, C. K. (2008). An Overview of Online Exhibitions. DESIDOC Journal of Library &amp; Information Technology, 28(4), 7\u201321. https://doi.org/10.14429/djlit.28.4.193", "Kinshuk, Chen, N.-S., Cheng, I.-L., &amp; Chew, S. W. (2016). Evolution Is not enough: Revolutionizing Current Learning Environments to Smart Learning Environments. International Journal of Artificial Intelligence in Education, 26(2), 561\u2013581. https://doi.org/10.1007/s40593-016-0108-x", "Kirch, S. A., Bargerhuff, M. E., Turner, H., &amp; Wheatly, M. (2005). Inclusive Science Education: Classroom Teacher and Science Educator Experiences in CLASS Workshops. School Science and Mathematics, 105(4), 175\u2013196. https://doi.org/10.1111/j.1949- 8594.2005.tb18157.x", "Kuschmierz, P., Meneganzin, A., Pinxten, R., Pievani, T., Cvetkovi\u0107, D., Mavrikaki, E., Graf, D., &amp; Beniermann, A. (2020). Towards common ground in measuring acceptance of evolution and knowledge about evolution across Europe: A systematic review of the state of research. Evolution: Education and Outreach, 13(1), 18. https://doi.org/10.1186/s12052-020-00132- w", "La Belle, T. J. (1982). Formal, nonformal and informal education: A holistic perspective on lifelong learning. International Review of Education, 28(2), 159\u2013175. https://doi.org/10.1007/BF00598444", "Land, M. H. (2013). Full STEAM Ahead: The Benefits of Integrating the Arts Into STEM. Procedia Computer Science, 20, 547\u2013552. https://doi.org/10.1016/j.procs.2013.09.317", "Laurentino, T. G., Xavier, M., Ronco, F., Pina-Martins, F., Domingues, I., Penha, B., Dias, M., de Sousa, A., Carrilho, T., Rodrigues, L. R., Pinheiro, C., Rato, D., Balata, D., Ayala-Botto, G., Matos, M., Campelo, M., &amp; Botelho, R. (2021). evALLution: Making basic evolution concepts accessible to people with visual impairment through a multisensory tree of life. Evolution: Education and Outreach, 14(1), 5. https://doi.org/10.1186/s12052-021-00143-1", "Li, M.-C., &amp; Tsai, C.-C. (2013). Game-Based Learning in Science Education: A Review of Relevant Research. Journal of Science Education and Technology, 22(6), 877\u2013898. https://doi.org/10.1007/s10956-013-9436-x", "Lloyd, R., Neilson, R., King, S.