DOI: 10.5061/dryad.7d7wm3843
title: Data from: Fun surveys? Developing an innovative approach to assessing learning through citizen science

description: Assessing the impact of citizen science participation on volunteers is
 increasingly important. However, traditional methods for evaluating
 volunteer experiences—such as quantitative surveying—have drawbacks.
 Response rates are often low, pre- and post-tests provide a coarse metric
 for assessing learning, and static evaluation techniques have failed to
 align with the participatory spirit of citizen science, where volunteers
 are involved in understanding their role in scientific inquiry. In
 response to these problems, we developed a more engaging type of
 longitudinal survey aimed at achieving higher response rates and better
 involving volunteers in understanding the scientific value of their survey
 efforts. We first reviewed the literature on “fun” and found that fun
 activities embody six characteristics: autonomy, social connection,
 playfulness, challenge, transport, and curiosity. We used this “fun
 heuristic” to design “SciQuest,” an exploratory survey tool that evaluated
 volunteers’ pro-environmental attitudes and behaviors across citizen
 science projects hosted on the platform SciStarter.org. SciQuest proved to
 be a reliable instrument that captured common learning outcome–related
 constructs measured by conventional, validated instruments. Although most
 beta-testers had no preference between SciQuest and a conventional survey,
 among those expressing a preference, nearly twice as many favored
 SciQuest. SciQuest also achieved much higher retention rates (49–67%) than
 past conventional surveys on SciStarter (19%). Embedded, creative
 approaches to citizen science evaluation are thus a viable, and perhaps
 preferable, complement to traditional surveys, particularly on citizen
 science platforms. Citizen science facilitators should extend their
 volunteer-centric perspectives to include participant evaluation and seek
 evidence-based strategies for enriching the citizen science experience.
description: The attached CSV contains 243 anonymized survey responses from
 Qualtrics "Panelists" (paid surveytakers). These surveytakers
 completed a survey in May, 2021 where they answered a variety of questions
 about their connection to nature, their pro-enviornmental behaviors, and
 other conservation-related constructs. Each respondent completed two
 different types of survey analyzing the same construct: one using
 existing, validated constructs, and one using a new, untested instrument
 called "SciQuest."  The goal of collecting
 this survey data was to quasi-validate the SciQuest version of these
 constructs. SciQuest is designed to provide a more fun, engaging way of
 answering survey questions for citizen science evaluators. However, to
 make sure the SciQuest questions are assessing the same constructs as
 conventional instruments, we compared responses to the same constructs
 among these Panelists who took both a conventional survey and a SciQuest
 version of the same survey constructs.  This dataset
 has already been processed to remove 197 incomplete/low-quality survey
 responses.  We have also attached an R script that
 analzes this dataset to calculate the results presented in our
 manuscript.  We have also included a supplemental video
 file. This file is a screen recording of the SciQuest landing page and
 annotated walk-throughs of each survey module on SciQuest. The video shows
 a participant completing the three available survey modules on SciQuest:
 Journey Prep, Module 1: Nature, and Module 2: Environment. The journal to
 which we are submitting this manuscript does not allow for video-type
 files. I am uploading this supplemental video file on Dryad so that we can
 have a stable reference to this supplemental video. 
 Lastly we have included three .qsf files of each of the three
 Qualtrics survey modules (Supplemental Files 4 corresponds to the Journey
 Prep module, Supplemental File 5 corresponds to Module 1 and Supplemental
 File 6 corresponds to Module 2). These files can be directly loaded into
 Qualtrics by others interested in using the surveys. The journal to which
 we are submitting this manuscript does not allow for submission of the
 .qsf file type, so I am uploading these supplemental files on
 Dryad.
description: # Data from: Fun surveys? Developing an innovative approach to assessing
 learning through citizen science
 [https://doi.org/10.5061/dryad.7d7wm3843](https://doi.org/10.5061/dryad.7d7wm3843) ## Description of the data and file structure This CSV contains survey data from 243 Qualtrics Panelists (paid survey takers). These Panelists completed a survey evaluating their pro-environmental attitudes and beliefs. The Panelists completed two different survey types assessing the same pro-environmental constructs. The first survey type used existing, validated instruments (e.g. Manfredo's "Wildlife Value Orientation"). The second survey type used variations of these existing instruments designed to be more engaging, fun, use interesting graphic elements, etc., for a new type of citizen science survey tool called "SciQuest." The goal of this survey effort was to quasi-validate the survey questions used in SciQuest against the existing, conventional instruments.  There is also an attached R script for replicating our data analysis. There is also an attached supplemental video file that is a screen recording of someone completing the "SciQuest" citizen science survey modules. We have included this here because the journal we are submitting to doesn't allow video files to be uploaded in their manuscript submission software. Including the video here (as Supplemental Information) allows us to create a stable link to which we can reference this file. Lastly, there are three Supplemental Files of file type .qsf corresponding to the three survey modules referenced in our study (Journey Prep module is in Supplemental File 4, Module 1 is in Supplemental File 5, and Module 2 is in Supplemental File 6). The file type .qsf is the Qualtrics Survey file type, and this file can be directly loaded into the Qualtrics software to view or adapt our survey modules. The files can also be viewed using a text editor.  ### Files and variables #### File: survey_data_final.csv **Description:** Results of Qualtrics Panelist survey effort evaluating respondents' pro-environmental attitudes and beliefs and quasi-validating new survey instruments for a citizen science survey tool called "SciQuest." Many of the columns start with either "SI" or "VI". This indicates whether the survey question(s) is a conventional validated survey instrument ("VI") or a new, experimental SciQuest instrument ("SI"). Citations related to validated instruments will be provided in the supplemental table included in the publication. ##### Variables * gender: Numeric value representing the gender of survey respondent ("Man" or "Woman"). Genders were randomly assigned a numeric value (1 or 2) to anonymize respondents. * race: Race/ethnicity of survey respondent * age: Age of survey respondent, binned to anonymize respondents. * education: Highest educational attainment of survey respondent * politics: Political views of survey respondent on a five point scale from "Conservative Republican" to "Liberal Democrat". Note that we randomly assigned the direction of this scale (either from Conservative Republican (1) to Liberal Democrat (5) or from Liberal Democrat (1) to Conservative Republican (5) to anonymize responses. * career: Binary variable indicating whether the respondent worked in a STEM (science, technology, engineering, mathematics) career * cs_experience: Variable indicating whether the respondent has prior experience participating in citizen science * live_in_usa: Binary variable indicating whether the respondent lives in the United States * prefer_ab: Variable indicating whether the respondent **preferred** the SciQuest version ("A") or the conventional version ("B") of the **first** set of survey questions. * prefer_cd: Variable indicating whether the respondent **preferred** the SciQuest version ("C") or the conventional version ("D") of the **second** set of survey questions. * enjoy_ab: Variable indicating whether the respondent **enjoyed** the SciQuest version ("A") or the conventional version ("B") of the **first** set of survey questions. * enjoy_cd: Variable indicating whether the respondent **enjoyed** the SciQuest version ("C") or the conventional version ("D") of the **second** set of survey questions. * SI_aff_enviro_concern: Average of three items assessing respondents' affective environmental concern using a set of content to distraught emoji faces. The scale for this set of items is 1 (low environmental concern) to 7 (high environmental concern).  * VI_enviro_concern: Average of nine items assessing respondents' environmental concern using a conventional instrument from Cruz and Manata. The scale for this set of items is 1 (low environmental concern) to 7 (high environmental concern).  * SI_dist_from_nature: Average of three items assessing respondents' connection to nature by asking respondents to place themselves close to or far from a nature-related word. The scale for this set of items is 1 (low nature connection) to 9 (high nature connection).  * VI_ill_inclusion_nature_in_self: Single item measuring respondents' connection to nature by asking them to move two circles representing them and nature either close to or far from one another. Based on the Illustrated Inclusion of Nature in Self Scale by Kleespies. The scale for this item is 1 (low nature connection) to 7 (high nature connection).  * SI_ecocentrism_slider: Single item measuring respondents' wildlife valuation by having them indicate which of a set of images represents their valuation of nature. The images represent a spectrum of ecocentrism to anthropocentrism. The scale for this item is 1 (low wildlife valuation) to 5 (high wildlife valuation).  * SI_basketfish_narrative: Single item measuring respondents' wildlife valuation by having them respond to a fictional narrative about the perils posed to a rare fish by a dam-building project. The scale for this item is 1 (low wildlife valuation) to 4 (high wildlife valuation).  * SI_basketfish_slider: Single item measuring respondents' wildlife valuation by having them move a slider indicating whose rights they thought were most important (people or a rare fish). The scale for this item is 0 (low wildlife valuation) to 100 (high wildlife valuation).  * VI_wildlife_value_orient: Average of eight items measuring respondents wildlife valuation based on the Wildlife Value Orientation instrument by Manfredo. The scale for these items is 1 (low wildlife valuation) to 7 (high wildlife valuation). * SI_this_or_that_pref: Average of eight items measuring respondents' interest in nature by having them choose which activities they would prefer-- activities involving nature/the environment and those not involving nature/the environment. The scale for these items is 0 (low nature interest) to 1 (high nature interest).  * SI_cartoon_nat_int: Average of four items measuring respondents' interest in nature by having them respond to a cartoon person inviting them to do an activity related to nature/the environment. Possible responses range from high to low excitement to participate. The scale for these items is 1 (low nature interest) to 5 (high nature interest). * SI_nature_photo_feel: Average of six items measuring respondents' nature affinity by having them respond to a nature-related photo. Respondents indicate whether the photo provokes a negative or positive emotional state. The scale for these items is 0 (low nature affinity) to 100 (high nature affinity).  * VI_interest_affinity_nat: Average of ten items measuring respondents' interest in nature/nature affinity using adaptations of established scales from Kals et al., Cheng and Monroe, and Phillips et al. The scale for these items is 1 (low nature interest) to 5 (high nature interest). * SI_dnd_conservation: Average of two items measuring respondents' conservation behavior by having them indicate how likely they are to do a conservation behavior in a set of different circumstances. The scale for these items is 0 (low conservation behavior) to 1 (high conservation behavior).  * SI_food_behavior: Single item measuring respondents' conservation behavior by asking them how frequently they eat meat. The scale for this item is 1 (low conservation behavior) to 5 (high conservation behavior).  * SI_consumer_choice: Average of two items measuring respondents' conservation behavior by asking them about their likelihood of making various consumer choices that help the environment. The scale for these items is 0 (low conservation behavior) to 1 (high conservation behavior).  * VI_conservation_behav: Average of four items measuring respondents' conservation behavior using a validated instrument from Larson et al. The scale for these items is 1 (low conservation behavior) to 5 (high conservation behavior).  * SI_land_steward_graphics: Average of three items measuring respondents' land stewardship by responding to graphic questions asking questions about the frequency with which they engage in different land stewardship activities. The scale for these items is 1 (low land stewardship) to 5 (high land stewardship). * VI_land_stewardship: Average of three items measuring respondents' land stewardship using a validated instrument from Larson et al. The scale for these items is 1 (low land stewardship) to 5 (high land stewardship). * SI_avatar_soc_enviro_slider: Single item measuring respondents' social environmentalism by reading about different cartoon people with different environmental habits and indicating which person best represents their views. The scale for this item is 1 (low social environmentalism) to 5 (high social environmentalism).  * VI_social_enviro: Average of three items measuring respondents' social environmentalism using a validated instrument from Larson et al. The scale for these items is 1 (low social environmentalism) to 5 (high social environmentalism). * SI_avatar_enviro_cit_slider: Average of two items measuring respondents' environmental citizenship by reading about different cartoon people with different environmental citizenship behaviors and indicating which person best represents their habits. The scale for this item is 1 (low environmental citizenship) to 5 (high environmental citizenship).  * VI_enviro_cit: Average of four items measuring respondents' environmental citizenship using a validated instrument from Larson et al. The scale for these items is 1 (low environmental citizenship) to 5 (high environmental citizenship). 
 
 Note that we removed the column "end_date" (the date the survey
 was completed) from this file to help with anonymizing these survey
 responses (suggestion from datadryad). #### File: Supplemental_File_4.qsf
 (Zenodo) **Description:** This is the "Journey Prep" Qualtrics
 survey. #### File: Supplemental_File_5.qsf (Zenodo) **Description:** This
 is the "Module 1" Qualtrics survey. #### File:
 Supplemental_File_6.qsf (Zenodo) **Description:** This is the "Module
 2" Qualtrics survey. ## Code/software Data can be viewed in Excel, R,
 etc. (it's a simple CSV file). Anyone is also welcome to use the
 included R script to run the analyses used in this submission to calculate
 correlation coefficients, and other summary statistics.  ## Access
 information Other publicly accessible locations of the data: * NA Data was
 derived from the following sources: * NA ## Change Log * **9 Sept 2025**:
 Uploaded Qualtrics files to Supplemental Information