Americans' Views of US Transportation Policy - 6 month follow-up
Creators: name: Klein, Nicholas
name: Ralph, Kelcie
name: Thigpen, Calvin
name: Brown, Anne
Publisher: name: Open Science Framework
Year: 2022
Subjects: subject: Transportation Engineering
subject: Transportation
subject: Public Affairs, Public Policy and Public Administration
subject: Civil and Environmental Engineering
subject: Social and Behavioral Sciences
subject: Urban Studies and Planning
subject: Engineering

description: Study Information Experiment: Can Information Change Opinions even after 6 months Hypothesis 1: Views about transportation funding and congestion mitigation policy become more favorable after reading refutation texts during an online experiment six months earlier. Design Plan Study design We previously conducted a mixed subject experiment with two factors (congestion and finance refutation texts). In this study, we are inviting the same participants to answer a subset of questions relating to our experiment Randomization We randomize—using the built-in Qualtrics randomization function—the order of survey questions assessing support or opposition to changing the transportation status quo. We also randomized the order of answer options (responses) to these questions. Sampling Plan Existing Data We have already conducted the initial survey but have not yet conducted the follow-up survey. Data collection procedures We are recruiting participants who completed valid responses in our October 2020 survey via an online opt-in survey panel. We will be contracting the same participants via Prolific.co. Participants will be paid $2 per survey to participate in the survey. Sample size Our target sample size is 599 respondents, the same number who participated in our October 2020 survey. However, we expect fewer will elect to participate in our survey. As of April 10th, 2021, 508 of these respondents were active on Prolific.co in the past 90 days. Power Analysis for 6-mo follow-up In the previous analysis, we had &gt;80% power at alpha = 0.05, given our study design and the prevalence of the policy preferences we examined. This was especially true for the topic of induced demand, since the log-odds of the refutation text were so high and the pre-existing prevalence of alternatives to road building was fairly low. Effect sizes are likely to decline over 6 months, as are the number of eligible respondents, both of which will lead to lower statistical power. Currently 500 respondents are still available on the Prolific platform of our original sample. For the topic of transportation finance, if the effect size dropped by half from the effect size estimated from the immediate post-refutation text question, we would have: 81% statistical power (alpha = 0.05) with a sample size of 500 68% statistical power (alpha = 0.05) with a sample size of 400 53% statistical power (alpha = 0.05) with a sample size of 300 For the topic of induced demand / congestion, if the effect size dropped by half, we would have: 91% statistical power (alpha = 0.05) with a sample size of 500 76% statistical power (alpha = 0.05) with a sample size of 400 57% statistical power (alpha = 0.05) with a sample size of 300 We are therefore likely to have low statistical power for both topics, and especially for the gas tax. These numbers are all based on the effect sizes dropping by half, so the actual mileage may vary. Stopping rule We plan to leave the survey open for one month. Variables Measured variables Questions about preferred transportation policies for transportation finance and addressing congestions 7 questions Questions about transportation facts: 2 questions about transportation in the US with a correct answer rather than opinion-based Demographic questions Age: birth year Analysis Plan Statistical models For the experiment, we will estimate a Bayesian ordinal logit model with proportional odds, with policy agreement as the dependent variable and the treatment and all of the covariates included as independent variables. We will include person-specific varying effects to account for different response patterns by individual (e.g. some individuals may be more likely to use the full agree/disagree scale, others may be more likely to agree). Data exclusion We will exclude individual responses under the following conditions: Respondents who did not participate in our earlier survey Respondents that completed the survey too fast (3 SD outside the mean). Respondents that completed the refutation text portions too fast (3 SD outside the mean). Responses where the year born in our follow-up survey is different from the response in the initial survey Missing data We include a "force response" for all of the survey questions.