Design and evaluation of online technologies for societal decision making
Keywords: Engineering.
Management Science and Engineering.

Access to digital devices is increasingly prevalent and after the 2020 pandemic, more people than ever before know how to use the basic functionalities of these devices. Governments (more generally, decision makers) have an opportunity to engage in a dialogue with their residents (more generally, stakeholders) that may otherwise not find the time or opportunity to participate. Bringing citizen engagement processes online comes with a responsibility to ensure that they are well understood and that their implementation does not exacerbate inequities in society. It is essential that decision makers have access to tools, methods and manuals to implement these processes in a practical setting. This thesis describes online technologies that have already helped decision makers to engage their stakeholders in participatory budgeting, city budgeting and deliberation. In Participatory Budgeting, decision makers make a portion of their budget available to residents to propose and prioritize projects that they see as the most important for their community. I present data collected from 124 budgeting elections on the PB Stanford online voting platform which allows us to compare voting methods (including both elicitation and aggregation) and ballot designs. We find that while ballot complexity is significantly correlated with the median time spent on the ballot by the voter, there is no such correlation with the abandonment rate. By comparing two ballots from the same voters, we can compare elicitation methods. We confirm that a knapsack ballot can be extracted from a sufficiently large K-ranking elicited ballot (facilitating implementation on a paper ballot) and can compare allocations from different aggregation methods from the same K-ranking elicited ballots. We are also able to establish a better understanding of how the voting method and interface affects the average cost of selected projects. Earlier work observed for a small number of elections that K-approval voting tends to result in higher average cost in aggregate. We confirm that for most ballot pairs in our dataset, the average cost is higher with K-approval voting than with knapsack voting -- but also find examples where knapsack voting results in significantly higher average cost. We measure both the average cost across all projects selected by a voter, and across the most expensive projects selected by a voter -- providing a more nuanced insight in voting behaviors. Organizers often have an express goal to make these democratic engagement processes inclusive and equitable. Online advertising can be used by decision makers to reach out to segments of the population that are otherwise hard to reach. We report findings from field studies in Durham and Greensboro (North Carolina) from the perspective of an advertiser trying to achieve a demographically balanced cohort of respondents. We report that these targeting methods are inaccurate and can result in fewer people (in total and from the targeted group) participating than if a general targeting was used with the same budget. We analyze the data to inform assumptions that a possible targeting algorithm would have to consider, and explore how audiences based on a list of registered voters segmented on known attributes could be used for equitable advertising. When participants are entirely self-selected, or when demographics are not able to distinguish populations of interest, analysis after the fact may be especially important. We report a study with data from a 2020 city budgeting exercise from Austin (Texas) that saw a hundredfold increase in responses after the killing of George Floyd by police officers during the feedback process. We analyze the data from this and a subsequent process, and find that this exogenous shock to the voting process resulted in a change of opinion (rather than a differential turnout). We show how clustering of opinions can be used to offer additional insights into the nature of the shift, and propose a framework to approach for analyzing minority opinions in such processes. Finally, we describe a video conferencing platform that facilitates deliberations without the need for a human moderator, overcoming the need to convene in-person and to train human moderators. The platform has been used in more than 2,000 small group discussions with about 20,000 unique participants as of April 2023, on topics ranging from the Chilean constitution to the Metaverse Terms of Service. We present the design of the platform and evaluate its efficacy. We find by analyzing surveys and metrics from an online exercise with a previous comparable offline exercise that the platform performed on par. We provide preliminary evidence that the platform leads to increased gender participation compared to in-person deliberation, and also performs well on equitable participation across income and education level.