Citizen Involvement in Urban Planning - How Can Municipalities Be Supported in Evaluating Public Participation Processes for Mobility Transitions? (2021)

Abstract: Public participation processes allow citizens to engage in municipal decision-making processes by expressing their opinions on specific issues. Municipalities often only have limited resources to analyze a possibly large amount of textual contributions that need to be evaluated in a timely and detailed manner. Automated support for the evaluation is therefore essential, e.g. to analyze arguments. In this paper, we address (A) the identification of argumentative discourse units and (B) their classification as major position or premise in German public participation processes. The objective of our work is to make argument mining viable for use in municipalities. We compare different argument mining approaches and develop a generic model that can successfully detect argument structures in different datasets of mobility-related urban planning. We introduce a new data corpus comprising five public participation processes. In our evaluation, we achieve high macro F1 scores (0.76 - 0.80 for the identification of argumentative units; 0.86 - 0.93 for their classification) on all datasets. Additionally, we improve previous results for the classification of argumentative units on a similar German online participation dataset.

## introduction in many democratic countries, political decisions are increasingly developed through the participation of citizens. public participation processes allow citizens to voice their suggestions and concerns on specific issues, for example in urban planning, and thus influence decision-making processes. participation can take place in formats that vary from on-site events such as citizen workshops, to written submissions via letter or e-mail, and to online platforms where citizens can discuss proposals digitally. building on @xcite , we can distinguish two main goals of public participation processes. on the one hand, the additional input provided by citizens can influence the decision-making process and, potentially, lead to more effective policies. on the other hand, citizens are assumed to develop a higher acceptance of the output when given an opportunity to participate and, ultimately, the resulting decisions have a higher legitimacy. in order to be able to include citizen comments in the further decision-making process, those comments first have to be evaluated. however, both offline and online participation formats have the potential to generate a high number of responses @xcite , e.g., thousands of contributions. along with stringent schedules in decision-making processes, this often poses major challenges for municipalities. still, participation contributions are commonly evaluated manually with considerable effort. therefore, if municipalities do not have enough resources (human or monetary) to shoulder this effort, the detailed evaluation will have to be cut back. as a result, opinions might be completely omitted or not been taken into account equally. this in turn can have a negative influence on the goals of public participation processes. filtering out individual or mass opinions risks loosing important clues for effective policies. it can also endanger citizens' confidence in the opportunity to participate in decision-making and weaken civic engagement @xcite . besides, decision acceptance is influenced by perceived fairness @xcite . automating the evaluation of public participation processes can help overcome these problems @xcite and has been addressed by research initiatives such as the cornell erulemaking initiative (ceri) 1 and, more recently, the citizen participation and machine learning for a better democracy project 2 . over the years, several tasks that arise in the evaluation process have been high-lighted. these include thematic classification and clustering of citizen contributions (e.g. @xcite @xcite , summarization of similar content (e.g @xcite , detection of duplicates (e.g. @xcite , and analysis of arguments and opinions (e.g. @xcite @xcite . in this paper, we focus on arguments in public participation processes that address sustainable mobility and land use in germany. german cities have involved their citizens in hundreds of decisionmaking processes on these issues in recent years. 3 we look at five of them in detail, four of which are processes for concrete improvements to cycling infrastructure and one of which is a strategic process for creating a general mobility concept for a city. at the same time, we consider two very different participation formats, namely online platforms and questionnaires. this paper's first objective is to analyze the strengths and weaknesses of previously published argument mining approaches for public participation processes when they are applied to different german datasets. our attention is focused on the classification of text segments as argumentative or non-argumentative, as well as on the downstream classification of argumentation components. in addition to our datasets, we include the only other german public participation dataset (to the best of our knowledge) for argument mining @xcite in the evaluation. our second objective is to improve the results obtained on the datasets under consideration by the previous approaches for both classification tasks. for this we apply bert @xcite which is known to perform very well on many tasks including argument mining. in practice, the use of argument mining to evaluate public participation processes only adds value when the benefits outweigh the effort. manual coding of data and the training or fine-tuning of machine learning models are costly. in addition, machine learning requires expert knowledge and usually cannot be performed directly by the municipalities. an optimal solution would be a universally valid model that can be applied flexibly to new datasets. our third objective is hence to investigate the extent to which trained models can recognize argument structures in other public participation processes that were not part of the training process. our contributions are: (1) we present a new data corpus of five mobility-related public participation processes that vary in content and format. the german corpus comprises 17, 306 sentences coded with an argument scheme tailored to informal public participation processes. (2) we perform a broad comparison of previously published best approaches for argument mining in public participation processes, which so far have been evaluated mostly on distinct datasets. we compare the algorithms directly on our data corpus and compare the performances. (3) we show that bert surpasses previously published argument mining approaches for public participation processes on german data for both tasks. especially when classifying argument components, macro f 1 results improve by between 0.05 and 0.12 depending on the dataset. (4) in a cross-dataset evaluation, we show that bert models trained on one dataset can recognize argument structures in other public participation datasets (which were not part of the training) with comparable goodness of fit. this finding is an important step towards practical application in municipalities. ## related work mining arguments in the domain of citizen participation has been the subject of several studies. much of this work centers on u.s. e-rulemaking initiatives, where citizens are given the opportunity for feedback on rule proposals. an early attempt to identify, classify, and relate arguments in e-rulemaking was made by @xcite ; @xcite . arguments were built as trees of claims and subclaims or main-support with support relations. @xcite extended the detection of generic argument components (support and opposition) with corpus-specific argument types. niculae et al. (2017), galassi et al. (2018) and cocarascu et al. (2020 ) differentiate between five proposition types (fact, testimony, value, policy, and reference) and evidence or reason relations. in addition, other research examined specific properties of argumentation and discourse in public participation processes. @xcite identified the lack of appropriate justifications as a common problem in the analysis of citizen contributions and tried to predict whether and by what means a proposal is verifiable. subsequent work was presented by @xcite and @xcite . furthermore, @xcite and @xcite investigated discourse analysis in more detail and measured controversy and divisiveness in argument graphs. besides e-rulemaking initiatives, informal public participation processes were considered. our work shares most similarity to @xcite who focused on a german-language process about the restructuring of a former airport area. the authors developed an argumentation scheme specifically adapted to discursive online public participation processes. with regard to languages other than german, @xcite and in a followup work @xcite studied a corpus consisting of over 200, 000 political arguments in chilean spanish dialect, derived from a participatory process to form a new constitution for chile. the arguments were classified thematically according to constitutional concepts and also as either policies, facts or values. further work @xcite paid attention to the complex structure of arguments in public online participation. relying on a japanese dataset, the authors presented an annotation scheme for discussion threads taking care of inner-post relations and inter-post interactions. although the work to date has produced encouraging results, most approaches are not yet mature for practical use (e.g. with german public participation processes). only few previous research addressed the development of general models (see @xcite , who perform a cross-dataset comparison of baselines for relation prediction). therefore, this paper investigates the cross-data transferability of trained models for the identification and classification of argument components in public participation processes, an investigation that is highly relevant for practical use. ## methodology argument mining can be divided into three subtasks: segmentation, segment classification, and relation identification @xcite . first, argumentative text is split into argument discourse units (adus). second, adus are classified according to their function in the argument. third, relations between adus are identified. peldszus and stede (2013) assume here that it is known which texts are argumentative or relevant for the argumentation. @xcite widen the first task and include the distinction between argumentative and non-argumentative units. in this work, we focus on (a) the classification of discourse units as argumentative (adu) and non argumentative (non-adu) and (b) the classification of adus according to contextual clausal properties for informal public participation processes. in the following, these two tasks will be referred to as task a and task b. we define each sentence as discourse unit, so that both tasks are sentence-level classification tasks. ## conclusion and future work we investigated (a) the distinction of adus and non-adus and (b) the classification of major posi- in future work, we plan to investigate whether our best model can generalize to non-mobility public participation processes in urban planning to cover a broader range of topics. to further improve our models, we will concentrate on improving the detection of argumentative discourse units. although we were able to achieve promising results, it has become apparent that distinguishing adus from non-adus is a particular challenge. additionally, we will extend the classification for sentences that include multiple argument components (major position and premise) and address stance detection.