Learning to Elect

1 Introduction . Voting systems are highly prevalent in our daily lives . Examples range from large scale democratic elections to company or family-wide decision making , recommender systems and product design [ Boutilier et al. , 2015 ] . As with any social decision-making process , the goal of designing voting rules is to reconcile differences and maximize some collective objective . The area of research that studies different voting rules and the approaches to designing them is called voting theory . A vast number of voting rules have been proposed over the years . Among them is the widely applied plurality rule . Despite being simple and intuitive , the plurality rule is very limited in that it does not consider the strength of voters ’ preferences . Other examples of voting rules , such as Borda and Copeland , take into consideration the ranked preferences of the voters . Voting theorists have developed different approaches to designing voting rules . For example , the axiomatic approach constrains the voting rules to satisfy certain desired properties ( axioms ) such as anonymity ( treating all voters equally ) and neutrality ( treating all candidates equally ) . The utilitarian approach , on the other hand , aims to maximize a pre-defined notion of social welfare — a scalar quantity that measures the quality of the elected candidate in the eyes of the electorate . There are major hurdles to overcome in the traditional way of designing and implementing voting rules . First , the celebrated Arrow ’ s Theorem states the nonexistence of non-dictatorship voting rules that simultaneously satisfy a set of seemingly sensible axioms [ Arrow et al. , 1951 ] . Second , for some voting rules such as the ones based on pairwise comparisons , finding the winner can be computationally expensive , making them infeasible for large-scale applications . Last but not least , for the utilitarian approach , it is not obvious how to design voting rules that maximize a given notion ∗Equal contribution . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . of social welfare.2 There might be a diverse set of social welfare functions of interest , but theory is lacking in finding their corresponding optimal voting rules . In this paper , we aim to tackle the latter two limitations using neural networks . While doing so , we also seek to preserve certain desired properties such as voter anonymity . In particular , we identify three permutation-invariant neural network ( PIN ) architectures : DeepSet [ Zaheer et al. , 2017 ] , Set Transformer [ Lee et al. , 2019 ] , and Graph Isomorphism Network ( GIN ) [ Xu et al. , 2018 ] , and apply them to learn the voting rules . As compact and universal function approximators , such trained neural networks not only address the computational burden of some voting rules , but also provide a flexible approach to maximize a diverse set of social welfare functions . The main contributions of this paper include : • We show that PIN architectures are theoretically and empirically well-suited for learning to represent voting rules . Theoretically , we show that the three PIN architectures are universal approximators in the space of permutation-invariant functions . This includes a novel proof on the universality of Graph Isomorphism Networks with fully-connected graphs ( as in learning voting rules ) . • We apply the aforementioned PIN models to mimic five classical voting rules : plurality , Borda , Copeland , Maximin and Kemeny . We show that they can mimic these voting rules to compelling accuracy and can generalize seamlessly to unseen real datasets and elections with an unseen number of voters . • We train the networks to maximize two different social welfare functions — utilitarian and egalitarian — on elections sampled using three different underlying voter utility distributions . We show that the PIN models discover novel voting rules that maximize social welfare better than classical voting rules . In cases where theoretical optimal voting rules are known ( i.e . for the utilitarian social welfare function ) , the PIN models match the optimal performance . The organization of this paper is as follows . Section 2 provides background on voting theory . Section 3 describes our method of using PINs to learn voting rules . Specifically , we introduce the permutation-invariant network architectures ( Section 3.1 ) , show the universality results ( Section 3.2 ) , and describe the proposed the training procedure for learning voting rules in detail . In Section 4 , we show comprehensive experiment results on the effectiveness of PIN models in learning voting rules . We discuss related works in Section 5 , limitations in Section 6 and conclude with Section 7 . 2 Background . 2.1 Voting theory preliminaries . We adopt the formalism used by Boutilier et al . [ 2015 ] . Let N = { 1 , . . . , n } be a set of voters , A = { a1 , . . . , am } be a set of candidates and R = { 1 , . . . , m } be the integer rankings each voter can assign to a candidate . Each vote i is represented as a bijection from candidates to rankings : σi : A 7→ R. The vector of candidate preferences # σ = ( σi , . . . , σn ) ∈ ( Sm ) n is called a preference profile . A voting rule f : ( Sm ) n 7→ A maps preference profiles to candidates . The utilitarian approach to voting makes the assumption that voters have utility functions ( u : A 7→ R+ ) over the alternatives which quantify how much a voter prefers a candidate . Utility functions are consistent with the rankings — that is , candidates with higher utilities are ranked higher by voters . The vector of all voters ’ utility functions # u = ( u1 , . . . , un ) is called a utility profile . The social welfare of an alternative sw ( a , ~u ) : A× Rn×m 7→ R quantifies the “ desirability '' of a candidate a under a given preference profile ~u . The utilitarian approach posits that the ultimate goal of a voting rule is to pick the candidate that maximizes social welfare . Note that since voting rules only have access to rankings and not the utilities , this is often impossible to achieve without further assumptions . A popular notion of social welfare is the utilitarian social welfare function , which computes the sum of all the utilities the voters assign a candidate swutil ( a , ~u ) = ∑ i∈N ui ( a ) . There exists , however , many different social welfare functions . For example , the Rawlsian social welfare function aims to make even the least happy voter as happy as possible : swrawl ( a , ~u ) = mini ui ( a ) , and the egalitarian social 2We consider average-case social welfare maximization , unlike the worst-case analysis in the distortion literature [ Boutilier et al. , 2015 , Caragiannis et al. , 2017 ] . welfare function aims to maximize the utilitarian welfare , regularized by a penalty term promoting equality : swegal ( a , ~u ) = ∑ i∈N ui ( a ) − λ ∑ i∈N ( ui ( a ) −minj ( uj ( a ) ) ) [ Allcott , 2011 ] . It is up to the designer of the voting system to pick a notion of social welfare best suited for the task at hand . 2.2 Classical voting rules . Most classical voting rules can be classified under two groups : score based rules and comparison based rules . Score based rules ( also called scoring functions ) are defined by a score vector ~s = ( s1 . . . sm ) . Each candidate a ∈ A is assigned the score ∑ i∈N sσi ( a ) and the candidate with the largest score is picked as the winner [ Boutilier et al. , 2015 ] . Famous examples include Plurality ( with a score vector of ~splr = ( 1 , 0 , . . . , 0 ) ) and Borda ( with a score vector of ~sborda = ( m − 1 , m − 2 , . . . , 0 ) ) . Comparison based rules operate using pairwise comparison matrices R ∈ Rm×m whose entries are filled based on how candidates fare against each other in pairwise comparisons . Well-known examples include the Copeland , Maximin and Kemeny rules . The Copeland rule picks the candidate who fares better in pairwise comparisons the largest number of times : argmaxi ( ∑ j rij ) where rij stands for the number of times candidate i fares better against candidate j in the voter preferences . Maximin , also known as Simpson ’ s rule , picks the candidate for whom the candidate who fares the best against him/her in pairwise comparisons has the least pairwise score : argmini ( maxj rji ) [ Bubboloni et al. , 2020 ] . The Kemeny rule [ Kemeny , 1959 ] first computes a ranking that maximizes the sum of all pairwise wins : σ∗ = argmaxσ ∑ i σj rij , where i σ j means that candidate i is preferred against j according to ranking σ . The Kemeny winner is the candidate that ranks the first in σ∗ . Note that computing the Kemeny ranking or Kemeny winner is NP-hard [ Bartholdi et al. , 1989 ] . 2.3 Average-case optimal voting rules . In cases where the social welfare function takes a very simple algebraic form , it might be possible to derive the voting rule that achieves the largest expected social welfare under a given utility distribution Pu . For example , it is known that the average-case optimal voting rule for the utilitarian social welfare funtion is a score-based voting rule with the score vectors s∗k = Eu∼Pu [ u ( a ) | ( σ ( u ) ) ( a ) = k ] ( or , the average utility of the candidates that are ranked at kth position ) [ Boutilier et al. , 2015 ] . 3 Permutation-Invariant Networks ( PINs ) to learn set-input functions . We review the fundamentals of learning set-input functions through the use of permutation-invariant networks ( PINs ) . Learning set-input functions is desirable in the context of learning voting rules : 1 ) it enables processing elections of varying sizes ( i.e . different number of voters ) 2 ) it makes it possible to satisfy desirable properties such as anonymity ( invariance to the shuffling of voter identities ) through architectural constraints . We review three permutation-invariant architectures ( DeepSets , fully-connected graph networks and Set Transformers ) and show that they ’ re universal approximators of set-input functions . We also detail the training procedure for learning voting rules . 3.1 Constructing permutation-invariant architectures . Functions defined on sets are by definition permutation-invariant [ Zaheer et al. , 2017 ] . Property 1 . Let 2X represent the powerset of X . Any set-input function f : 2X 7→ Y must be invariant to the reordering of its inputs by any permutation σ : f ( { x1 , . . . , xM } ) = f ( { xσ ( 1 ) , . . . , xσ ( M ) } ) Zaheer et al . [ 2017 ] showed that one can practically train expressive set-input neural networks by chaining together a permutation-equivariant feature extractor ( encoder ) and a permutation-invariant decoder.3 permutation-equivariance is the property that when the inputs of a function are permuted , the outputs get permuted the same way : Property 2 . A function f : RM×din 7→ RM×dout is permutation-equivariant if f ( [ xσ ( 1 ) , . . . , xσ ( M ) ] ) = [ fσ ( 1 ) ( x ) , . . . , fσ ( M ) ( x ) ] 3Composition of permutation-equivariant and permutation-invariant functions results in permutation-invariant functions . All of the architectures described below follow the central design principle of composing permutationequivariant building blocks with permutation-invariant ones to build expressive set-input networks . DeepSets [ Zaheer et al. , 2017 ] This architecture first encodes each element of the set independently using an encoder network ( e.g . a multilayer perceptron ) , pools the outputs of the encoder ( e.g . using sum , max or mean pooling ) and finally passes the result through a decoder network ( e.g . another multilayer perceptron ) . Since the encoder treats the elements of the set independently to achieve permutation-equivariance , the pooling operation is the only step that can model interactions between the elements in the set . ( Fully-connected ) graph neural networks ( GNNs ) Instead of achieving permutation-equivariance through the independent processing of the set elements , we can instead view the input set as a fully-connected graph and build a GNN-based [ Scarselli et al. , 2008 ] encoder that takes into account all of the interactions between different elements of the set . Each GNN layer transforms the nodes in the graph by concatenating a permutation-invariant aggregation of features of the neighbour nodes and a transformation that produces the next-layer features given the current-layer features and the result of aggregation . A GNN consists of multiple GNN layers , and the graph-level embedding is obtained by pooling all the node features using a permutation-invariant function . In our experiments , we use a powerful variant of GNN — the Graph Isomorphism Network ( GIN ) [ Xu et al. , 2018 ] . Detailed background on GNNs and GIN can be found in Supplementary Material . For a fully-connected graph , the graph structure remains unchanged under node permutation . Since both the neighbourhood aggregation and the graph-level pooling operations are permutation-invariant , the whole GNN represents a permutation-invariant function . Set Transformers [ Lee et al. , 2019 ] The computational building block of set-transformers is the Query-Key-Value ( QKV ) attention [ Vaswani et al. , 2017 ] , which can be interpreted as a differentiable dictionary retrieval operation . The neural network controls both the input and output space of the QKV operation using multihead attention , which multiplies all the input and output tensors by learnable weight matrices . The multihead attention possess desirable permutation-equivariance/invariance properties for learning set-input functions . Set Transformer builds highly expressive encoder-decoder architecture with multiple multihead attention blocks , resulting in the overall permutation-invariance [ Lee et al. , 2019 ] . Detailed background and the full encoder-decoder architecture are found in Supplementary Material .

Summary: This paper empirically studies the ability of PIN (Set Transformer, GIN, DeepSets and MLP) applied to predicting the (top-1) winner of voting rules (Plurality, Borda, Copeland, Maximin and Kemeny) and predicting the social-welfare-maximizing (utilitarian and egalitarian) candidates. The PINs are trained on synthetic data, agents whose utility over candidates are generated from Dirichlet distributions. The PINs are tested on synthetic data (Table 1 & 3), synthetic data with more voters than in training time (Table 2) and real world data (Table 4).