Exploring social theory integration in agent-based modelling using multi-objective grammatical evolution
Abstract
In Generative Social Science, modellers design agents at the micro-level to generate at the macrolevel a target social phenomenon.
In the Inverse Generative Social Science (iGSS), from a target phenomenon, the goal is to search for possible explanatory model structures.
This model discovery process is a promising tool to improve the explanatory capability and theory exploration of computational social science.
This paper presents a framework for iGSS and applies Grammatical Evolution to an empirically-calibrated agentbased model of alcohol use.
Results of the model discovery process find many alternative rules for agent behaviours with different trade-offs.
Future work should involve domain experts to evaluate the discovered structures in terms of theoretical credibility and knowledge contribution.
1. Introduction 1.1. Inverse generative social science
Agent-based modeling (ABM) is a bottom-up methodology that models a complex system as a collection of heterogeneous agents and their interactions.
ABM has become an established tool for Generative Social Science (Epstein, 1999).
If an agent-based model with a defined set of mechanisms can produce the target social phenomenon, the model is a candidate explanation for the phenomenon.
However, there can be other models that can generate the same target phenomenon.
In the inverse problem, from a target phenomenon, the aim is to find possible explanatory agentbased models.
The process is known as Inverse Generative
Social Science (iGSS) or model discovery .
In the develop- ment of agent-based models, modellers usually manually experiment with different sets of mechanisms, often derived from subjective interpretations of theory.
The iGSS methodology allows modellers to explore the space of model structures automatically with the aim of better explanatory capability.
iGSS usually utilizes techniques from Artificial
Intelligence and provides a powerful tool for social scientists to explore combinations of social theories or even develop new social theories.
The nascent field of iGSS has limited literatures.
Smith (2008) used a genetic algorithm to identify simplified rules of behavior that could reproduce the observed social assortativity of birds.
Zhong et al.
(2014) used gene expression programming to identify a individual reward function to reproduce observed human crowd dynamics.
Gunaratne &
Garibay (2017; 2020) used genetic programming to revise agents’ farm selection rules to reproduce the archeological population demography.
Vu et al.
(2019) used Genetic Programming to identify alternative situational mechanisms for a social norms model of alcohol use.
In these works, the model discovery limits to a single rule in agent decision making, usually manipulating a single equation.
Recently,
Greig & Arranz (2021) extends that by working with a series of equations, basic mathematical and conditional operations in two examples of flocking and opinion dynamics. Finally, instead of focusing on a single mechanism, Vu et al.
(2020) took a step further by manipulating multiple mechanisms of agent drinking decision within a social roles model of alcohol use.
A limitation of this work is that it includes only few constants in the model discovery process.
This means that, for every discovered alternative structure, there is a limit to the ability of the model discovery process to improve the fitness further. Ideally, for each discovered structure, a parameter calibration should be carried out to accurately evaluate its fitness. In this paper, we want to propose a solution to this issue by including parameter distribution selection in the model discovery process.
1.3. Aim and organisation of the paper
This paper aims to demonstrate a bi-level model discovery process that explore both model structures and parameter distributions to explain a social phenomenon.
As a case study, the paper uses an established agent-based model based on social norm theory.
Section 2 briefly introduces the existing model of social norm theory for alcohol use.
Section 3 provides background on multi-objective Grammatical Evolution.
In Section 4, the model discovery methodology is explained.
Results and discussion of the process are provided in Section 5.
Lastly, Section 6 concludes the paper and suggests possible future works.
2. A calibrated norm-theory agent-based model of alcohol use
This paper uses an agent-based model of alcohol use described in detail in Buckley et al.
The individual decision making was based on frequently used psychological models of behaviour and uses social norm theory to explain alcohol use.
The agents in the model are individuals, and are assumed to be representative of adults in the US.
Agent drinking behaviors are typologized using five schema representing different daily drinking practices: abstaining, light drinking, moderate drinking, heavy drinking and very heavy drinking.
The following explains the agent drinking decision, including several important concepts that will be used later on in the model discovery process.
The decision process begins with a probabilistic selection process between two pathways: habitual or intentional pathway.
The trigger of a pathway is based individual automaticity .
Automaticity ranges from 0 (always intentional) to(always habitual).
If a sampled random number betweenand 1 is lower than an individual’s automaticity, they will behave according to their previous drinking patterns on that day.
Otherwise, the intention pathway is triggered, and intentions will be calculated to determine the probability of different drinking decisions.
In the intention pathway, social norm theory was implemented.
The drinking decision is affected by descriptive norms, injunctive norms and autonomy.
The injunctive norm refers to the perceived acceptability of drinking in society for an individual like them (based on membership of socio-demographic subgroups).
The descriptive norm is an individual’s appraisal of actual drinking behavior for an individual like them.
Autonomy refers to the individual’s desire to ignore the norms.
An autonomy of 0.6 would indicate that the agent only pays attention to norms 40% of the time.
The model was calibrated to the US Behavioral Risk Factor
Surveillance System (BRFSS) survey data for New York, adjusted to per-capita New York State alcohol sales data for each year using a method described in Rehm et al.
Three alcohol use targets were defined for each year describing (1) prevalence — the overall proportion of individuals consuming alcohol at least once during the previous year, (2) quantity — the average grams of alcohol consumed per day among drinkers, and (3) frequency — the average number of drinking days per month among drinkers.
Targets were calculated separately for each year and split by sex.
Models were calibrated using data for the years 1984and validated using reserved data from 2011-2015.
3. Multi-objective Grammatical Evolution
Genetic programming (GP) (Koza, 1992) is used to generate computer programs 1 automatically with the help of genetic operators. It is usually applied to problems where there is an underlying requirement for structural optimization, besides the need (or not) to find the optimal parameters of the problem. An example is the design of a digital filter where besides having to find a set of optimal filter parameters, one also needs to determine the order of the filter.
Although traditional genetic algorithms (GAs) are well suited for finding the optimal parameters of an optimization problem, they are not able to represent the structural requirements of a solution in chromosomes.
A chromosome in GP besides supporting constants, it has provision for variables and functions (including algebraic operators, such as + and − ), which can be combined in different ways to generate a syntax tree .
In a similar fashion as it is done for GAs, a GP algorithm evolves a population of computer programs (or candidates) over many generations where selection and variation operators (e.g. crossover and mutation) take turns to find the fittest individuals.
Traditionally in GP, Lisp prefix notation was used to represent a syntax tree in a chromosome but it has several shortcomings, such as the chromosome can have variable length, and that it is very easy for crossover and mutation operators to generate illegal offsprings.
To mitigate these shortcomings, this paper uses Grammatical
Evolution (GE) (O’Neill & Ryan, 2001; 2003) which relies on the Backus–Naur form (BNF) syntax.
This ensures chromosomes with a fixed length, restricts the search space in a way that it is possible to apply standard crossover and mutation operators more freely, and prevents the generation of illegal offsprings.
Multi-objective evolutionary algorithms (MOEAs) are very popular nowadays for dealing with multi-objective optimization problems.
An MOEA evolves a set of solutions (also known as a population) over several generations by applying operators based on the principles of natural evolution such as selection, crossover, and mutation, until some termination criterion is satisfied (e.g. a maximum number of generations has been exceeded).
There are many MOEAs in the literature and existing ones can be categorised as Pareto based, decomposition based and indicator based.
For a more detailed discussion about the different types of MOEAs, including their strengths and weaknesses, the reader is referred to a recent tutorial by Emmerich & Deutz (2018).
In this study we employ a GP version of a popular Pareto based MOEAs known as NSGA-II (Deb et al., 2002), and the implementation can be found in the PonyGE2 toolkit (Fenton et al., 2017). NSGA-II relies on a non-dominated sorting algorithm to assign ranks to solutions, and on a diversity preservation mechanism that ensures a good spread across the Pareto frontier. To evaluate the performance of a non-dominated solution set obtained by a multi-objective optimization algorithm there are several performance indicators in the literature that could be used.
The hypervolume indicator (Zitzler & Thiele, 1998) is very popular in the evolutionary multiobjective optimization community and will be used in this study.
This is also known as the Lebesgue measure and it is determined by quantifying the region in the objective space enclosed by the front of the non-dominated solutions and an upper bounded reference point (assuming minimisation). Improving the hypervolume means increasing the area between the non-dominated solutions and the reference point, and this could be achieved by either improving the convergence (i.e. solutions with better performance with respect to both objectives) or improving the spread across the front (i.e. generating more evenly spread non-dominated solutions). To determine the exact hypervolume value we use a dimension-sweep algorithm (Fonseca et al., 2006). 4. Model discovery method 4.1. Overview of model discovery process
A schematic of the overall model discovery process is shown in Figure 1.
Step 0 represents a pre-condition for the model discovery process.
We defined a library of theory building blocks implemented as model components and a grammar to guide the search process.
Step 1 to 6 is the Grammatical Evolution process.
In Step 1, a initialized population of models is generated.
In Step 2, variation operators, e.g. crossover and mutation, are applied to produce new candidate models.
In Step 3, the models in the current population are evaluated for generative sufficiency.
Step 4 selects the models to retain in the population.
If convergence is not achieved, go back to Step 2.
If the convergence is achieved, the Pareto optimal model structures will be assessed for their theoretical contribution.
Afterward, if improvement is needed, the whole process can be restarted by adjusting the grammar or the library of components.
Libraries of model components: entities, attributes and mechanisms
Grammars defining permissible integrations of model components
Step 0. Generative social science ontologies and End To search for the combinations of social theories that offer the best explanation to the macro-level phenomenon (drinking behaviors in New York State), we make use of GE with bi-level multi-objective optimization .
The model structure and the model parameters are separated in a bi-level formulation, where for each model structure identified by the GE algorithm at the upper-level, a separate calibration process is conducted to identify the best parameters at the lower-level.
For clarification, all model parameters were calibrated in the original model (Section 2), only some parameters relevant to the alternative structure will be re-calibrated in the model discovery process.
This is known as a nested approach in the bi-level optimization literature (Sinha et al., 2018), which is commonly used in conjunction with evolutionary computation algorithms, such as genetic algorithms, differential evolution, and swarm intelligence.
However, to our knowledge, this is the first time that this approach is used in conjunction with a GE algorithm.
4.2. Objectives
We use the following three objectives in this study:
First, we consider the overall goodness-of-fit between the model output and the target data, accounting for sampling uncertainty in both the output and target measurement.
Here, our objective function for model structure x is an implausibility metric (Vernon et al., 2010) that measures the error between the m th simulated output y ⋆m , averaged over N model replications, and equivalent empirical target data y m over a sequence of temporal observations k defined by:
[Equation]
where M is the number of output measures, K is the number of observations, s m [ k ] is the observed standard error for output m at time point k , and ( d m ) 2 is the variance of the model discrepancy for output m , which is taken as 10% of the possible output range for each output.
‘Model discrepancy’ is the error in a model output that arises because the model is not a perfect representation of reality.
Since we are interested in whether the iGSS process might work better to fit to targets based on male drinking patterns than female drinking patterns, and vice versa, we also decompose this first objective into two sub-objectives: [Equation]
[Equation]
where M male and M female are the subset of targets relating to men and women respectively.
The final objective we consider is a complexity measure, which aims to promote model parsimony for the purposes of interpretability and avoiding over-fitting.
This type of approach was introduced to genetic programming by (Rodr´ıguez-V´azquez et al., 2004) to combat the issue of bloat —where the tree lengths in the genetic program tend to drift upwards over time.
We calculate interpretability therefore as the number of nodes in the grammar-based tree: [Equation] [Equation] nodes(
[Equation]
) , (4) where nodes(.) calculates the number of nodes in the tree encoding model structure x .
4.3. Bi-level optimization
In this study, our interest is to find model structures and parameters that offer the best trade-offs between the complexity of the model and the goodness-of-fit with respect to multiple phenomenon targets.
In total there are three objectives in this GE problem that has to be dealt with simultaneously, and these are: patterns of female alcohol use; patterns of male alcohol use, and complexity.
When dealing with multiple conflicting objectives the problem does not contain a single optimal solution, which is often the case with single-objective problems.
Besides the existence of a single optimal solution for each objective function, there is also a set of trade-off solutions where an improvement in one objective is earned at the expense (deterioration) of another objective(s).
The best trade-offs in terms of Paretooptimality can be captured by the concept of dominance.
Consider two solutions a ∈ R M and b ∈ R M where M is the number of objectives, a is said to dominate b if a is not worse than b in all objectives (i.e. a i ≤ b i ∀ i = 1 , . . .
, M ), and a is strictly better than b in at least one objective (i.e.
∃ i ∈{ 1 ,...,M } : a i < b i ).
From a set of solutions, any subset that contains only solutions that are not dominated by any other solution in the set constitutes a non-dominated set, and the non-dominated set with respect to the entire search space is known as the Pareto-optimal set.
The next section will describe in detail how the GE incorporates both model structures and parameters via the use of grammar.
4.4. Grammar
The grammar describes what components in agent behavioural rules will be included and how they can be combined together.
This study focuses on agents’ daily decision of the number of alcoholic drinks to consume (Section 2).
The structure of agents’ intention pathway will be exposed to the model discovery process.
It is a function that decides the number of drinks based on agent desire to drink and social norm theory concepts (injunctive norm, descriptive norm).
The grammar also includes the individual parameters of the agents that are relevant to the number-of-drinks decision: the automaticity that control the trigger between habitual and intentional pathway and the autonomy which refers to individual desire to ignore the norm.
For these two parameters, the grammar will have several options for their distributions in all agents.
Listing 1 describes the grammar that guides the model discovery process.
Each candidate (a program < p > ) contains 10 expressions: 1 for the log odds intention, 3 distribution options for the automaticity of three different drinker types, and 6 distribution options for the autonomy of three different drinker types split by male and female.
Each expression can be formed only by a defined combination of expressions, variables, constants, and distribution options.
This hierarchical grammar captures the complexity of different expressions.
[Table] The intention pathway uses a log odds function for each schema because, in the model, the probabilities of instantiating each drinking schema are represented using a multinomial logit equation.
The log odds expression <lo exp> can be modified by multiplying log odds with a variable/constant expression <vc> , or by summing two log odds.
The variable/constant expression <vc> can be constructed from Autonomy , (1-Autonomy) , constant between 0 and 1 <c 01> , taking a square root, or raising to the power of 2.
For the expression within the odds <exp> , it can be constructed with norm-theory concepts
Desire , Descriptive , Injunctive and several operation (sum, multiplication, square root, raising to the power of 2).
For simplification, automaticity and autonomy distributions can be selected from four distribution options <autonomy opt> : skewed-right, centre, skewedleft, bi-modal.
5. Results and Discussion
The model discovery evolutionary setup was as follows:candidates per population for 50 generations, 75% subtree crossover, 25% subtree mutation, and other default settings of PonyGE2.
It is computationally intensive to do a complete run of GE; the process took about 60 hours on an Intel i9 9980XE processor with 36 cores.
The hypervolume plot, Figure 2, shows the convergence of the model discovery process over 50 generations.
There are 97 non-dominated model structures in the Pareto front of the last generation.
However, some structures are duplicated (in terms of the three fitness) because they have the equivalent functional form of log odds intention, and the simulation was run with the same seed.
After removing the duplicated structures, there are 82 unique structures.
Figure 3 shows the trade-offs between three objectives (male goodness-of-fit, female goodness-of-fit, and complexity) of 82 non-dominated structures in a parallel coordinates plot.
Because of the trade-offs between objectives, model selection is not a straightforward process.
[Figure]
[Figure]
[Figure]
Let us discuss the model structures in detail.
There are two parts the log odds intention and the parameters.
The discussion will focus on the structure of the log odds intention.
In the non-dominated structures, there are structures with the same log odds intention but different parameters which leads to a slightly different goodness-of-fit.
So when showing an example of a log odds intention, the model structure with the least average goodness-of-fit of male and female will be presented.
The simplest structure has a complexity measure of 26, and the most complex one is 232.
Bringing attention to the normtheory concepts.
For structures with a complexity of less than 35, most structures only contain Desire (for example,
Listing 2).
The only different case is that Injunctive appears by itself in the simplest structure of 26 (Listing 3).
These
Injunctive-only structures result in unbalanced goodness-offit: low male goodness-of-fit but high female goodness-of-fit (0.888 and 2.255 on average).
On the other hand, Desireonly structures have a balance goodness-of-fit between male and female (1.578 and 1.470 on average).
In the context of social norm theory, when the log odds intention consists of only Desire and not Injunctive and Descriptive norm, agents follow their past behaviour initialised at baseline.
As the complexity increase, injunctive and descriptive norms start to appear.
In complexity from 44 to 71, there is either
Injunctive or Descriptive in the log odds intention.
From complexity 72, both appear at the same time in most structures.
This means that to achieve better male and female fitness (as the complexity is higher), injunctive and descriptive norms must be included.
It is noted that Desire is still a necessary component since it is present in all structures Listing 2. Non-dominated
[Table] id 95 LogOddsIntention=Autonomy*log(odds(Desire));
[Table]
with size greater than 27. Table 1 summarizes the count of Desire, Injunctive, and
Descriptive in the log odds intention function, averaged over the number of structures in different ranges of complexity.
The count of all three concepts increases as the complexity increases.
Desire concepts are always greater than the other two concepts.
Injunctive and Descriptive counts are not greatly different.
[Table]
Models with high complexity are extremely difficult to interpret.
For example, Listings 4 shows a model structure with a complexity of 72.
The log odds intention is complex but still manageable.
It shows intention as a function of
Desire, the interaction of Desire and Injunctive, and the interaction of Desire and Descriptive. However, looking at
[Table]
the most complex structure in Listing 5, it is near impossible to interpret the function.
Thus, if the modellers only want the best goodness-of-fit and does not interest in theoretical interpretability, complex structures are selected.
However, to interpret the agent behaviour and underlying social theory, it is better to pick simpler structures.
Other strategies can be used during model discovery process for theoretical interpretability include simplifying the function before evaluation, limiting the tree depth in GE configuration
[Table]
or restricting the grammar.
With the grammar, modellers have to be careful to balance exploration capability and interpretability.
6. Conclusion
This paper presents a model discovery method that utilizes multi-objective GE to explore different combinations of social theory concepts and parameter distributions to search for alternative agent behavioural rules.
The case study of alcohol use modeling has shown that different realizations of theoretical mechanisms can result in trade-offs between different metrics.
Our model discovery method offers a promising approach to generating novel combinations of social theory components.
In the research frontier of inverse generative social science, our work shows the feasibility of performing a bi-level optimization, i.e. joint structureparameter model discovery.
Future research should involve domain experts to interpret the discovered model structures.
This work also highlights the challenge of meaningful model discovery, requiring the involvement of the domain expert during grammar design as well as the theoretical credibility assessment.
[Table]