Cox Bayesian Optimization for Police Patrolling
Abstract
This paper presents a generalization of Bayesian Optimization (BO) that can cope with objective functions deﬁned over discrete spaces.
Typically, BO assumes a latent model represented by a Gaussian Process (GP) prior under Gaussian likelihood assumptions.
This works well for continuous real valued objective functions that are Lipschitz continuous.
However, discontinuous objective functions, which are a consequence of discrete inputs and outputs, present important challenges.
The contribution in this paper is to address these challenges by making use of a Log Gaussian Cox Process (LGCP) in the BO framework, which has a GP as an argument of the link function, and uses Monte Carlo Tree Search (MCTS) to deal with discrete inputs.
We evaluate the proposed algorithm over a synthetic objective function and apply it to a real world problem – police patrolling, where observations are integers at a set of discrete locations.
Introduction
Over the last decade, Bayesian Optimization (BO) [3, 14] has gained popularity among the Machine Learning community due to its ﬂexibility and efﬁciency for solving expensive optimization problems, spanning a wide range of areas in academia and industry.
Some interesting applications of BO include environmental monitoring [9], experimental design [2] and interactive user interface [3].
In this paper we propose Cox BO (CBO) which allows BO to optimise functions with observations arising from the realisation of discrete events.
In the spatial-temporal scenario, this can be represented by an in-homogeneous Poisson Point Process, whose intensity is commonly modelled by a non parametric
Bayesian model, particularly a GP giving rise to the popular Log Gaussian Cox Process [11, 6].
Various methods have been proposed to approximate posterior distributions with non-Gaussian likelihoods, such as Laplace Approximation [13], Expectation Propagation [13] and Variational
Bayes (VB) [12].
A unifying framework for constructing ﬂexible models using a generic exponential family likelihood was proposed by [15] and deﬁned as Generalized Gaussian Process Model (GGPM).
GGPM assumes y follows an exponential distribution where the expected value of y , denoted by η = g − 1 ( f ( x )) , where g is a link function, varies according to the distribution of y .
If y is Poisson then g ( x ) = log ( x ) .
The likelihood function in LGCPs lacks of closed form solution, and there has been seen signiﬁcant research into the efﬁcient estimation of the intensity function given possibly large number of observations, using VB [8, 1] and other Bayesian inference techniques [16].
Despite latest developments in BO and the existing LGCP theory, to the best of our knowledge, there does not exist a piece of work which bridges both areas.
Our paper describes a framework for Bayesian Optimization to handle a Poisson likelihood and build on the work by [8, 15, 14].
An important contribution of this paper is the application of the proposed methodology to monitoring spatial temporal event data.
Particularly, we have identiﬁed the problem of optimal police patrolling to better understand and uncover areas with higher number of crimes.
In the more general setting, one would ideally build a continuous representation of λ that can accommodate multiple patrolling modalities (helicopters, foot, drone, car, etc). However, in this paper, we have discretised the action space over street networks to reﬂect car patrolling. We deal with this discrete input space by applying a Monte Carlo Tree Search (MCTS) [4] approximation in the Sequential BO setting described by [10]. Presently, there is an ongoing concern about over-policing resulting from maximising only the expected number of crimes.
By using the following technique, it is possible to reduce the bias in the decision making process by taking into account the uncertainty over these estimations.
Cox Bayesian Optimization
Cox Bayesian Optimization (CoxBO) is an adaptation of vanilla BO that is able to deal with discrete event data over a continuous space.
For a detailed background on BO we refer the reader to the comprehensive review by [14].
In this paper we discretise the input space and aggregate observations into counts of events, y , which breaks the Gaussian assumption in vanilla BO. A statistical model for the data generating process is given by
[Equation]
where λ ( x n ) represents the input-dependent intensity for the in-homogeneous point process used to describe y n .
Following the principles behind a LGCP model, CoxBO places a latent model over the log of the intensity function.
Since the goal is to ﬁnd the locations for which the counts y are higher, the log-intensity will be used as a proxy optimization routine.
A critical characteristic of every BO algorithms is the acquisition function u ( x ) ∈ R , which is responsible for the active learning nature of the solution and guides the search for the location of the optimum.
As in vanilla BO, the acquisition function u ( x ) is designed to be rapidly evaluated though non-convex auxiliary goal function, effectively minimizing the number of samples and automatically controlling the balance between exploitation and exploration.
Algorithm 1 presents the CoxBO algorithm, which incrementally gathers samples from the goal function and uses active decision making to select the locations of where to evaluate next.
The data-set D represents the set of N location-sample pairs, by D = { x n , y n } i =1 ...
N . procedure BO ( f, u, GP , T ) for t = 1 , 2 , ..., T do
Find x t = argmax x u ( x | D 1: t − 1 )
Obtain a realization of the objective function: y t ∼ p ( y | log λ ( x t ))
Augment the observed data-set with new data: D 1: t = D 1: t − 1 ∪ { ( x t , y t ) }
Update the GP model of η ( x )
In practice, the hyper-parameters of the GP model are pre-trained with existing data and can be re-trained over time depending on available computational resources.
Since the likelihood is not tractable and there are a large number of observations, we have conducted inference by using
Stochastic Variational Inference [7, 1, 8].
The predictive distribution for y at x (cid:63) , p ( y (cid:63) | x ) is computed in two stages:
ﬁrst predicting the latent function by marginalizing over all possible function values,
[Equation]
[Equation]
then estimating the actual prediction over y by using the Poisson likelihood function, which becomes
[Equation]
The acquisition function is deﬁned in terms of the predictive distribution for the latent function, given by Equation 2.
Since the log-intensity is modelled with a GP, as in Equation 1, it is possible to evaluate the posterior mean estimate µ ( x (cid:63) ) and the variance σ 2 ( x (cid:63) ) for log λ ( x (cid:63) ) .
Similar to other
In the more generic setting, observations correspond to events in a space-time continuum (or any other domain). For the ﬁrst iteration of this work we discretise the space x , where y i represents a count of events over cell i in the space.
We avoid the use of myopic approximations and implement sequential setting proposed by [10].
Effectively, this leads to evaluating the objective function at a ﬁxed set of locations.
A Partially
Observable Markov Decision Process (POMDP) is deﬁned as an analog to the discrete-action BO scenario, which is solved using MCTS.
Experiments 3.1 Synthetic Poisson Process Data
The ﬁrst experiment illustrates the behaviour of CoxBO for synthetic data with known ground truth.
For evaluation purposes, we use a known log-intensity function λ ( x ) = 9 sin(0 .
7 x ) + 9 .
The input space for x corresponds to a one-dimensional space x ∈ [0 , 19] .
We executed Algorithm 1 using an UCB [5] acquisition function, with parameter κ = 7 . 5 .
As with most BO techniques, this parameter was manually tuned and represents the trade off between exploration-exploitation.
The results, illustrated in Figure 1, show that the application of CoxBO to this data is highly beneﬁcial in multiple ways.
Firstly, it can be appreciated that the model has converged after 9 iterations, which is highly beneﬁcial for the evaluation of costly functions.
It is also worth mentioning that the uncertainty over the counts remains positive for any x , given the real valued exponential link function.
I.e. even if the estimate of the GP provides a negative estimate, the ﬁnal sample for the counts y is a positive integer.
3.2 Crime Simulation over Street Networks
The ultimate goal is to better understand the areas for which a higher number of crimes occur, while exploring the latent function of crime intensity across space.
The active search procedure corresponds to decision making over street networks, where the counts (aggregated observations) are at the cells and actions are along an interconnected graph of streets.
The objective function is the number of crimes (Domestic Violence related assaults) in a given area for the city of Sydney in New South Wales, Australia.
The data are 467 crime occurrences, within a one month period – December 2014. Figure 2 shows a ground truth density from the raw crime locations.
The environment was discretised into 425 cells and the street network graph was obtained by using Open Street Maps API.
The active learning nature of the CoxBO algorithm over count data was used to decide actions of a police patrolling car within the street network.
Actions correspond to driving along edges of the graph, highlighting the discrete notion of the input domain, and observations y corresponds to a number of crimes observed in the neighbourhood region.
The covariance function used to represent the latent function for the log-intentisy of the GP was the Matern32, for with hyper-parameters were estimated from random samples of the environment and result in (cid:96) = 0 .
156 and σ f = 7 . 732 .
Similar to 3.1 we use an UCB acquisition function u ( x ) .
The goal is to identify the changing behavioural patterns of the patrolling, in terms of exploration and exploitation.
At each time-step, the maximization of the acquisition function was conducted by building an approximated solution using MCTS.
The parameters for MCTS are a 4 step look-ahead, with β = 50 , where β is the exploration exploitation trade-off in the expansion function UCT for growing the tree.
We ran 400 simulations per experiment.
It can be seen that the resulting paths are considerably different between each other and serve different purposes.
Figure 3 illustrates the effect of modifying the exploration-exploitation trade off parameter from the acquisition function, κ .
A value of κ = 50 (left in Figure 3), results in a pure exploration behaviour, i.e. a maximum coverage scenario, where not much importance is given to the expected value of crime. For a value of κ = 2 there is an over-importance to the expected value of crime intensity and the patrolling car gets stuck and known high risk locations without exploring.
κ = 8 . 5 , shown in the centre of Figure 3, presents a more sensible and parsimonious approach for simultaneously dealing with exploration and exploitation.
It can be seen how the police patrolling car meets two of the most important requirements: exploitation around areas with expected high crime and also exploration to achieve discovery of other height crime areas.
By taking an evidence based approach for decision making under uncertainty, and particularly ﬁnding a convenient value for κ , it is possible to reduce the existing bias and over-policing.
The reinforcing phenomena between over-policing and data bias can get reduced and potentially removed if the trade off between exploration and exploitation is handled in a principled manner.
Conclusion
This paper presents Cox Bayesian Optimization, which corresponds to the generalization of BO for the case when the objective function is deﬁned over the positive integers, following a Poisson distribution.
We use Stochastic Variational Inference, which coupled with a logarithmic link function and MCTS give rise to practical implementation of CoxBO that is demonstrated to work on real data-set.
We have validated the methodology on synthetic data and on a real scenario consisting of a police patrolling environment.
We envisage several areas for future work that could increase the range of applications of this work.
Particularly expand the models to generic exponential distributions with multidimensional parameters paces by using muti-output GsP. Another area of improvement is to model the data generation process following a mixture of Poisson distributions, which will leverage the strong limitation of identical mean and variance arisigin from a single Poisson distribution.  Acknowledgment
This work was supported by the Sydney Informatics Hub (SIH) at The University of Sydney and the data was provided by the New South Wales Police Force