From: ~Ransalu_Senanayake1
comment: The rest of the review. 3/3

Considering the global optimization perspective of the BO algorithm, what are we trying to achieve here? The police vehicle will explore a bit and converge to a place where the crime count is high. How does this help the police in their decision-making process?

On another aspect, although it is not clear, I believe MCTS is used to make sure the police patrol in a more structured way in the space rather than wondering all over the space with arbitrary step sizes. What are the detrimental factors of adding this constraint? Will this loose all the theoretical convergence guarantees comes with the UCB algorithm? (Are the typical convergence guarantees still applied after the LGCP, in the first place?) Because road networks are represented as a graph, it is not clear how sampling was performed? How about performing BO on a graph rather than on the Euclidean space?


4. Minor issues:
line 42 - "If the data are non-Gaussian then" - I believe what was attempted to say is "if the likelihood is non-Gaussian, then the predictive distribution does not have a closed form solution under a GP prior". This is also not always true because there are cases where the Gaussian is a conjugate prior for some non-Gaussian likelihoods such as the log-normal distribution.
line 42 - pred distribution should be p(y*|X*,y,X,\phi)
line 45 - y \inZ^+ \union {0} -- counts can be zero?
line 51 - also mention MCMC
line 64 - "about the By using"
line 64 - this line does not belong here "By using the following technique, it is possible to remove bias in the decision making process by taking into account the uncertainty estimation and the acquisition function outputs as a decision support system."
line 77 - "he input-dependent intensity for the in-homogeneous point process" not clear
line 86 - not clear "though non-convex auxiliary goal function,"
line 133 - "I.e." - Block letter
line 155 - "with hyper-parameters were estimated from random samples of the environment and result" - not clear. How did you learn these?
line 186- "Particularly expand the models" - grammar
line 189 - "arisigin"- a  typo
Shrink the figures and show a side-by-side comparision.

I deeply enjoyed reading the paper, despite the ambiguities. However, unfortunately, the manuscript is well over the page limit making it unfair to the other papers.

End of the review. 
title: On behalf of NIPS 2018 Workshop Spatiotemporal Paper33 AnonReviewer1

From: ~Ransalu_Senanayake1
comment: On behalf of NIPS 2018 Workshop Spatiotemporal Paper33 AnonReviewer1

The rest of the review. 2/3

... The Upper Confidence Bound (UCB) has been the choice for the acquisition function of the BO algorithm: u = \mu + \kappa \sigma^2 where \kappa is the parameter that controls the exploration-exploitation trade-off and \mu and \sigma^2 are estimates of the predictive distribution of the underlying stochastic process. I found the choice of \mu and \sigma^2 in the proposed approach somewhat problematic. Instead of using the statistical moments of the predictive distribution, this paper uses the mean and variance of the log of the intensity which is a Gaussian process. Using the Gaussian predictive variance as a proxy is not correct as Gaussian to Poisson is not a linear transformation. If we ask a human to tell where to sample next intuitively, he/she might possibly focus on the upper bound of the predictive variances in Fig 2. Plausibly, the authors might incorporate the higher-order statistical moments such as skewness and kurtosis into the acquisition function to account for the asymmetry of the Poisson distribution (note that mean=var=\lambda for a Poisson distribution).

Although a commonly asked question, this reviewer is wondering how the hyperparameters of the underlying GP were picked. I am keen to know about this because the optimization routine in the proposed formulation does not seem to be easy. Further, how does the cell size affect the intensity function and overall result?

It is not clear how the Monte-Carlo tree search comes into the play.

3. Applications
Attempting to reduce the existing bias and over-policing is unarguably a great move. However, I am not confident if the paper utilizes the proper statistical tools for this purpose. From the title, the paper proposes an algorithm to solve a real-world problem, and therefore, the problem at hand is more important than the technique. Hence, it is at utmost important to describe why BO is a good choice over other statistical inference techniques, tools discussed in operations research, and rule-based AI techniques. As the research seems to be in the rudimentary stage, it is advisable to make sure that  BO is a good choice for solving this multi-faceted problem. On the other hand, the horizon of this research is obscure to me. For instance, how would multiple police vehicles be used within a limited time frame (duty period of the police officers) under the proposed framework? I can think of a scenario where different vehicles with an aggregated data model use different \kappa values so that some vehicles can explore while the others exploit.

Although I appreciate using real-world datasets, the dataset used is quite confusing. Firstly, how would police patrolling be used for sampling domestic violence? For example, it is hard to imagine a cop knocking someone's door and ask if some domestic violence has been occurred in the household, merely because the BO algorithm wants to explore more. The proposed algorithm would be more useful for patrolling drug abusers than domestic violence. If my perspective is incorrect, the paper should clarify this point. Secondly, it is not verified that a crime count at a given cell in the proposed model is a Poisson process as every counting process cannot be explained by a Poisson process. On the other hand, various factors such as the severity of crimes should be taken into consideration rather than the count. The algorithm would say "if an assault happens in this house, there is a very high chance that assaults would happen in neighbor-houses" which is neither correct nor ethical. This is not an outcome based on data, but a structure carved into the model by using kernel functions which are nothing but similarity functions.

TBC
title: On behalf of NIPS 2018 Workshop Spatiotemporal Paper33 AnonReviewer1

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper33/AnonReviewer1
relevance: 4: high
impact: 1: very low
novelty: 3: borderline

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper33/AnonReviewer1
evaluation: 1. Summary:
This paper proposes an algorithm for police patrolling if the subject of interest is the number of occurrence of an activity/crime. To this end, Bayesian optimization (BO) with an underlying log-Gaussian Cox process (LGCP) is proposed. The proposed model is tested on a domestic violence dataset.

2. Methods:
It is interesting to see attempts to extend BO/multi-armed bandits beyond  Gaussian processes. Although the high-level idea of replacing the Gaussian process in BO with LGCP to account for non-negative count data is straightforward, I have not previously come across the same formulation. However, there are many similar alternatives that are worth discussing. Firstly, it is unfortunate that the extensive amount of work on various formulations of Cox processes done by Peter J. Diggle has neither been cited nor acknowledged. Although the term "Bayesian optimization" is not commonly encountered outside the contemporary machine learning community, authors are encouraged to look at Statistics and Geostatistics literature carefully. For instance, "preferential sampling" is a valid choice over Bayesian optimization because the locations and the underlying process itself are stochastically dependent. As another example, Steven K. Thompson (1990) proposes Adaptive Cluster Sampling on a grid with binomial distributions. Note that, binomial distributions with N approaches infinity is a Poisson distribution. The following papers might also be useful:
[1] Bayesian Optimization algorithm for nurse scheduling: https://arxiv.org/pdf/0804.0524.pdf
[2] Botev, An Efficient Algorithm for Rare-event Probability Estimation, Combinatorial Optimization, and Counting (2008)

Stochastic variational inference (SVI) is used to compute the hierarchical predictive equations given in eqs. 4 and 5. Since the reference given is generic, it is not clear how exactly the predictive distribution was computed or the hyperparameters were optimized. Since a BO routine starts with a small number of samples, it is unnatural to opt for SVI which is designed for parameter estimation with big data. I am wondering why the authors did not choose MCMC or vanilla variational inference. If SVI was used for some reason, what were the choices for the approximate posterior distributions? I believe this choice is crucial for the performance of the BO.

The introduction in lines 75-81 is somewhat confusing, primarily because of the terminology and notation. I would instead introduce theory in the following order,
i) if y is a count, then we assume y follows a Poisson process with a constant rate/intensity/lambda and sampling locations are independent (if both conditions are satisfied, then the process is homogenous).
ii) However, in a spatial process such as ..., this rate is the average count in a given area. Hence, it is not reasonable to assume a constant intensity throughout the space.
iii) A Poisson process with location-dependent intensity is known as a Cox process.
iv) One commonly used Cox process is LGCP where the intensity is the exponential of a spatial Gaussian process.

Because the paper uses a variant of the Cox process known as LGCP, it would be more accurate to name the proposed process as LGCP-BO over CoxBO.

The spatio-temporal treatment is through sampling of the BO rather than through LGCP, although the explicit representation of a spatio-temporal process is possible within the LGCP framework. It is important to make this point clearer.

... 

Note: The rest of the review will be sent to the PCs because of the 5000 character limit.