From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper43/Authors
comment: Thanks very much for these comments that will help to improve the quality of the final paper. Below are our response:

#1 "To this end locations are linked, however, it is never made clear why these connections are needed, as opposed to just treating them as independent observations associated with a single individual."

R.: The locations are linked to show the general version of the problem. In this paper we treated the locations independently associate with a single individual. We use the places visited during the period of analysis as informative of the potential risk places.

#2 " For example the visit model makes the assumption that the zones are independent. This seems rather unlikely,
for example if they are along the same river."

R.: The review is correct that the locations are unlikely to be independent. However, given the sparse nature of geo-tagged social media data this assumption is not bad at all. More realistically, what we have here is a Dirichlet-multinomial, where each individual has their own separate multinomial distribution that is peaked at a small number of locations. This is in our future developments. 

#3 The infection model has the risk of infection inside a zone r(Z) and then another one for outside the zone r(\bar Z). This seems to indicate that there can only ever be one zone captured by the model, as high infection risk inside another second zone and infection risk outside of both zones would not be able to represented in this manner.

R.: r(Z) is obtained for each fixed region Z by maximum likelihood. The score of each zone is contrasted with the reference distribution obtained when we are certain Ho holds (using permutation test). All zones below the significance level are detected as riskier zones. The procedure is analogous to the one for the Logistic Model in section 2.2. 

#4 "Another confusing bit is that in section 2 the paper makes the comment that biasing the results based on number of observations is not ideal. Yet the justification for equation 2 is given as this same fact being desirable."

R.: The function g() in equation 2 is exactly used to control for the effect that the reviewer mentioned. It prevent that a user with a high odd of being labeled as case due to the number of observations dominates the analysis. 

#5 "The function g() seems to play an important role but is never properly defined other then being an "arbitrary and unspecified function"

R.: g() controls the odds of users being labeled as a case given their number of points. As we do not know this function in advance we estimate it using a non-parametric regression model such as loess from the data and is used as a plug-in offset. 

#6 "The logistic model also employs a ratio beta = lambda_in / lambda_out but it is never made clear how these values are obtained. "

R.: For each fixed zone Z, we obtain a corresponding beta by maximum likelihood as described in section 2.2. The likelihood function is given in equation (5).

#7 "There is no explanation given for why the particular hypothesis test was selected and the assumption implicitly made by it."

R.: The test contrasts the null hypothesis that the risk is equally spread over the map, in which case staying longer in any of the regions will make no difference on the likelihood of being infected, against the hypothesis that there is a region where the risk within this region \lambda_in is higher than in the rest of the map \lambda_out.

#8 "The case study mentions that Monte Carlo simulation was used, though it's unclear for what and how."

R.: The Monte Carlo simulation is used to obtain a reference distribution from the data, when we are certain that the null hypothesis hold. The procedure is explained in the last paragraph of section 2.2. 


title: Response

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper43/Authors
comment: We wish to thank the reviewer for the comments. Below are our response:

#1 Yes. Likelihood ratio tests have been used in the field of anomalous spatial pattern detection for a while. The seminal paper by Kulldorf and Nagarwalla (1995) [1] developed a practical way to take into account the multiple testing involved in the search of anomalous regions. They showed how a Monte Carlo reference distribution could be obtained from the data and how it controls the false positive level of the potentially infinite statistical tests involved. This idea opened the door to many novel developments.

#2 Also, we would like to call the attention to the fact that we are identifying places where people have higher risk of being infected, rather than focusing on residential address locations, which may be key to surveillance for vector-borne diseases such as malaria and dengue, allowing public health officials to focus mitigation actions. Dengue is just one of many infectious diseases with a well-known etiology but a huge number of uncertain and difficult to obtain parameters that quantify factors such as infected mosquito population, likelihood of being bitten by an infected mosquito, and human movement in the mosquito-infested areas. We expect that our method will also be useful to other spatial public health surveillance problems where movement data can bring relevant information.

#3 As we mentioned in the previous comment, the main novelty of our work is the use of each individual’s entire location trace rather than a single location, which is critical when trying to identify areas of increased infectious disease risk. The stochasticity of location data is not appropriate for typical spatial cluster detection tools such as the traditional spatial scan statistic. Each user is represented by a different number of geographic points and the variability of these numbers is large; traditional approaches can be easily misled if not extended to account for this special structure.

[1] M Kulldorf and N. Nagarwalla (1995) Spatial disease clusters: detection and inference. Statistics in Medicine. 
title: Response

From: ~Roberto_Souza1
comment: We wish to thank the reviewer for the comments. Below are our response:

#1 Yes. Likelihood ratio tests have been used in the field of anomalous spatial pattern detection for a while. The seminal paper by Kulldorf and Nagarwalla (1995) [1] developed a practical way to take into account the multiple testing involved in the search of anomalous regions. They showed how a Monte Carlo reference distribution could be obtained from the data and how it controls the false positive level of the potentially infinite statistical tests involved. This idea opened the door to many novel developments.

#2 Also, we would like to call the attention to the fact that we are identifying places where people have higher risk of being infected, rather than focusing on residential address locations, which may be key to surveillance for vector-borne diseases such as malaria and dengue, allowing public health officials to focus mitigation actions. Dengue is just one of many infectious diseases with a well-known etiology but a huge number of uncertain and difficult to obtain parameters that quantify factors such as infected mosquito population, likelihood of being bitten by an infected mosquito, and human movement in the mosquito-infested areas. We expect that our method will also be useful to other spatial public health surveillance problems where movement data can bring relevant information.

#3 As we mentioned in the previous comment, the main novelty of our work is the use of each individual’s entire location trace rather than a single location, which is critical when trying to identify areas of increased infectious disease risk. The stochasticity of location data is not appropriate for typical spatial cluster detection tools such as the traditional spatial scan statistic. Each user is represented by a different number of geographic points and the variability of these numbers is large; traditional approaches can be easily misled if not extended to account for this special structure.



title: Response

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper43/Authors
comment: Thanks for the message, will reply to all comments right now.

All the best,
The authors
title: Response

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper43/AnonReviewer1
relevance: 3: borderline
impact: 2: low
novelty: 2: low

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper43/AnonReviewer1
evaluation: The problem addressed by the problem is that of finding regions of high
risk of certain events occurring, infection with dengue fever in the
paper's case study. The methods covered in the paper attempt to identify
such regions based on labelled data of cases, i.e. affected people, and
control, i.e. non-affected people, using simple statistical methods.

The paper makes the case for using mobility data, i.e. traces of how
people move through space, instead of only single locations such as home
or work place. To this end locations are linked, however, it is never
made clear why these connections are needed, as opposed to just treating
them as independent observations associated with a single individual.

There are many assumptions made throughout the paper and none of them
are explained or justified. For example the visit model makes the
assumption that the zones are independent. This seems rather unlikely,
for example if they are along the same river. The impact of such
potentially invalid assumptions in the model need to be explained in
more detail. The infection model has the risk of infection inside a zone
r(Z) and then another one for outside the zone r(\bar Z). This seems to
indicate that there can only ever be one zone captured by the model, as
high infection risk inside another second zone and infection risk
outside of both zones would not be able to represented in this manner.

Another confusing bit is that in section 2 the paper makes the comment
that biasing the results based on number of observations is not ideal.
Yet the justification for equation 2 is given as this same fact being
desirable. This contradiction is rather confusing and should be cleared
up. The function g() seems to play an important role but is never
properly defined other then being an "arbitrary and unspecified
function".

The logistic model also employs a ratio beta = lambda_in / lambda_out
but it is never made clear how these values are obtained, or if they are
tuning parameters, and if so how they'd be optimized or selected.

There is no explanation given for why the particular hypothesis test was
selected and the assumption implicitly made by it. Furthermore, H0 is
unclear as it claims to encode that staying longer in an infected zone
has no effect, while it would appear that H0 only encodes spending equal
amount of time inside and outside a zone Z.

The case study mentions that Monte Carlo simulation was used, though
it's unclear for what and how.

The basic premise of the paper is interesting, however, there are many
aspects that simply are not explained clearly enough. Furthermore, the
abstract indicates that three methods would be compared, when in fact
there is only a single qualitative result for one method presented.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper43/Authors
title: Comments on #3.
comment: Thanks very much for these comments!  

The main novelty of our work is the use of each individual’s entire location trace rather than a single location, which is critical when trying to identify areas of increased infectious disease risk.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper43/AnonReviewer2
evaluation: Summary: the authors want to find spatial regions with high risk of infection, by likelihood ratio tests with p(data | spatial box) / p(data | no special region). They compare three probabilistic models p and evaluate one in detail. The idea is that the infection risk level is correlated with human mobility patterns, that are recorded via e.g., spatial Twitter data. The authors describe the results of their method on a dengue dataset. 

Overall, the discussion is nice, but the evaluation feels a bit lacking in insight in terms of comparing the three methods. The main evaluation seems to show that the method finds regions where people often interact socially / congregate, but does not offer more insights.

    1. What did you learn from reading this paper that you did not know before?
Likelihood ratio tests work for (dense?) spatial data. Are they stable?

    2. What was the most exciting part of the work?
The process of using streaming Twitter data to quickly identify risk zones.

    3. What in your opinion makes this work stand apart from other papers in this area?
Not so clear.

    4. Does the paper fit into the scope of the workshop?
Yes.

    5. Are you aware of relevant literature that you may want to make the authors aware of?
No

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper43/AnonReviewer2
relevance: 5: very high
impact: 4: high
novelty: 3: borderline