Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Authors propose a spatio-temporal model for modelling evolution of SARS-CoV2 and then they use that model with the a semi-parametric model to analyse risks of COVID-19 at individual levels. 
main_review: Authors propose a spatio-temporal model for modelling infections, hospitalisations and deaths related to COVID19. They then further extend their analysis by using a semi-parametric model to asses individual level risks by using EHR data from two hospitals in NYC. The  paper is written in a very elegant and simple way that makes it easier for reader to parse through the ideas and get to the main goals. The experiments and methodology is sound. 

I think, authors are overplaying their contributions in terms of coming up with a spatio-temporal models. Temporal models are back bone of infectious diseases, they are not just limited to citation 26 in the paper. The history of models using delay distributions could go way back to use of renewqal equations in Willy Feller. “On the Integral Equation of Renewal Theory”. In:The An-nals of Mathematical Statistics(1941).issn: 0003-4851.doi:10.1214/aoms/1177731708 or Richard Bellman and Theodore Harris. “On Age-Dependent Binary Branch-ing Processes”. In:The Annals of Mathematics(1952).issn: 0003486X.doi:10.2307/1969779. Practical use of these equations could be seen in Fraser, C. Estimating Individual and  Household Reproduction  Numbers in an Emerging 
Epidemic. PLOS ONE 2, e758 (2007)., E. Goldstein et al.: Proc. Natl. Acad. Sci. U.S.A. 106, 21825 (2009) and A. Cori et al.: Am. J. Epidemiol. 178, 1505 (2013). In fact one of the very first model for modelling covd19 in Europe, S. Flaxman et al.: Nature 584, 257 (2020),  is also based on approach the authors claim to be novel. I would urge them to downplay this claim and even the claim that infectious disease models generally are not temporal, at best it is actually exactly the other way around. Similarly various spatial models have also been around there for modeling SARS-COV2 for long like J. S. Jia et al.: Nature 582, 389 (2020), Giuliani, D., Dickson, M.M., Espa, G. et al. Modelling and predicting the spatio-temporal spread of COVID-19 in Italy. BMC Infect Dis 20, 700 (2020). https://doi.org/10.1186/s12879-020-05415-7, F. Bartolucci et al.: Spatial Stat. 100504 (2021), https://localcovid.info/data-methods.html and many others. So again I willask authors to downplay their claims. 

Similarly like modelling of spread the model to account for individual level risk  is standard and something used in the public health for a long-long time. SO again these contributions are not novel, hence request is to downplay this part. 

However, the results and analysis is quite detailed and looks for things of interest. I will urge authors to talk about them more and push that as their main contribution. Lastly, I will like authors to release their code at least. I can see why data they can't but see no reason behind not releasing the code. 
limitations_and_societal_impact: Yes they have
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This study aims to estimate the daily infection rate and individual risk of hospitalization through offering a modeling of COVID-19 pandemic. They find correlations between the individual characteristics (e.g., age and race) with the community neighborhood. They have use real-world data of NYC cases during the 2020 pandemic.
main_review: - My main concern with this paper is its relevance to the NeurIPS community. The paper is more suited for submission to data analytics and knowledge discovery venues. 

- The topic of this paper is among one of the most studied during the past year. Modeling the Covid-19 and estimating the new cases are well-studied and there are plethora of models to benchmark the results of this study against. However, authors do not perform a thorough research on the related studies and offer no comparison with any benchmarks. It is hard to judge the merit of the proposed model in the absence of any baseline.

- The study offers marginal novelty over the existing methods, as evident from the amount of methods borrowed in the paper from the previous studies. More specifically, the impact of communities on a viral spread (and especially on COVID-19) is well-studied. So is the relationship between the individual features and the risk of severe outcomes upon diagnosis. 

- Although the study claims to perform casual inference, the results only find correlation between the selected features. Moreover, the authors do not discuss any of the discovered correlations in detail (for example, what might be underlying cause of increased hospitalization among minority groups). The study does not offer much more than a simple analytic study.

- In figure C.3 in appendix where the difference between observed and predicted daily cases are given, for some zip codes the prediction is much weaker than the observed value (e.g., 11375, 11368, 11229). These cases are the interesting ones that show an underlying factor(s) change the pattern of infection unexpectedly. Deeper analysis to find the difference between these zipcodes and the rest help to understand the possible shortcomings of the proposed model (and addressing them).


Minor comments:
    - Line 24: "health system" --> "healthcare system" 
limitations_and_societal_impact: See above.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This work proposes (a) a community-transmission model, and (b) an individual risk-assessment model for COVID-19. The model is general enough to be applicable to any infectious disease. The model for (b) is enriched by inferred variables from (a) by (i) using them as covariates and (ii) addressing the selection bias in the data used for (b). 

### community-transmission model

First, this work links the temporal model for forecasting the number of infectious cases with a spatial model to account for disease propagation between regions. 
The connection is made via modelling the relative infection rate as a Gaussian Process parameterized by a covariance matrix that depends on the expected number of infectious cases in the neighbouring regions. 
The simulation results in the Appendix demonstrate that the model is able to uncover the true underlying parameters. 
This spatiotemporal model is then fitted to the reported number of daily COVID-19 cases in NYC from March to July, and the results show a substantial spatial correlation in disease propagation. 

### individual risk-assessment model

Finally, an individual risk-assessment model is constructed using the inferred variables from the above fitted spatiotemporal model.
The data used to build this model is from two hospitals in New York. 
The authors addressed the issue of selection bias in the data via subject-specific weights by making the sample representative of true underlying prevalence and spatial population.
The results seem to suggest a superior fit to the data compared to the unweighted model. 


main_review: ### Writing

The paper could have been written or structured better. It was hard to follow through the description at times, for example, it was difficult to follow the description in Methods at places. Some equations use variables that are explained after quite a while, for example, $\rho_t$ in eq. 5. Some variables are not explained at all, for example, $\tau_t$ in eq.5.

It was difficult to connect the two models. See my last comment in Experiments. 

### Novelty

There have been several discussions in the community about growing inequalities due to the COVID-19 pandemic.
As pointed out in the introduction, and to the best of my knowledge, there aren’t studies that forecast infection or assess individual risks by incorporating socio-economic and spatial factors even though these factors are deemed crucial in the transmission of viruses.
This research attempts to mathematically model and assess the correlation between various socioeconomic variables to individual disease severity and spatial propagation. 
I think this work could be useful both practically and to make further progress in the field of modelling infectious diseases with socioeconomic factors. 

### Methods

1. Line 113: Survival function seems similar to assuming characteristics of an infectious individual’s likelihood to infect susceptible individuals. Following this line of thought, can the survival function follow the same distribution as viral load curve, i.e., a gamma distribution with the same mean (akin to the modelling of infectiousness in most published studies)? Do the authors think that will result in better estimations? 
2. Eq. 5: Is the choice $\Sigma_t$ motivated from prior work? Can the authors briefly describe the intuition behind such a choice in Appendix? It is hard to find this in the resources referenced.
3. Line 135: Can the denominator be 0? If so, what should be $h_{ij}$ in that case? Will this not happen because the summation in Eq. 6 is from the day of observing a min of 1 case across all areas?
4. Line 142-143: I don’t get how $\rho_t$ is a scalar given that it depends on i,j,k. Can the authors provide proof of this statement or refer to the relevant text in the referenced papers? Even an intuitive explanation might suffice for the readers. 
5. Line 173: Can the authors suggest a way to modify Eq. 7 to account for the reversal of trend (e.g., waves of infections after some decline in cases)? 
6. Line 233 - 245: Can the authors comment whether PCA could be as useful for dimensionality reduction as the proposed factor model? Did they try PCA? Are there results on PCA as well? 
7. Line 246-263: What is the relation between $\tilde{\mathbf{X}}_j$ and $\hat{\mathbf{z}}_j$?
8. Eq.6: Is the first term any different from $(R_i(t) - Y_i(t))^2$? It doesn't seem like, but if so, can the authors explain so that the reader is aware?

### Experiments

1. Line 282: Can it be shown visually? For example, show a spatial map of the density of the minority population. This will help the reader understand this sentence better. 
2. Line 283: Is it $\rho_t$ instead of $beta_t$?
3. Line 287: The procedure to construct confidence bounds is not explained in the supplementary material.
4. It is not entirely clear how exactly the spatio-temporal model in section 2.1 is related to the individual-risk model in section 2.2. The connection is through the correction for sampling bias ($w_{2j}$) and using $M(t-7)$ as a covariate. For a complete understanding of this connection, can the authors compare their regression results with a risk model that includes only $w_{1j}$ and doesn’t use $M(t-7)$ as a covariate?

### Minor comments

1. Line 244: Figure C.4 (not C.3) corresponds to the factor loadings
2. Line 291: Figures C.1 and C.2 are exactly the same. I think you meant Figure C.3. 
3. Line 308: IRB could mean different things to people from different countries. Can the authors please provide a link to this organization?

### Broader comments

Can the authors comment on what would be the pitfalls or advantages of jointly training the two models? Does it sound reasonable to do so? 

Can the authors comment on whether their model would be appropriate to forecast waves of infection? Will it require additional variables in modelling?



limitations_and_societal_impact: The paper has discussed the potential pitfalls of their modelling approach.
review_title: null

Meta-review: This paper presents novel ML approaches to examining the socioeconomic disparity of infection risks and interaction among the risk factors, to assist public health decision-making and facilitate better clinical management of COVID patients. This study is highly relevant to the NeurIPS community. As one reviewer states, this work pushes the boundary of merging the epidemiological analysis with socio-economic factors and is quite ground breaking in this respect, with the potential for huge societal impact. The paper is methodologically solid and the authors have presented a strong rebuttal and discussion with the reviewers
Decision: Accept (Poster)