# Towards Improving Transparency of Count Data Regression Models for Health Impacts of Air Pollution
## 2 Illustration of False Inference and Overfitting Due to pmf Misspecification
For our synthetic dataset, which we analyzed in Table 1 below, the code provided through the link was applied in R version 4.0.0 [8] to generate 1000 values for each of three covariates, \(x_{1}\), \(x_{2}\), and \(x_{3}\), from the normal distribution with the mean \(\mu=10\) and standard deviation \(\sigma=1\). Parameter values were then assigned to create 1000 conditional mean values as follows, with \(x_{3}\) excluded:  
\[\hat{y}_{i}=e^{1+0.1x_{1i}+0.1x_{2i}}\ for\ i=1,\ 2,\ldots,\ 1000 \tag{6}\]  
Observed \(y_{i}\) values were distributed about these \(\hat{y}_{i}\) values according to the negative binomial pmf, which has a standard deviation of \(\sigma_{i}=\sqrt{\hat{y}_{i}+\alpha\hat{y}_{i}^{2}}\). (This is in contrast with the Poisson distribution, which is less spread out, with \(\sigma_{i}=\sqrt{\hat{y}_{i}}\).) A value of 0.5 was chosen for \(\alpha\), the dispersion parameter. Figure 2: Schematic for the generation of synthetic data with count data as the response variable. pmf: probability mass functions; CMM: conditional mean model. The results for each of the three CMMs are shown in Table 1\(\mathcal{L}_{optimal}\) columns. In each case, the optimal pmf is, not surprisingly, the same one used to generate the data. In some cases, adding covariates may cause a switch to a pmf with a less spread structure [1]. As expected, \(x_{3}\), which was not used to generate the response variable, has a coefficient with a \(p\)-value well above 0.05, and slightly increases the AIC. It is to be excluded from the CMM. For comparison, results for the Poisson pmf, often used in the literature, appear in the \(\mathcal{L}_{poisson}\) columns. The \(p\)-values are now falsely low, sometimes by several orders of magnitude. The false inference would now lead to including \(x_{3}\). The lowering of the AIC value by including \(x_{3}\) shows that the AIC is inadequate for preventing CMM overfitting. [PERSON], an author of more than 10 books on statistical modeling, has cautioned that \"Many analysts have been deceived into thinking that they have developed a well-fitted model\" because the spread of the residuals was greater than represented in their count data regression model [1]. In our own dataset of childhood asthma and air quality in Houston, the distribution of the daily arrivals to the emergency department appears to be zero-inflated, i.e., there is an inexplicably high number of days with zero arrivals if the observed values are assumed to be Poisson distributed about the conditional mean. The zero-inflated Poisson pmf accounted for what is in effect an increase in the spread of the residuals, thereby giving a more realistic \(p\)-value (0.051), which is many orders of magnitude higher than that suggested by the Poisson pmf (\(<10^{-15}\)). Utilizing the most appropriate pmf can dramatically reduce the risk of false inference and overfitting. However, it must be noted that the AIC and related criteria do not establish appropriateness in any absolute sense but only identify the best choice among a set of choices. It could be that none of the choices is ultimately appropriate. In recognition of this limitation of such criteria, and in recognition of limitations among various software packages to select the most appropriate pmf, and to provide an intuitively appealing visual check on the selected pmf and CMM, a predicted-and-observed count histogram (POCH) is discussed in the following section.