Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Authors: Spencer Young
Porter Jenkins
Longchao Da
Jeff Dotson
Hua Wei
Venue: ICML 2025 poster (2025)
Keywords: Predictive uncertainty Deep Ensembles
Heteroscedastic regression
Primary area: probabilistic_methods
TL;DR: A state-of-the-art method to quantify aleatoric and epistemic uncertainty on count regression tasks with fully heteroscedastic predictive variance and ensembles.

Abstract: Neural networks capable of accurate, input-conditional uncertainty representation are essential for real-world AI systems. Deep ensembles of Gaussian networks have proven highly effective for continuous regression due to their ability to flexibly represent aleatoric uncertainty via unrestricted heteroscedastic variance, which in turn enables accurate epistemic uncertainty estimation. However, no analogous approach exists for $\textit{count}$ regression, despite many important applications. To address this gap, we propose the Deep Double Poisson Network (DDPN), a novel neural discrete count regression model that outputs the parameters of the Double Poisson distribution, enabling arbitrarily high or low predictive aleatoric uncertainty for count data and improving epistemic uncertainty estimation when ensembled. We formalize and prove that DDPN exhibits robust regression properties similar to heteroscedastic Gaussian models via learnable loss attenuation, and introduce a simple loss modification to control this behavior. Experiments on diverse datasets demonstrate that DDPN outperforms current baselines in accuracy, calibration, and out-of-distribution detection, establishing a new state-of-the-art in deep count regression.

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Spencer Young" "1
Porter Jenkins"
Longchao Da
Jeffrey Dotson 4 Hua Wei
Abstract
detecting out-of-distribttion data (Amini et al. 2020 Liu
ct al. 2020 Kang et al.) 2023), active learning (etttee
Neural networks capable of accurate, input- conditional uncertainty representation are essen- tial for real-world Al systems. Deep ensembles of Gaussian networks have proven highly effective for continuous regression due to their ability to flexibly represent aleatoric uncertainty via unre- stricted heteroscedastic variance, which in turn enables accurate epistemic uncertainty estimation. However, no analogous approach exists for count regression, despite many important applications. To address this gap, we propose the Deep Double Poisson Network (DDPN), a novel neural discrete count regression model that outputs the parame- ters of the Double Poisson distribution, enabling arbitrarily high low predictive aleatoric uncer- tainty for count data and improving epistemic un- certainty estimation when ensembled. We formal- ize and prove that DDPN exhibits robust regres- sion properties similar to heteroscedastic Gaus- sian models via learnable loss attenuation, and in- troduce a simple loss modification to control this behavior. Experiments on diverse datasets demon- strate that DDPN outperforms current baselines in accuracy, calibration, and out-of-distribution detection, establishing a new state-of-the-art in deep count regression.
2009
Ziatdinov 2024). reinforcement learning YYu et al. Jenkins et al.
2020
(022), and real-world decision decision-making
under uncertainty (Abdar et al.) 202T). While uncertainty
quantification applied to regression on continuous outputs is well-studied, training neural networks to make probabilistic predictions over discrete counts has traditionally received less attention, despite multiple relevant applications. In re- cent years, neural networks have been trained to predict the size of crowds (Zhang et al., 2016 Lian et al.
2019 Zou
et al. 2019 Luo et al. 2020 /Zhang & Chan
2020
Lin
& Chan 2023), the number of cars in a parking lot (Hsich
et al. 1), traffic flow (Lv et al.
2014 Li et al. 2020 Liu
et al. 2021 agricultural yields (You et al. 2017 ). inventory
of product shelves (Jenkins et al.
2023
and bacteria in
microscopic images (Marsden et al.
2018).
Uncertainty is often decomposed into two quantities: epis- temic, which refers to uncertainty due to misidentification of model parameters, and aleatoric. which is uncertainty due to observation noise (Der Kiureghian & Ditlevsen, 2009). In regression tasks, popular and effective approach for capturing epistemic uncertainty is to use deep ensembles (DEs) [aakshminaryyanan et al., 2017) where a set of M neural networks are independently trained from different initializations and combined to make predictions. Aleatoric uncertainty, meanwhile, is accounted for in each individ- ual member, which outputs a distinct predictive distribution over the target. Aleatoric uncertainty can be categorized as homoscedastic, where the predictive uncertainty is con- stant for all inputs, and heteroscedastic, where predictive uncertainty varies as a function of the input. Existing work on DEs for regression trains each member of the ensemble to predict the mean and variance of a Gaussian distribu- tion, pm on distributions are then combined into a single prediction.
1. Introduction
The pursuit of neural networks capable of learning accu- rate and reliable uncertainty representations has gained sig- nificant traction in recent years (I [Lakshminarayanan et al. 2017 Kendall& & Gal 2017 Gawlikowski et al. [2033 Dheur
(8) fe... (x.). Each of the M individual
& Taieb 2023). Input-dependent uncertainty useful for
"Equal contribution 'Delicious AI Brigham Young Univer- sity, Department of Computer Science 'Arizona State University, School of Computing and Augmented Intelligence "Ohio State University, Department of Marketing. Correspondence Spencer Young <spencer. young @ delicioussi.comm com,, Porter Jenkins <pjenk- ins See@es..yuu bdd..uuu
Full heteroscedasticity from unrestricted predictive variance is critical for DEs to produce calibrated output distribu- tions. The total variance of ensemble prediction can be described as Varly, xx] = E,, [o (i) + Varm. (8) which is equal to the average aleatoric uncertainty of the mem- bers, plus the variance of the predicted means (Appendix C.6p. If each member of the ensemble is not fully het-
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Prisson Ensemble
Ensemble
DDPN Ensamble (Durs)
ous outputs is well-studied (Nix & Weigend 1994 Bishop
Prode tiet.
1994), no such method exists for count data. Previous work trains a network to predict the À parameter of Poisson distribution and minimize its negative log likelihood (NLL) (Fallah et al. 2009). However, the Poisson parameteriza- tion of the neural network suffers from the equi-dispersion restriction: the predictive mean and variance are the same (² = P = 62). Another common alternative is train the network to minimize Negative Binomial (NB) NLL (Xie) 2022). The Negative Binomial breaks equi-dispersion by introducing another parameter to the PMF This helps dis- entangle the mean and variance, but suffers from the over- dispersion restriction: 8² 2 p. Neither of these methods can produce unrestricted heteroscedastic variance.
N
hutited
Abastoric
Epocsamic
A plausible alternative to introduce full heteroscedasticity to the count regression setting is to violate distributional assumptions and simply apply models with Gaussian like- lihoods. However, this decision neutralizes crucial in- ductive bias, since it outputs a probability density func- tion, p(v (x)), E R over a discrete output space, i.e. y E Zzo- This can impact both the accuracy and calibration of the predictive distribution, since the model is constrained to assign probability infeasible real values and thus can- not maximally concentrate center its predictions around the ground truth. Such model will also exhibit various pathologies: 1) it will assign nontrivial probability to neg- ative values (impossible for counting problems) when the predicted mean is small, and 2) the boundaries of its predic- tive intervals (i.e. highest density interval or 95% credible interval) are likely to fall between two valid integers, dimin- ishing their interpretability and utility.
Figure 1: Simulation experiment demonstrating a het- eroscedastic data-generating process with discrete outputs. The predictive distributions of three ensemble methods are compared. The mean predictions are shown in black, the 95% credible intervals shaded in blue, along with the cor- responding Mean Absolute Error (MAH) and Continuous Ranked Probability Score (CRPS) in the top left (Top). Un- certainty is decomposed into its aleatoric (Middle) and epis- temic (Bottom) components, with the ground-trutt -ruth alcatoric uncertainty represented by a dashed line. Existing dis- crete regression methods are unable to accurately capture aleatoric uncertainty, which impacts both epistemic and total predictive uncertainty. In contrast, ensembles compris- ing Deep Double Poisson Networks (DDPNs) effectively represent epistemic and aleatoric uncertainty, while also improving mean fit.
Our Contributions To address these issues, we introduce the Deep Double Poisson Network (DDPN), a novel dis- crete neural regression model (see Figure D.. DDPN models aleatoric uncertainty by outputting the parameters of the Double Poisson distribution (Efron) 1986), a highly flexi- ble predictive distribution over Z2 Unlike existing count regression methods based on the Poisson and Negative Bi- nomial distributions, DDPN exhibits full heteroscedasticity, allowing it to model a broader range of uncertainty patterns. At the same time, DDPN avoids the misspecification issues of Gaussian-based models on count data, enabling sharper, more reliable predictions. In addition to these desirable characteristics, we demonstrate that DDPN possesses ro- bust regression properties akin to those of heteroscedastic Gaussian models. To do so, we propose a formal detini- tion of learnable loss attenuation, concept first studied by Kendall & Gal (2017) in which model can adaptively modify its loss function to lower the impact of outlier points during training, and prove that DDPN satisfies this def- inition. Furthermore, we introduce a discrete analog of the 3-modification proposed by [Seizer et al. (2022). en-
eroscedastic (i.e.. the regressor produces constrained vari- ance predictions) the aleatoric term can be misspecified, which induces a miscalibrated predictive distribution of the ensemble. Additionally, recent work has shown that aleatoric and epistemic uncertainty are largely entangled in practice (Mucsányi et al., 2024), further compounding miscalibration. Consequently, poor aleatoric uncertainty quantification will likely also give rise poor estimates of epistemic uncertainty. For a concrete example, see Fig- ure| [ In this experiment we train three count regression DES using a synthetic dataset and plot their predictive dis- tributions, along with the estimated aleatoric and epistemic uncertainty decompositions. Two of these DEs, Poisson and Negative Binomial, are constituted of members that not fully heteroscedastic (Proposition 2.3p. This leads to overestimated aleatoric uncertainty and miscalibrated predictive distributions. We also note that such misspecifi- cation impacts epistemic uncertainty, which is high in these ensembles despite adequate training data.
While full heteroscedasticity in deep regression on continu-
2

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
abling controllable attenuation strength. Across variety of datasets, our experiments show that DDPN (both individu- ally and as an ensemble method) outperforms all baselines in terms of mean accuracy, calibration, and out- out-of-distribution detection, establishing a new state-of-the-art for deep count regression.
networks via Gaussian NLL due to the the influence of d² the gradient of the mean, fi. [mmer et al. (024) reparam- eterize their model to output the natural parameters of the Gaussian distribution. /Seizzer et al. (2022) propose mod- ified objective and introduce a hyperparameter, 3 E [o. 1], which tempers the impact of 6² on the gradient of ju and offers tunable control over the loss attenuation properties of the Gaussian NLL. Stirn et al. (2023) modify their architec- ture to include separate sub-networks for ja and d², then use stop gradient operation to neutralize the impact of 6² on the j(() sub-network. In the count setting, Poisson (Fallah) ct al. 2009) and Negative Binomial (Xie) (022) likelihoods have been used to train heteroscedastic neural networks
2. Modeling Predictive Uncertainty with Neural Networks
A large body of work has developed methods to represent both epistemic and aleatoric uncertainty in deep learning.
2.1. Epistemic Uncertainty
2.2.2. HETEROSCEDASTIC REGRESSION WITH GENERALIZED LINEAR MODELS
Epistemic uncertainty refers to uncertainty due to model misspecification. Modern neural networks tend be sig-
Historically, Generalized Linear Models (GLMs) have been used to model predictive uncertainty on tabular data. GLMs specify a conditional distribution, p(u Mr Ø), where is a member of the exponential family, 1// = w x represents the natural parameter of p. and 6 is the dispersion term (MCuullagh 1989 Murphy 2023). A link function, ((.), is selected to specify mapping between the natural parameter and the mean such that I(44) n w æz The model is then fit by minimizing NLL. Many common models can be viewed under this general framework, including logis- tic regression, Poisson regression, and binomial regression (Fahrmeir et al., 2013). The Double Poisson distribution we employ in this work was originally developed the context of GLMS and proposed constrained dispersion term with an explicit dependence on the mean (Efron) 1986). More re- cent work employs Joint GLMs with separate covariates for mean and dispersion, allowing for more degrees of freedom (Aragon et al. 2018). All of these models are restricted to linear families of functions.
nificantly underspecified by the data, which introduces a high degree of uncertainty (Wilson & Izmailov 2020). A variety of techniques have been proposed to explicitly rep- resent epistemic uncertainty including Bayesian inference
(Chen et al. 2014 Hoffman et al. 2014] Wilson & Izmailov
2020), variational inference (Grraves 2011; Blundell et al.
2015), Laplace approximation (Daxberger et al. 202 1), and
Epistemic Neural Networks (Osband et al. 2023). Recently,
deep ensembles have emerged as simple and popular so-
lution (Lakshminarayanan al. 2017: D'Angelo & For-
tuin 2021 Dwaracherla et al. 2022). Other work connects
Bayesian inference and ensembles by arguing the latter can be viewed as a Bayesian model average where the posterior sampled at multiple local modes (Fort et al. 2019 Wilson Izmailov 2020 This approach has number of attractive properties: 1) generally improves predictive performance (Dietterich] 2000); 2) it can model more complex predic- tive distributions; and 3) it effectively represents uncertainty over learned weights, which leads to better calibration.
2.2.3. FULL HETEROSCEDASTICCTY
2.2. Aleatoric Uncertainty
Because misspecification of aleatoric noise often corrupts overall estimates of uncertainty, regression models should be unconstrained in the range of variances they can output. We formalize this property as sull heteroscedasticity, which ensures that a model's predictive variance is both input- dependent and unconstrained in scale.
Aleatoric uncertainty quantifies observation noise and gen- erally cannot be reduced with more data (Der Kiureghian & Ditlevsen, 2009 Kendall & Gal 2017). In practice, this uncertainty can be introduced by low resolution sensors, blurry images, or the intrinsic noise of signal.
2.2.1. HETEROSCEDASTIC REGRESSION IN DEEP LEARNING
Definition 2.1. A family of distributions Q, parametrized by &b E Rd, is said to have unrestricted variance if, for any random variable Z ~ Q, if we condition on E[Z] = /a for any valid μr, for any o² E (0, oo) there exists a setting of sb such that Var[Z] = o².
Aleatoric noise is commonly modeled in deep learning by learning the parameters of a probability distribution over the label. This often takes the form of heteroscedastic regres- sion, where the network learns an input-dependent disper- sion parameter, Φ;, in addition to an estimate of the mean, f NNx & Weigend| 1994 Bishop 1994). In the Gaussian case, 6 07. Recent work identifies issues with training such
Definition 2.2. A probabilistic regression model f is called fully heteroscedastic if its predictive variance depends on the input x, and its output distribution family Q has unrestricted variance.
3

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
pb fo (x). Ma %)
learnable weights @ (stacked into L layers). In practice, we model fe X vb. Given such network, we obtain predictive distribution, p(y: (x:)), for any input x,-
M
Over Stppession
Hi
In particular, suppose that we restrict our output space to PDr C P. the family of Double Poisson distributions over y- Any distribution p E Ppp is uniquely parameterized by 2b [p. > O, with mean j and inverse dispersion 7 ø. The distribution function, p : Zzo + [0, 1]. is defined as:
Vi
Figure An overview of the Deep Double Poisson Network (DDPN). DDPN is neural network that can process com- plex data and outputs the parameters of Double Poisson distribution, já; and i/- The resulting predictive distributions exhibit unrestricted variance such that the network can learn over-, equi-, and under-dispenn diser-dispersion. We ensemble a set of M DDPNs to estimate alcatoric and epistemic uncertainty.
p(yle,)) - TM
(e (e"y²) y! (2))")
ct
(I)
du. 7)
where c is normalizing constant. Let Z denote a random variable with a Double Poisson distribution function, then we say Z DP( 7). In line with previous work (Zou) et al. 2013 Aragon et al., 2018). we assume the moment
These definitions allow to characterize existing methods through the lens of full heteroscedasticity.
approximations proposed by Efron (1986): B|Z] = μ and Var[Z] = 4. We specify model, [log fi, log %d fe (x,). as follows: let 2, fe (x,), be the d- dimensional hidden representation of the input x; produced by the first L 1 layers of neural network. We apply two separate affine transformations to this representation to obtain our distribution parameters: log(f4) = w 2; + by and log(f:) = : z, + by.
Proposition 2.3. Gaussian regressors are fully hel- eroscedastic, whereas Poisson and Negative Binomial re- gressors are not.
See Appendix C.I for proof. While Gaussian regressors provide the flexibility needed for uncertainty quantification in the continuous setting, existing methods for neural count regression place restrictions on their variance and thus lack full heteroscedasticity. This limitation reduces these models' ability to capture the complex aleatoric noise present in real- world count data.
The flexibility induced by the parameter in DDPN yields a powerful model with the capacity to represent any form of aleatoric noise. Referring back to Definition 2.2 we concretely state this property in the following proposition (with a proof in Appendix C.2
Proposition 3.1. DDPN regressors are fully heteroscedastic under approximate moment assumptions.
3. Deep Double Poisson Networks (DDPN)
To address the limitations described in the previous section, we propose the Deep Double Poisson Network (DDPN), otate-of-the--at deep regression model that represents fam- ilies of non-linear functions over complex, non-negative count data. DDPN outputs the parameters of the Double Poisson distribution (Efron, 1986), which results in fully heteroscedastic predictive distribution under approximate moment assumptions (Proposition 3.1). We demonstrate that DDPN inherits robust regression properties through learnable loss attenuation, similar to Gaussian based net- works. Additionally, we introduce a discrete analog of the .3 modification from Seitzer et al. (2022), allowing tunable control over self-attemuation to improve overall model fit.
See Appendix A.3 for thorough assessment of the quality of Efron's 's moment approximations. In practice, we observe that these estimates are almost exact for nearly all relevant values of μ and o² (Figure 7
3.1. DDPN Objective
To learn the weights of a DDPN, we minimize the following NLL objective (averaged across all prediction / target tuples (ja, fs, ve) in the dataset):
L, log f1
Sufe Sr + log vog
(2)
We assume access to a dataset, 2, with N training examples {xx, ys / where each y/ E Zzo is drawn from some unknown count distribution p(us(x;) Let X' denote the space of all possible inputs x. let P denote the space of all possible distributions over Zzo, and let ab E IRd denote a vector of parameters identifying a specific p E P. We wish to model P with neural network fo X + P with
2
In accordance with convention, we define y4 log = 0 when y/- 0 (Cover 1999). During training, we mini-
For both á and & we apply the log "link" function to ensure positivity and numerical stability. We simply exponentiate when- ever fa or is are needed (i.c., to evaluate the P.M.F))
4

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
mize [[ iteratively via stochastic gradient descent (or common variants). Note that this objective drops the nor- malizing constant, c(ps, 7... Prior work demonstrates that setting c(U4, x)) = 1 provides an accurate approximation of the Double Poisson density and is easier to optimize (Efron 1986, Chow & Steenhard 2009). We provide a full deriva- tion of Equation 2 from the Double Poisson distribution function in Appendix A.1
models can dynamically adjust to downweight the in-
fluence of outliers and noisy labels. This robustness likely contributes to the superior performance of DDPN observed in Section 4
3.3. 3-DDPN: Controllable Loss Attentuation
Seitzer et al. (2022) argue that the "learned loss attenu- ation" property of neural networks trained via Gaussian NLL can sometimes result in premature convergence due to inflated predicted variance in hard-to-fit regions of the training data, which in turn can give rise to suboptimal mean fit. The mechanism that drives this behavior is the presence of the predicted variance term in the partial deriva- tive of the NLL with respect to the mean. We observe that these same phenomena exist with DDPN. Our loss has the following partial derivatives: a.; = 7/2 (1 az
3.2. Loss Attenuation Dynamics of DDPN
Previous work (Kendall & Gal 2017 states that Gaussian heteroscedastic regressors exhibit learnable loss attenuation, property where predictive uncertainty in the NILL objective reduces the influence of outliers. We will demonstrate that the DDPN objective has this same property. To support this claim, we first introduce a formal definition of learnable attenuation -a a concept that, to the best of our knowledge, has not been rigorously defined in prior literature:
"4 ps) and
ap:
3% 25. I + ju: (1 + log i. log vs).
Definition 3.2. The loss function of regressor (A,, dz), taking as input the predicted mean jú E U and disper- sion 6, E (0, oo). exhibits learnable attenuation if it ad- mits the form L L(i,, (.) - d(()) + ((())r(((ss Wz), where g d(()) = oc, lim i -_ a(δ) = O, and r 2 with equality holding iff م. = y/- We call d the dispersion penalty, a the atlenuation factor, and r the residual penalty.
Notice that the predicted inverse dispersion, 5%. is suffi- ciently small (corresponding to large variance), it can com- pletely zero out 0% regardless of the current value of jú; Thus, during training. neural network can converge to (and get "stuck" in) suboptimal solutions wherein poor mean fit is explained away via large uncertainty values. To remedy this behavior, we propose a modified loss function, _2(3) also called the Double Poisson 3-NLL:
A key consequence of this formulation is that a model trained with such loss can actively dampen the impact of certain data points phenomenon captured in the fol- lowing proposition:
(0()) [5=") (- log 94 + Gaps 71 + log j.: log v:)
2
(3)
Proposition 3.3. a loss function L(i di) exhibits learn- able attenuation, then a(p:)r((..v)) + 0 as 6. + 20.
where |-] denotes the stop-gradient operation and we once again average across all prediction/ target tuples (p.. %r. y:) in the dataset. With this modification we can effectively temper the loss attenuation behavior of DDPN. We now have partial derivatives (_
d((x))
This result (proved in Appendix
reveals a fundamen- tal mechanism of learnable attenuation: by increasing the predicted dispersion de, a model can effectively nullify the residual penalty's 's contribution to the total loss. This allows it to "ignore" high-error outliers, leading to robust regres- sor with better overall accuracy. Meanwhile, the dispersion penalty in the loss discourages the model from universally inflating di ensuring that it does not suppress all errors.
a
(1)- (1
and
ap.
+ Aa (1 + log tas - log y:). The Double Poisson 3-NLL is parameterized by ,3 E [0, 1]. where 3 = 0 recovers the original Double Poisson NLL and 3 = corresponds to fitting the mean, μ,, with no respect to 7 (while still performing normal weight updates fit the value of 7). Thus, we can consider the value of B as providing smooth interpolation between the natural DDPN likelihood and a more mean- mean-focused focused loss. For an empirical demonstration of the impact of B on DDPN, see Figure in Section 4.5
0%
24
The Gaussian NLL commonly used to train heteroscedastic regressors naturally satisfies Definition 3.22 as first observed by Kendall & Gal (0117). Specifically, if set 6. = o?- we obtain d(o:)) = 2 log di. r(pr, yi) = (fi- - ) and a(g:) 28 Proposition 3.4. The DDPN objective in Equation - ex-
3.4. DDPN Ensembles
hibits learnable attenuation of the form d( = log d.. a(g.) à.' and r(js, y:) (f4-9:) --- y:: (log j.. log yı).
The formulation in the previous sections describes a net- work with a single forward pass. As noted in Section 1 multiple independently-trinn trained neural networks can be com- bined to improve mean fit and distributional calibration.
proof is provided in Appendix C.4 Because DDPN satis- fies the learnable attenuation property, it- like Gaussian
5

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
We create DDPN ensembles by combining the predictive distributions of M distinct DDPNs in a uniform mixture as follows: Given M settings of weights { { JM we model y: as p(y/|xx/ 1 == p(y fe... (x.)). Section 4 con- vincingly demonstrates that combining model predictions in this way yields the best overall performance.
Polaecn DAN
0'0% (Qurs!
MAC: $43
MAC 1539
150
38.0
25
25
4. Experiments
(a) Predictive distributions of a Poisson DNN and DDPN on data where Y| ~ Poisson.
In our experiments, we aim to answer the following research questions: (I) Is DDPN flexible enough to fit any count dis- tribution, even under known misspecification? (2) How does DDPN compare to existing deep regression methods as mea- sured by accuracy and calibration? (3) Does the enhanced uncertainty quantification of DDPN from full heteroscedas- ticity combined with proper inductive biases of a discrete predictive distribution lead to better out-of-distribution de- tection? (4) What the effect of the 3 hyperparameter on the training dynamics of DDPN?
NB DAN
D2P HOUPs!
275
156
125
28.5
7.5
55:
(b) Predictive distributions of a Negative Binomial (NB) DNN and DDPN on data where Y|X ~ NegBinom.
4.1. Evaluation Metrics
Figure 3: Even when explicitly misspecified, DDPN recov- ers the data-generating distribution.
We evaluate each regression method along two key dimen- sions:
accuracy, measured by mean fit, and calibration, which indicates the overall quality of the predictive distri- bution. Accuracy is quantified using the Mean Absolute Error (MAE). Calibration is assessed using the Continuous Ranked Probability Score (CRPS), a strictly proper scoring rule (Matheson& & Winkler 1976). with lower values signi- fying closer alignment between the predictive CDF F; and the observed label W/- Detailed explanations and definitions of these evaluation metrics are provided in Appendix [B.2]
Dataset
Modality
Backbone
Length of Stay COCO-Pcople Inventory
Tabular
MLP
Image
ViT-B-16
Point Cloud
CountNet3D
Reviews
Text
DistiBBRRT Base Cased
Table 1: Summary of datasets used, along with their corre- sponding backbones.
4.2. Misspecification Recovery
DDPN is trained by minimizing Double Poisson NLL, which makes a distributional assumption about the targets. In this section, we test whether DDPN is robust to vio- lations of this assumption. To do so, we generate data from two processes: 1) Y|X ~ Poisson Poisson(exp(3>))) (ex((=X)); and 2) Y|X ~ NegBinom(X?, }); neither of which conform to the assumptions in Section 3 For each dataset, we train both a DDPN and model explicitly matched to the noise distribution of the data (a Poisson DNN for the first process and Negative Binomial DNN for the second). Figures 3a and 3b compare the predictive distributions learned by these models. Within each figure, we report MAE and CRPS.
4.3. Real-World Datasets
We compare DDPN to baselines on number of real- world, complex count datasets. A summary of the datasets used and the corresponding backbone architectures is given in Table | Notably, we run experiments across various modalities, including tabular, image, point cloud, and text data.
Length of Stay (Microsofi) 2016) is tabular dataset where the task is to forecast the number of days of a pa- tient will spend in a hospital. The features consist of pa- tient measurements and attributes of the healthcare facility. COCO-Peopl e (Lin et al., 2014) is an adaptation of the MS- MS-COCO dataset, where the task is to predict the num- ber of people in each image. Inventory (Jenkins et al.) 2023) consists of point clouds that capture retail shelves. The goal is to predict the number of products in the point cloud. Finally, Reviews is taken from the "Patio, Lawn, and Garden" category of the Amazon Reviews dataset (Ni ct 2019). The goal is to predict the discrete 1-5 product rating from the text review.
Remarkably, DDPN is flexible enough to accurately cap- ture the aleatoric structure of the data in both experiments. achieves performance that meets or exceeds models ex- plicitly tailored to the true noise distribution. These results highlight the robustness of DDPN and suggest that it offers a reliable and versatile approach to neural count regression, even in the face of potential misspecification.
6

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Table 2: Results across four datasets of differing modalities: Length of Stay (tabular), COCO-People (image), Inventory
(point cloud), and Amazon Reviews (language). We denote the best performer in each subgroup (aleatoric only vs. alcatoric + epistemie) in bold and the second-best with an underline. In each case, DDPN outperforms all baselines in terms of MAE and CRPS, achieving new state-of-the-an on count regression tasks.
Length of Stay
COCO-People
Inventory
Reviews
MAE (1)
CRPS (L)
MAE (4)
CRPS (L)
MAE (L)
CRPS (J)
MAE (
CRPS (4)
Poisson DNN
0.664 (0.01)
0.553 (0.01)
1.099 (0.02)
0.851 (0.01)
1.023 (0.04)
0.706 (0.01)
0.818 (0.01)
0.559 (0.00)
NB DNN
0.685 (0.00)
0.570 (0.00)
1.143 (0.05)
0.867 (0.01)
1.020 (0.04)
0,708 (0.01)
0.855 (0.01)
0,562 (0.00)
Alcatoric Only
Gaussian DNN
0.599 (0.01)
0.453 (0.02)
1.219 (0.12)
0.866 (0.07)
0.936 (0.01)
0.659 (0.00)
0.452 (0.01)
0,323 (0.00)
Faithful Gaussian
0.582 (0.00)
0.436 (0.01)
1.082 (0.01)
0.879 (0.01)
0.959 (0.03)
0.688 (0.02)
0.428 (0.00)
0.428 (0.00)
Natural Gaussian
0.597 (0.01)
0.439 (0.01)
1.157 (0.04)
0.848 (0.02)
0.958 (0.01)
0.675 (0.01)
0.428 (0.01)
0.312 (0.00)
Bo.s-Gaussian
0.600 (0.01)
0.427 (0.01)
1.055 (0.01)
0.786 (0.00)
0.935 (0.01)
0.669 (0.01)
0.420 (0.00)
0.306 (0.00)
PI. Gaussian
0.646 (0.01)
0.462 (0.01)
1.085 (0.01)
0.809 (0.00)
0.923 (0.01)
0.653 (0:01)
0.458 (0.01)
0.327 (0.00)
DDPN (ours)
0,502 (0.01)
0.390 (0.04)
1.135 (0.08)
0.810 (0.03)
0.906 (0.01)
0.632 (0.01)
0.392 (0.01)
0.277 (0.00)
8o.s-DDP (ours)
0.516 (0.01)
0.370 (0.01)
1.095 (0.03)
0.782 (0.02)
0,905 (0.02)
0.635 (0.01)
0.356 (0.01)
0,268 (0.00)
B1o-DDPN (ours)
0.558 (0.01)
0:407 (0.01)
1.006 (0.01)
0.759 (0.01)
0.909 (0.01)
0.634 (0.01)
0.356 (0.00)
0.263 (0.00)
Alcatoric + Epistemic (DEs)
Poisson DNN
0.650
0.547
1.046
0.817
0,996
0.683
0,823
0.556
DNN
0.681
0,567
1.066
0.824
0,982
0.686
0.857
0.560
Gaussian DNN
0.590
0,450
1,148
0.815
0.902
0.634
0,447
0.319
Faithful Goussian
0.571
0.429
1.042
0,841
0.909
0.643
0.424
0.324
Natural Gaussian
0,582
0.428
1.090
0,800
0.916
0.643
0.423
0.307
Ra.o Gaussian
0.591
0.420
1.019
0.740
0.879
0.619
0,414
0.302
fL.- Gausssin
0.633
0.453
1,050
0.765
0.887
0.624
0.455
0.324
DDPN (ours)
0.485
0,361
1.024
0.744
0.861
0.604
0.373
0.268
Bo.s DDPN (ours)
0.495
0.359
1.029
0.729
0.840
0.590
0.358
0.261
8.a-DDPN (ours)
0,543
0.393
0.959
0.712
0.859
0.597
0.344
0.257
4.3.1. .. BASELINES
4.3.2. ANALYSIS OF EMPIRICAL RESULTS
We compare to two discrete baselines: Poisson DNN Fal
We observe that DDPN (or one of its 3 variants) achieves the best accuracy and calibration on every task we benchmark. In almost all cases, the margin is substantial across both individual models and ensembles, COCO- PPoole e is the only dataset in which a non-DDPN model even places in the top three. In line with previous results LLakshminarayanan et al. 2017: Fort et al., 2019)) we find that marginalizing over multiple plausible sets of weights through deep ensembles yields improvements in both mean fit and probabilistic alignment. Ensembled DDPN also out- performs all baselines in terms of accuracy and calibration.
lah et al. 2009) and 2) Negative Binomial DNN Xie 2022). We also include fully heteroscedastic continuous models in our experiments: 1) Gaussian DNN (Nix & Weigend 1994). 2) 3-Gaussian (Seitzer al. 2022 3) Faithful Gaus- sian (Stirn et al. 2023), and 4) Natural Gaussian (Immer jet al. 2024). For 3-Gaussian we use the prescribed values of B = 0.5 and B = 1.0 from Seitzer et al. (2022). and mirror this behavior for our discrete adaptation, 3-DDDNN. A subscript marks the specific setting of / (c.g. Bo.s).
We also evaluate ensembles for all methods to highlight the effects of modeling both aleatoric and epistemic uncertainty. Gaussian ensembles are produced according the technique of ofLaashminarayaaan et al. (2017), while DDPN, Poisson, and Negative Binomial ensembles follow the prediction strategy outlined in Section 3.4 See Table for results.
On most datasets, we observe that Negative Binomial and Poisson DNNs struggle with fitting the mean in addition to uncertainty quantification. In contrast to Gaussian and DDPN models that can perform loss attenuation during training (see Section 3.2). Negative Binomial and Poisson regressors are limited in their ability to discount outliers. Whenever these models predict a high count, the equidis- persion and overdispersion assumptions built into each re- spective distribution force a high uncertainty to be predicted as well. Thus, training data points are weighted accord- ing to their predicted labels without truly accounting for observation noise, and outliers are not properly tempered.
For each dataset, we generate train/val/test splits with a fixed random seed. We select models according to lowest average loss the validation split and report results on the test split. We train and evaluate 5 models per technique and record the empirical mean and standard deviation of each metric. To form ensembles, these same 5 models are combined. All experiments are implemented in Py'oorch (Paszke et al., 2017). Details related to data splits, network architecture, hardware, and hyperparameter selection are reported in Appendix B.3 Source code available online?
The gap in performance between Gaussian neural regressors and DDPN models, meanwhile, can largely be attributed to the lack of an inductive bias: Gaussian models naïvely assign probability mass to continuous values that are known priori to be infeasible, while DDPNs concentrate their
https: // whu.tthhb..oom com i ous- a /ddpp
7

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
1.0
1.0
1.0
0.5
0.
0.6
0.4
0.4
04
02
0.0
Figure 4: Predictive PMFs from a -1.0-DDPN ensemble on samples from the test split of COCo-People, Individual member predictions are in gray, while the ensemble prediction is in blue. DDPN is able to flexibly and accurately represent counts of various magnitudes.
probabilities on discrete counts by explicit construction. This mismatch produces predictive CDFs that are more misaligned with observations compared to those produced by DDPNs. See Figure for two illustrative examples from Length of stay.
used for aut-of-distribution (OOD) detection. In accordance with common practice, we form scoring threshold-baan -baee OOD detectors and evaluate them in the binary classifi- cation setting (Hendrycks & Gimpel 2016 Alemi et al.) 2018 Ren et al. 2019). We define our in-distribution (ID) dataset Din to be Amazon Reviews and form our out-of- distribution dataset Dout by compiling verses from the King James Version of the Holy Bible. We refer to the train and test splits of Din as ((trr) and D (teat) respectively.
1.0-
1.0
0.5
We use the total variance of ensemble predictions as our OOD score, since both aleatoric and epistemic uncertainty are helpful OOD indicators (Wang Aitchison 202T). The OOD threshold To is chosen such that expect false positive rate of data drawn from Din- Specifically, we randomly select 20% of "D(teat) and define Ta to be the (1 11-oj a) quantile of the predictive uncertainties on that holdout set. Any inputs from D (test) (excluding the held- out data) or Dout that produce a predictive variance above Ta are classified as OOD, while the rest are marked as ID.
Gaussian
Gaussian DDPN
DOPN
0.0
Figure 5: Predictive CDFs from Go.s-Gaussian (blue) and 8o.r-DDPN (green) model on two test points from Length of Stay. Observed labels are marked with a vertical dashed line. When y is discrete, Gaussian models' pre- dictions suffer from having to assign probability mass to infeasible continuous values. DDPN is free from this con- straint.
For each ensemble, we vary a from 0 to I and report the resultant area under the ROC curve (AUROC), the area under the precision-recall curve (AUPR), and the false posi- tive rate at the 80% true positive rate (FPR80). To account for variability due to randomness in the calculation of Ta we run each evaluation 10 times, re-sampling the holdout set with each iteration, and report the mean and standard deviation of each metric. Results are presented in Table 3
In Section 33 explain how SSitzer al. (2022) 's 3 mod- ification can be adapted to improve mean fit in the discrete heteroscedastic setting by defining 3-DDPN. Our results demonstrate that this adjustment to our training objective generally results in superior MAE, although the strength of the effect varies by dataset. We note that CRPS often im- proves alongside MAE, indicating that can simultaneously boost both accuracy and overall probabilistic fit.
Our results demonstrate that uncertainties obtained from DDPN (and 3-DDPN) ensembles are the best-suited among all other models evaluated for identifying OOD inputs. This is strong evidence that uncertainty estimates obtained from DDPN models are robust, yielding important context for practitioners around how much an individual prediction can be trusted. For plots showing how distributions differ for ID OOD data, Appendix [.2] In particular, Figure highlights the effective OOD behavior of DDPN.
dieure/ddepiies several test instances from COCO-People and the predictive distributions produced by 31o-DDDN ensemble, the highest scoring method. Further examples, including predictions from each baseline model, can be found in Appendix [II]]
4.4. Dut-of-Distribution Detection
In this section, we investigate how DDPN models perform relative to baselines when their predictive distributions are
8

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
0.5
1,0
50
42
32-
20
30-
32"1
15
Figure 6: The effect of 8 on the convergence of a DDPN during training, inspired by Fig. 2 of SStim et al. (003)). Dotted lines indicate ground truth values of i and y respectively, while solid lines show the model's learned distribution. Shaded regions illustrate training data coverage. With standard NLL (3 O), poor mean fit on the rightmost isolated point "explained away" via high uncertainty (low values of 7), leading to subpar convergence. Increasing the value of changes training priorities and allows the network to adequately model the mean without exploding uncertainty estimates. Higher values of B lead to faster convergence the mean; when 0.5, the mean is fit by epoch 800, but when 3 1.0, the mean is fit by epoch 600.
Table 3: OOD detection results. We denote the best performer for each metric bold and the second-best underlined.
values of y for DDPN) to explain the poor fit. This is con- sequence of the learnable loss attentuation property from Definition 3.2 This raises the possibility that model training converges without producing a function that fits the outlier points. We can temper the effect of such loss attenuation by training with 8-NLL. In Figure we see that increasing the value of 3 indeed addresses this behavior, with higher 3 leading faster convergence the true mean fit. This
AUROC (t)
AUPR (1)
FPRSO (L)
Poisson DNN
0.330 (0.001)
0.413 (0.000)
0.793 (0.001)
DNN
0.280 (0.001)
0.397 (0.000)
0.819 (0.002)
Gaussian DNN
0.840 (0.001)
0.812 (0.005)
0.318 (0.002)
Faithful Gaussian
0.731 (0.001)
0.670 (0.001)
0.380 (0.002)
Natural Gaussian
0.836 (0.001)
0.827 (0.002)
0.317 (0.002)
matches the analysis performed by (Seitzer et al. 2022 who use / to regularize the Gaussian NLL.
Bo.5 G-Gusssiian
0.829 (0.001)
0.797 (0.004)
0.323 (0.002)
BLo- Gaussian
0.817 (0.001)
0.806 (0.002)
0.338 (0.001)
DDPN (ours)
0.854 (0.001)
0.849 (0.003)
0.269 (0.002)
fo. DDPN (ours)
0.887 (0.001)
0.875 (0.003)
0.199 (0.001)
5. Conclusion
B DDDN (ours)
0,870 (0.001)
0.851 (0.002)
0.236 (0.002)
In this paper, we presented the Deep Double Poisson Net- work (DDPN) as a novel approach to enhance the quality of predictive distributions in deep count regression. DDPN achieves full heteroscedasticity through the unrestricted vari- ance of its output distribution, allowing it to accurately capture aleatoric uncertainty of any form. This capability results in uncorrupted estimates of epistemic uncertainty when predictions are combined an ensemble. addition, we formally defined the concept learnable loss attenua- tion and proved that DDPN exhibits this property. We also proposed the 3-DDPN modification, which enables control over the strength of this loss attenuation to further improve performance. Experiments on diverse datasets demonstrate that DDPN surpasses current baselines accuracy, calibra- tion, and out-of-distribution detection, establishing a new ofate-oftthe-ar in deep count regression.
4.5. The Effect of 3 on DDPN Training Dynamies
In this section, study how the 3 hyperparameter medi- ates the functional evolution of DDPN's predictive distri- bution over the course of training. Figure 6 shows one dimensional count regression problem with three settings of 3. We generate data from Y|X ~ DP( X sin(X) + 157.6 6 0.03.X?), where X Uniform [3, 8). We then produce a set of isolated points at (1, [sin(1) + 15]) and (10, [10) + 15]) and concatenate them with the gen- erated dataset. Since these points are relatively far from the rest of the training data, a neural network will struggle to initially fit them. If the network is heteroscedastic, it may downweight the contribution of these outlier points to the overall loss by assigning high uncertainty values (low

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Acknowledgments
Jebara, T. (eds.), Proceedings of the 31st Interna- tional Conference Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp. 1683-1691, Bejing, China, 22-24 Jun 2014. PMLR. URL htt.httss////occeddii ml r. press/v32] Cheni14.htmi
The work was partially supported by NSF awards #2421839, NAIRR #240120. The views and conclusions contained in this paper are those of the authors and should not be interpreted representing any funding agencies.
Impact Statement
Chow, N. and Steenhard, D. A flexible count data regression model using sas proc nlmixed. In SAS Global Forum: Statistics and Data Analysis, volume 250, pp. 1-14, 2009,
By enabling robust uncertainty estimation on count data, DDPN improves the reliability and robustness of Al sys- tems high-stakes applications such as healthcare, finance, and environmental modeling. The ability model both epis- temic and aleatoric uncertainty enhances trustworthiness, particularly in safety-critical systems. However, as with any predictive modeling approach, there are ethical consider- ations regarding bias, misuse, and overreliance on model outputs. Ensuring that DDPN is used responsibly requires careful dataset curation, bias mitigation, and transparent reporting of model uncertainties.
Cover, T. M. Elements of information theory. John Wiley & Sons, 1999.
Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., and Le, Q. V. Autoaugment: Learning augmentation policies from data, arXiv preprint IrXiv: 2018.
D' Angelo, F, and Fortuin, V. Repulsive deep ensembles are bayesian. Advances in Neural Information Processing Systems, 34:3451-34655 2021.
References
Daxberger, E., Kristiadi, A., Immer, A., Eschenhagen, R., Bauer, M., and Hennig, P. Laplace redux-effortless bayesian deep learning. Advances in Neural Informa- tion Processing Systems, 34:20089-20103 2021.
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khos- ravi, A., Acharya, U. R., et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information fusion, 76:243-297, 2021.
Deng, J., Dong, W., Socher, R., Li, L...., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Alemi, A. A., Fischer, I., and Dillon, J. V. Uncertainty in the variational information bottleneck. arXiv preprint arXiv:1807.00906, 2018.
Der Kiureghian, A. and Ditlevsen, O. Aleatory or epistemic? does it matter? Structural safety, 31(2):105-112 2009.
Amini, A., Schwarting, W., Soleimany, A., and Rus, D. Deep evidential regression. Advances in neural informa- tion processing systems, 33:14927-14937, 2020.
Dheur, V. and Taieb, B. A large-scale study of probabilis- tic calibration in neural network regression. In Interna- tional Conference on Machine Learning, pp. 7813-7836. PMLR, 2023.
Aragon, D. C., Achear, J. A., and Martinez, E. Maximum likelihood and bayesian estimators for the double poisson distribution. Journal of Statistical Theory and Practice, 12:886-911, 2018.
Dietterich, T. G. Ensemble methods in machine learning, In International workshop on multiple classifier systems, pp. 1-15. Springer, 2000.
Bishop, C. M. Mixture density networks. 1994.
Blasiok, J., Gopalan, P., Hu, L.,, and Nakkiran, P. A unifying theory of distance from calibration. In Proceedings of the 55th Annual ACM Symposium on Theory of Computing, pp. 1727-1740, 2023.
Dwaracherla, V., Wen, Z., Osband, I., Lu, X., Asghari, S. M., and Van Roy. B. Ensembles for uncertainty estima- tion: Benefits of prior functions and bootstrapping. arXiv preprint arXiv:2206.08:: 00633,,..33.... 2022.
Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. Weight uncertainty in neural network. In International conference on machine learning, pp. 1613-1622. PMLR, 2015.
Efron, B. Double exponential families and their use in generalized linear regression. Journal of the American Statistical Association, 81 (395):7709-771, 1986.
Fahrmeir, L., Kneib, T., Lang. S., Marx, B., Fahrmeir, L., Kneib, T., Lang, S., and Marx, B. Regression models, chapter Springer, 2013.
Chen, T., Fox. E., and Guestrin, C. Stochastic gra- dient hamiltonian monte carlo. In Xing, and
10

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Fallah, N., Gu, H., Mohammad, K., Seyyedsalehi, S. A., Nourijefyani, K., and Eshraghian, M. R. Nonlinear pois- son regression using neural networks: A simulation study. Neural Computing and Applications, 18:939-9433 2009.
Jenkins, P., Armstrong, K., Nelson, S., Gotad, S., Jenkins, J. S., Wilkey, W., and Watts, T. Countnet3d: A 3d com- puter vision approach to infer counts of occluded objects. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3008-3017, 2023.
Fort, S., Hu, H., and Lakshminarayanan. B. Deep en- sembles: A loss landscape perspective. arXiv preprint arXiv: 19tp:02772.0777 2019.
Kang, K., Settur, A., Tomlin, C., and Levine, S. Deep neural networks tend to extrapolate predictably. arXiv preprint arXivv2310.00877. 2023.
Fukushima, K. Visual feature extraction by a multilayered network of analog threshold elements. IEEE Transactions on Systems Science and Cybernetics, 5(4):322-333, 1969.
Kendall, A. and Gal, Y. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
Gawlikowski, J., Tassi, C. R. Ali, M., Lee, J., Humt, M., Feng. J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., et al. A survey of uncertainty in deep neural networks. Artificial Intelligence Review, 56(Suppl 1):1513-1589, 2023.
Kiefer, J. and Wolfowitz, Stochastic estimation of the maximum of a regression funetion. The Annals of Mathe- matical Statistics, pp. 462-466, 1952.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.698) 2014.
Gneiting, T. and Raftery, A. E. Strictly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477) 2007.
Koren, Y., Bell, R., and Volinsky, C. Matrix factorization techniques for recommender systems. Computer, 42(8): 30-37, 2009.
Gneiting, T., Balabdaoui, F., and Raftery, A. E. Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society Series B: Statistical Methodology, 69 (2):243-268. 22443-268,220 2007.
Kuleshov, V., Fenner, N., and Ermon, S. Accurate uncer- tainties for deep learning using calibrated regression. In International conference on machine learning, pp. 2796- 2804. PMLR, 2018.
Graves, A. Practical variational inference for neural net- works. Advances in neural information processing sys- tems, 24, 2011.
Lakshminarayannnn B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Hendrycks, D. and Gimpel, K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv: 161v.026 2016
Li, S., Chang, F., Liu, C., and Li, N. Vehicle counting and traffic flow parameter estimation for dense traffic scenes. IET Intelligent Transport Systems, 14(12):1517-1523, 2020.
Hill, T. Conflations of probability distributions. Trans- actions of the American Mathematical Society, 363(6);
3351-3372, 2011.
Hoffman, M. D., Gelman, A., al. The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. J. Mach. Learn. Res., 15(1): 1593-115921213 2014.
Lian, D., Li, J., Zheng, J., Luo, W., and Gao, S. Den- sity map regression guided detection network for rgb-d crowd counting and localization. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 1821-1830, 2019.
Hsieh, M.-R,, Lin, Y.L.,, and Hsu, W. H. Drone-based ob- ject counting by spatially regularized regional proposal network. In Proceedings of the IEEE international con- ference computer vision, pp.
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ra- manan, D., Dollár, P., and Zitncck C. L. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6- 2014, Proceedings, Part V 13, pp. 740- 755. Springer, 2014.
2017.
Immer, A., Palumbo, E., Marx, A., and Vogt, J. Effec- tive bayesian heteroscedastic regression with deep neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Lin, W. and Chan, A. B. Optimal transport minimization: Crowd localization on density maps for semi-supervised counting. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 21663- 21673, 2023.
Jenkins, P., Wei, H., Jenkins, J. S., and Li, Z. Bayesian model-based offline reinforcement learning for product allocation. In Proceedings of the AAAI Conference on Ar- tificial Intelligence, volume 36, pp. 12531-1255 -12377, 2022.

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Liu, C., Huynh, D. Q., Sun, Y., Reynolds, M., and Atkinson, S. A vision-based pipeline for vehicle counting, speed estimation, and classification. IEEE Transactions Intelligent Transportation Systems, 22(12):7577-556, Tttp:TTTTSS.2033004040
Mnih, A. and Salakhutdinov, R. R. Probabilistic matrix factorization. Advances in neural information processing systems, 20, 2007.
Mucsányi, B., Kirchhof, M., and Oh, S. J. Benchmarking uncertainty disentanglement: Specialized uncertainties for specialized tasks. arXiv preprint arXiv:2402. 19460, 2024.
2021. doi:
Liu, W. Wang, X., Owens, J., and Li, Y. Energy-based out- of-distribution detection. Advances in neural information processing systems, 33:21 13:21164-22147,, 2020.
Murphy, K. P. Probabilistic machine learning: Advanced lopics, chapter 15. MIT press, 2023.
Loshchilov, I. and Hutter, F. Sgdr: Stochastic gra- dient descent with warm restarts. arXiv preprint 1rX1v:1608.00388,, 2016.
Ni, J., Li, J., and McAuley, J. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference empirical meth- ods in natural language processing and the 9th interna- tional joint conference on natural language processing (EMNLP-IJCNLP), pp. 188-197, 2019.
Loshchilov, I. and Hutter, F. Decoupled weight decay regu- larization. arXiv preprint arXiv: 1.1100001, 2017.
Luo, A., Yang, F., Li, X., Nie, D., Jiao, Z., Zhou, S., and Cheng, H. Hybrid graph neural networks for crowd count- ing. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 11693-11700, 2020.
Nix, D. A. and Weigend, A. Estimating the mean and variance of the target probability distribution. In Pro- ceedings of 1994 ieee international conference on neural networks (ICNN'94) (94), volume 1, pp. 55-60. IEEE, 1994,
Lv, Y., Duan, Y., Kang, W., Li, Z., and Wang. F.-Y. Traffic flow prediction with big data: A deep learning approach. leee transactions on intelligent transportation systems, 16(2):865-873, 2014.
Osband, I., Wen, Z., Asghari, S. M., Dwaracherla, V., Ibrahimi, M., Lu, X., and Van Roy, B. Epistemic neural networks. Advances in Neural Information Processing Systems, 36:2795-2823, 2023.
Marron, S. and Wand, M. P. Exact mean integrated squared error. The Annals of Statistics, 20(2):122-736, 1992.
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang. E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. Automatic differentiation in pytorch. 2017.
Ren, J., Liu, P. J., Fertig, E., Snoek, J., Poplin, R., Depristo, M., Dillon, J., and Lakshminarayanan B. Likelihood ra- tios for of--iffiiittitionnnn detection. Advances in neural information processing systems, 32, 2019.
Marsden, M., McGuinness, K., Little, S., Keogh, C. E., and O'Connor, N. E. People, penguins and petri dishes: Adapting object counting models new visual domains and object types without forgetting. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pp. 1070-8079, 2018.
Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXiv, abs/1910.01 108, 2019.
Marx, C., Zalouk, S., and Ermon, S. Calibration by dis- tribution matching: trainable kernel calibration metrics. Advances in Neural Information Processing Systems, 36, 2024.
Seitzer, M., Tavakoli, A., Antic, D., and Martius, G. On the pitfalls of heteroscedastic uncertainty estima- tion with probabilistic neural networks In Interna- tional Conference on Learning Representations, April forum?
Marzban, C. The roc curve and the area under it as per- formance measures. Weather and Forecasting, 19(6): 1106-1114, 2004.
2022. URL
i id=aPOpXInV11
Settles, Active learning literature survey. 2009.
Matheson, E. and Winkler, R. L. Scoring rules for contin- uous probability distributions. Management science, 22 (10):1087-1096, 1976.
Song, H., Diethe, T., Kull, M., and Flach, P. Distribution calibration for regression. In International Conference Machine Learning, pp. 5897-5906. PMLR, 2019.
McCullagh, Generalized linear models. Routledge, 1989.
Stirn, A., Wessels, H., Schertzer, M., Pereira, I., Sanjana, N., and Knowles, D. Faithful heteroscedastic regression with neural networks. In International Conference on Ar- tificial Intelligence and Statistics, pp. 5593-5613. PMLR, 2023.
Microsoft.
rseeverhhopial-legghhoo--ssyy 2016.
URL httpss//gtthub..cc//lccccsssss
r- --server-hoosttta--eeeggh------e
12

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Toledo, D., Umetsu, C. A., Camargo, A. M., and de Lara, I. A. R. Flexible models for non-equidispersed count data: comparative performance of parametric models to deal with underdispersion. AStA Advances in Statistical Analysis, 106(3):473-497, 2022.
Zou, Z.., Shao, H., Qu, X., Wei, W., and Zhou, Enhanced 3d convolutional networks for crowd counting. arXiv preprint weXiv:1908.0412.1 2019.
Wang, X. and Aitchison, L. Bayesian ood detection with alcatoric uncertainty and outlier exposure. arXiv preprint arXiv:2102. 129599 2021.
Wilson, A. G. and Izmailov, P. Bayesian deep learning and probabilistic perspective of generalization. Advances in neural information processing systems, 15:4697-4700 2020.
Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., and Vajda, Visual transformers: Token-based image representation and processing for computer vision, 2020.
Xie, S.-M. neural network extension for solving the pareto/negative binomial distribution model. Interna- tional Journal of Market Research, 64(3):420-439) 2022.
You, J., Li, X., Low, M., Lobell, D., and Ermon, Deep gaussian process for crop yield prediction based on re- mote sensing data. In Proceedings the AAAI conference artificial intelligence, volume 31, 2017.
Young, and Jenkins, P. On measuring calibration of discrete probabilistic neural networks, 2024.
Yu, T., Thomas, G., Yu, L., Ermon, S., Zou, J. Levine, S.,
Finn, C., and Ma, T. Mopo: Model- based offline policy optimization. Advances in Neural Information Processing Systems, 33:14129-14142, 2020.
Zhang, Q. and Chan, A. 3d crowd counting via multi- view fusion with 3d gaussian kernels. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 12837-12844, 2020.
Zhang, Y., Zhou, D., Chen, S., Gao, S., and Ma, Y. Single- image crowd counting via multi-column convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 589-597, 2016.
Zhu, F. Modeling time series of counts with com-poisson ingarch models. Mathematical and Computer Modelling, 1609-10): 2012.
Ziatdinov, M. Active learning with fully bayesian neural networks for discontinuous and nonstationary data. arXiv preprint arXiv:2405. 09817,987771 010817,202.. 2024.
Zou,Y Y. Geedipally, S. R., and Lord, D. Evaluating the dou- ble poisson generalized linear model. Accident Analysis & Prevention, 59:497-505. 2013.
13

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
A. Deep Double Poisson Networks (DDPNs)
A.1. Derivation of the DDPN Objective
If we model our regression targets as yi ~ DP(Hs; where (p4, mul" = fo(x)), then the Double Poisson NLL for an individual prediction is obtained as follows:
argmax (esmax/p(yiieeer (folffe(())) (x:)) = argmax ppllpp 7:]
argmin
[- log p(y:lpss [ps
eepp (-y:)y!"
ejpe
7836
argmin Purt
log
Y exp (-7(11))
ys!
V/
1
ejtss
}
arrgmin Pifti
log
11 exp (-71(1s)
Vi
argiiin Purt
log 7/
74/44 + 74t6 (1 + log uai
log 1/))
2
Thus,
log 7:
Ldyiss Pi, 7x)
+ 7:44 - (1 + log pri - log yy)
(4)
2
During training, we minimize this NLL average across N data points (x,, M- N
Note that our derivation assumes c(us; 74) = 1, which is in line with Fact I (Equation 20) of Efron (1866). See also Chow & Steenhard (2009), who employ a similar assumption in their work with Double Poisson regression.
A.2. Limitations
DDPNs are general, easy to implement, and can be applied to variety of datasets. However, some limitations do exist, One limitation that might arise is on count regression problems of very high frequency (i.e., on the order of thousands or millions). In this paper, we don't study the behavior of DDPN relative to existing benchmarks on high counts. In this scenario, it is possible that the choice of a Gaussian as the predictive distribution may offer a good approximation, even though the regression targets are discrete.
Although we follow precedent (Efron 1986 Zou al. 2013 Aragon et al. 2018 in employing the general approximations [[Z] R μ and Var[Z] [r] Ra for some Z ~ DP 7) in this work, these are not guaranteed to hold under all conditions. See Appendix A.3 for detailed study of these approximations.
One difficulty that can sometimes arise when training a DDPN (or any model trained with NLL) is poor convergence of the model weights. In preliminary experiments for this research, we had trouble obtaining consistently high-performing solutions with the SGD (Kiefer & Wolfowitz, 1952) and Adam (Kingma Ba 2014) optimizers, thus AdamW (Loshchiloy] Hutter 2017) was used instead. Similar Seitzer et al. (2022), we found that the 8-NLL exhibited greater stability across wider variety of hyperparameter settings. Future researchers using the DDPN technique should be wary of this behavior.
In this paper, we performed single out-of-distributis (OOD) experiment on Amazon Reviews. This experiment provided encouraging evidence of the efficacy of DDPN for OOD detection. However, the conclusions drawn from this experiment may be somewhat limited in scope since the experiment was performed on single dataset and task. Future work should seek to build off of these results to more fully explore the OOD properties of DDPN on other count regression tasks.
A.3. Assessing the Quality of Efron's Moment Approximations
In line with prior work, we use Efron's approximations for the first two moments of the Double Poisson distribution in the proof of Proposition 3.] An obvious question that arises is how reliable are these approximate moment assumptions. We introduce the concept of moventtdeviation functions (MDFs) to assess this theoretically in Definition A.1
14

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Definition A.I. Let Q be family of univariate distributions parametrized by ub E R4, Suppose we are given o R R which outputs setting of parameters for Q, nh (14), such that the first moments of Z Q (M) ((Z)) (z*)), nearly equal to target moments μ (Ma,,.., pa)) Then for any pair (Q, + { 1
R] moment-deviation
functions if, for any valid μ., if Z (m)) we have KZ*) uul (p) for all 1 S S n2.
We focus on π = 2. (approximation error for the mean variance of a distribution). If we can pick v so that é1, 82 are small, it follows that Q is incredibly fiexible, as there exists a setting of parameters that can roughly achieve any mean and variance. the Gaussian case (N, I),, we have & = E2 = The Double Poisson case (using Efron's 's approximations for Φb), is handled in Proposition A.2
Proposition A.2. Let DP denote the Double Poisson family. Set 25 (No, oB) (no: #b) Letting to og (DP, 1222 are:
the MDFss for
Σy=o s(10 7o, 3y no)
(wo, ot))
Exzo s(100 To. y)
d(Ho: 70) Eyz s sMM: To: 9) - 70 70(2 s(1o: 7o. y) - mo))2
c2(po, on))
70 7o(_p= s(Mo 7o. y)
where we have
h(2) 2²
z!
r(u, 7,2) = 7(= - 1 + z blog 15 - z log=)
s(1,7,2) = 11=)) exp(r((,7))
Σ s/4 7: v. 12)2 y) + Σ s(14 7,44 11)
d(y7) = 1/2
Proof. All we have to show is that the mean and variance of Z ~ Q (2) are within the provided E and E2. We first re-cast the Double Poisson PMF into canonical form, with natural parameters 7 7 log μ and 7/2 7:
73e-
p(21,,7)
c(,,7)
z!
l1(2) exp {72 72 log / 72 log
7(14,)
ná exp 1 exp
m1 }) }, 92)
h(2) exp (me + 1/22 722 log
c(exp _
= h12))ex) exp (n T(:) A(n))
where h(2) matches the form specified in the proposition, T(x) = (z, x(1 - log is the sufficient statistic, and A(r)
log + log c(exp { m }. n2) is the cumulant function.
7/2 exp
m2
By properties of the canonical form, we then have E|Z obtain
84 Jep.
Var [Z]
8²A By!
Letting E(m, m2) c(exp ).92) 12), we
15

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
6 (m, 72)
B|2] = cxp
dm
1/2
6 (111 T2)
Var [
a
exp
71
+
dm
8 6 (m,,
Om
1/2
8 {m, 12)
exp
{
& (m 12)
Bm
0 C
(m, 112)
3m
a
((n, (mm,)))
E 77)2
We first solve for [Z Recall that c(p,) 7) o -Ph(y) 22 exp muy vy log - ryloggy. or equivalently c(p,7) 7) Ex=o 7 hly) exp m.)) Then
① Om
dc 0p
exp
[i
dc
6 (m 12)
Tps
0y dm
1/2
0p
1
hyy) exp (r(4.7 u)) 3z
Ör
7
7 $
h(yy) (r(u 1))
so, recalling that exp { [m]] = 14, we have
Ey=o hly) exp 7.)) -H)
E|Z| = +
Eyn h(y) exp (r(4, v))
2 s(u, 7, - 1))
= + +
Eyo 5(1,,73)
Thus, 6Z² =Huo, a)). ). we have
Σg== (o -g (y mo)
BiZ] 14
Σg (oo 4g.
This implies that
00
(mu og. y (y po)
uo
$
[||2] Mol
2gz
mo dg y/
E1 (No: a
which trivially yields [E[2]- - Pol I (1 Ri), as desired.
To evaluate Var (), we must first compute the second derivative of 6:
16

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
8
8e
8p
8 μ de Omf du
6-
3p2
7m
2 0
5/1, 7,8))
1 -ー-1)
7
s
7, y)
y
72 du
72
y=
2
Σ
s(p,,,)7
( (y-M))
2
s(1. 7.9)
+
1
(7
Σ
20
s
s(م 7.9)
1
112
7/2
y=U
y=U
Σ
$11, 7,9) (7(2 -1)² - v) + Σ
5/1, 7, (y 14)
0=0
7)
We now have an expression for the variance:
VarlZ] = exp 1 2
88 ((n, 12) 807
861 12))
1/2
6(11,72)2
a(s, 7)7 Σy s(11 7.9) v) - - Eng s(1, 7, 7. - 1))²
+
7
Eggo ((H47:1)
So
Exj (no. m. v) (y un) 2
d | 4o
noo .)
Varl [ - o
2
ug
Enn= s
No: !ot v)
which implies
d No: o
(1o.
Eg=
No, 5g mg v) - !
Eggor
(wo. #o y)
(y Ho)
Var[Z] - o!l
up %
\يح s
(No, mo. v)
E2 ao))
yielding |Var[Z] - obl S $2(1o, oî), as desired.
The MDFs in the proposition yield near-zero values for most target means and variances, implying that in most settings, the Double Poisson can be considered distribution with unrestricted variance.
Remark 1.
We now study the bounds obtained in Proposition A.2 empirically. In Figure 7 we plot the error, e, incurred via Efron's 's estimates on a grid of target means and variances, using 100th partial sums. Both and Eg are effectively 0 for nearly all values of μN and 0-- The only exception is for very small values of the target mean, Mo -> O; when the target variance subsequently large, we see an increase in both é1 and Eg. Under these rare conditions, the approximations begin to deteriorate.
17

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
1/A, %f)
C, of)
Function Wart
33
40
(a) Grid of values for ei (wo, ob), the moment-deviation function for the Double Poisson mean.
(b) Grid of values for e2 (g), the moment-deviatee function for the Double Poisson variance.
Figure 7:
We plot the error, e, incurred via Efron's approximations of the Double Poisson mean and variance, grid of target values for these moments. Values were obtained using 100th partial sums.
A.4. The effect y initialization
Another potential method for avoiding the loss attenuation trap (see Section 3.3 is to initialize 7 to some high value (via the initial bias in the output head). This essentially forces the network to begin training with low uncertainty values. We investigate the effectiveness of this method by initializing 7 to various high values at the start of training. We train on the same dataset as in Figure 6 Results with differing initializations are presented in Figure g Rather than improve mean fit, we observe that this strategy actually hurts overall convergence to the true function. The best performance comes, in fact, when log(?) initialized close to zero. Note that despite higher initialization of y, the point to the far right of the data still "explained" via low y (high uncertainty). Interestingly, the modification can help us avoid the impact of such poor initialization. In Figure 9 we mirror the experimental setup of Figure 8 but set 3 1 and train with 3-NLL. In all but the most extreme case (log(?a) 5), allows recover the true function.
B. Additional Experimental Details
B.1. Description of Datasets
In this section, we provide a brief description of each dataset used in our experiments and define the associated network architectures. For further details, we refer the reader to Appendix B.3 and our source code.
tasked with forecasting the number of days between initial admission and discharge for given hospital patient, conditioned collection of health metrics and indicators about the specific facility providing the treatment. This is inherently count prediction problem, since resource allocation and billing typically discretize time spent in a hospital. We train small MLPs to output the parameters of a predictive distribution over the number of days a patient expected to stay.
Length of Stay Regression models Length of Stay (Microsofi 2016)
We introduce an image regression task on the person class of MS-COCO Lin et 2014). which we call ccco-Pecple. In this dataset, the task is to predict the number of people in each image. Each model evaluate coco-People is trained with small MLP on top of the pooled output from ViT-B-16 backbone (Wu et al. 2020)).
COCO-People
Inventory This dataset comprises an inventory counting task (Jenkins et al. 2023). where the goal is to predict the number
of objects on a retail shelf from an input point cloud (see Figure 32 in Appendix D.3 for an example). For this task, we adapt CountNet3D Jenkins al. 2023) for probabilistic regression.
18

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
(0011)
40-
30
30-
20
SpooT
32
Figure 8: Results of training on a dataset with isolated points when we initialize 7 for a standard DDPN model to a high value (via the imitial bias the output head). Despite forcing low uncertainty at the beginning of training, the model still ignores difficult points and converges to a suboptimal solution. In fact, the smaller we force the uncertainty be initially (through high y values), the worse eventual fit is.
Ing
30-
22
22
Spact
200
600
18
1020
Figure 9: Rerun of the experiment depicted Figure where we train with 31.0-NLL instead of vanilla NLL. Interestingly, we find that allows the model to recover from poor weight initializations.
19

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Amazon Reviews We model user ratings from the "Patio, Lawn, and Garden" split of a collection of Amazon reviews Ni
al. 2019). The objective in this task is predict the discrete review value (1-5 stars) from input text sequence, which historically has been addressed with Gaussian NLL (Mnih & Salakhutdinov consist of small MLP on top of a DistilBert backbone (Sanh et al. 2019).
2007 Koren et al. 2009). All text regressors
B.2. Evaluation Metrics
In this section, provide detailed definition of each metric we employ for evaluation in our experiments. We also provide context for our choices around which metrics to use (when relevant).
B.2.1. MEAN ABSOLUTE ERROR
The Mean Absolute Error (MAE) quantifies how closely a model's point predictions align with the ground truth across a dataset. the probabilistic setting, we use the mode of the predictive distribution as our point prediction. Given a dataset {x,, Vi 1- and model fe, the MAE is then computed as:
MAE(fe,
argmax p(y: 2 (x:)
y/
71
zEsupport/wc
B.2.2. CONTINUOUS RANKED PROBABILITY SCORE
The Continuous Ranked Probability Score (CRPS) is strictly proper scoring rule that quantifies how aligned predictive distributions (sometimes referred to as "forecasts") are with ground-truth observations (Matheson & Winkler 1976 Gneiting & Raftery 2007). For a specific predictive cumulative density function (CDF), F,, and a ground-truie regression target y4 with CDF F. defined as follows:
CRPS(F..))) =
[P((2)- - P((z)]2 dz
[P((2) [=22011 dz
Fz(2)² dz + 1 [P((2) - 1]2 dz
Note that the second equality arises because Yi, once observed, is deterministic (its probability density is point mass at the observed value). When the predictive distribution is discrete, this integral further reduces to the summation:
Σ dz) + [ - 112
CRPS can also be computed from deterministic forecasts (i.c. those obtained from models that output only a point prediction). In this case, it is a known fact that CRPS reduces to the mean absolute error (Gneiting & Raftery, (007). which aids interpretability. In our experiments, when we report CRPS, we take the average value across all test points (since CRPS defined point-wise).
B.2.3. MEDIAN PRECISION
Gneiting al. (2007) observe that good condition for well-fit probabilistic model is "sharpness subject to calibration". In other words, our model must produce forecasts that exhibit statistical consistency with observations while also concentrating as much mass as possible around the true value of a regression target. Although sharpness alone is not useful metric for quantifying probabilistic fit, measuring this value can provide helpful insights into the typical shape of a model's predictions. To quantify sharpness for a model/dataset pair, we use the Median Precision (MP), where precision is defined as A, (xx))) We employ the median instead of the mean as summary statistic to increase robustness to outliers
20

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
(when the predictive variance is very small, the precision value blows up). In Appendix B.4.1 we report MP values for each model benchmarked in Section
B.2.4. ON THE OMISSION OF OTHER CALIBRATION METRICS
In this work, we choose not to quantify calibration via common metrics such as negative log likelihood (NLL) and expected calibration error (ECE). This is primarily due to the unique challenge of comparing probability forecasts across models with various distributional assumptions. NLLs obtained for continuous distributions are computed from probability densities, whereas discrete distributions output probability masses. These are fundamentally different quantities that cannot be directly compared. Meanwhile, the regression form of ECE (Kuleshov et al. 2018) has recently been shown favor continuous models over discrete ones due to implicit assumption about uniformity of the probabilityy integral transform (Young & Jenkins 2024). Other work suggests that ECE may not be reliable indicator of true calibration in the first place, since it is marginal measure (Song et al. 2019 Marx et al. 2024) and is sharply discontinuous as function of the predictor arbitrarily small perturbations to a model can cause large jumps in ECE (Btasiok et al. 2023
We note that CRPS (see Appendix B.2.2). which is defined solely in terms of the distance between a predicted and ground- truth CDF and is strictly proper scoring rule, overcomes the discrete vs. continuous comparison challenge and provides a reliable indicator of probabilistic fit. Thus, we include it the main measure of calibration in our experiments (Section 4.3)
B.2.5. CUT-OF-DISTRIBUTEE CLASSIFICATION METRICS
We use standard binary classification metrics to characterize each model's ability to identify out-of-distribution inputs, in line with Hendryeks & Gimpel 2016), Alemi et al. (2018), and Ren et al. (2019).
The Area Under the Receiver Operating Characteristic Curve (AUROC) evaluates the performance of a binary classifier across a range of thresholds. In our autoo--distribttion (OOD) experiment, we define these thresholds based on different expected false positive rates, i.e., a E [0, 1]. For each threshold, we plot the true positive rate (the percentage of true OOD samples correctly identified) against the false positive rate (the percentage of in-distribution samples mistakenly classified as OOD). This produces curve whose integrated area provides comprehensive measure of the classifier's performance. In this context, the AUROC can be interpreted as the probability that, given random in-distribution (ID) and OOD sample, the model assigns a higher predictive variance to the OOD sample than the ID sample. Thus, higher values of AUROC indicate more useful classifiers. A score of 0.5 corresponds to random guessing. For further details, see Marzban (2004).
AUROC
The Area Under the Precision Recall Curve (AUPR) is another metric for evaluating classifier performance, analogous to AUROC, but tailored for imbalanced datasets. Like AUROC, it involves computing the integral under a curve parameterized by a. This curve plots precision (the proportion of OOD classifications that were correct) against recall (the proportion of all true OOD instances that were correctly classified), illustrating the trade-off between these two metrics. Higher values are better. A model relying solely on class proportions in the data would achieve an AUPR equal to P(X is OOD), which in our experiment is 0.224.
AUPR
FPR80 The False Positive Rate at 80% True Positive Rate (FPR80) quantifies the proportion of in-distribution (ID) samples misclassified as diitr-idittiittionn (OOD) when the classification threshold a is set to achieve True Positive Rate (TPR) of 80% (Ren et al. 2019). This metric highlights the classifier's 's ability to minimize false positives while maintaining high sensitivity to OOD samples. Lower values indicate better performance, with a perfect classifier achieving an FPRS0 of zero.
B.3. Training Details
In all experiments, instead of using the final set of weights achieved during training with a particular technique, we select the weights associated with the best average loss held-out validation set. This can be viewed as form of early stopping.
The ReLU (Fukushimal 1969) activation is exclusively used for all MLPs. No dropout or batch normalization is applied. We employ the AdamW optimizer [oshchilov & Hutter 2017) for all training procedures, and anneal the initial learning rate to 0 following a cosine schedule (Loshchilov & Hutter 2016).
The hyperparameter settings we report are chosen such that each model converges to a stable training loss basin with minimal
21

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
overfitting (as quantified by validation loss). To ensure faimess across all models, use one set of training hyperparameters (including choice of underlying architecure) for each specific dataset.
In addition to the details we provide in this section, we refer the reader to our source code where configuration files for each model benchmarked can be found. The corresponding author can be contacted for access to our pre-generated splits of each dataset.
B.3.1. SIMULATION EXPERIMENT (INTRODUCTORY FIGURE)
To produce Figure 1 we generate dataset with the following procedure: First, we sample x from a uniform distribution, Uni for(0, 2T) Next, draw initial proposal for y from a conflation (Hill) 2011) of five identical Poissons, each with rate parameterized by X(x) 10 sin(2) + 10. We scale y by -1 and shift it by +30 to achieve high dispersion at low counts and under-dispersion at high counts while maintaining nonnegativity. We generate 800 points from this process for the training split, as well as 100 for validation and 100 for evaluation.
The models that make up the ensembles we report in this figure are small MLPs (with layers of width [128, 128, 128, ). We train each network for 200 epochs on the CPU of 2021 MacBook Pro with a batch size of 32 and an initial learning rate of 10 We set weight decay to 10m5
B.3.2. LENGTH OF STAY
Each model benchmarked for Length of Stay is a small MLP with layer widths [128, 128, 128, 64]. Models are trained for 15 epochs on a 2021 MacBook Pro CPU with a batch size of 128, an initial learning rate of 10 and a weight decay value of 10-4.
The Length of Stay dataset has been open-sourced by Microsoft under the MIT license. has 100,000 rows, which we divide into a 80/10/10 train/val/test split using fixed random seed.
Licensing information for Length of Stay can be found at http:://@tthub..ccmmcccroooooo r- c--aever-hoapitaal
The full
X- dataset can be downloaded from
httpps//ean.ghhbbberrrrteee.ccce frros
B.3.3. COCO-PEOPLE
We train Vision Transformers (Wu et al. 2020) on COco-People, passing their pooled patch representations to two-layer MLP regression head with layer widths of 1384, 256]. All weights are fine-tuned from the vit-base-pat 1122111-24-i222--
checkpoint (Deng et al. 2009). We set the initial learning rate to 10-4 and use weight decay of 10-3, We train with an effective batch size of 256 in distributed fashion, using an on-prem machine with 2 Nvidia GeForce RTX 4090 GPUs. Images are normalized with the ImageNet (Deng et al. 2009) pixel means and standard deviations and augmented during training with the AutoAugment transformation (Cubuk et al. 2018). Training is done with BFloat 16 Mixed Precision.
The COCO dataset from which we form the COCO COCO-People subset is distributed via the CCBY 4.0 license. It can be accessed at hhttts://coooccaaeeet org/ thome Our subset has 64,115 images in total, which we divide into 70/10/20 train/val/test split.
B.3.4. INVENTORY
For the Inventory task, we use a modified version of CountNet3D (Jenkins et al., 2023) that ouputs the parameters of a probability distribution instead of regressing the mean directly. We train models for 50 epochs with an effective batch size of 16, an initial learning rate of 10-3, and weight decay of 10-5, Training performed on an internal cluster NVIDIA GeForce RTX 2080 Ti GPUs.
The Inventory dataset is made available us via industry collaboration and is not publicly accessible.
https://github http://thhha.ccoee/////////oose/
22

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
B.3.5. AMAZON REVIEWS
We fine-tune a BestilBert Base Cased (Sanh al. 2019) backbone with [384, 256] MLP regression head for the Amazon Reviews task. To avoid overfitting, freeze all but the final transformer block in the backbone. The regression head is not frozen. All networks are trained for 10 epochs across 2 on-prem Nvidia GeForce RTX 4090 GPUs with an effective batch size of 2048. We use initial learning rate of 10-4 and weight decay of 10-6, and run computations with BFloat 16 Mixed Precision.
Amazon Reviews is publicly available at /ttps::/ uosd. hww.dl/ccccc.cccallaaaaaaattttte
v2/ The "Patio, Lawn, and Garden" subset we employ in this work is hosted httt:pp://daaarepp/ edu/mcauley. wwwccullyggruuu///ee /att amazon v2 wttptt/ttorraatteeeee lesSma wtt.SSaaaaaaatttaaaaaaaaaaaaaeaa
uosd ucsd
Our
subset contains 793,966 total reviews, which divide into 70/10/20 train/val/test.split..
B.4. Additional Results
B.4.1. SHARPPESS
To provide a notion of the sharpness of predictive distributions produced by each model we benchmark, we measure and report the median precision (MP) across all datasets in Table See Appendix [B.ee3 for more details about how we compute this metric. Higher values correspond to sharper distributions (those with lower predictive variance), We note that sharpness alone is not an indication of good probabilistic fit: thus we do not mark "winners" in this metric. We simply provide sharpness values to facilitate improved understanding of each technique's 's behavior on our datasets.
[B.2.3]
Table 4: Median Precision (MP) values across main experiments.
Length of Stay
COCO-People
Inventory
Reviews
Poisson DNN
0.262 (0.00)
0.437 (0.01)
0.263 (0.01)
0.211 (0.00)
NB DNN
0.265 (0.00)
0.392 (0.04)
0.261 (0.01)
0.211 (0.00)
Gaussian DNN
2.031 (0.09)
0.913 (0.43)
0.689 (0.02)
3.833 (0.28)
Alcatoric Only
Faithful Gaussian
1.258 (0.09)
0.647 (0.05)
1.072 (0.13)
3.907 (0.04)
Natural Gaussian
1.456 (0.07)
0.745 (0.20)
0.713 (0.02)
4.545 (0.29)
$0.5 GGuussin
2.125 (0.07)
2.174 (0.18)
1,011 (0.07)
4.604 (0.45)
$.oo-Gaussian
1.646 (0.21)
1.558 (0.09)
(0.06)
3.935 (0.04)
DDPN (ours)
2.663 (0.14)
0.915 (0.30)
0.680 (0.04)
3.960 (0.13)
Bo.s-DDPN (ours)
2.184 (0.14)
1.006 (0.13)
0.818 (0.04)
3.642 (0.18)
BL.o-DDNN (ours)
1.565 (0.11)
1.986 (0.04)
0.690 (0.03)
3.806 (0.04)
Poisson DNN
0.261
0.413
0.253
0,210
Alcatoric + Epistemic
NB DNN
0.263
0.354
0.252
0.210
Gaussian DNN
1.933
0.548
0.631
3.643
Faithful Gaussian
1.211
0.592
0.907
3.927
Natural Gaussian
1.386
0.571
0.639
4.217
30.s Gaussiinn
2.003
1.785
0.828
4.362
31.0 Gaussiann
1.531
1.360
0.734
3.819
DDPN (ours)
2.441
0.626
0,623
3.694
Po.s-DDPN (ours)
2.046
0.821
0.709
3.645
B1u-DDPP (ours)
1.463
1.680
0.620
3.569
B.4.2. GLM RESULTS ON LENGTH OF STAY
Since Length of Stay is tabular, may compare GLM baselines in addition to the neural networks benchmarked in
Section 4.3 We train and evaluate a Poisson Generalized Linear Model (GLM), a Negative Binomial GLM, and Double
Poisson GLM [Efron] 1986. Zhu 2012, Zou et al. 2013 Aragon et al.) 2018 Toledo et al.) (022). As in Table 2 we train 5 independent models with each technique and report the mean and standard error of both MAE and CRPS across all trials under the "Alcatoric Only" header. We also provide the median precision values as in Table 4 Although GLMs are not deep networks, we may ensemble their predictions similar fashion as deep ensembles (DEs) to quantify epistemic uncertainty.
23

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
Table 5: GLM results for the Length of Stay dataset.
Aleatoric Only
Aleatoric + Epistemic
MAE (4)
CRPS (4)
MP
MAE ())
CRPS ())
MP
Poisson GLM
1.284 (0.08)
0.882 (0.06)
0.319 (0.01)
1,256
0,850
0.309
NB GLM
3.783 (0.27)
1.692 (0.14)
0.241 (0.00)
3,906
1.685
0.239
DP GLM
1.320 (0.15)
0.894 (0.08)
0.543 (0.44)
1.185
0.829
0.340
We do so, and report these results under the "Aleatoric + Epistemic" header.
The limited flexibility of linear models appears to hinder the GLMs we evaluate, as they struggle with both mean fit and calibration when compared to their respective neural network counterparts.
C. Proofs and Derivations
In this section we provide full proofs of the propositions presented in the main body of the paper, along with derivations of key equations (excepting the Double Poisson NLL derivation, which located in Appendix A.T
C.1. Proof of Proposition 2.3
Proof. We will first demonstrate that Gaussian regressors are fully heteroscedastic. Since they model both the predicted mean μ4 and variance of conditionally, all we have to show is that the Normal distribution exhibits unrestricted variance. To see this, note that if Z is distributed normally, if we fix μ for any μt E R, then for arbitrary o² (0, oo), the parameter setting ab- [u, "²)T achieves Var = 0². Thus the Normal distribution has unrestricted variance, and Gaussian regressors are indeed fully heteroscedastic.
To show that Poisson and Negative Binomial regressors are not fully heteroscedastic, we will prove that their respective output distributions do not have unrestricted variance, i.e. there exists least one 0² E (0, oo) such that no setting of parameters &p can achieve this variance when the expected value is fixed.
If we say Z is Poisson, then for any À > O, fixing E[2] À requires Z Poisson(A), i.c. zb [A]. For any o² E (0, oo) \ {A}, we see that there is no setting of 2/ such that Var[Z] = O², since Var[ E1|Z] A. So the Poisson distribution does not have unrestricted variance.
The Negative Binomial case is similar. We employ the common (r, parametrization, though other alternatives are possible. Recall that in this parametrization, r > 0 is the number of successful trials until an experiment is stopped, and p E (0. 1) is the probability of success in each trial of the experiment, Pick any μ > 0. If Z is Negative Binomial, then fixing E[Z] 4 means that ub = |r, must satisfy μ r(1-p) Now choose any o² E (0, For Var [2] o² to be true, +b must satisfy r(1-p) Multiplying both sides by the p
0² r(1-p) Noting that we chose a² such that o² < μ., this implies r(1-p) 8²
p2 <
positive quantity マ|ーッー yields 1 < p. But p < 1 by definition. Thus we have identified range of 0² values such that there exists no ub that can satisfy Var[2] = o² if B[Z] = So the Negative Binomial distribution does not have unrestricted variance.
We conclude that any regression model outputting either the Poisson or Negative Binomial distribution not fully heteroscedastic.
C.2. Proof of Proposition 3.1
Proof. We assume the first and second moment approximations proposed by Efron 1986); E[Z] = μ and Var _.4 To see that DDPN is fully heteroscedastic, note that for any input x1, DDPN predicts log( (x)
w zi + by, where z is a neural network's 's input-conddiional hidden representation of x, Since DDPN's s predictive variance directly depends on & (which we just showed depends on x²), all we have left to demonstrate is that the Double Poisson distribution has unrestricted variance. To see this, note that if Z is Double Poisson with expectation p (where μ > O), then for any o² E (0, oc), choosing
24

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
jr # 1 implies Var[Z] #/4 =².. Thus, DDPN fully heteroscedastic.
C.3. Proof of Proposition 3.3
Proof. Suppose that have loss function Lipr, 9i) that can be written in the form d((,)) + a(p))r((s, y1) as stated in Definition 3.2
We have
a(o.)r(Mu
lim a(o.) 0r d(
lim
r(us, Wi)
10000
d(d.)
= r(p4, (0)
(since a -> 0 and d - oc)
0
where for the second equality, we rely on the conditions imposed for and d in Definition 3.2
C.4. Proof of Proposition 3.4
Proof. To see that DDPN exhibits learnable loss attenuation and conforms with Definition 3.2 Equation 2
recall the loss function from
log 2 + Mp: -1 + log ü. log ys)
L
(5)
2
Because DDPN is expressed in terms of inverse dispersion, = loss function in terms of dispersion:
1 d, we can substitute 1 6;
in for y/ to re-parameterize the
P4
y: (1 + log f.s log ys)
L
log
(6)
2
@
6.
di
We can rearrange terms to put L in the standard form with dispersion penalty d((b)), attenuation factor a())), and residual penalty r(44, V:):
1
L log
+
[p= yo(1 + log f.s log ys)
2
Φi
1
log
[(A: vd) y:(log iu: log v:)
1
1
log 1
log à;
+
1(a: -y:) v. cs log
2
1
1
log Φ,
[(p: v.) y (log d4 log
2
Φi
Let d(δ.) log d. a(δ.) 1 and r(pr, y1) (Ai 91) yr (log A. log We have d(pi) alp:)r((n..)) log &. + 1/ [i:: - 31) - y (log j4, logs:)) Clearly, d(()) = log & is monotonically increasing w.n.t & and a(d:) 62 is both monotonically decreasing w.r.t & and bounded from below by 0 (since 3
0).
All that is left show is that (i) r(βy, 91) = A = ys and (ii) is nonnegative:
(i) We first show that fú = ys = r(p;, y1) 0. To see this, note that when üu; = sr both the terms our residual penalty, (f. - y:) and yrooog fig - log yi). are So their difference, r, must be 0.
see that r(it, = = fi M/ we handle the case of yı = O and y > 0 separately:
25

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
= (j: yr) V:olog i4 logyi ys) 0. But y/: cancels the second term in this difference, leaving us with 0 (ju: y:) ia; Vi-
Case 1 If y: = O, then
Case 2 If > O, we can make the substitution
P. and rewrite the residual penalty: r == ugy - ys log u log u) 0. We now claim that lu(u)
yi(u 1 log u). Since y/ 0. this yields
0
(u
(u (u- 1u-1-log log u) =0
1. To see this, note that (u) 1 - 1 is negative when and positive when
1.
Since h(u) = 0 when
1, strictly increasing to the right of u, and strictly decreasing to the left of u, 0 is uniquely
achieved by h at u = 1. So = = u = 1. Thus, u 1 - log = 0 2 =
Pi =
Jss = y/-
In either case, have رم; y/- Since have proved both directions, we conclude that r r(ps, y:)
P Vi
iii) We separately handle the cases of y/4 0 and y; 0, and may now assume (by virtue of the iff) that fú + ys:
Case If ys 0, then r= رs > 0, where the last inequality comes from the restrictions on the parameters of a Double Poisson distribution.
Case If y/ > O, then substituting u = As as before yields r = ye(u - log u), which is positive if h(u)
(u-18 1- - log u) is positive. Recall that hu()) 0 iff = 1, and that h is strictly decreasing to the left of and strictly increasing to the right of u. Since (u(u) = 0, this implies (by the first derivative test) that u = 1 is the unique minimizer of h. with hu()) 0. But we only have u = if fl = ys, which we can assume is not the case because (i). So h > > (u
log u) > 0 = r >
In both cases, we are guaranteed that 0.
Thus Equation conforms with Definition 3.2
C.5. Computing the Mean and Variance of a Mixture Model
In Section 3.4 we describe how the ensembled predictive distribution is a uniform mixture of the M members of the ensemble: p( ) t Em= (fe. (x:)).
Letting Pm EE ffe. (x:)] and of Var[yi (x:)). we can get the mean and variance of the predictive distribution as [x = M ΣM Mm and Varly: x.] Σm a +
(IM
We note that the ensembling technique used throughout our work can be applied to form an ensemble from any collection of neural networks outputting probability distribution, regardless of the specific parametric form (Marron Wand 1992).
C.6. Decomposing the Predictive Variance of a Mixture Model
For the mixture model described above, we can decompose the predictive variance for single regression target y/ as follows:
M
(6)
(1)
M
2
0
+
pkm
Varly: [x]]=
m
M
M
m==1
mm=1
M
27
M
1
m
2
(6)
+
M
1
um
(6)
M
(t)
Em
(i)
Em [n(t)
(Em 11
Em (6)
[arm(u(!)
The first term in this sum, [o 19) is the average predictive variance of each ensemble member and is often considered to be a proxy for aleatoric uncertainty, while Var, [pmi]] represents the amount of disagreement between ensemble members regarding the predicted mean and interpreted as epistemic uncertainty (Kendall & Gal 2017).
26

---

Figure 12: Predictive PMFs from 11.0--DDPPPP on samples from the test split of COCO People. COCO-PPeplee Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
D. Additional Case Studies
D.I. Example Predictive Distributions on COCo-People
19 depict the predictive distributions of one model trained under that framework on coco-People, while 20 show the combined predictions 5-model ensemble. We select instances from the test split and visualize outputs each. A black star indicates the count of people in the image.
For each baseline, Figures
D.1.1. INDIVIDUAL MODELS
0.2
0.0
0.0
Figure 10: Predictive PMFs from DDPN on samples from the test split of COCO -Poplee
1.2
10
10-
0.5
0.0
0.8
0.5
0.5
0.4
0.4
12
Figure 11: Predictive PMFs from 9o.5-DDPN on samples from the test split of COCO- People.
O
0.2
0.0
5.0
10
0.
0.8
0.5
0.6
0.6
0.6
0.6
0.4
0.4
0.2
0.2
0.2
0.0
0.0
Figure 13: Predictive PMFs from a Poisson DNN on samples from the test split of 10000-Poppee
27

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
0.1
0.2
0.2
0.0
0.0
Figure 14: Predictive PMFs from a Negative Binomial DNN on samples from the test split of COCO-Pecple.
1.0
10-
100
0.5
08
0.8
0.5
06
0.6
04-
0.4
0.2
02
Figure 15: Predictive PMFs from a Gaussian DNN on samples from the test split of COCO-
102
10
15
15
Figure 16: Predictive PMFs from Natural Gaussian DNN on samples from the split of COCO Peoo-e.
0.0
0.0
0.0
Figure 17: Predictive PMFs from Faithful Gaussian DNN on samples from the test split COCO People.
1.2
0.5
0.8
0.5
0.5
0.6
0.6
0.8
0.4
04
0.2
0.2
02
Figure 18: Predictive PMFs from a -au-sian DNN on samples from the test split of COCO-People
1.
3.0
0.5
O
0.-
5.6
5.4
15
10
Figure 19: Predictive PMFs from Gaa-Saann DNN samples from the test split of COCO- PPc0--Peple.
28

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
D.1.2. ENSEMBLES
1.0
1.0
1.0
0.6
0.5
0.6
0.4
0.0
0.0
6.0
Figure 20: Predictive PMFs from a DDPN ensemble on samples from the split of COCO Peoope.
1.-
1.0
1.0
0.8
O
0.6
0.4
0.4
04
0.0
0.0
0.0
Figure 21: Predictive PMFs from PD.N ensemble on samples from the test split of COCO People.
1.0
0.8
0.8
0.8
O
0.4
0.4
0.4
02
Figure 22: Predictive PMFs from a 3DDPN ensemble on samples from the test split of COCO People.
0.8
0.8
0.8
0.6
0.4
0.2
0.2
0.2
10
10
10
Figure 23: Predictive PMFs from Poisson DNN ensemble on samples from the test split of 1:00--Poppe:
1.0
0.6
0.5
0.6
0.2
0.2
6.2
0.0
0.9
10
10
10
Figure 24: Predictive PMFs from Negative Binomial DNN ensemble on samples from the test split of -Poole.
29

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
1.0
1.0
0.-
0.5
0.4
0.2
0.2
0.2
10
Figure 25: Predictive PMFs from Gaussian DNN ensemble on samples from the test split of COco-People.
0.8
0.8
0.8
0.4
0.4
0.2
0.2
0.2
Figure 26: Predictive PMFs from a Natural Gaussian DNN ensemble on samples from the test split of COCO-People.
1.0
1.0
1.0
0.6
0.4
0.4
0.4
0.0
0.0
Figure 27: Predictive PMFs from Faithful Gaussian DNN ensemble samples from the test split of COco-People.
1.0
0.5
9.6
0.2
0.2
0.2
00
Figure 28: Predictive PMFs from 30.5 Gaussiann DNN ensemble on samples from the test split of CocO-People.
1.5
1.0
0.8
0.8
0.8
ORO
0.4
0.4
0.4
02
Figure 29: Predictive PMFs from Gauussin DNN ensemble on samples from the test split of COCO- People.
30

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
D.2. OOD Predictive Distributions on Amazon Reviews
Figures 30 and 3T visualize the difference (or lack thereof) between ensemble predictions on in-distributton data (in this case, the test split of Amazon Reviews) and -ittt--diittitttionn data (verses from the King James Bible). We include predictive distributions for each ensemble bascline presented in Section 4.3
sere and siomn ladd
Heages Brsiccas. enccomted
Jelle the
pacted Janne des Sie
coserne LoRd
aor brrm to als"
(a) DDPN
(b) DDDN
bor sxcoralii boailea yorls
eeci Eil orsdl
Be-001 Losl and Leth
Cartasss 2013 and
Da 1000, and rall Lert.
ce them Mboiuu
worres The Aomboc
Cratte e 16- abal
(c) 81.0- BDDNNDDN
(d) Poisson
"Iid good grilac
aeoola tae the tn
Sgan ofthe
pangal sie Coal
Negative Binomial
Figure 30: "Discrete" ensemble predictions in-distribution (ID) vs. out-of-distribution (OOD) examples. Individual ensemble members were trained on Amazon Reviews and then fed verses from the King James Version of the Bible. Note the desirable behavior of DDPN (and 3-DDPN) ensembles, which produce concentrated probabilities on ID examples but appear to revert to the optimal constant solution (OCS) when faced with OOD examples (Kang et al. 2023). Meanwhile, the predictive variance exhibits little difference between ID and OOD examples for the Poisson and Negative Binomial ensembles.
31

---

Fully Heteroseedastic Count Regression with Deep Double Poisson Networks
and Autte sded and
bosk muse and Cendinl
Madite pribed
ba cdedia
Mick picks
(a) Gaussian
ladd mestaaeet
Sevepuend and coneea
Jedia fre
pi pro abel
anttaned por AETE corcar"
parveetane of LORD.
goog fo to gas"
(b) Natural Gaussian
(c) Faithful Gaussian
"Sate jol
En orttl bendd
de Dil and Lnd.
Cars-as WIT3 and
pediveerry them Mioiiu
wocres the Cembor
darie e 16.1 abil
(d) Gausssan
(c) -Gausian
Figure 31: "Continuous" ensemble predictions on in-distribution (ID) vs. out-t--distributiinn (OOD) examples. Individual ensemble members were trained Amazon. Reviews and then fed verses from the King James Version of the Bible.
32

---

Fully Heteroscedastic Count Regression with Deep Double Poisson Networks
D.3. Example Point Cloud from Inventory
In Figure 32 provide an example point cloud from the Inventory dataset used in the experiments of Section 4.3 Further examples can be viewed in Jenkins et al. (2023)
Figure 32: Example point cloud from Inventory. Each green box represents an inventory slot which is segmented into point beam (see Jenkins et al. (2033) for details and further examples). Models predict the product count within each point beam.
33


Reviews:
[{"review_id": "MiryjWsihX", "reviewer": "Reviewer_AhTw", "summary": "The paper focuses on outputting distributions for non-negative integer predictions (i.e., count data). To do so, the paper has a model output the parameters for a Double Poisson distribution, which admits separate mean and variance parameterizations. Then the paper further utilize ensembles to include epistemic model uncertainty. Results comparing against other predictive distributions, such as a typical Gaussian, show that the proposed predictive distribution outperforms on count tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the recognition of our work.", "claims_and_evidence": "Yes, the paper is quite clear on the proposed approach, the different uncertainties involved (e.g., the diff between aleatoric and epistemic, which is often confused or muddled), and the experimental setup. The reasoning for using the Double Poisson instead of a regular Poisson is clear and backed by both theory and empirical evidence.", "methods_and_evaluation_criteria": "Yes, the proposed Double Poisson makes sense for count data and for the goal of heteroscedastic variance (i.e., per-example variance controlled by the model). The evaluation datasets are fine and varied, and the baselines being other predictive distributions is appropriate and expected.", "theoretical_claims": "The formal definitions of the distributions (e.g., 2.1, 2.2, 3.2) and the propositions appear to be correct. In general, the approach is straightforward: have a model output the parameters of the Double Poisson distribution and then optimize that distributions NLL; this is generally the same type of approach used in modern models, just with a different distributional family.", "experimental_designs_or_analyses": "Yes, the experimental setup, including datasets, metrics, and baselines, are appropriate and expected for the type of approach being proposed. E.g., accuracy and a proper scoring rule is an ideal combination (often, the latter is missed), and the baselines are appropriately other predictive distributions and do not conflate that with other modeling choices.", "relation_to_broader_scientific_literature": "This fits well within the broader literature on uncertainty quantification. Accordingly, the appropriate papers are generally referenced. Most existing work has focused on categorical or continuous problems. This paper's novelty is in focusing on count data and using the less common Double Poisson for full heteroscedasticity.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "This paper's novelty is in focusing on count data and using the less common Double Poisson for full heteroscedasticity. It's not a surprisingly result and is fairly straightforward, but it's a useful paper to have in the literature. In particular, I'm pleased by the way in which the paper is very clear on the various uncertainty concepts and does not confuse or conflate any terms.\n\nI am assigning a \"4: Accept\" to mean that it's a solid paper; a 5 would be for an exceptionally exciting result, such as showing that this pushes on SoTA in some current frontier model.", "other_comments_or_suggestions": "None", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qQ4pFuLbhP", "reviewer": "Reviewer_MbuP", "summary": "In this paper, the authors consider the problem of estimating heteroscedastic uncertainty within the context of counting tasks, where the final outputs should represent positive integer numbers. While many successful solutions have been proposed for heteroscedastic uncertainty in general (real-valued) regression tasks, this is not the case for counting, as it requires a different parametrization of the output distribution. Earlier solutions for the counting setting, such as the Poisson distribution, suffer from restricted heteroscedastic variance, meaning that the parameter defining the mean value of the distribution significantly restricts the possible predicted variance. In this paper, the authors propose using the Double Poisson distribution for counting tasks and prove that it resolves the issue of the former method, namely, it has unrestricted variance. Additionally, they demonstrate that the proposed loss has the property of adaptive loss attenuation, which lowers the impact of outlier points during training. Finally, they propose a way to make this attenuation controllable through $\\beta$-DDPN. The effectiveness of the proposed method is demonstrated on several datasets from different domains, showing that the proposed parametrization outperforms other parametrizations for counting tasks in terms of uncertainty quality (calibration) and accuracy.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s thoughtful comments. \n\n##  Introducing additional uncertainty approaches, such as ensembling methods (Deep Ensembles, Batch Ensembles, etc.), could be beneficial.\n\nAn important aspect of our work is the interplay between DDPN and Deep Ensembles. We demonstrate this connection throughout the paper. In Section 3.4 we show how individual DDPNs can be combined as ensembles. Then, in the bottom half of Table 2 we present results with ensemble DDPNs. Table 2 demonstrates that learning ensembles of DDPNs improves accuracy **and** the quality of predictive uncertainty.\n\nWe suspect that similar results will hold for BatchEnsembles, as this type of ensemble just changes how the weights of each member are derived: combining slow shared weights and fast, independent rank one weights. The principles we propose in this paper could easily be applied to other types of ensembles. We leave the validation of this hypothesis to future work. \n\nFinally, in the discussion with Reviewer 68NH, we discuss how DDPN can be combined with other UQ methods such as Bayesian Neural Nets, Evidential Methods, Laplace Approximation and Monte Carlo Dropout.", "claims_and_evidence": "The authors present their claims and contributions in a clear manner while also supporting them with both theoretical (for example, proving that the proposed DDPN regressors are fully heteroscedastic) and experimental results.", "methods_and_evaluation_criteria": "The authors primarily compare against other loss-based heteroscedastic approaches on various counting tasks, clearly demonstrating the effectiveness of the proposed approaches in the discussed counting setups.", "theoretical_claims": "The main theoretical contributions of the paper could be considered Propositions 2.3 and 3.1, which prove that previously proposed parameterizations, such as Poisson and Negative Binomial, are not fully heteroscedastic, while DDPN is. The proposed proof appears to be correct and valid, with no observable issues.", "experimental_designs_or_analyses": "The experimental design and analysis are adequate and rigorous, with no issues.", "relation_to_broader_scientific_literature": "The paper clearly positions itself within the existing literature by thoroughly discussing prior work on heteroscedastic uncertainty estimation, particularly in regression and counting tasks. It provides sufficient detail on previous parameterizations, such as Poisson and Negative Binomial, highlighting their limitations and demonstrating how the proposed DDPN framework overcomes these constraints.", "essential_references_not_discussed": "No, there are no critical references missing in the paper.", "other_strengths_and_weaknesses": "In short, the major Strengths of the paper are:  \n\n* The paper clearly discusses the problem of heteroscedastic uncertainty estimation in the counting context, its existing problems, and solutions.  \n* The proposed method is clear, easy to implement, and demonstrates good performance on a number of different tasks.  \n* In contrast to many other uncertainty methods, it does not require a significant increase in computation/memory during inference while still producing high-quality uncertainty estimations.  \n\nOne of the potential Weaknesses:  \n\n* The paper mostly compares the method against other loss-based uncertainty methods. Introducing additional uncertainty approaches, such as ensembling methods (Deep Ensembles, Batch Ensembles, etc.), could be beneficial.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uLxAkDjiaC", "reviewer": "Reviewer_68NH", "summary": "The work introduces deep double poisson networks for the count regression problem. The proposed approach can quantify both the aleatoric and epistemic uncertainty with ensemble. Also, double poisson network allows unrestricted variance to model discrete count data, and can show robustness to outiers. Authors carry out experiments where the approach performs better in terms of calibration, out-of-distribution detection, and accuracy.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s thoughtful comments and helpful feedback.\n\n## Comparison to the Natural Posterior Network\nWe have followed the official repository to download the `bike-sharing` dataset file and pre-process exactly as used in the paper reviewer mentioned. For training, the paper mentioned they perform training after the grid search in the space `[1e−2, 5e−4]`, since the search step is not specified, we took the log-scale space step as below: [0.01, 0.005623, 0.003162, 0.001778, 0.001, 0.000708, 0.0005], and found the lr as: 0.003162 in our model’s configuration. Following the exact same settings, we conducted evaluation on 5 rounds of training and report the mean and standard deviation of our model’s performance on RMSE, the results are:\n\n\n| Method         | RMSE         |\n|----------------|--------------|\n| Dropout-N      | 70.20 ± 1.30 |\n| Ensemble-N     | 48.02 ± 2.78 |\n| EvReg-N        | 49.58 ± 1.51 |\n| NatPN-N        | 49.85 ± 1.38 |\n| Dropout-Poi    | 66.57 ± 4.61 |\n| Ensemble-Poi   | 48.22 ± 2.06 |\n| NatPN-Poi      | 51.79 ± 0.78 |\n| **DDPN (ours)**| **47.87 ± 0.42** |\n\n## How does DDPN compare to other uncertainty methods?\n\nIn short, the objective function proposed in Equation 1 enables the network to capture aleatoric uncertainty over count data. We show how this can be combined with Deep Ensembles to better capture epistemic uncertainty. Table 2 shows this combination is effective. DDPN is presented in our paper in terms of maximum likelihood + ensembles for 1) simplicity, 2) effectiveness, and 3) likelihood of community adoption.  DDPN can easily be combined with other UQ methods. \n\n\n### Bayesian Neural Networks\n\nOne could put a prior over the weights of the network (ideally an isotropic Gaussian prior). Let $\\theta$ denote the neural network parameters, $f_\\Theta(x_i)$, and $\\mathcal{D}$ denote the training dataset. The log posterior is:\n\n$\\log p(\\theta | \\mathcal{D})) = \\log \\frac{1}{Z} + \\log p(\\mathcal{D} | \\theta) + \\log p(\\theta)$\n\nwhere $\\frac{1}{Z}$ is the normalizing partition function and is usually dropped during inference.\n\nFortunately, the negative log likelihood, $-\\log p(\\mathcal{D} | \\theta)$ is already defined in Equation 1 of our paper, and the log gaussian prior is easy to compute, $\\log p(\\theta) = -\\frac{1}{2} \\log (2\\pi \\sigma_0^2) - \\frac{1}{2\\sigma_0^2} (\\theta - \\mu_0)^2$, where the prior hyperparameters are $\\mu_0$ and $\\sigma_0^2$. \n\nThen one could choose the preferred inference algorithm: MAP, HMC, SGLD etc… and estimate the posterior.\n\nEmpirically, Bayesian neural networks can outperform Deep Ensembles when using high fidelity inference algorithms such as Hamiltonian Monte Carlo. However, in practice MCMC-based inference is impractical and DEs often outperform less exact inference methods (i.e., SGLD, Variational Inference etc…) [Izmailov et al. What Are Bayesian Neural Network Posteriors Really Like? ICML’21]. We suspect the same results hold for DDPNs.\n\nMoreover, many recent works have directly connected Deep Ensembles to Bayesian Inference by showing that DEs are a coarse approximation of the posterior, sampled and multiple modes with no local uncertainty [Fort et al. Deep Ensembles: A Loss Landscape Perspective. 2019][Wilson and Izmailov. Bayesian Deep Learning and a Probabilistic Perspective of Generalization. NeurIPS’20]. \n\nThe effectiveness, simplicity and attractive theoretical properties of DEs motivated our decisions to use them in our experiments.\n\n### Evidential Approaches\n\nDDPN could also be easily applied with evidential regression techniques [Amini et al. Deep Evidential Regression. NeurIPS’20]. Because DDPN uses a likelihood function during training, one would simply have to specify evidential priors over the parameters of the DDPN, $p(\\mu)$ for the mean and $p(\\gamma)$ for the inverse dispersion. Then, the network would be trained to predict the parameters of the higher-order evidential distribution. \n\n### Laplace Approximation (LA)\n\nThis is perhaps the easiest since LA is typically a post-hoc technique. One would train the DDPN in the standard way described in our paper. Then, one could apply any of the post-hoc second-order, covariance approximation methods described in [Daxberger et al. Laplace Redux – Effortless Bayesian Deep Learning. NeurIPS’21]\n\n### Dropout-based uncertainty\n\nMonte Carlo dropout estimates epistemic uncertainty by randomly dropping out weights at test time and approximates the Bayesian posterior. \n\nDDPN can easily be combined with MC dropout by 1) training the single-member DDPN to convergence, and 2) applying the MC dropout procedure with $T$ different forward passes through the dropped out model [Gal and Ghahramani. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. ICML’16]\n\nHowever, [Lakshminarayanan et al. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. NeurIPS’17] show that MC dropout is clearly inferior to DEs.", "claims_and_evidence": "- The authors claim that the proposed deep double poisson network can perform well on the count regression task. The claims are empirically validated through experiments on benchmark datasets and some baseline methods.", "methods_and_evaluation_criteria": "The methods and evaluation criteria look reasonable. The authors consider discrete count regression problem, and look at the different metrics, evaluating the method along different dimensions including OOD detection, calibration, and accuracy.", "theoretical_claims": "- Authors present some theoretical claims,  but these seem to be derived from standard double poisson networks.", "experimental_designs_or_analyses": "The experimental design looks sound.", "relation_to_broader_scientific_literature": "The work is likely to have limited impact to a narrow subfield in the scientific community.", "essential_references_not_discussed": "The work Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions [Bertrand Charpentier, Oliver Borchert, Daniel Zügner, Simon Geisler, Stephan Günnemann]) introduces a general evidential approach that can be effective for a wide range of problems including uncertainty-aware classification, uncertainty-aware regression, and uncertainty-aware count regression. Discussion and comparison with the work could be beneficial.", "other_strengths_and_weaknesses": "Strengths\n- The paper is easy to follow and I found it to be a pleasant read.\n- The work introduces deep double possion network which seems to be effective in discrete count regression based on the experimental results The authors show the robustness to outliers of the proposed approach. Also, the approach performs well in terms of accuracy, calibration, and ood detection.  \nWeaknesses:\n- Beyond ensembling, there are other approaches (e.g., Bayesian neural networks, evidential approaches, dropout-based uncertainties). While the authors compare with standard Poisson, NB, and Gaussian-based heteroscedastic networks, a thorough comparison can help better illustrate the effectiveness of the approach.", "other_comments_or_suggestions": "- Figures, labels and captions can be better presented. Many labels/legend texts are too small and not clearly legible. Also, the captions are too long and could be shortened for a better read.", "questions_for_authors": "- How does the work perform compared to natural posterior networks and the presented baselines on the benchmark Bike Sharing dataset (Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions [Bertrand Charpentier, Oliver Borchert, Daniel Zügner, Simon Geisler, Stephan Günnemann]) ?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q7Whjw7vUF", "reviewer": "Reviewer_bN97", "summary": "The paper introduces the Deep Double Poisson Network (DDPN), a novel neural network model for count regression that provides accurate input-conditional uncertainty quantification. The main conceptual idea is that DDPN extends deep ensembles to count regression by using the Double Poisson distribution, which allows for heteroscedastic variance in count data. This flexibility enables improved estimation of aleatoric uncertainty (inherent variability in data) and, consequently, better epistemic uncertainty (model uncertainty) estimation. The paper proves that DDPN exhibits properties similar to heteroscedastic Gaussian models. The authors introduce a loss modification to control the learnable loss attenuation mechanism, allowing for more precise uncertainty calibration. Experiments on diverse datasets show that DDPN outperforms existing count regression baselines in accuracy, calibration, and out-of-distribution detection.\n\n## update after rebuttal\nI acknowledge that the authors have improved the proofs (my point 1.) but do not provide a strong argument for point 2. I will increase my score to 2, but still think the work does not reach the acceptance bar.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful feedback.\n\n\n## Monotonicity vs tend to infinity\n\nWe agree with this remark and propose to change Def. 3.2 to:\n\n>  where $\\lim_{\\hat{\\phi} \\to \\infty} d(\\hat{\\phi}) = \\infty$ and $\\lim_{\\hat{\\phi} \\to \\infty} a(\\hat{\\phi}) = 0 $\n\nFortunately, the proof in Appendix C.3 holds under this new definition, as these limits are in fact used to show that the residual error tends towards 0 (lines 1108-1109). With this change, our proof of Prop. 3.4 also remains valid (since logx tends to infinity and 1/x tends to zero).\n\n## Normalizing constant\n\nTo simplify our objective, we followed previous work (see below) and assumed $c(\\mu, \\gamma) = 1$. This facilitates easier differentiation. We will make this more clear in App. A.1.\n\n- See Fact 1 (Eqn. 2.10) of [Efron, B. \"Double exponential families and their use in generalized linear regression.\" Journal of the American Statistical Association. 1986]\n- Follow-up work has also set $c(\\mu, \\gamma)=1$ [Chow, N., and David Steenhard.  2009. \"A flexible count data regression model using SAS Proc nlmixed”]\n\n## Max/Min in Appendix 1\n\nWe propose two changes to the derivation of our objective to increase clarity and align with convention:\n\n1. Replace max/min with argmax/argmin\n2. Clarify that we maximize over $N$ training examples (see below) and derive the per-instance loss defined in Equation 1 (see discussion below)\n\nThe NLL becomes: $\\arg\\min_{\\mu_i, \\gamma_i} [ -\\sum_{i=1}^N \\log p(y_i | \\mu_i, \\gamma_i)]$.\n\n##  Lack of Log link in Appendix A.1\n\nIn App. A.1 we derive the training objective. In contrast, Section 3 describes how it can be used to train DDPN. The connection between the two is trivial (exponentiate the log link to evaluate Eq. 1) and is stated in Footnote 1 (pg. 4).\n\n## Lack of Summation in Equation 1\n\nEq. 1 expresses the loss for a single training example, $x_i$. We state in lines 183-184 that the loss is:\n> averaged across all prediction / target tuples…in the dataset\n\nTo be more explicit, we propose to change $\\mathcal{L}$ to $\\mathcal{L}_i$. \n\n## Prop. 3.1: DDPN regressors and full heteroscedasticity\n\nIn line with prior work, we use Efron's approximations for the first two moments in the proof of Prop. 3.1. We propose to revise this proposition: *With mild assumptions*, DDPN regressors are fully heteroscedastic (where we assume that Efron's approximations hold).\n\nHow good are Efron's approximations? We introduce the concept of moment-deviation functions (MDFs) to assess this theoretically:\n\nLet $Q$ be a family of distributions parametrized by $\\psi\\in\\mathbb{R}^d$. Suppose we are given $\\hat{\\psi_n}:\\mathbb{R}^n\\to\\mathbb{R}^d$, which outputs $\\hat{\\psi_n}(\\boldsymbol{\\mu})$ s.t. the first $n$ moments of $Z\\sim Q_{\\hat{\\psi_n}(\\boldsymbol{\\mu})}$ are nearly equal to target moments $\\boldsymbol{\\mu}=(\\mu_1,...,\\mu_n)$. Then for any pair $(Q,\\hat{\\psi_n})$, {$ \\varepsilon_i:\\mathbb{R}^n\\rightarrow\\mathbb{R}$} for $i=1..n$ are moment-deviation functions if, for any valid $\\boldsymbol{\\mu}$, if $Z\\sim Q_{\\hat{\\psi_n}(\\boldsymbol{\\mu})}$, we have $|\\langle Z^i\\rangle -\\mu_i|\\leq\\varepsilon_i(\\boldsymbol{\\mu})$ for all $1\\leq i\\leq n$. \n\nWe focus on n=2, (error for the mean/variance). If we can pick $\\hat{\\psi}$ s.t. $\\varepsilon_1,\\varepsilon_2$ are small, $Q$ is flexible, as there are parameters that can roughly achieve any mean and variance. In the Gaussian case ($\\mathcal{N},\\mathbb{I}$), we have $\\varepsilon_1 =\\varepsilon_2=0$.\n\n### Proposition\n\nLet $DP$ denote the Double Poisson family. Set $\\hat{\\psi_2}(\\mu_0, \\sigma_0^2) = (\\mu_0,\\frac{\\mu_0}{\\sigma_0^2})$. Letting $\\gamma_0=\\frac{\\mu_0}{\\sigma_0^2}$, the MDFs for $(DP,\\hat{\\psi_2})$ are:\n    \\begin{align*}\n        \\varepsilon_1(\\mu_0,\\sigma_0^2) &= \\left|\\frac{\\sum_{y=0}^{\\infty}s(\\mu_0, \\gamma_0, y)(y - \\mu_0)}{\\sum_{y=0}^{\\infty}s(\\mu_0,\\gamma_0,y)}\\right|  \\\\\\\\\n        \\varepsilon_2(\\mu_0, \\sigma_0^2)&=\\left|\\frac{d(\\mu_0,\\gamma_0)\\gamma_0^{\\frac{1}{2}}\\sum_{y=0}^{\\infty}s(\\mu_0,\\gamma_0,y)-\\gamma_0(\\sum_{y=0}^{\\infty}s(\\mu_0, \\gamma_0,y)(y-\\mu_0))^2}{\\gamma_0(\\sum_{y=0}^{\\infty}s(\\mu_0,\\gamma_0,y))^2}\\right|\n    \\end{align*}\n    where: $h(z)=\\frac{e^{-z} z^z}{z!}, r(\\mu,\\gamma,z)=\\gamma(z-\\mu +z\\log\\mu -z\\log z), s(\\mu,\\gamma,z)=h(z)\\exp(r(\\mu,\\gamma,z)),$ and $d(\\mu,\\gamma)=\\gamma^{-1/2}\\left[\\sum_{y=0}^{\\infty}s(\\mu, \\gamma, y)(\\gamma(y-\\mu)^2-y)+\\sum_{y=0}^{\\infty}s(\\mu,\\gamma,y)(y-\\mu)\\right]$.\n   \nIf desired, we can provide the proof to this proposition in the follow-up response. We plot the error incurred via Efron’s estimates on a grid of target means and variances, using 100th partial sums (https://anonymous.4open.science/r/ddpn-651F/deep_uncertainty/figures/artifacts/epsilon_1.png). To see epsilon_2, change the filepath to epsilon_2.png. Except for the case of small μ, high σ², the error is essentially zero. Thus, in most settings we can treat DDPN as fully-heteroscedastic. Empirically, DDPN produces flexible, well-fit distributions (Fig. 3/4, Table 2).", "claims_and_evidence": "Overall, the claims made in the submission are supported by clear evidence, but I found that some theoretical claims are not supported by convincing mathematical arguments (see below).", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "I checked the theoretical claims and found significant flaws. Some may be fixable, but others appear more critical.\n\n- One issue concerns Proposition 3.3: the stated convergence to 0 does not seem valid under the proposed definition of the learnable attenuation loss function (Definition 3.2). A monotonically increasing function does not necessarily tend to infinity, just as a monotonically decreasing function does not necessarily tend to 0—both cases can have a constant asymptote. This flaw undermines the argument in the proof of Proposition 3.3.\n\n- Another issue concerns the derivation of the DDPN objective. The derivation in Appendix A.1 omits the normalizing constant of the Double Poisson (DP) distribution, denoted c(\\mu, \\gamma) at the beginning of Section 3. Since c(\\mu, \\gamma) is not constant with respect to (\\mu, \\gamma), the stated objective does not properly learn the parameters of a DDPN.\n\n- Additionally, most equations in Appendix A.1 fail to hold because the maxima and minima do not align due to omitted constants between successive lines. Using \\arg\\max and \\arg\\min would provide more precision. Also, the parameterization of the network f_\\Theta(x_i) is inconsistent: while a log link function is used in the main text, this log transformation is omitted in the first line of Appendix A.1.\n\n- Finally, it is unclear why the DDPN objective loss in Equation (1) of Section 3.1 does not include a summation over all training examples i = 1, \\dots, N.", "experimental_designs_or_analyses": "Owing to the previous flaws, I did not check the soundness of the experimental analyses.", "relation_to_broader_scientific_literature": "The paper builds on prior work in deep ensembles for uncertainty estimation in regression, extending heteroscedastic modeling from Gaussian outputs to count data using the Double Poisson distribution, addressing a gap in discrete uncertainty modeling and improving epistemic uncertainty quantification.", "essential_references_not_discussed": "Not that I am aware of.", "other_strengths_and_weaknesses": "I identify a weakness in Proposition 3.1, which claims that DDPN regressors are fully heteroscedastic. In reality, the proposition is derived under the moment approximations proposed by Efron (1986), where the first two moments are approximated as \\mu and \\mu / \\gamma. Given this approximation, full heteroscedasticity is unsurprising. It would be more meaningful to establish this result without relying on Efron’s approximation. I suspect that the exact Double Poisson distribution is inherently fully heteroscedastic, and proving this directly would be a more valuable contribution.", "other_comments_or_suggestions": "- The second line of the displayed equation in app C.6 is hard to parse: adding parentheses to the rhs 2nd sum would help.\n\n- There are a few (math) typos that could easily be fixed.", "questions_for_authors": "Given the identified flaws and weaknesses, I would likely revise my evaluation if the authors:\n1. Provide convincing proofs that correct the identified issues.\n2. Establish a stronger version of full heteroscedasticity for Double Poisson regressors without relying on moment approximations.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work considers the problem of non-negative integer prediction (count regression) with heteroscedastic uncertainty by fitting neural networks to the parameters of a double Poisson distribution. The resulting regressor is shown to be fully heteroscedastic with learnable loss attenuation, a property also enjoyed by Gaussian heteroscedastic regressors. To get clearer uncertainty quantification, ensembling is performed. These models are shown to be competitive through a variety of experiments. Reviewers were generally positive about the work; while not necessarily found to be paradigm-shifting, reviewers agreed with the soundness of the model, appreciated the comprehensive analysis, and the variety of experimental findings. 

The most major concerns identified during the review period have been addressed in the author rebuttal, however, some concerns remain regarding the proof of heteroscedasticity, as this was found to hold only under approximations for the moments. I agree that this is an inherent limitation of the work, but can appreciate that proving general heteroscedasticity in this case may be out of reach. I would suggest that the authors include the evidence provided in their rebuttal, but adjust their wording to acknowledge they only show "approximate heteroscedasticity" under the moment assumptions. This needs to be _very_ clear and precisely defined. This is not imposing "mild assumptions", this is showing that the approximate moments are unrestricted. With these inclusions, as well as those discussed throughout this review period, I believe this is is a good contribution to ICML.
Decision: Accept (poster)