Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Summary:

This paper presents a new network architecture for learning a regression of probability distributions.

The distribution output from a given node is defined in terms of a learned conditional probability function, and the output distributions of its input nodes. The conditional probability function is an unnormalized distribution with the same form as the Boltzman distribution, and distributions are approximated from point estimates by discretizing the finite support into predefined equal-sized bins. By letting the conditional distribution between nodes be unnormalized, and using an energy function that incorporates child nodes independently, the approach admits efficient computation that does not need to model the interaction between the distributions output by nodes at a given level.

Under these dynamics and discretization, the chain rule can be used to derive a matrix of gradients at each node that denotes the derivative of the discretized output distribution with respect to the current node's discretized distribution. These gradients are in turn used to calculate updates for the network parameters with respect to the Jensen Shannon divergence between the predicted distribution and a target distribution.

The approach is evaluated on three tasks, two synthetic and one real world. The baselines are the state of the art triple basis estimator (3BE) or a standard MLP that represents the output distribution using a softmax over quantiles. On both of the synthetic tasks --- which involve predicting gaussians --- the proposed approach can fit the data reasonably using far fewer parameters than the baselines, although 3BE does achieve better overall performance. On a real world task that involves predicting a distribution of future stock market prices from multiple input stock marked distributions, the proposed approach significantly outperforms both baselines. However, this experiment uses 3BE outside of its intended use case --- which is for a single input distribution --- so it's not entirely clear how well the very simple proposed model is doing.

Notes to authors:

I'm not familiar with 3BE but the fact that it is used outside of its intended use case for the stock data is worrying. How does 3BE perform at predicting the FTSE distribution at time t + k from the FTSE distribution at time t only? Do the multiple input distributions actually help?

You use a kernel density estimate with a Gaussian kernel function to estimate the stock market pdf, but then you apply your network directly to this estimate. What would happen if you built more complex networks using the kernel values themselves as inputs?

Could you also run experiments on the real-world datasets used by the 3BE paper?

What is the structure of the DRN that uses > 10^3 parameters (from Fig. 4)? The width of the network is bounded by the two input distributions, so is this network just incredibly deep? Also, is it reasonable to assume that both the DRN and MLP are overfitting the toy task when they have access to an order of magnitude more parameters than datapoints.

It would be nice if section 2.4 was expanded to actually define the cost gradients for the network parameters, either in line or in an appendix.
review_title: This paper introduces a simple but neat network architecture for representing probability distributions that is supported by preliminary experiments. However, the paper would be much stronger with more robust evaluation on real-world data.

final_score: rating: 7: Good paper, accept
review_content: This is an intriguing paper on running regressions on probability distributions: i.e. a target distribution is expressed as a function of input distributions. A well-written manuscript, though the introduction could have motivated the problem a little better (i.e. why would we want to do this). The novelty in the paper is implementing such a regression in a layered network. The paper shows how the densities at each nodes are computed (and normalised). Optimisation by back propagation and discretization of the densities to carry out numerical integration are well explained and easy to follow. The paper uses three problems to illustrate the idea -- a synthetic dataset, a mean reverting stochastic process and a prediction problem on stock indices.  
My only two reservations of this paper is the illustration on the stock index data -- it seems to me, returns on individual constituent stocks of an index are used as samples of the return on the index itself.  But this cannot be true when the index is a weighted sum of the constituent assets.  Secondly, it is not clear to me why one would force a kernel density estimate on the asset returns and then bin the density into 100 bins for numerical reasons -- does the smoothing that results from this give any advantage over a histogram of the returns in 100 bins?
 
review_title: Distribution Regression Network

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper considers distribution to distribution regression with MLPs.  The authors use an energy function based approach.  They test on a few problems, showing similar performance to other distribution to distribution alternatives, but requiring fewer parameters.

This seems to be a nice treatment of distribution to distribution regression with neural networks. The approach is methodological similar to using expected likelihood kernels.  While similar performance is achieved with fewer parameters, it would be more enlightening to consider accuracy vs runtime instead of accuracy vs parameters.  That’s what we really care about.  In a sense, because this problem has been considered several times in slightly different model classes, there really ought to be a pretty strong empirical investigation.  In the discussion, it says 
“For future work, a possible study is to investigate what classes of problems DRN can solve.”  It feels like in the present work there should have been an investigation about what classes of problems the DRN can solve.  Its practical utility is questionable.  It’s not clear how much value there is adding yet another distribution to distribution regression approach, this time with neural networks, without some pretty strong motivation (which seems to be lacking), as well as experiments.  In the introduction, it would also improve the paper to outline clear points of methodological novelty.  

review_title: Potentially promising paper but difficult to see the practical significance

Meta-review: The paper proposes a method to map input probability distributions to output probability distributions with few parameters. They show the efficacy of their method on synthetic and real stock data. After revision they seemed to have added another dataset, however, it is not carefully analyzed like the stock data. More rigorous experimentation needs to be done to justify the method.
Decision: Reject