Paper:
Compact Representation of Uncertainty in Clustering
For many classic structured prediction problems, probability distributions over the dependent variables can be efficiently computed using widely-known algorithms and data structures (such as forward-backward, and its corresponding trellis for exact probability distributions in Markov models). However, we know of no previous work studying efficient representations of exact distributions over clusterings. This paper presents definitions and proofs for a dynamic-programming inference procedure that computes the partition function, the marginal probability of a cluster, and the MAP clustering-all exactly. Rather than the N th Bell number, these exact solutions take time and space proportional to the substantially smaller powerset of N . Indeed, we improve upon the time complexity of the algorithm introduced by Kohonen and Corander [11]  for this problem by a factor of N . While still large, this previously unknown result is intellectually interesting in its own right, makes feasible exact inference for important real-world small data applications (such as medicine), and provides a natural stepping stone towards sparse-trellis approximations that enable further scalability (which we also explore). In experiments, we demonstrate the superiority of our approach over approximate methods in analyzing real-world gene expression data used in cancer treatment.

Review:
Post-rebuttal: thank you for clarifying / correcting the figure with experimental results.   Main idea:   Exactly compute the partition function of a multicut / correlation clustering problem using dynamic programming.   Strengths:   Signed graph partitioning is an extremely versatile tool, so any theoretical or practical progress is welcome and may (eventually) have large impact. Studying a Gibbs distribution over solutions rather than just the minimal energy / MAP solution is of fundamental interest with a view to passing meaningful uncertainty estimates on to downstream processing. In essence, I think this is meaningful work.   The paper is clearly written and thorough (though I did not check all mathematical details).   Overall, I think the work has more substance than many NIPS papers I see, which is why I count it among the better 50%.   Weaknesses:   The method is of limited practical usefulness, with exact computations limited to 25 elements or so.   The comparison with approximate methods (notably Kwikcluster) is not really fair. The state of the art to compare to is probably a set of multicut MAP solutions obtained on noisy versions of the original edge weights (ref. 7). This is not difficult to implement in practice and should give better results.   Comments:   I don't think the title summarizes the contribution well. There is not really much study or discussion of uncertainty, beyond the use of marginals. While marginals are a compact representation, the trellis certainly is not.   Similarly, section 2 is entitled "Uncertainty in clustering" even though that topic is not discussed there at all.   The core result which makes dynamic programming applicable is fact 1, proved in Appendix E. This being at the very heart of the contribution, I would recommend moving an extended version of Appendix E into the main text, at the expense of other parts. For instance, the MAP solution (section 3.2) which does not recur to branch & bound, is unlikely to be competitive with SOA ILP solvers in conjunction with efficient column generation. In fact, I wouldn't mind having the parts on sparse trellises omitted altogether. Without detailed analysis of what choice of trellis gives what kind of approximation quality, this section is not so useful. Bottom line: Fact 1 is all-important for this paper, and merits more extensive explanation / discussion in the main text.   To give a better feeling for the data used, it would be good to show both the MAP clustering as well as all the marginals (in an 11x11 matrix).   Fig 3: One would expect 11*(11-1)/2 = 55 entries, but there are fewer points in the scatter plot?    Minor comments:   Changing the definition of energy such that higher is preferable to lower (line 97) is not a good idea.   The data used in the experiments is not a very convincing example for correlation clustering (coordinates in some space, with repulsive interactions obtained "artifically"; in particular, this choice induces a spherical shape bias on the resulting clusters).  typo: elemnts line 241: unresolved reference    Note on my confidence score:  I did not check the maths; and I cannot guarantee that dynamic programming has not been used previously to compute the exact partition function of clusterings. I am confident of my other comments.