Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The authors present a decentralized algorithm for test-time model combination (collective prediction) wherein each party has an private model and dataset, and only has query access to other party's models. The algorithm is a novel application of DeGroot's consensus model which models how several agents iteratively influence each other's predictions based on mutual trust. The authors use the mean squared error on a local set of points as a proxy for mutual trust: each party queries each other party's models using points neighboring the test instance from its private dataset and measures the MSE on this set. The authors also show that the algorithm easily lends itself to a decentralized jackknife algorithm.

The authors prove several intuitive properties of their algorithm and prove that, asymptotically, it recovers an optimal inverse-MSE weighting. Empirically, they show that the method outperforms a naive equally-weighted average of the classifier predictions on synthetic and real data. Through synthetic experiments, the authors show that the proposed algorithm is particularly beneficial when there is significant heterogeneity in the private data or learners because of the adaptively computed weighting (based on MSE in a neighborhood around the test instance).
main_review: This is a well-written, well-executed paper. The authors present a novel algorithm and characterize it both theoretically and through experiments. The experiments highlight where the model is expected to work well and where it reduces to existing baselines. The authors also provide an excellent survey or related work.

My main concern is the motivation for this learning setup: the key motivation for decentralized collective prediction is to keep allow parties to keep their data and models private. However, as parties use their own data to query other models, they are in affect sharing their dataset piece by piece with others. Is it possible to share data between parties in a differentially private way?

Minor typos:

Lines 116-119: recommend using \(x\) instead of \(x'\).
limitations_and_societal_impact: The author should mention or address the limitation regarding agents leaking private data by providing query access mentioned above.

I do not foresee any negative societal impact from this work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper addresses test-time collective prediction. As far as I know, this is an underexplored topic, yet it is interesting. The paper proposes a decentralized consensus algorithm for multi-agent model aggregation. The paper includes both a nice theoretical analysis and empirical validation. Although I did not check the proofs, the theoretical analysis seems sound.
main_review: Overall, I tend to favor accepting this paper, so I am tentatively voting with a 7. However, I have some concerns below that I would like the authors to clarify.

KEY CONCERN:
I would like the authors to clarify one aspect of the proposed algorithm. It seems like the trust model relies on the agent's querying each other on a validation subset of their private data. How does this not violate the premise that the agents are not willing to share data: "[the mechanism] does not require any data sharing, communication of model parameters, or labeled validation data."? 

With respect to the above, I might recommend that the authors modify the protocol such that the collaborative agents use cryptographic tools like secure 2-party communication or FHE in order for agent i and agent j to agree on MSE_ij without agent i learning what agent j's model predicts on any particular data point in D_i(x') and without agent j learning what any particular data point in D_i(x') is. By only agreeing on MSE_ij, agent i and j are sharing the minimum amount of information necessary for the protocol to work. 

The authors should either (1) clarify why this type of data sharing still is acceptable given their own premise,  (2) modify the protocol as described above (or any other way that achieves the objective laid out in their premise), or (3) clarify how I am misunderstanding the protocol if there is no data sharing.
limitations_and_societal_impact: Yes, included social impact.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper proposes an interesting approach based on DeGroot’s consensus model for combining predictions of multiple pre-trained models to collectively make a final prediction. The proposed method is useful in cases where data pooling, model sharing, or external validation is difficult. Experiments show that the proposed method is better than classical baselines of model averaging under heterogeneous conditions, without assuming access to additional labeled data for validation or learning a meta-model. 

main_review: Pros:
- The motivation of collective prediction from multiple independent models without model/weight sharing has a lot of practical relevance. Thus, this paper might be of interest to a broader community.
- The proposed method does not require additional validation data for learning aggregation to do collective predictions and scales well with the number of models in the ensemble.
- Sufficient and convincing experiments in the main paper and appendix demonstrating the usefulness of the proposed method for regression tasks. The authors also analyze the conditions when the proposed method may not be significantly better than simple ensembling approaches.

Limitations:
- Experiments are limited to the task of regression and involve only simple record-based datasets. The experimental section could be made stronger with more complex regression/classification tasks. 
- Computing trust scores for each test example might affect the latency in comparison to other simpler baselines. 

Questions:
- Is there any specific reason why classification tasks were not considered?



limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper considers collective prediction at test time, and proposes a novel decentralized mechanism by leveraging each agent's pretrained model and data without sharing them with other agents, inspired by human consensus in social science. In particular, for each test example, the proposed approach computes trust scores between two agents and aggregate the scores for a final collective prediction via DeGroot aggregation. In theory, this paper proves that the proposed DeGroot consensus mechanism is asymptotically optimal under some conditions. The efficacy of the proposed approach is empirically demonstrated by showing that the proposed scheme is better than an un-weighted consensus baseline. 

The major contributions include the following: 
(1) this paper introduces the concept of "mutual trust scores among agents" in social science (which is similar to PageRank algorithm in data science) for collective prediction at test time, 
(2) the proposed approach is advantageous over existing methods since it does not require agents to share data or model parameters, which are beneficial in terms of communication cost and privacy perspective, 
(3) the proposed approach is advantageous over existing methods since it does not require additional hold-out validation set to aggregate predictions of each agent, 
(4) the proposed approach is advantageous over existing methods since it does not have any restriction on training procedure and a trained model of each agent, 
(5) this paper theoretically proves the optimality of the proposed scheme (i.e., DeGroot consensus mechanism) is optimal under some conditions, and
(6) this paper empirically demonstrates that the proposed approach is superior than the baseline approach (which does not weight the predictions of agents). 
main_review: **Originality:**
This paper introduces, for the first time, the known DeGroot consensus model in social science to address collective prediction at test time in the "decentralized" environment. PageRank can be considered as one application of the DeGroot consensus model, but it is not specifically used for collective predictions at test time. 

I think the key claim is that the proposed approach has its "decentralization" properties, e.g., each agent does not need to share its data and model to conclude the final, collective prediction via predictions from multiple agents. But, one concern in computing mutual scores in Eq (2) is that each agent actually needs to see other agents' local data (assuming models are not shared); in particular, Eq (2) means that the i-th agent computes the mutual scores of other agents by evaluating the j-th agent prediction over the i-th agent data, which implies the i-th agent data needs to be transferred to the j-th agent for the prediction. 
- Considering this, the paper's claim that each agent's data is not shared (e.g., line 35) could be overstated, and its unclear that the proposed approach addresses learning/inference issues in decentralized settings (since actually data sharing happens).
- Assuming each agent need to transmit 1% of its data (as mentioned in line 611), i.e., |D_i(x')| = 0.01|D_i|. in Eq (2), each agent can obtain most of other agent's covariate data about 100 inference iterations. This is equivalent to having one agent all data without labels across agents, then what's the benefit of the proposed approach compared to conventional semi-supervised learning approaches, i.e., one agent just trains a model using its labeled data and unlabeled data from other agents, or the ensemble of agent's models, each of which is trained using semi-supervised learning?


**Quality:**
This paper looks technically sound, and empirically well-justified on the importance of estimating "weight" of agent's predictions in aggregation.


**Clarity:**
This paper is clearly-written. 
- line 609: is "1 − 10%" typo?

**Significance:**
I guess the DeGroot consensus idea in collective prediction could fertilize interesting ideas in other domains (e.g., classification or uncertainty estimation under distribution shift, where ensemble of models could be effective, but the models are combined with equal-weights).
limitations_and_societal_impact: Yes.
review_title: null

Meta-review: Strengths:
- Novel approach and application of the DeGroots consensus model
- Thorough theoretical analysis
- Effective empirical demonstration
- As stated by one reviewer: “well-written, well-executed”

Weaknesses (mild):
- Motivation for decentralized collection prediction could be improved
- The way in which privacy and data sharing properties of the approach are described

Summary:

Reviewers were unanimous in their view that this is a strong and complete submission. This was evident both in the original reviews as well as in correspondences with the authors. Authors are encouraged to discuss in the final version issues raised by the reviewers, including (i) how privacy and data sharing are considered, (ii) experimenting with classification (in addition to regression), and (iii) possible future relations to crypto and differential privacy.

Decision: Accept (Poster)