What should an AI assessor optimise for?

Authors: Daniel Romero-Alvarado, Jose Hernandez-Orallo, Fernando Martínez-Plumed
Primary area: datasets and benchmarks
Keywords: Assessor Models, Predictable AI, Regression problems, Error metrics

Abstract:
An AI assessor is an external, ideally independent system that predicts an indicator, e.g., a loss value, of another AI system. Assessors can leverage information from the test results of many other AI systems and have the flexibility of being trained on any loss function: from squared error to toxicity metrics. Here we address the question: is it always optimal to train the assessor for the target loss? Or could it be better to train for a different loss and then map predictions back to the target loss? Using ten regression problems with tabular data, we experimentally explore this question for regression losses with monotonic and nonmonotonic mappings and find that, contrary to intuition, optimising for more informative losses is not generally better. Surprisingly though, some monotonic transformations, such as the logistic loss used to minimise the absolute or squared error, are promising.

Decision: reject
Reviewer scores: 5, 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1730383441534

thread_type:
official_review

<thread_object>
creator: reviewer_rvan

<content>
summary:
The paper investigates the question of what an AI assessor should optimize for. Specifically, could it be better to use something other than the target loss for the assessor. The question is considered empirically in the domain of tabular data using regression methods with a variety of methods of parameter optimization.

strengths:
- The paper is generally clearly written.
- A fair number of experiments are conducted.

weaknesses:
- The big weakness of the paper is that it is framed around assessing AI systems, but focuses on regression with tabular data. I don't see how the latter is relevant to the former.

questions:
- Given the generality of the question, why are we focusing on tabular data? 
- Line 39: please introduce the variable v. 
- Line 47 "routing(Hu" missing space. 
- I don't understand what Figure 1 is intended to illustrate. On the picture there are "target assessor" and "proxy assessor" but the caption text doesn't clearly define these. It would be helpful to see an example that illustrates the notion of better that the authors appear to be aiming for? Also, when might this be true?  
- Why are the six losses chosen the right ones? 
- If we are interested in AI broadly, why are we using decision trees, Random forests, etc for parameter optimization? 
- It seems likely that they heuristic methods used for hyper parameter optimization or tabular data or regression might really be more of the story here than anything about AI.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730676165328

thread_type:
official_review

<thread_object>
creator: reviewer_nvkt

<content>
summary:
In this paper the authors investigate the question as to whether assessor models should be trained using a proxy metric rather than the metric of interest. In doing so, they find mixed results. They also demonstrate that training assessors using more informative losses is not always better than training with uninformative versions.

strengths:
I think the paper has several strengths:
1. The explanations of concepts and methodology is quite clear. I like the use of diagrams such as Figures 1, 3, 4 and 8.
2. The number of datasets used is quite commendable. 10 seems like a sufficient number to generate some useful trends if there was a large effect at play.
3. The subject matter is of interest although I think the focus on tree-based methods restricts the utility of the paper.

weaknesses:
I do not think this paper is sufficiently novel and broadly applicable in order to warrant publicaton in ICLR.

I believe there are several weaknesses:
1. Unless I missed something, I believe all models used in the training set for the assessors are tree-based and are not neural networks. I know that tree-based methods can be better for tabular data but the most interesting application of assessors (to me at least) is neural networks. The most interesting assessor models are the ones that produce scaling laws.
2. The results of the paper do not seem particularly surprising. We know that changing the geometry of the loss landscape by using a different loss function can sometimes improve pefromance in regression. I don't think applying this logic to assessor models is paticularly interesting especially given the limited scope of exploration of the paper to tree based methods and the lack of mathematical theory.
3. Outside of intuition, there is no theory developed to explain the results. It seems that for a paper at ICLR, I would have expected more here.

questions:
There may be some assumptions in my review which are incorrect. Please correct me if I am wrong but I am assuming that:
1. There are no neural network models which the assessors are applied to but neural networks are used to do asessing.
2. There are no proofs or more formal justifications for the relationships observed

I have a few other questions / requests:
1. Could the authors replace the figures with PDFs or SVGs intead of what is currently being used (easier to zoom in on)?
2. Do the authors think that there might be different relationships in the case of assessors applied to predicing things about over-parameterised models?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730688218494

thread_type:
official_review

<thread_object>
creator: reviewer_hkoa

<content>
summary:
The paper studies the effects of training different prediction models on proxy assessors instead of the target metric. The study focuses on regression problems and tabular features. Results show that training for more informative losses is not always better. In addition, the paper provides a decomposition of results on different directions of pairwise combinations of loss functions.

strengths:
S1: The exploration of what loss functions are most effective for a target lost is generally important and can be informative if best choices are known and documented for practitioners.

S2: The work uses a variety of tabular models and datasets.

weaknesses:
W1: The work engages only with regression models, which makes the scope somewhat narrow.

W2: The work does show some interesting outcomes, with some of them summarized in Figure 8. The results are however exclusively explained from an empirical perspective only. There is no theoretical discussion about for example how ML optimization and different optimization algorithms may interfere and interact with the results. For example, the fact that signed simple error -> signed squared error is green (proxy easier), may be explained by optimization difficulties. For this one in particular, it would be useful to experiment with different optimization algorithms and settings of such algorithms. It would also be useful to understand whether things like learning rate etc have an impact on results.

questions:
Q1: Are the artifacts of this work going to be made available to the public (e.g. code)?

Q2: Are there any findings or insights that are specific to some of the datasets or do all results apply to all datasets individually? It would be useful to have some breakdown per datasets and see if there are any characteristics of the data that strengthen the results (or otherwise make them less applicable).

Q3: Which optimization algorithms did you use?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730733901106

thread_type:
official_review

<thread_object>
creator: reviewer_n3pr

<content>
summary:
This paper explores the efficacy of training assessor models, which are predictive systems designed to estimate the performance of other models (base systems) on various metrics before deployment. The authors question the common assumption that directly optimizing assessors for a specific validity metric always leads to the best outcomes for that metric. They investigate this through a series of experiments involving regression problems, comparing the performance of assessors trained on different error metrics, including absolute error, squared error, and logistic error. The results reveal that optimizing for a proxy metric can sometimes outperform direct optimization for the target metric, with logistic error frequently yielding the best performance.

strengths:
1. **Innovative Approach:** The paper addresses a critical gap in the understanding of how assessors can be trained, providing a fresh perspective on the relationship between the optimization of assessor models and the performance metrics of interest.
2. **Comprehensive Experimental Design:** The authors conduct several experiments that systematically evaluate the impact of different training metrics on assessor performance, leading to insightful findings.
3. **Interesting Findings:** The consistent superiority of logistic error in various situations prompts further investigation into the characteristics of error distributions, which is a valuable contribution to the field.

weaknesses:
1. **Limited Metric Scope:** While the paper examines several regression metrics, it may benefit from a broader exploration of other types of metrics across different applications, such as classification or ranking tasks, to enhance the generalizability of the findings.
2. **Theoretical Foundation:** The paper would be strengthened by a more robust theoretical explanation of why certain metrics outperform others in the context of assessor training, particularly for the surprising results related to logistic error.
3. **Clarification Needed on Assessor Application:** The practical applications of assessors in real-world scenarios could be discussed in greater detail, including how to implement these findings in various domains, such as healthcare or autonomous systems.

questions:
- Can the authors elaborate on the specific characteristics of the logistic error that made it the most effective metric across the experiments?
- How might the findings change if applied to more complex models or different domains? Are there limitations in the current experimental setup that could affect generalizability?
- It would be beneficial for the authors to provide concrete examples of how their findings could inform the design of assessor models in practice.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732741290498

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Authors' rebuttal

comment:
We are thankful for the comments received but if these scores are kept we will likely withdraw the paper in order to fully implement the suggested changes (i.e., increase the scope to other kinds of tasks and methods, include dataset-level results and develop some theoretical analysis). We could do some of them for the camera ready deadline, but not during the rebuttal period.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper investigates whether AI assessors should optimize directly for a target loss or a proxy loss with a transformation to the target loss. The authors conduct experiments on regression tasks with tabular data, comparing various loss functions, including squared error, absolute error, and logistic loss. The key finding is that optimizing for proxy losses, particularly logistic loss, can yield better performance than direct optimization, challenging conventional assumptions.

**Strengths**  
- The paper poses an innovative and practically relevant question regarding the optimization of assessor models
- Comprehensive experimental evaluation across ten datasets
- Interesting findings showing that logistic loss can serve as an effective proxy for assessor models

**Weaknesses**  
- The study is limited to regression tasks with tabular data (with restricted generalizability)  
- The paper lacks theoretical explanations
- The choice of tree-based models for assessors limits applicability in more general scenarios
- The practical relevance of the findings in real-world scenarios needs further elaboration.

In summary, while the paper presents an interesting and well-motivated problem with promising empirical findings, its limited scope, lack of theoretical insights, and restricted applicability weaken its overall impact.