The regression-based debiasing approach used to control for length bias in automated evaluations can be extended to address other biases by incorporating additional features into the regression model. For biases like model self-preference, we can introduce a feature that quantifies the degree of self-preference exhibited by each model. This feature can capture how often a model selects its own output as superior in pairwise comparisons. By including this self-preference feature in the regression model, we can control for and mitigate the impact of this bias on the evaluation metric.
Similarly, for biases related to the presence of lists in outputs, we can introduce a feature that indicates the list structure or complexity of the generated text. This feature can help the regression model differentiate between outputs with lists and those without, allowing for a more nuanced evaluation that accounts for the influence of list-related biases. By including such features in the regression analysis, we can effectively control for multiple biases simultaneously and produce more reliable and robust automated evaluation metrics.