Title: Streaming algorithms for evaluating noisy judges on unlabeled data - binary classification.
Venue: NeurIPS 2023
Accepted: no
Keywords: unlabeled data

evaluation

ensembles

stream algorithms

algebraic geometry
Abstract:  The evaluation of noisy binary classifiers on unlabeled data is treated as a
  streaming task - given a data sketch of the decisions by an ensemble, estimate
  the true prevalence of the labels as well as each classifier's accuracy on them.
  Two fully algebraic evaluators are constructed to do this. Both are based on the assumption that
  the classifiers make independent errors on the test items. The first is based on
  majority voting. The second, the main contribution of the paper, is guaranteed
  to be correct for independent classifiers. But how do we know the classifiers
  are error independent on any given test? This principal/agent monitoring paradox
  is ameliorated by exploiting the failures of the independent evaluator to
return sensible estimates.  Some of these failures can be traced to producing
algebraic versus real numbers while evaluating a finite test. A
  search for nearly error independent trios is empirically carried out on the 
 \texttt{adult}, \texttt{mushroom}, and \texttt{twonorm} datasets by using
these algebraic failure modes to reject potential evaluation ensembles as
too correlated. At its final steps, the searches are refined by constructing
a surface in evaluation space that must contain the true value point.
The surface comes from considering the algebra of arbitrarily correlated
classifiers and selecting a polynomial subset that is free of any correlation variables.
Candidate evaluation ensembles are then rejected if their data sketches produce
independent evaluation estimates that are too far from the constructed surface.
The results produced by the surviving evaluation ensembles can sometimes be as good as 1\%. 
But handling even small amounts of correlation remains a challenge. A Taylor expansion
of the estimates produced when error independence is assumed but the classifiers are, in fact,
slightly correlated helps clarify how the proposed independent evaluator has algebraic `blind spots'
of its own. They are points in evaluation space but the estimate of the independent evaluator
has a sensitivity inversely proportional to the distance of the true point from them.
How algebraic stream evaluation can and cannot help when done for safety or economic 
reasons is briefly discussed.

Abstract

The evaluation of noisy binary classifiers on unlabeled data is treated as a streaming task - given a data sketch of the decisions by an ensemble, estimate the true prevalence of the labels as well as each classifier's accuracy on them. Two fully algebraic evaluators are constructed to do this. Both are based on the assumption that the classifiers make independent errors on the test items. The first is based on majority voting. The second, the main contribution of the paper, is guaranteed to be correct for independent classifiers. But how do we know the classifiers are error independent on any given test? This principal/agent monitoring paradox is ameliorated by exploiting the failures of the independent evaluator to return sensible estimates. Some of these failures can be traced to producing algebraic versus real numbers while evaluating a finite test. A search for nearly error independent trios is empirically carried out on the adult , mushroom , and two-norm datasets by using these algebraic failure modes to reject potential evaluation ensembles as too correlated. At its final steps, the searches are refined by constructing a surface in evaluation space that must contain the true value point. The surface comes from considering the algebra of arbitrarily correlated classifiers and selecting a polynomial subset that is free of any correlation variables. Candidate evaluation ensembles are then rejected if their data sketches produce independent evaluation estimates that are too far from the constructed surface. The results produced by the surviving evaluation ensembles can sometimes be as good as 1%. But handling even small amounts of correlation remains a challenge. A Taylor expansion of the estimates produced when error independence is assumed but the classifiers are, in fact, slightly correlated helps clarify how the proposed independent evaluator has algebraic 'blind spots' of its own. They are points in evaluation space but the estimate of the independent evaluator has a sensitivity inversely proportional to the distance of the true point from them. How algebraic stream evaluation can and cannot help when done for safety or economic reasons is briefly discussed.

1 Introduction

Streaming algorithms compute sample statistics of a data stream. A data sketch , selected to fit the 29 sample statistic one wants to compute, is updated every time a new item appears in the stream. A 30 simple example of such a streaming algorithm is the use of two counters to compute the average 31 value of a stream of numbers, 32

(2)

1

B

B

β

α

α

β

α

β

α

α

B

β

β

α

α

B

β

α

α

B

β

β

β

α

α

α

β

α

β

2

B

α

α

α

β

β

α

α

α

α

α

β

α

β

β

α

α

α

α

α

α

β

α

α

α

B

α

α

3

α

B

α

α

B

α

α

α

α

α

α

β

α

α

α

α

β

B

α

α

β

α

α

α

α

β

β

B

β

β

true

β

β

β

β

β

β

α

α

α

α

β

B

α

β

α

β

β

α

β

β

β

β

α

β

α

α

β

β

β

α

label

The first integer counter tallies how many numbers have been observed so far. The second keeps 33 a running total of the observed values. This data sketch is then used to compute the mean of the 34 observed stream, 35

Two things are notable about this simple algorithm. It only uses observed variables, and its computa36 tion is purely algebraic with them. There are no free parameters to tune or know beforehand. This 37 paper discusses evaluation algorithms for binary classifiers that act similarly. 38

An evaluation on a finite set of labeled data is defined by the sample statistics it computes based on 39 knowledge of the true labels. Unlabeled evaluation , the problem considered here, is the estimation of 40 the values of these same sample statistics when the true labels are not known. This paper looks at 41 how to do so when all we have are statistics of the decisions the members of an ensemble of noisy 42 binary classifiers make - it is a 'black-box' algorithm. There are 2 n possible prediction events when 43 we are concerned only about per-item evaluation statistics for n binary classifiers. Figure 1 shows 44 how an example of a stream labeled by three classifiers. To keep track of the decisions events at the 45 per-item level, only 8 integer counters are needed. 46

47

48

49

50

51

52

53

54

For three binary classifiers, a basic set of evaluation statistics is defined by the prevalence of one of the labels, say label α , ˆ P α , and the label accuracies, ˆ P i,α and ˆ P i,β , for each of the classifiers. Note that these are per item statistics. They cannot quantify performance across items in the stream. Theorem 1 asserts that these variables are complete to explain the data sketches formed from the aligned decisions of independent classifiers. If we knew the value of the value of these evaluation statistics, we can predict exactly the value of the per-item data sketch counters. The challenge in unlabeled evaluation is to go the other way - to obtain estimates of the basic evaluation statistics starting from the data sketch.

55

56

57

58

59

60

61

62

63

64

65

66

67

68

Two evaluators for binary classifiers that are fully algebraic are built. The first is based on majority voting (MV). It decides what the correct answer key must be for the test. This makes it nearly impossible for it to return a correct answer even when its assumptions are satisfied. Even worse, it always provides seemingly correct estimates even when its assumptions are violated. The second evaluator is fully inferential. It never decides what the true label is for any items. Theorem 2 proves that this approach will correctly trap the true evaluation point to just two point candidates in evaluation space. Unlike the MV evaluator, it can return obviously incorrect estimates.

This paper assumes that these streaming evaluators are being deployed in an environment that has a principal/agent monitoring paradox. Evaluation ensembles, like decision ensembles, work best when they are independent in their error. Just as one would not want to incur the technical debt of having an ensemble of classifiers that always agreed, it makes little sense to deploy evaluation ensembles that are highly correlated. This raises two challenges when working on unlabeled data - how do we find these error independent evaluation trios, and how do we know, on any given evaluation, that they are still independent, or nearly so?

The approach taken here to ameliorate this monitoring paradox is that the failures of the independent 69 evaluator can be used to exclude evaluation trios that are too correlated. The independent evaluator, 70 by construction, is deterministic and always returns algebraic numbers. But its answers do not always 71 make sense. We can detect this because we have prior knowledge about what seemingly correct 72 evaluation estimates look like. 73

74

1.1 Properties of the true evaluation point

All the basic evaluation statistics are integer ratios by construction. For example, ˆ P α , must be the ratio 75 of two integers. Its numerator is some integer between 0 and the size of the test. The denominator, 76 the size of the test. By construction, their ratio lies inside the unit interval. Similar considerations 77 apply to any of the label accuracies for the classifiers - their true value must be an unknown integer 78 ratio in the unit interval. 79

Seemingly correct estimates are estimated values that seem to be correct because they have this real, 80 integer ratio form. Estimates that do not have this form are obviously incorrect. The naive evaluator 81 built using majority voting by the ensemble always returns seemingly correct answers, never alerting 82 its users that it is wrong and its evaluation assumptions do not apply on a given test. 83

84

85

86

87

88

The independent evaluator constructed from Theorems 1 and 2 is not like that. It fails, with varying degrees, to return seemingly correct estimates when the assumptions of a test are violated. The failures vary in their severity. The empirical hypothesis explored here is that their severity are indicative of the magnitude of the unknown decision correlations that are needed to correctly predict the observed data sketch.

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

1.2 A sample definition of decision error independence

Theorem 2 provides a closed, algebraic solution to the evaluation variety defined by the data sketch of error independent classifiers. The variety, the geometrical object in variable space that contains all points that satisfy a polynomial ideal, consists of just two points for these classifiers. But as noted above, independent evaluation ensembles are rare.

Handling correlation correctly requires that we introduce new evaluation variables to quantify it. Theorem 3 provides a a constructive proof of how to connect the data sketch of correlated classifiers to polynomials using these new correlation statistics plus those in the basic evaluation set. The evaluation variety, the set of values for the evaluation statistics that solves the polynomials, is not solved here for correlated classifiers. Nonetheless, it can be proven that the variety exists, its exact shape in evaluation space to be determined in future work. Nonetheless, a partial characterization of its shape is possible because of Theorem 3. The same process that solved the polynomial system for independent classifiers in Theorem 2 achieves a partial disentanglement of the variables when they are correlated. This defines a subset of the generating set that defines a surface computable without any knowledge of the correlations between the classifiers. This surface is not the evaluation variety but is guaranteed to contain it. This surface is used in the experiments to construct nearly independent evaluation ensembles.

1.3 Previous work and related topics

A mathematical treatment of the correctness of the decisions made when correct labels are assigned to majority voting dates back to Condorcet's analysis of the correctness of human juries during the French Revolution. But it was not until almost two centuries later that a mathematical treatment of using juries, this time human doctors, to evaluate themselves using only their aligned decisions was published by Dawid and Skene [1]. It proposed a probabilistic solution to evaluation by minimizing a likelihood function using the EM algorithm. This work was followed up in the early 2010s by a succession of papers in the NeurIPS conferences that took a Bayesian approach to constructing stream evaluator ([2], [3], [4], [5], [6]). Applications to evaluating workers in crowd-sourcing platforms was a big motivator for some of this research. The algebraic methodology proposed here would be economically impractical in such applications since it requires all the classifiers labeling every item in the stream.

The algebraic approach proposed here is closest to another probabilistic method, one proposed by Parisi et al. [7]. Rather than minimizing a likelihood function, it considers the spectral properties of matrices created by moments of the observed decisions. By hypothesizing hidden distributions, it then tries to carry out a matrix decomposition that eventually gives evaluation estimates. In contrast, the independent evaluator proposed here is purely algebraic. It invokes no assumptions about distributions. Nonetheless, Theorems 1 and 2 discussed here should be contrasted with Theorem 1 in the paper by Jaffe et al. [8], a solution for distribution-independent classifiers. The Supplement details the mathematical similarities and differences between the two solutions.

But research on direct evaluation seems to have waned since the 2010s. The 1st NeurIPS workshop 126 on AI safety occurred last year and its RFP had a section on monitoring and anomaly detection that 127 does not cite any of the above work [9]. Instead, research has focused on other aspects of monitoring 128 that are important for AI safety - for example, carrying out risk minimization computations given 129 unknown operating points for an ensemble as was done by Steinhardt et al. in [10]. 130

131

132

133

134

135

Although stream algorithms do not usually calculate hidden knowledge statistics in a stream, some do. Good-Turing frequency smoothing [11] is one such algorithm. Its core intellectual idea - that you can estimate the probability of seeing hitherto unseen types of items using only the count for observed types - is used by LLMs whenever they want to estimate token sequences that were never observed during training.

Finally, most of the mathematical tools used in this paper come from algebraic geometry (AG) [12]. 136 The use of algebraic concerns to study statistical problems was pioneered by Pistone et al. [13]. The 137 topic is known as algebraic statistics . Like most of statistics, it is focused mostly on topics related to 138 inferring distributions. This paper uses AG to estimate sample statistics. 139

140

2 Using majority voting to evaluate noisy classifiers

The correctness of decisions made by an ensemble that majority votes (MV) depends, roughly 141 speaking, on two things. They must be error independent, and their labeling accuracies must be 142 greater than 50% on each label. Item labels decided by majority voting will be correct more often 143 than not if these conditions are met. This section details how a naive algebraic stream evaluator can 144 be built on the basis of this decision algorithm. It works by imputing the correct labels or answer key 145 for the observed items. 146

The integer counters of the per-item data sketch can be trivially turned into frequency variables. Each 147 of the counters in a decision sketch tallies how often a decision event has been seen in the stream so 148 far. There are only 8 decision events when considering the per-item decisions of three classifiers. The 149 sum of their fractional frequencies, f ℓ 1 ,ℓ 2 ,ℓ 3 , must sum to one, 150

151

152

153

154

155

156

The logic of MV evaluation is straightforward. The true label is given by MV, therefore the prevalence of a label is equal to the frequency that label was the majority vote. The estimate for the α label is thus a simple linear equation of these ensemble decision frequencies,

Similarly, we can write down algebraic formulas of the decision frequencies for each classifiers label accuracy. For classifier 1, those estimates are,

The MV evaluator considers a classifier wrong if it votes against the majority.

These are algebraic functions of the frequencies derived from the data sketch, there are no free 157 parameters. In addition, the estimates returned by them are always seemingly correct. All the MV 158 estimates of prevalence and label accuracies are integer ratios inside the unit interval. The MV 159 evaluator will never be able to alert its user that its assumptions are incorrect even when they are 160 wildly off the mark. 161

162

2.1 The drawbacks of evaluating by deciding

Decision and inference are traditionally recognized as separate concerns in Machine Learning. 163 Avoiding decisions and its hard choices until they are absolutely required typically leads to better 164 performance. So it is here. Making a hard choice on the true label is going to be incorrect on some 165 unknown fraction of the events that produced a particular voting pattern. Some of the times the 166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

ensemble voted ( α , β , α ) it could have been a β item, not an α one. This is expressed by the following equation,

The number of times we saw a voting pattern is equal to the sum of times the items were α plus the times it was β . For any voting pattern by the ensemble, both labels are possible for any one item, no matter what the majority says. Zeroing out one term in this sum is an approximation. The supplement works out how this decision step means that the MV evaluator is hardly ever right even though it always seems so. Fixing this naive MV evaluator is easy - include both terms when expressing data sketch frequencies. Carrying out evaluation with these full equations is much harder but leads to an evaluator that is guaranteed to be correct when its assumptions are true.

3 Fully inferential evaluation of sample independent binary classifiers

Systems of equations can be wrong. Care must also be taken that they they define objects that exist so as to avoid making statements about non-existing entities. The two mathematical objects of concern here are systems of polynomial equations and the geometrical objects consisting of the points that solve them. The following theorem does this for error independent classifiers. It establishes that the basic evaluation statistics are sufficient to explain all observed data sketches created by them.

Theorem 1. The per-item data sketch produced by independent classifiers is complete when expressed as polynomials of variables in the basic evaluation set.

These polynomial expressions of the data sketch form a generating set for a non-empty polynomial ideal, the evaluation ideal . The evaluation variety , the set of points that satisfy all the equations in the ideal is also non-empty and contains the true evaluation point.

Sketch of the proof. The assumption that true labels exist for the stream items underlies the algebraic work required for the proof. The correct label of each item can be encoded in indicator functions, ✶ s ( ℓ ) , that are 1 if its argument is the correct label for item s, and zero otherwise. The existence of a true label for an item s is then equivalent to the equation,

✶ s ( α ) + ✶ s ( β ) = 1 . (17) Consider now the first term in Equation 8, #( ℓ 1 , ℓ 2 , ℓ 3 | α ) , as it relates to, say, the decisions event ( α , β , α ). By using the predicted labels by the classifiers for a given items s , the following expression is exactly equal to one precisely at those decisions events but zero otherwise,

✶ s ( ℓ 1 ,s ) (1 -✶ s ( ℓ 2 ,s )) ✶ s ( ℓ 3 ,s ) . (18) The proof of having a complete representation using the basic evaluation variables then hinges in equating the average of this equation to the variables as follows,

✶ This equality is only true for independent classifiers because we have substituted the average of 196 products of the indicator functions by products of their averages. New correlation variables are 197

introduced and then set to zero to define rigorously a sample definition of decision correlations. For 198 example, the definition of the pair correlation variable on a label is given by, 199

200

201

202

203

204

205

206

207

Setting these pair correlations to zero then guarantees that we can write averages of the product of the indicator functions for two classifiers as the product of their label accuracies. Similar considerations apply to the product of the indicators for three classifiers. The consequence is that any decision event frequency by independent classifiers is complete when written in terms of the basic evaluation statistics. All data sketches produced by independent classifiers are predicted by the basic statistics. Since the proof is constructive and starts from expressions for the true evaluation point, we know that there is at least one point that satisfies all these polynomial equations. We conclude that the evaluation variety for independent classifiers exists and it contains the true evaluation point.

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

Theorem 2 details exactly what the evaluation variety for independent classifiers must be.

Theorem 2. The polynomial generating set for independent classifiers has an evaluation variety that has two points, one of which is the true evaluation point.

Sketch of the proof. The quartic polynomials of the independent generating set are not trivial to handle. A strategy for solving them is to obtain algebraic consequences of them that isolate the variables. Using the tools of AG, this can be accomplished for independent classifiers. Solving their polynomial system is accomplished by calculating another representation of the evaluation ideal, called the Gröebner basis, that does this. It can be arranged to isolate the ˆ P α in a quadratic

The coefficients a, b, and c are polynomials of the decision frequencies. Since this is a quadratic, it follows from the quadratic formula that it can only contain two solutions. This, coupled with the fact that other equations in the evaluation ideal are linear equations relating ˆ P α to ˆ P i,α or ˆ P i,β variables leads one to conclude that only two points exist in the evaluation variety of independent classifiers.

Table 1 compares the prevalence estimates of the fully inferential independent evaluator with the naive MV one. By construction, it will be exact when its assumptions apply. But unlike the naive MVevaluator, this formula can return obviously wrong estimates. The next section details how one can carry out experiments on unlabeled data to find evaluation ensembles that are going to be nearly independent.

4 Experiments with the failure modes of the independent stream evaluator

If perfect evaluations are not possible, one should prefer methods that alert us when they fail or their assumptions are incorrect. The experiments discussed here show how we can lever the self-alarming failures of the independent evaluator to reject highly correlated evaluation ensembles. There are four failure modes for the independent evaluator,

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

%observed

%observed

100

40

adult

80

mushroom

30

60

twonorm

20

40

adult

mushroom

20

10

twonorm

100

200

300

400

500

600

700

800

9001000

门

100200300400

500

600

700

800

9001000

n

The evaluation variety corresponding to the independent evaluation ideal is the empty set no points in evaluation space can zero out the equations in the evaluation ideal.

The two evaluation points contain complex numbers.

The two evaluation points lie outside the real, unit cube.

The estimated values contain unresolved square roots.

The fourth failure mode is interesting theoretically but not as practical. The Supplement details how an unresolved square root in the evaluation estimates can be used to prove that the classifiers were not error independent in the evaluation. Its theoretical interest lies in demonstrating that algebraic numbers, unlike real ones, can be used to self-alarm, in an almost perfect fashion, when its assumptions are violated.

The first three failure modes are more practical. The first set of experiments will look at how the failures can be used to estimate what test sizes are less likely to have failed evaluations when we use the independent evaluator. The second set of experiments profiles how well another rejection criteria, this time based on the polynomial generating set for correlated classifiers, can help identify nearly independent evaluations.

4.1 Rejecting highly correlated evaluation ensembles

Given a set of unlabeled data and a smaller set of labeled training data, the goal of the first set of experiments is to construct and identify nearly independent evaluation ensembles on a larger, unlabeled portion of data. This simple experimental set-up is meant to mimic a possible Auto-ML application of stream evaluation. The experiments are meant to answer the question - how big should an evaluation test be? This is done by profiling the algebraic failures as a function of test size.

Since the goal is to construct and then test if an evaluation ensemble is near enough independence to give seemingly correct estimates, a generic training protocol was applied to the three datasets studied. A training sample of 600 items was selected and the rest of the dataset was then held-out to carry out unlabeled evaluations. The rate of failures on the held-out data is then used as a guide to select test sizes that have low failure rates as observed empirically.

Independence in the candidate ensembles was maximized by training each member on features disjoint with those used by the others. Disjoint partitions of the small 600 training set, each of size 200, were then used to train each candidate ensemble. A single profiling run selected 300 disjoint feature partitions to test as the size of the held out data was changed. Averaging successive profiling runs then gives an empirical measure of the failure rates as a function of test size. Each disjoint feature partition was trained and evaluated ten times.

Figure 2 shows profiles of algebraic failures for the adult , mushroom , and twonorm dataset experiments. Figure 2a plots the percentage of feature partitions that resulted in evaluation ensembles returning seemingly correct estimates. The twonorm and adult experiments suggest that nearly independent ensembles will be easier for them than in mushroom . Figure 2b plots the percentage of

267

268

269

270

271

272

273

(a) The

adult

dataset.

(b) The

mushroom

dataset.

2.0

0.14

0.07

0.12

1.8

0.06

0.10

0.08

0.05

90'0

1.4

0.04

0.04

1.2

0.02

0.03

4.×10-8

6.×10-8

8.x10-8

1.x10-7

1.2×10-7

8.x10-8

9.x10-8

1.x10-7

1.1 ×10-7

1.2×10-7

feature partitions that never produced data sketches explainable by the independent assumption. This plot also suggests that mushroom good evaluation ensembles will be harder to find.

5 A containing variety for arbitrarily correlated binary classifiers

The rejection of evaluation ensembles that have algebraic failures is no guarantee that the remaining ones will return somewhat accurate evaluation estimates. A different criteria has to be found to identify those that are close to independence. We can do this by considering the evaluation ideal of arbitrarily correlated classifiers.

Theorem 3 in the Supplement shows how a complete polynomial representation for their data sketch is 274 possible when we include variables for each of the correlation statistics. Its corresponding evaluation 275 variety remains an open problem. But another variety that contains it can be defined. And most 276 importantly, it can be constructed without knowledge of the correlation statistics. Theorem 4 in the 277 Supplement details how the basic evaluation statistics can be disentangled from the correlation ones 278 by finding a suitable Gröbner basis for the generating set. The polynomials of the disentangled set 279 have the forms, 280

Since this generating set is a subset of the complete generating set for correlated classifiers, it is guaranteed to contain their evaluation variety, which, in turn, must contain the true evaluation point.

281

282

283

284

285

286

287

288

289

290

291

By self-consistency, if the evaluation ensemble was truly independent, it would be on this 4dimensional surface. The second set of experiments looked at the hypothesis that data sketches from correlated classifiers would return independent evaluator estimates whose distance from the containing variety was related to their unknown amount of correlation. Figures 3a and 3b show the observed relation between the distance to the containing variety and an ensemble's maximum absolute pair correlation for the adult and mushroom experiments. As in the first experiments, the training data was 200 per label items. But the evaluation was carried out on held-out data with 2000 per label items. As was expected by the test size profiling runs, the mushroom seemingly correct evaluations were harder to find. The trend in these plots is suggestive but not conclusive.

The Supplement contains some of the evaluations from the least-distance evaluation ensembles. The 292 evaluations on twonorm perform best, perhaps because that dataset is synthetic. But challenges 293 remain when handling correlated ensembles. Perhaps these will be resolved with further work. This 294 may be possible because we have a complete representation for correlated classifiers. Using that 295 representation one can expand the independent evaluator estimates as Taylor series on the unknown 296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

correlations. The linear term in Γ i,j,ℓ has the inverse,

Consequently, independent evaluator estimates becomes worse the closer one is to the 'blindspots' in evaluation space at f i,ℓ . A look at the Gröbner basis for correlated classifiers shows how the blindspots shunt off the correlation variables by eliminating them from the basis. An evaluator whose evaluation statistics lie at the blindspots is thus unable to capture correlation effects - its sketch is explainable by an independent ensemble hypothesis that is not correct. This extreme case happens at a finite number of points, so its occurrence would be correspondingly rare. But as the Taylor expansion shows, it can affect the quality of the independent evaluator estimate severely if the true evaluation point lies near them.

6 Advantages and disadvantages of algebraic stream evaluation

The main advantage of algebraic evaluation is that it bypasses the representation and out-ofdistribution problems in ML. Its focus is on estimating sample statistics with no concern for inferring models of the phenomena being classified or how the classifiers do it. There are no unknown unknowns in algebraic evaluation.

But algebraic evaluation cannot resolve the principal/agent monitoring paradox, only ameliorate it. Its batch approach only estimates average performance on a test. This may not be sufficient to handle anomalies or identify important subsets of the test where the classifiers perform much worse. In addition, sample statistics are not enough to identify the causes of poor performance or predict performance in the future. These are important considerations in settings one bothers to monitor with evaluation ensembles. Algebraic evaluators should be used in conjunction with other evaluation methods, such as the ones discussed in Section 1.3, that do encode more information about the application context.

Finally, all evaluation methods on unlabeled data are ambiguous. This is seen here by the two-point variety associated with independent classifiers. Additional assumptions about the evaluation must be made to 'decode' the true evaluation point. For example, in contexts where the prevalences are not expected to vary greatly their known value can be used to select to correct point. Such is the case in the adult dataset where the rare label corresponds to tax record features for people earning more than 50K US dollars annually. Fewer higher income records is a reasonable assumption for future random samples of US tax records. Conversely, if one could have high assurances of the quality of the classifiers and then use them to select the one point that aligns with it. In that case, stream evaluation is being used to monitor the environment and not the classifiers.

7 Broader Impacts

Evaluation on unlabeled data is a perennial problem in ML. As this conference and others discuss the impact AI agents have on our safety and society, it becomes necessary to have safeguards that can protect us from their decisions. The framework proposed here should have a positive impact across multiple application areas for ML since it is based on generic considerations.

References

P. Dawid and A. M. Skene. Maximum likelihood estimation of observer error-rates using the em algorithm. Applied Statistics , pages 20-28, 1979.

Vikas C. Raykar, Shipeng Yu, Linda H. Zhao, Gerardo Hermosillo Valadez, Charles Florin, Luca Bogoni, and Linda Moy. Learning from crowds. Journal of Machine Learning Research , 11(43):1297-1322, 2010.

Fabian L Wauthier and Michael I. Jordan. Bayesian bias mitigation for crowdsourcing. In J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 24 , pages 1800-1808. Curran Associates, Inc., 2011.

Qiang Liu, Jian Peng, and Alexander T Ihler. Variational inference for crowdsourcing. In 342 F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural 343 Information Processing Systems 25 , pages 692-700. Curran Associates, Inc., 2012. 344

Dengyong Zhou, Sumit Basu, Yi Mao, and John C. Platt. Learning from the wis345 dom of crowds by minimax entropy. In F. Pereira, C. J. C. Burges, L. Bottou, and 346 K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25 , pages 347 2195-2203. Curran Associates, Inc., 2012. URL http://papers.nips.cc/paper/ 348 4490-learning-from-the-wisdom-of-crowds-by-minimax-entropy.pdf . 349

350

351

352

353

Yuchen Zhang, Xi Chen, Dengyong Zhou, and Michael I Jordan. Spectral methods meet em: A provably optimal algorithm for crowdsourcing. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 27 , pages 1260-1268. Curran Associates, Inc., 2014.

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

Fabio Parisi, Francesco Strino, Boaz Nadler, and Yuval Kluger. Ranking and combining multiple predictors without labeled data. Proceedings of the National Academy of Sciences , 111(4): 1253-1258, 2014.

Ariel Jaffe, Boaz Nadler, and Yuval Kluger. Estimating the accuracies of multiple classifiers without labeled data. In Guy Lebanon and S. V. N. Vishwanathan, editors, Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics , Proceedings of Machine Learning Research, pages 407-415, San Diego, California, USA, 2015. PMLR.

Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ml safety, 2021.

Jacob Steinhardt and Percy S Liang. Unsupervised risk estimation using only conditional independence structure. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 29, pages 3657-3665. Curran Associates, Inc., 2016.

Wikipedia. Good-Turing frequency estimation - Wikipedia, the free encyclopedia. http://en.wikipedia.org/w/index.php?title=Good%E2%80%93Turing% 20frequency%20estimation&oldid=1141712803 , 2023. [Online; accessed 16-May-2023].

D. Cox, J. Little, and D. O'Shea. Ideals, Varieties, and Algorithms: An Introduction to 370 Computational Algebraic Geometry and Commutative Algebra . Springer-Verlag, 4th edition, 371 2015. 372

G. Pistone, E Riccomagno, and H. P. Wynn. Algebraic Statistics: Computational Commutative 373 Algebra in Statistics . Chapman and Hall, 2001. 374

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the problem of estimating the accuracy of noisy judges/classifiers in a streaming fashion, using only unlabeled data. Specifically, the goal is to compute the accuracy of each judge while processing items and the judge predictions for each item as part of a stream, without any associated labels for the items.\n\nSTRENGTHS:\n1. The problem of evaluating noisy judges that is being explored by the paper is an interesting and important one.\n\nWEAKNESSES:\n1. The main method presented in this paper (i.e., section 3) is not novel. For example, it can be seen as a special case of the approach presented in *Platanios, E. A., Blum A., and Mitchell T. \"Estimating accuracy from unlabeled data\" UAI (2014)*, which relaxes the independence assumption (though does not consider the streaming setting) and is also not cited in this paper. In fact, a lot of related work is missing including derivatives of the aforementioned paper (e.g., a direct follow-up in ICML 2016).\n\n2. The premise of section 4 is weak. The idea that the method of section 3 is “self-alarming” because when you have dependent classifiers the accuracy estimates will be invalid is not completely correct. While this may capture some cases, there are still a lot of cases where you can have dependent classifiers, and where there exists a valid solution to the presented system of equations. Thus, I am not convinced by the main claim of this section.\n\n3. The paper considers a streaming setting but it does not provide motivation for it. For example, it was not clear to me why we cannot store the predictions of the classifiers as items are processed in a database and then perform accuracy estimation periodically. If we have 8 classifiers and 2 possible labels, this would require 1MB per 1 million items, which does not seem expensive (and we can also perform random sampling if space becomes an issue).\n\n4. The paper is presented in a manner that is hard to follow and could be significantly improved. I The whole paper would be presented in a simpler and more organized manner, but section 5 was particularly hard to follow without spending a significant amount of time to understand the argument that was being made.\n\n5. The experimental evaluation is a bit lacking in that only toy datasets are being used, there is no explanation for what they are and why they are interesting, and there are very limited results being presented. Ideally, I’d like to see an “Experiments” section in the paper that describes the setup targeted at testing some hypotheses, the datasets, and the evaluation metrics, and then presents and discusses the evaluation results.\n\nQUESTIONS:\n1. Regarding the majority vote estimator, I don’t understand why you need 2^n variables in your sketch. I think all you need is n+1 variables which are defined as follows: (i) the number of items that have been processed thus far, and (ii) for each classifier, the number of times its predicted label matches the majority vote label. Is my understanding correct or am I missing something?\n\n2. I didn’t understand section 2.1 and I also disagree with the statement in lines 163-164. I can see decision being framed as inference and I am also confident that oftentimes making hard decisions as opposed to keeping soft values around (which I assume is what you are referring to as “decision”) can be helpful. Can you please elaborate and also provide some reference for this claim if I am mistaken?\n\n3. I didn’t understand what you mean by the “principal/agent monitoring paradox”. Can you please explain what that is and also why it is a paradox?\n\nLIMITATIONS:\nThere is no discussion of limitations and potential negative social impact in this paper. One recommendation would be to try and think about what the implications could be for say voting systems, and also in situations where these methods are used to evaluate people whose income may depend on this evaluation (e.g., crowdworkers). In this case, the independence assumption being made by the paper may be too strong and yield in incorrect evaluations that could negatively and unfailrly affect the income of those people.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new inferential evaluator for evaluating noisy binary classifiers on unlabeled data in a streaming manner. Specifically, compared to the evaluator based on majority votes, the new evaluator gives a more complete and reasonable modeling of the true label prevalence and each classifier’s accuracy. In addition, the property of the new evaluator is also mathematically discussed, and the relationship between error dependence and the evaluator estimate is empirically discovered through experiments.\n\nSTRENGTHS:\n1. The paper addresses a significant problem in machine learning - evaluating the performance of binary classifiers on unlabeled data.\n2. The proposed methods could have wide-ranging applications in various fields where machine learning is used, making the paper highly relevant.\n3. The author provides mathematical proof to support the proposed methods and conducts empirical tests on several datasets.\n4. The proposed generic framework that is based on algebraic geometry can cast a positive influence on evaluation methods on unlabeled data.\n5.  The new algebraic evaluation method bypasses the representation and OOD problems in ML.\n\nWEAKNESSES:\n1. The paper is very hard to read and lacks the background to help the reviewer understand and improve the reading experience. Moreover, the supplement mentioned in lines 124, 171, 236, and 292 is missing from the paper. There are no detailed proofs for all the theorems.\n2. The organization of the paper is confusing. The logic chain of the whole paper needs to be improved, better briefly introduce the outline and main content of each chapter at the beginning.\n3. The aiming research problem needs to be explained formally in math language and to be explained clearly with intuitive explanations, better with a toy example or case study.\n4. The current limitations of existing research, the proposed solutions (contributions of this paper), and the aiming experimental questions are not clearly listed, making it hard to catch the author's idea.\n5. More experiments are needed. There are no experiments supporting that the performance of a majority vote-based evaluator is worse than that of the inferential one. And from the perspective of experiments, the advantage of the new evaluator is not made clear.\n6. Only label prevalence is formalized in the paper, while there are no formulas for classifier accuracy.\n7. The complexity of the concepts and the heavy use of mathematical proofs might affect their clarity. The author could consider providing more background information, intuitive explanations, or visual aids to improve the paper's accessibility.\n8. The experiment part fails to show the superiority of the proposed method compared with other baselines.\n9. The equations need to be carefully edited using formal math language. The space should be used for some meaningful and essential equations, not for simple ones such as summation or average operations.\n10. Typos: In line 167, \"it could have been a βitem, not an αone\" should be \"it could have been a β item, not an α one\".\n\nQUESTIONS:\nIs it possible to compare the proposed method to other baselines on the same datasets, in order to show the superiority and significance of the new method?\n\nLIMITATIONS:\nThe paper concludes with a brief discussion of how algebraic stream evaluation can and cannot help when done for safety or economic reasons.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper considers the problem of evaluating noisy binary classifiers on unlabeled streaming data. It aims to estimate the prevalence of the labels and the accuracy of each classifier on them, given a data sketch of label predictions by the members of an ensemble of noisy binary classifiers. The authors propose two algebraic evaluators based on the assumption of error-independent classifiers: the first is based on an additional assumption of majority voting, and the second is fully inferential and is guaranteed to contain the true evaluation point.

STRENGTHS:
- The results of this paper seem to be well supported by the rigorous mathematical analysis as well as empirically demonstrated on three benchmark datasets.

WEAKNESSES:
- The independence assumption may not be satisfied in practice, especially when the ensemble of classifiers consists of different models trained on the same or overlapping datasets, or even the same model but trained for different durations.
- The significance of this paper is unclear. It would be better for the authors to provide some concrete real-world examples that fit the problem setting of this paper and explain the possible uses of the quantities desired to be estimated in these examples.
- The proposed evaluators may be sensitive to noise or corruption in the data sketch. The solutions of the algebraic equations will change if the data sketch is not perfectly recorded, leading to mistakes in distinguishing between independent and correlated evaluations.

QUESTIONS:
Please see Weaknesses.

LIMITATIONS:
The authors has discussed the limitations of this paper.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-15

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses making decisions based on the outputs of three binary classifiers. More precisely, it focuses on evaluating the performances of noisy classifiers. It considers majority voting on one hand, and a proposed evaluation scheme based on the classifiers' accuracies. The paper establishes several theorems so as to demonstrate the superiority of the second (proposed) evaluation scheme. Then, experiments are conducted to test the ability of the proposal to avoid making decisions in problematic situations—e.g., correlated classifiers.\n\nSTRENGTHS:\nI see no particular strength in the paper that would mitigate its flaws.\n\nWEAKNESSES:\nThe proposal suffers from several major flaws. \n\nFirst of all, it is badly written. The problem is not clearly stated. The mathematical objects (typically, the prevalence of the labels, and the label accuracies) are not properly introduced and defined. Some key notions (such as the \"evaluation variety\", the precise definition of correlated classifiers, among others) are also not defined. There are frequent references in the text to notions which have not been exposed yet (e.g., \"the evaluators for binary classifiers\" in the introduction, Theorems 1 and 2 in the introduction as well, Theorem 3 in Section 1.2). \n\nBesides, the paper ignores a large amount of literature. The problem addressed has connections with computational social choice (voting schemes), of which some works are mentioned. But it also relates to classifier combination (boosting, error-correcting output codes, weighted averaging, racing algorithms, etc): the problem of evaluating the performance of an ensemble has been addressed in a number of works which are ignored here. \n\nLast, but not least, the paper focuses on a very specific case—the ensemble has only three classifiers. This is very restrictive and such ensembles are hardly used in practice. Some claims are not supported—e.g., in Section 1.1, \"Seemingly correct estimates are estimated values that seem to be correct because they have this real, integer ratio form. Estimates that do not have this form are obviously incorrect.\" This is not the case of the $F_\\beta$ measure, for instance.\n\nQUESTIONS:\nHow would your approach compare to classifier combination techniques where classifiers are combined based on their accuracy (e.g. racing algorithms or boosting) ? \n\nHow could the approach be generalized to more than 3 classifiers in the ensemble, or to multi-class classification problems ?\n\nLIMITATIONS:\nThe authors have addressed the limitations of their approach, but in my opinion only in a restricted way. They have not addressed the potential negative societal impact of their work, but I do not think that this is crucial here.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of evaluating an ensemble of binary classifiers on unlabeled data in a streaming setting. The authors first describe a baseline which treats the majority vote as the correct label and evaluate each classifier accordingly. Then they propose an evaluator based on an assumption that the classifiers are independent. The algebraic expression for this evaluator should return rational numbers if the assumption holds (as they should correspond to ratios of integer counts). Thus, this evaluator has failure modes that can be detected clearly unlike the majority-voting baseline that may return incorrect but seemingly sensible values.\n\nSTRENGTHS:\nWhile I am not an expert in evaluation using unlabeled data and cannot speak definitively, the proposed algebraic evaluation and characterizing failure modes by algebraic failures seem creative and novel.\n\nWEAKNESSES:\nI found the paper overall quite difficult to follow, and thus my assessment of its technical contributions may be limited. More thorough motivation and background on the problem setting (e.g. using real-world applications), as well as careful characterization of the proposed algebraic evaluator in contrast with existing approaches, would help make the paper much more approachable. I also had a hard time following most of Section 1 (especially 1.2) without the technical details in Section 3.\n\nThe paper is also missing some related work discussion, and its contributions with respect to prior work is not very clear. I struggled to see the connection to the works mentioned in the first paragraph of Section 1.3. Another work that appears very relevant is [1]; how does this paper relate to their approach?\n\nThroughout the paper, only the setting with three binary classifiers is considered. A more general formulation may be helpful. As far as I can tell, this approach would scale exponentially in the number of classifiers which could limit its impact in practical settings.\n\nEmpirical evaluation was limited only to analyzing the failure rates, and there were no experiments on how well the proposed approach performs as an evaluator (i.e., how close are the error rate estimates to the true error rates?). \n\n[1] Platanios, Emmanouil Antonios, Avrim Blum, and Tom Mitchell. \"Estimating Accuracy from Unlabeled Data.\" 2014.\n\nQUESTIONS:\nPlease see above for the main questions and suggestions. As a minor suggestion, I think Theorem 1 would be very intuitive to see using probabilities. It involves the probability of true label being $\\alpha$ and the conditional probabilities of each outcome given the true label being $\\alpha$ or $\\beta$; the independence assumption allows turning the probability of a joint outcome into a product of probabilities for each classifier’s outcome.\n\nLIMITATIONS:\nOverall, yes. One limitation that was not discussed is that the approach seems to scale exponentially in the number of classifiers.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-27"}