Original: Active Labeling: Streaming Stochastic Gradients
Revised: Active Labeling: Streaming Stochastic Gradients

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: The workhorse of machine learning is stochastic gradient descent.
Revised: The workhorse of machine learning is stochastic gradient descent.

Original: To accessstochastic gradients, it is common to consider iteratively input/output pairs of atraining dataset.
Revised: To access stochastic gradients, it is common to consider iteratively input/output pairs of a training dataset.

Original: Interestingly, it appears that one does not need full supervisionto access stochastic gradients, which is the main motivation of this paper.
Revised: Interestingly, it appears that one does not need full supervision to access stochastic gradients, which is the main motivation of this paper.

Original: Afterformalizing the “active labeling” problem, which focuses on active learning withpartial supervision, we provide a streaming technique that provably minimizesthe ratio of generalization error over the number of samples.
Revised: After formalizing the “active labeling” problem, which focuses on active learning with partial supervision, we provide a streaming technique that provably minimizes the ratio of generalization error over the number of samples.

Original: We illustrate ourtechnique in depth for robust regression.
Revised: We illustrate our technique in depth for robust regression

Original:  Introduction
Revised:  Introduction

Original: A large amount of the current hype around artificial intelligence was fueled by the recent successesof supervised learning.
Revised: A large amount of the current hype around artificial intelligence was fueled by the recent successes of supervised learning.

Original: Supervised learning consists in designing an algorithm that maps inputs tooutputs by learning from a set of input/output examples.
Revised: Supervised learning consists in designing an algorithm that maps inputs to outputs by learning from a set of input/output examples.

Original: When accessing many samples, and givenenough computation power, this framework is able to tackle complex tasks.
Revised: When accessing many samples, and given enough computation power, this framework is able to tackle complex tasks.

Original: Interestingly, many ofthe difficulties arising in practice do not emerge from choosing the right statistical model to solvethe supervised learning problem, but from the problem of collecting and cleaning enough data (see
Revised: Interestingly, many of the difficulties arising in practice do not emerge from choosing the right statistical model to solve the supervised learning problem, but from the problem of collecting and cleaning enough data (see

Original: Chapters 1 and 2 of Géron, 2017, for example).
Revised: Chapters 1 and 2 of Géron, 2017, for example).

Original: Those difficulties are not disjoint from the currenttrends toward data privacy regulations (Council of European Union, 2016).
Revised: Those difficulties are not disjoint from the current trends toward data privacy regulations (Council of European Union, 2016).

Original: This fact motivates thiswork, where we focus on how to efficiently collect information to carry out the learning process.
Revised: This fact motivates this work, where we focus on how to efficiently collect information to carry out the learning process.

Original: In this paper, we formalize the “active labeling” problem for weak supervision, where the goal isto learn a target function by acquiring the most informative dataset given a restricted budget forannotation.
Revised: In this paper, we formalize the “active labeling” problem for weak supervision, where the goal is to learn a target function by acquiring the most informative dataset given a restricted budget for annotation.

Original: We focus explicitly on weak supervision that comes as a set of label candidates foreach input, aiming to partially supervise input data in the most efficient way to guide a learningalgorithm.
Revised: We focus explicitly on weak supervision that comes as a set of label candidates for each input, aiming to partially supervise input data in the most efficient way to guide a learning algorithm.

Original: We also restrict our study to the streaming variant where, for each input, only a singlepartial information can be collected about its corresponding output.
Revised: We also restrict our study to the streaming variant where, for each input, only a single partial information can be collected about its corresponding output.

Original: The crux of this work is toleverage the fact that full supervision is not needed to acquire unbiased stochastic gradients, andperform stochastic gradient descent.
Revised: The crux of this work is to leverage the fact that full supervision is not needed to acquire unbiased stochastic gradients and perform stochastic gradient descent.

Original: The following summarizes our contributions.
Revised: The following summarizes our contributions.

Original: First, we introduce the “active labeling” problem, which is a relevant theoretical framework that encompasses many useful problems encountered by practitioners trying to annotate their data in the most efficient fashion, as well as its streaming variation, in order to deal with privacy preserving issues.
Revised: First, we introduce the “active labeling” problem, which is a relevant theoretical framework that encompasses many useful problems encountered by practitioners trying to annotate their data in the most efficient fashion, as well as its streaming variation, in order to deal with privacy preserving issues.

Original: This is the focus of Section 2. 
Revised: This is the focus of Section 2. 2.

Original: Then, in Section 3, we give a high-level framework to access unbiased stochastic gradientswith weak information only.
Revised: Then, in Section 3, we give a high-level framework to access unbiased stochastic gradients with weak information only.

Original: This provides a simple solution to the streaming “active labeling”problem.
Revised: This provides a simple solution to the streaming “active labeling” problem.

Original: Finally, we detail this framework for a robust regression task in Section 4, and provide analgorithm whose optimality is proved in Section 5.
Revised: Finally, we detail this framework for a robust regression task in Section 4, and provide an algorithm whose optimality is proved in Section 5.

Original: As a proof of concept, we provide numerical simulations in Section 6.
Revised: As a proof of concept, we provide numerical simulations in Section 6.

Original: We conclude with a high-leveldiscussion around our methods in Section 7.
Revised: We conclude with a high-level discussion around our methods in Section 7.

Original: Related work.
Revised: Related work.

Original: Active query of information is relevant to many settings.
Revised: Active query of information is relevant to many settings.

Original: The most straightforwardapplications are searching games, such as Bar Kokhba or twenty questions (Walsorth, 1882).
Revised: The most straightforward applications are searching games, such as Bar Kokhba or twenty questions (Walsorth, 1882).

Original: Werefer to Pelc (2002) for an in-depth survey of such games, especially when liars introduce uncertainty,and their relations with coding on noisy channels.
Revised: We refer to Pelc (2002) for an in-depth survey of such games, especially when liars introduce uncertainty, and their relations with coding on noisy channels.

Original: But applications are much more diverse, e.g. fornumerical simulation (Chevalier et al., 2014), database search (Qarabaqi and Riedewald, 2014), orshape recognition (Geman and Jedynak, 1993), to name a few.
Revised: But applications are much more diverse, e.g. for numerical simulation (Chevalier et al., 2014), database search (Qarabaqi and Riedewald, 2014), or shape recognition (Geman and Jedynak, 1993), to name a few.

Original: In terms of motivations, many streams of research can be related to this problem, such as experimentaldesign (Chernoff, 1959), statistical queries (Kearns, 1998; Fotakis et al., 2021), crowdsourcing (Doanet al., 2011), or aggregation methods in weak supervision (Ratner et al., 2020).
Revised: In terms of motivations, many streams of research can be related to this problem, such as experimental design (Chernoff, 1959), statistical queries (Kearns, 1998; Fotakis et al., 2021), crowdsourcing (Doan et al., 2011), or aggregation methods in weak supervision (Ratner et al., 2020).

Original: More precisely,“active labeling” 1 consists in having several inputs and querying partial information on the labels.
Revised: More precisely, “active labeling” 2 consists in having several inputs and querying partial information on the labels.

Original: Itis close to active learning (Settles, 2010; Dasgupta, 2011; Hanneke, 2014), where there are severalinputs, but exact outputs are queried; and to active ranking (Valiant, 1975; Ailon, 2011; Bravermanet al., 2019), where partial information is queried, but there is only one input.
Revised: It is close to active learning (Settles, 2010; Dasgupta, 2011; Hanneke, 2014), where there are several inputs, but exact outputs are queried; and to active ranking (Valiant, 1975; Ailon, 2011; Braverman et al., 2019), where partial information is queried, but there is only one input.

Original: The streaming variantintroduces privacy preserving constraints, a problem that is usually tackled through the notion ofdifferential privacy (Dwork et al., 2006).
Revised: The streaming variant introduces privacy preserving constraints, a problem that is usually tackled through the notion of differential privacy (Dwork et al., 2006).

Original: In terms of formalization, we build on the partial supervision formalization of Cabannes et al.(2020), which casts weak supervision as sets of label candidates and generalizes semi-supervisedlearning (Chapelle et al., 2006).
Revised: In terms of formalization, we build on the partial supervision formalization of Cabannes et al. (2020), which casts weak supervision as sets of label candidates and generalizes semi-supervised learning (Chapelle et al., 2006).

Original: Finally, our sequential setting with a unique final reward is similar tocombinatorial bandits in a pure-exploration setting (Garivier and Kaufmann, 2016; Fiez et al., 2019).
Revised: Finally, our sequential setting with a unique final reward is similar to combinatorial bandits in a pure-exploration setting (Garivier and Kaufmann, 2016; Fiez et al., 2019).

Original:  The “active labeling” problem
Revised:  The “active labeling” problem

Original: Supervised learning is traditionally modeled in the following manner.
Revised: Supervised learning is traditionally modeled in the following manner.

Original: Consider X an input space, Yan output space, ℓ : Y × Y → R a loss function, and 𝜌 ∈ Δ X × Y a joint probability distribution.
Revised: Consider X an input space, Y an output space, ℓ : Y × Y → R a loss function, and 𝜌 ∈ Δ X × Y a joint probability distribution.

Original: Thegoal is to recover the function
Revised: The goal is to recover the function

Original: [Equation]
Revised: [Equation]

Original: yet, without accessing 𝜌 , but a dataset of independent samples distributed according to 𝜌 , D 𝑛 =( 𝑋 𝑖 ,𝑌 𝑖 ) 𝑖 ≤ 𝑛 ∼ 𝜌 ⊗ 𝑛 .
Revised: yet, without accessing 𝜌 , but a dataset of independent samples distributed according to 𝜌 , D 𝑛 = ( 𝑋 𝑖 ,𝑌 𝑖 ) 𝑖 ≤ 𝑛 ∼ 𝜌 ⊗ 𝑛 .

Original: In practice, accessing data comes at a cost, and it is valuable to understand thecheapest way to collect a dataset allowing to discriminate 𝑓 ∗ .
Revised: In practice, accessing data comes at a cost, and it is valuable to understand the cheapest way to collect a dataset allowing to discriminate 𝑓 ∗ .

Original: We shall suppose that the input data ( 𝑋 𝑖 ) 𝑖 ≤ 𝑛 are easy to collect, yet that labeling those inputs to getoutputs ( 𝑌 𝑖 ) 𝑖 ≤ 𝑛 demands a high amount of work.
Revised: We shall suppose that the input data ( 𝑋 𝑖 ) 𝑖 ≤ 𝑛 are easy to collect, yet that labeling those inputs to get outputs ( 𝑌 𝑖 ) 𝑖 ≤ 𝑛 demands a high amount of work.

Original: For example, it is relatively easy to scrap the web ormedical databases to access radiography images, but labeling them by asking radiologists to recognizetumors on zillions of radiographs will be both time-consuming and expensive.
Revised: For example, it is relatively easy to scrap the web or medical databases to access radiography images, but labeling them by asking radiologists to recognize tumors on zillions of radiographs will be both time-consuming and expensive.

Original: As a consequence, weassume the ( 𝑋 𝑖 ) 𝑖 ≤ 𝑛 given but the ( 𝑌 𝑖 ) 𝑖 ≤ 𝑛 unknown .
Revised: As a consequence, we assume the ( 𝑋 𝑖 ) 𝑖 ≤ 𝑛 given but the ( 𝑌 𝑖 ) 𝑖 ≤ 𝑛 unknown .

Original: As getting information on the labels comes at acost ( e.g. , paying a pool of label workers, or spending your own time), given a budget constraint, whatinformation should we query on the labels?
Revised: As getting information on the labels comes at a cost ( e.g. , paying a pool of label workers, or spending your own time), given a budget constraint, what information should we query on the labels?

Original: To quantify this problem, we will assume that we can sequentially and adaptively query 𝑇 informationof the type 1 𝑌 𝑖𝑡 ∈ 𝑆 𝑡 , for any index 𝑖 𝑡 ∈ { 1 , · · · , 𝑛 } and any set of labels 𝑆 𝑡 ⊂ Y (belonging to a specifiedset of subsets of Y ).
Revised: To quantify this problem, we will assume that we can sequentially and adaptively query 𝑇 information of the type 1 𝑌 𝑖𝑡 ∈ 𝑆 𝑡 , for any index 𝑖 𝑡 ∈ { 1 , · · · , 𝑛 } and any set of labels 𝑆 𝑡 ⊂ Y (belonging to a specified set of subsets of Y ).

Original: Here, 𝑡 ∈ { 1 , · · · ,𝑇 } indexes the query sequence, and 𝑇 ∈ N is a fixed budget.
Revised: Here, 𝑡 ∈ { 1 , · · · ,𝑇 } indexes the query sequence, and 𝑇 ∈ N is a fixed budget.

Original: The goal is to optimize the design of the sequence ( 𝑖 𝑡 , 𝑆 𝑡 ) in order to get the best estimate of 𝑓 ∗ interms of risk minimization (1).
Revised: The goal is to optimize the design of the sequence ( 𝑖 𝑡 , 𝑆 𝑡 ) in order to get the best estimate of 𝑓 ∗ in terms of risk minimization (1).

Original: In the following, we give some examples to make this setting moreconcrete
Revised: In the following, we give some examples to make this setting more concrete.

Original: Example 1 (Classification with attributes) .
Revised: Example 1 (Classification with attributes) .

Original: Suppose that a labeler is asked to provide fine-grainedclasses on images (Krause et al., 2016; Zheng et al., 2019), such as the label “caracal” in Figure 6.
Revised: Suppose that a labeler is asked to provide fine-grained classes on images (Krause et al., 2016; Zheng et al., 2019), such as the label “caracal” in Figure 6.

Original: This would be difficult for many people.
Revised: This would be difficult for many people.

Original: Yet, it is relatively easy to recognize that the imagedepicts a “feline” with “tufted-ears” and “sandy color”.
Revised: Yet, it is relatively easy to recognize that the image depicts a “feline” with “tufted-ears” and “sandy color”.

Original: As such, a labeler can give the weakinformation that 𝑌 belongs to the set “feline”, 𝑆 1 = { “cat”, “lion”, “tiger” , . . . } , and the set “tuftedears”, 𝑆 2 = { “Great horned owl”, “Aruacana chicken” , . . . } .
Revised: As such, a labeler can give the weak information that 𝑌 belongs to the set “feline”, 𝑆 1 = { “cat”, “lion”, “tiger” , . . . } , and the set “tufted ears”, 𝑆 2 = { “Great horned owl”, “Aruacana chicken” , . . . } .

Original: This is enough to recognize that𝑌 ∈ 𝑆 1 ∩ 𝑆 2 = { “caracal” } .
Revised: This is enough to recognize that 𝑌 ∈ 𝑆 1 ∩ 𝑆 2 = { “caracal” } .

Original: The question 1 𝑌 ∈ 𝑆 1 , corresponds to asking if the image depicts a feline.
Revised: The question 1 𝑌 ∈ 𝑆 1 , corresponds to asking if the image depicts a feline.

Original: Literature on hierarchical classification and autonomic taxonomy construction provides interestingideas for this problem ( e.g. , Cesa-Bianchi et al., 2006; Gangaputra and Geman, 2006).
Revised: Literature on hierarchical classification and autonomic taxonomy construction provides interesting ideas for this problem ( e.g. , Cesa-Bianchi et al., 2006; Gangaputra and Geman, 2006).

Original: Example 2 (Ranking with partial ordering) .
Revised: Example 2 (Ranking with partial ordering) .

Original: Consider a problem where for a given input 𝑥 , characterizing a user, we are asked to deduce their preferences over 𝑚 items.
Revised: Consider a problem where for a given input 𝑥 , characterizing a user, we are asked to deduce their preferences over 𝑚 items.

Original: Collecting such a label requiresknowing the exact ordering of the 𝑚 items induced by a user.
Revised: Collecting such a label requires knowing the exact ordering of the 𝑚 items induced by a user.

Original: This might be hard to ask for.
Revised: This might be hard to ask for.

Original: Instead,one can easily ask the user which items they prefer in a collection of a few items.
Revised: Instead, one can easily ask the user which items they prefer in a collection of a few items.

Original: The user’s answerwill give weak information about the labels, which can be modeled as knowing 1 𝑌 𝑖 ∈ 𝑆 = 1 , for 𝑆 the setof total orderings that satisfy this partial ordering.
Revised: The user’s answer will give weak information about the labels, which can be modeled as knowing 1 𝑌 𝑖 ∈ 𝑆 = 1 , for 𝑆 the set of total orderings that satisfy this partial ordering.

Original: We refer the curious reader to active ranking anddueling bandits for additional contents (Jamieson and Nowak, 2011; Bengs et al., 2021).
Revised: We refer the curious reader to active ranking and dueling bandits for additional contents (Jamieson and Nowak, 2011; Bengs et al., 2021).

Original: Example 3 (Pricing a product) .
Revised: Example 3 (Pricing a product) .

Original: Suppose that we want to sell a product to a consumer characterizedby some features 𝑥 , this consumer is ready to pay a price 𝑦 ∈ R for this product.
Revised: Suppose that we want to sell a product to a consumer characterized by some features 𝑥 , this consumer is ready to pay a price 𝑦 ∈ R for this product.

Original: We price it 𝑓 ( 𝑥 ) ∈ R ,and we observe 1 𝑓 ( 𝑥 ) <
Revised: We price it 𝑓 ( 𝑥 ) ∈ R , and we observe 1 𝑓 ( 𝑥 ) <

Original: 𝑦 , that is if the consumer is willing to buy this product at this price tag or not(Cesa-Bianchi et al., 2019; Liu et al., 2021).
Revised: 𝑦 , that is if the consumer is willing to buy this product at this price tag or not (Cesa-Bianchi et al., 2019; Liu et al., 2021).

Original: Although, in this setting, the goal is often to minimizethe regret, which contrasts with our pure exploration setting.
Revised: Although, in this setting, the goal is often to minimize the regret, which contrasts with our pure exploration setting.

Original: As a counter-example, our assumptions are not set to deal with missing data, i.e. if some coordinatesof some input feature vectors 𝑋 𝑖 are missing (Rubin, 1976).
Revised: As a counter-example, our assumptions are not set to deal with missing data, i.e. if some coordinates of some input feature vectors 𝑋 𝑖 are missing (Rubin, 1976).

Original: Typically, this happens when input datacomes from different sources ( e.g. , when trying to predict economic growth from country informationthat is self-reported).
Revised: Typically, this happens when input data comes from different sources ( e.g. , when trying to predict economic growth from country information that is self-reported).

Original: Streaming variation.
Revised: Streaming variation.

Original: The special case of the active labeling problem we shall consider consists inits variant without resampling.
Revised: The special case of the active labeling problem we shall consider consists in its variant without resampling.

Original: This corresponds to the online setting where one can only ask onequestion by sample, formally 𝑖 𝑡 = 𝑡 .
Revised: This corresponds to the online setting where one can only ask one question by sample, formally 𝑖 𝑡 = 𝑡 .

Original: This setting is particularly appealing for privacy concerns, insettings where the labels ( 𝑌 𝑖 ) contain sensitive information that should not be revealed totally.
Revised: This setting is particularly appealing for privacy concerns, in settings where the labels ( 𝑌 𝑖 ) contain sensitive information that should not be revealed totally.

Original: Forexample, some people might be more comfortable giving a range over a salary rather than the exactvalue; or in the context of polling, one might not call back a previous respondent characterized bysome features 𝑋 𝑖 to ask them again about their preferences captured by 𝑌 𝑖 .
Revised: For example, some people might be more comfortable giving a range over a salary rather than the exact value; or in the context of polling, one might not call back a previous respondent characterized by some features 𝑋 𝑖 to ask them again about their preferences captured by 𝑌 𝑖 .

Original: Similarly, the streamingsetting is relevant for web marketing, where inputs model new users visiting a website, queries modelsets of advertisements chosen by an advertising company, and one observes potential clicks.
Revised: Similarly, the streaming setting is relevant for web marketing, where inputs model new users visiting a website, queries model sets of advertisements chosen by an advertising company, and one observes potential clicks.

Original:  Weak information as stochastic gradients
Revised:  Weak information as stochastic gradients

Original: In this section, we discuss how  stochastic gradients can be accessed through weak information.
Revised: In this section, we discuss how unbiased stochastic gradients can be accessed through weak information.

Original: Suppose that we model 𝑓 = 𝑓 𝜃 for some Hilbert space Θ ∋ 𝜃 .
Revised: Suppose that we model 𝑓 = 𝑓 𝜃 for some Hilbert space Θ ∋ 𝜃 .

Original: With some abuse of notations, let usdenote ℓ ( 𝑥, 𝑦, 𝜃 ) : =
Revised: With some abuse of notations, let us denote ℓ ( 𝑥, 𝑦, 𝜃 ) : =

Original: ℓ ( 𝑓 𝜃 ( 𝑥 ) , 𝑦 ) .
Revised: ℓ ( 𝑓 𝜃 ( 𝑥 ) , 𝑦 ) .

Original: We aim to minimize R ( 𝜃 ) =
Revised: We aim to minimize R ( 𝜃 ) =

Original: E ( 𝑋,𝑌 ) [ ℓ ( 𝑋,𝑌, 𝜃 )] .
Revised: E ( 𝑋,𝑌 ) [ ℓ ( 𝑋,𝑌, 𝜃 )] .

Original: Assume that R isdifferentiable (or sub-differentiable) and denote its gradients by ∇ 𝜃 R .
Revised: Assume that R is differentiable (or sub-differentiable) and denote its gradients by ∇ 𝜃 R .

Original: Definition 1 (Stochastic gradient) .
Revised: Definition 1 (Stochastic gradient) .

Original: A stochastic gradient of R is any random function 𝐺 : Θ → Θ suchthat E [ 𝐺 ( 𝜃 )]
Revised: A stochastic gradient of R is any random function 𝐺 : Θ → Θ such that E [ 𝐺 ( 𝜃 )]

Original: = ∇ 𝜃 R ( 𝜃 ) .
Revised: = ∇ 𝜃 R ( 𝜃 ) .

Original: Given some step size function 𝛾 : N → R ∗ , a stochastic gradient descent(SGD) is a procedure, ( 𝜃 𝑡 )
Revised: Given some step size function 𝛾 : N → R ∗ , a stochastic gradient descent (SGD) is a procedure, ( 𝜃 𝑡 )

Original: ∈
Revised: ∈

Original: Θ N , initialized with some 𝜃 0 and updated as 𝜃 𝑡 + 1 =
Revised: Θ N , initialized with some 𝜃 0 and updated as 𝜃 𝑡 + 1 =

Original: 𝜃 𝑡 − 𝛾 ( 𝑡 ) 𝐺 ( 𝜃 𝑡 ) ,where the realization of 𝐺 ( 𝜃 𝑡 ) given 𝜃 𝑡 is independent of the previous realizations of 𝐺 ( 𝜃 𝑠 ) given 𝜃 𝑠 .
Revised: 𝜃 𝑡 − 𝛾 ( 𝑡 ) 𝐺 ( 𝜃 𝑡 ) , where the realization of 𝐺 ( 𝜃 𝑡 ) given 𝜃 𝑡 is independent of the previous realizations of 𝐺 ( 𝜃 𝑠 ) given 𝜃 𝑠 .

Original: In supervised learning, SGD is usually performed with the stochastic gradients ∇ 𝜃 ℓ ( 𝑋,𝑌, 𝜃 ) .
Revised: In supervised learning, SGD is usually performed with the stochastic gradients ∇ 𝜃 ℓ ( 𝑋,𝑌, 𝜃 ) .

Original: Moregenerally, stochastic gradients are given by
Revised: More generally, stochastic gradients are given by

Original: [Equation]
Revised: [Equation]

Original: for 𝜏 : T → Θ with T ⊂ 2 Θ a set of subsets of Θ , and 𝑇 a random variable on T , such that
Revised: for 𝜏 : T → Θ with T ⊂ 2 Θ a set of subsets of Θ , and 𝑇 a random variable on T , such that

Original: [Equation]
Revised: [Equation]

Original: Stated otherwise, if you have a way to image a vector 𝜃 from partial measurement 1 𝜃 ∈ 𝑇 such thatyou can reconstruct this vector in a linear fashion (3), then it provides you a generic strategy to get aunbiased stochastic estimate of this vector from a partial measurement (2).
Revised: Stated otherwise, if you have a way to image a vector 𝜃 from partial measurements 1 𝜃 ∈ 𝑇 such that you can reconstruct this vector in a linear fashion (3), then it provides you a generic strategy to get an unbiased stochastic estimate of this vector from a partial measurement (2).

Original: For 𝜓 : Y → Θ a function from Y to Θ ( e.g. , 𝜓 = ∇ 𝜃 ( 𝑋, · , 𝜃 ) ), a question 1 𝜓 ( 𝑌 )∈ 𝑇 translates into aquestion 1 𝑌 ∈ 𝑆 for some set 𝑆 = 𝜓 − 1 ( 𝑇 ) ⊂
Revised: For 𝜓 : Y → Θ a function from Y to Θ ( e.g. , 𝜓 = ∇ 𝜃 ( 𝑋, · , 𝜃 ) ), a question 1 𝜓 ( 𝑌 )∈ 𝑇 translates into a question 1 𝑌 ∈ 𝑆 for some set 𝑆 = 𝜓 − 1 ( 𝑇 ) ⊂

Original: Y , meaning that the stochastic gradient (2) can be evaluatedfrom a single query.
Revised: Y , meaning that the stochastic gradient (2) can be evaluated from a single query.

Original: As a proof of concept, we derive a generic implementation for 𝑇 and 𝜏 in
Revised: As a proof of concept, we derive a generic implementation for 𝑇 and 𝜏 in

Original: Appendix B. This provides a generic SGD scheme to learn functions from weak queries when there isno constraints on the sets to query.
Revised: Appendix B. This provides a generic SGD scheme to learn functions from weak queries when there are no constraints on the sets to query.

Original: Remark 2 (Cutting plane methods) .
Revised: Remark 2 (Cutting plane methods) .

Original: While we provide here a descent method, one could also developcutting-plane/ellipsoid methods to localize 𝜃 ∗ according to weak information, which corresponds tothe techniques developed for pricing by Cohen et al.
Revised: While we provide here a descent method, one could also develop cutting-plane/ellipsoid methods to localize 𝜃 ∗ according to weak information, which corresponds to the techniques developed for pricing by Cohen et al.

Original: (2020) and related literature.
Revised: (2020) and related literature.

Original:  Median regression
Revised:  Median regression

Original: In this section, we focus on efficiently acquiring weak information providing stochastic gradients forregression problems.
Revised: In this section, we focus on efficiently acquiring weak information providing stochastic gradients for regression problems.

Original: In particular, we motivate and detail our methods for the absolute deviation loss.
Revised: In particular, we motivate and detail our methods for the absolute deviation loss.

Original: Motivated by seminal works on censored data (Tobin, 1958), we shall suppose that we query halfspaces .
Revised: Motivated by seminal works on censored data (Tobin, 1958), we shall suppose that we query halfspaces .

Original: For an output 𝑦 ∈ Y
Revised: For an output 𝑦 ∈ Y

Original: = R 𝑚 , and any hyper-plane 𝑧 +
Revised: = R 𝑚 , and any hyper-plane 𝑧 +

Original: 𝑢 ⊥ ⊂ R 𝑚 for 𝑧 ∈ R 𝑚 , 𝑢 ∈ S 𝑚 − 1 , we canask a labeler to tell us which half-space 𝑦 belongs to.
Revised: 𝑢 ⊥ ⊂ R 𝑚 for 𝑧 ∈ R 𝑚 , 𝑢 ∈ S 𝑚 − 1 , we can ask a labeler to tell us which half-space 𝑦 belongs to.

Original: Formally, we access the quantity sign (⟨ 𝑦 − 𝑧, 𝑢 ⟩)for a given unit cost .
Revised: Formally, we access the quantity sign (⟨ 𝑦 − 𝑧, 𝑢 ⟩) for a given unit cost .

Revised: Such an imaging scheme where one observes summations of its components rather than a vector itself bears similarity with compressed sensing.

Revised: To provide further illustration, this setting could help to price products while selling bundles: where the context 𝑥 characterizes some users, web-pages or/and advertisement companies; the label 𝑦 ∈ R 𝑚 corresponds to the value associated to 𝑚 different items, such as stocks composing an index, or advertisement spots; and the observation sign (⟨ 𝑦, 𝑢 ⟩ − 𝑐 ) (with 𝑐 = ⟨ 𝑧, 𝑢 ⟩ ) captures if the user 𝑥 buys the basket with weights 𝑢 ∈ S 𝑚 − 1 when it is priced 𝑐 .

Original: Least-squares.
Revised: Least-squares.

Original: For regression problems, it is common to look at the mean square loss 
Revised: For regression problems, it is common to look at the mean square loss ℓ

Original: [Equation]
Revised: [Equation]

Original: where 𝐷 𝑓 𝜃 ( 𝑥 ) ∈ Y ⊗ Θ denotes the Jacobian of 𝜃 → 𝑓 𝜃 ( 𝑥 ) .
Revised: where 𝐷 𝑓 𝜃 ( 𝑥 ) ∈ Y ⊗ Θ denotes the Jacobian of 𝜃 → 𝑓 𝜃 ( 𝑥 ) .

Original: In rich parametric models, it is preferableto ask questions on 𝑌 ∈ Y rather than on gradients in Θ which is a potentially much bigger space.
Revised: In rich parametric models, it is preferable to ask questions on 𝑌 ∈ Y rather than on gradients in Θ which is a potentially much bigger space.

Original: Ifwe assume that 𝑌 and 𝑓 𝜃 ( 𝑋 ) are bounded in ℓ 2 -norm by 𝑀 ∈ R + , we can adapt (2) and (3) throughthe fact that for any 𝑧 ∈ Y , such that ∥ 𝑧 ∥ ≤ 2 𝑀 , as proven in Appendix B,
Revised: If we assume that 𝑌 and 𝑓 𝜃 ( 𝑋 ) are bounded in ℓ 2 -norm by 𝑀 ∈ R + , we can adapt (2) and (3) through the fact that for any 𝑧 ∈ Y , such that ∥ 𝑧 ∥ ≤ 2 𝑀 , as proven in Appendix B,

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: for 𝑈 uniform on the sphere S 𝑚 − 1 and 𝑉 uniform on [ 0 , 2 𝑀 ] .
Revised: for 𝑈 uniform on the sphere S 𝑚 − 1 and 𝑉 uniform on [ 0 , 2 𝑀 ] .

Original: Applied to 𝑧 = 𝑓 𝜃 ( 𝑋 ) − 𝑌 , it designs an
Revised: Applied to 𝑧 = 𝑓 𝜃 ( 𝑋 ) − 𝑌 , it designs an

Original: SGD procedure by querying information of the type 1 ⟨ 𝑌,𝑈 ⟩ < ⟨ 𝑓 𝜃 ( 𝑋 ) ,𝑈 ⟩− 𝑉 .
Revised: SGD procedure by querying information of the type 1 ⟨ 𝑌,𝑈 ⟩ < ⟨ 𝑓 𝜃 ( 𝑋 ) ,𝑈 ⟩− 𝑉 .

Original: A case for median regression.
Revised: A case for median regression.

Original: Motivated by robustness purposes, we will rather expand on medianregression.
Revised: Motivated by robustness purposes, we will rather expand on median regression.

Original: In general, we would like to learn a function that, given an input, replicates the outputof I/O samples generated by the joint probability 𝜌 .
Revised: In general, we would like to learn a function that, given an input, replicates the output of I/O samples generated by the joint probability 𝜌 .

Original: In many instances, 𝑋 does not characterize allthe sources of variations of 𝑌 , i.e. input features are not rich enough to characterize a unique output,leading to randomness in the conditional distributions ( 𝑌 | 𝑋 ) .
Revised: In many instances, 𝑋 does not characterize all the sources of variations of 𝑌 , i.e. input features are not rich enough to characterize a unique output, leading to randomness in the conditional distributions ( 𝑌 | 𝑋 ) .

Original: When many targets can be linked to avector 𝑥 ∈ X , how to define a consensual 𝑓 ( 𝑥 ) ?
Revised: When many targets can be linked to a vector 𝑥 ∈ X , how to define a consensual 𝑓 ( 𝑥 ) ?

Original: For analytical reasons, statisticians tend to use theleast-squares error which corresponds to asking for 𝑓 ( 𝑥 ) to be the mean of the distribution ( 𝑌 | 𝑋 = 𝑥 ) .
Revised: For analytical reasons, statisticians tend to use the least-squares error which corresponds to asking for 𝑓 ( 𝑥 ) to be the mean of the distribution ( 𝑌 | 𝑋 = 𝑥 ) .

Original: Yet, means are known to be too sensitive to rare but large outputs (see e.g. , Huber, 1981), and cannotbe defined as good and robust consensus in a world of heavy-tailed distributions.
Revised: Yet, means are known to be too sensitive to rare but large outputs (see e.g. , Huber, 1981), and cannot be defined as good and robust consensus in a world of heavy-tailed distributions.

Original: This contrasts withthe median, which, as a consequence, is often much more valuable to summarize a range of values.
Revised: This contrasts with the median, which, as a consequence, is often much more valuable to summarize a range of values.

Original: For instance, median income is preferred over mean income as a population indicator (see e.g. , US
Revised: For instance, median income is preferred over mean income as a population indicator (see e.g. , US

Original: Census Bureau, 2021).
Revised: Census Bureau, 2021).

Original: Median regression.
Revised: Median regression.

Original: The geometric median is variationally defined through the absolute deviationloss, leading to
Revised: The geometric median is variationally defined through the absolute deviation loss, leading to

Original: [Equation]
Revised: [Equation]

Original: Similarly to the least-squares case, we can access weakly supervised stochastic gradients through thefact that for 𝑧 ∈ S 𝑚 − 1 , as shown in Appendix B,
Revised: Similarly to the least-squares case, we can access weakly supervised stochastic gradients through the fact that for 𝑧 ∈ S 𝑚 − 1 , as shown in Appendix B,

Original: E 𝑈 [ sign (⟨ 𝑧,𝑈 ⟩) · 𝑈 ] = 𝑐 2 · 𝑧, where𝑐 2 = E 𝑈
Revised: [Equation] where

Original: [ sign (⟨ 𝑒 1 ,𝑈 ⟩) · ⟨ 𝑒 1 ,𝑈 ⟩] = √ 𝜋 Γ ( 𝑚 − 12 ) 𝑚 Γ ( 𝑚 2 ) , (5) where 𝑈 is uniformly drawn on the sphere S 𝑚 − 1 , and Γ is the gamma function.
Revised: [Equation] where 𝑈 is uniformly drawn on the sphere S 𝑚 − 1 , and Γ is the gamma function.

Original: This suggests
Revised: This suggests

Original: Algorithm 1.
Revised: Algorithm 1.

Original: Data: A model 𝑓 𝜃 for 𝜃 ∈ Θ , some data ( 𝑋 𝑖 ) 𝑖 ≤ 𝑛 , a labeling budget 𝑇 , a step size rule 𝛾 : N → R +
Revised: Data: A model 𝑓 𝜃 for 𝜃 ∈ Θ , some data ( 𝑋 𝑖 ) 𝑖 ≤ 𝑛 , a labeling budget 𝑇 , a step size rule 𝛾 : N → R +

Original: Result:
Revised: Result:

Original: A learned parameter ˆ 𝜃 and the predictive function ˆ 𝑓 = 𝑓 ˆ 𝜃 .
Revised: A learned parameter ˆ 𝜃 and the predictive function ˆ 𝑓 = 𝑓 ˆ 𝜃 .

Original: Initialize 𝜃 0 .
Revised: Initialize 𝜃 0 .

Original: for 𝑡 ← 1 to 𝑇 do
Revised: for 𝑡 ← 1 to 𝑇 do

Original: Sample 𝑈 𝑡 uniformly on S 𝑚 − 1 .
Revised: Sample 𝑈 𝑡 uniformly on S 𝑚 − 1 .

Original: Query 𝜀 = sign (⟨ 𝑌 𝑡 − 𝑧,𝑈 𝑡 ⟩) for 𝑧 = 𝑓 𝜃 𝑡 − 1 ( 𝑋 𝑡 ) .
Revised: Query 𝜀 = sign (⟨ 𝑌 𝑡 − 𝑧,𝑈 𝑡 ⟩) for 𝑧 = 𝑓 𝜃 𝑡 − 1 ( 𝑋 𝑡 ) .

Original: Update the parameter 𝜃 𝑡 = 𝜃 𝑡 − 1 + 𝛾 ( 𝑡 ) 𝜀 · 𝑈 ⊤ 𝑡 ( 𝐷 𝑓 𝜃 𝑡 − 1 ( 𝑋 𝑡 )) .
Revised: Update the parameter 𝜃 𝑡 = 𝜃 𝑡 − 1 + 𝛾 ( 𝑡 ) 𝜀 · 𝑈 ⊤ 𝑡 ( 𝐷 𝑓 𝜃 𝑡 − 1 ( 𝑋 𝑡 )) .

Original: Output ˆ 𝜃 = 𝜃 𝑇 , or some average, e.g. , ˆ 𝜃 = 𝑇 − 1 (cid:205)
Revised: Output ˆ 𝜃 = 𝜃 𝑇 , or some average, e.g. , ˆ 𝜃 = 𝑇 − 1 (cid:205)

Original: 𝑇 𝑡 = 1 𝜃 𝑡 .
Revised: 𝑇𝑡 = 1 𝜃 𝑡 .

Original:  Statistical analysis
Revised:  Statistical analysis

Original: In this section, we quantify the performance of Algorithm 1 by proving optimal rates of convergencewhen the median regression problem is approached with (reproducing) kernels.
Revised: In this section, we quantify the performance of Algorithm 1 by proving optimal rates of convergence when the median regression problem is approached with (reproducing) kernels.

Original: For simplicity, wewill assume that 𝑓 ∗ can be parametrized by a linear model (potentially of infinite dimension).
Revised: For simplicity, we will assume that 𝑓 ∗ can be parametrized by a linear model (potentially of infinite dimension).

Original: Assumption 1.
Revised: Assumption 1.

Original: Assume that the solution 𝑓 ∗ : X → R 𝑚 of the median regression problem (1) and (4)can be parametrized by some separable Hilbert space H , and a bounded feature map 𝜑 : X → H ,such that, for any 𝑖 ∈ [ 𝑚 ] , there exists some 𝜃 ∗ 𝑖 ∈ H such that ⟨ 𝑓 ∗ (·) , 𝑒 𝑖 ⟩ Y
Revised: Assume that the solution 𝑓 ∗ : X → R 𝑚 of the median regression problem (1) and (4) can be parametrized by some separable Hilbert space H , and a bounded feature map 𝜑 : X → H , such that, for any 𝑖 ∈ [ 𝑚 ] , there exists some 𝜃 ∗ 𝑖 ∈ H such that ⟨ 𝑓 ∗ (·) , 𝑒 𝑖 ⟩ Y

Original: =
Revised: =

Original: (cid:10) 𝜃 ∗ 𝑖 , 𝜑 (·) (cid:11) H , where( 𝑒 𝑖 ) is the canonical basis of R 𝑚 .
Revised: (cid:10) 𝜃 ∗ 𝑖 , 𝜑 (·) (cid:11) H , where ( 𝑒 𝑖 ) is the canonical basis of R 𝑚 .

Original: Written into matrix form, there exists 𝜃 ∗ ∈ Y ⊗ H , such that𝑓 ∗ (·)
Revised: Written into matrix form, there exists 𝜃 ∗ ∈ Y ⊗ H , such that 𝑓 ∗ (·)

Original: =
Revised: =

Original: 𝜃 ∗ 𝜑 (·) .
Revised: 𝜃 ∗ 𝜑 (·) .

Original: The curious reader can easily relax this assumption in the realm of reproducing kernel Hilbert spacesfollowing the work of Pillaud-Vivien et al.
Revised: The curious reader can easily relax this assumption in the realm of reproducing kernel Hilbert spaces following the work of Pillaud-Vivien et al.

Original:  Under the linear model of Assumption 1, Algorithm 1is specified with 𝑢 ⊤ 𝐷 𝑓 𝜃 ( 𝑥 ) =
Revised: (2018a). Under the linear model of Assumption 1, Algorithm 1 is specified with 𝑢 ⊤ 𝐷 𝑓 𝜃 ( 𝑥 ) =

Original: 𝑢 ⊗ 𝜑 ( 𝑥 ) .
Revised: 𝑢 ⊗ 𝜑 ( 𝑥 ) .

Original: Note that rather than working with Θ = Y ⊗ H which ispotentially infinite-dimensional, empirical estimates can be represented in the finite-dimensionalspace Y ⊗ Span { 𝜑 ( 𝑋 𝑖 )} 𝑖 ≤ 𝑛 , and well approximated by small-dimensional spaces to ensure efficientcomputations (Williams and Seeger, 2000; Meanti et al., 2020).
Revised: Note that rather than working with Θ = Y ⊗ H which is potentially infinite-dimensional, empirical estimates can be represented in the finitedimensional space Y ⊗ Span { 𝜑 ( 𝑋 𝑖 )} 𝑖 ≤ 𝑛 , and well approximated by small-dimensional spaces to ensure efficient computations (Williams and Seeger, 2000; Meanti et al., 2020).

Original: One of the key points of SGD is that gradient descent is so gradual that one can use noisy or stochasticgradients without loosing statistical guarantees while speeding up computations.
Revised: One of the key points of SGD is that gradient descent is so gradual that one can use noisy or stochastic gradients without loosing statistical guarantees while speeding up computations.

Original: This is especiallytrue when minimizing convex functions that are nor strongly-convex, i.e. , bounded below by aquadratic, nor smooth, i.e. , with Lipschitz-continuous gradient (see, e.g. , Bubeck, 2015).
Revised: This is especially true when minimizing convex functions that are nor strongly-convex, i.e. , bounded below by a quadratic, nor smooth, i.e. , with Lipschitz-continuous gradient (see, e.g. , Bubeck, 2015).

Original: In particular,the following theorem, proven in Appendix A.1, states that Algorithm 1 minimizes the populationrisk at a speed at least proportional to 𝑂 ( 𝑇 − 1 / 2 ) .
Revised: In particular, the following theorem, proven in Appendix A.1, states that Algorithm 1 minimizes the population risk at a speed at least proportional to 𝑂 ( 𝑇 − 1 / 2 ) .

Original: Theorem 1 (Convergence rates) .
Revised: Theorem 1 (Convergence rates) .

Original: Under Assumption 1, and under the knowledge of 𝜅 and 𝑀 tworeal values such that E [∥ 𝜑 ( 𝑋 )∥ 2 ] ≤ 𝜅 2 and ∥ 𝜃 ∗ ∥ ≤ 𝑀 , with a budget 𝑇 ∈ N , a constant step size𝛾 = 𝑀𝜅 √ 𝑇 and the average estimate ˆ 𝜃 = 1 𝑇 (cid:205)
Revised: Under Assumption 1, and under the knowledge of 𝜅 and 𝑀 two real values such that E [∥ 𝜑 ( 𝑋 )∥ 2 ] ≤ 𝜅 2 and ∥ 𝜃 ∗ ∥ ≤ 𝑀 , with a budget 𝑇 ∈ N , a constant step size 𝛾 = 𝑀𝜅 √ 𝑇 and the average estimate ˆ 𝜃 = 1 𝑇 (cid:205)

Original: 𝑇 − 1 𝑡 = 0 𝜃 𝑡 , Algorithm 1 leads to an estimate 𝑓 that suffers froman excess of risk
Revised: 𝑇 − 1 𝑡 = 0 𝜃 𝑡 , Algorithm 1 leads to an estimate 𝑓 that suffers from an excess of risk

Original: [Equation]
Revised: [Equation]

Original: (6) where the expectation is taken with respect to the randomness of ˆ 𝜃 that depends on the dataset ( 𝑋 𝑖 ,𝑌 𝑖 )as well as the questions ( 𝑖 𝑡 , 𝑆 𝑡 ) 𝑡 ≤ 𝑇 .
Revised: ) where the expectation is taken with respect to the randomness of ˆ 𝜃 that depends on the dataset ( 𝑋 𝑖 ,𝑌 𝑖 ) as well as the questions ( 𝑖 𝑡 , 𝑆 𝑡 ) 𝑡 ≤ 𝑇 .

Original: While we give here a result for a fixed step size, one could retake the extensive literature on SGD toprove similar results for decaying step sizes that do not require to know the labeling budget in advance ( e.g. setting 𝛾 ( 𝑡 ) ∝ 𝑡 − 1 / 2 at the expense of an extra term in log ( 𝑇 ) in front of the rates), as well asdifferent averaging strategies (see e.g. , Bach, 2023).
Revised: While we give here a result for a fixed step size, one could retake the extensive literature on SGD to prove similar results for decaying step sizes that do not require to know the labeling budget in advance ( e.g. setting 𝛾 ( 𝑡 ) ∝ 𝑡 − 1 / 2 at the expense of an extra term in log ( 𝑇 ) in front of the rates), as well as different averaging strategies (see e.g. , Bach, 2023).

Original: In practice, one might not know a priori theparameter 𝑀 but could nonetheless find the right scaling for 𝛾 based on cross-validation.
Revised: In practice, one might not know a priori the parameter 𝑀 but could nonetheless find the right scaling for 𝛾 based on cross-validation.

Original: The rate in 𝑂 ( 𝑇 − 1 / 2 ) applies more broadly to all the strategies described in Section 3 as long as theloss ℓ and the parametric model
Revised: The rate in 𝑂 ( 𝑇 − 1 / 2 ) applies more broadly to all the strategies described in Section 3 as long as the loss ℓ and the parametric model

Original: 𝑓 𝜃 ensure that R ( 𝜃 ) is convex and Lipschitz-continuous.
Revised: 𝑓 𝜃 ensure that R ( 𝜃 ) is convex and Lipschitz-continuous.

Original: Althoughthe constants appearing in front of rates depend on the complexity to reconstruct the full gradient∇ 𝜃 ℓ ( 𝑓 𝜃 ( 𝑋 𝑖 ,𝑌 𝑖 )) from the reconstruction scheme (3).
Revised: Although the constants appearing in front of rates depend on the complexity to reconstruct the full gradient ∇ 𝜃 ℓ ( 𝑓 𝜃 ( 𝑋 𝑖 ,𝑌 𝑖 )) from the reconstruction scheme (3).

Original: Those constants correspond to the secondmoment of the stochastic gradient.
Revised: Those constants correspond to the second moment of the stochastic gradient.

Original: For example, for the least-squares technique described earlier onewould have to replace 𝑐 2 by 𝑐 1 in (6).
Revised: For example, for the least-squares technique described earlier one would have to replace 𝑐 2 by 𝑐 1 in (6).

Original: Theorem 2, proven in Appendix A.3, states that any algorithm that accesses a fully supervisedlearning dataset of size 𝑇 cannot beat the rates in 𝑂 ( 𝑇 − 1 / 2 ) , hence any algorithm that collects weakerinformation on ( 𝑌 𝑖 ) 𝑖 ≤ 𝑇 cannot display better rates than the ones verified by Algorithm 1.
Revised: Theorem 2, proven in Appendix A.3, states that any algorithm that accesses a fully supervised learning dataset of size 𝑇 cannot beat the rates in 𝑂 ( 𝑇 − 1 / 2 ) , hence any algorithm that collects weaker information on ( 𝑌 𝑖 ) 𝑖 ≤ 𝑇 cannot display better rates than the ones verified by Algorithm 1.

Original: This provesminimax optimality of our algorithm up to constants.
Revised: This proves minimax optimality of our algorithm up to constants.

Original: Theorem 2 (Minimax optimality) .
Revised: Theorem 2 (Minimax optimality) .

Original: Under Assumption 1 and the knowledge of an upper bound on∥ 𝜃 ∗ ∥ ≤ 𝑀 , assuming that 𝜑 is bounded by 𝜅 , there exists a universal constant 𝑐 3 such that for anyalgorithm A that takes as input D 𝑇 =
Revised: Under Assumption 1 and the knowledge of an upper bound on ∥ 𝜃 ∗ ∥ ≤ 𝑀 , assuming that 𝜑 is bounded by 𝜅 , there exists a universal constant 𝑐 3 such that for any algorithm A that takes as input D 𝑇 =

Original: ( 𝑋 𝑖 ,𝑌 𝑖 ) 𝑖 ≤ 𝑇 ∼ 𝜌 ⊗ 𝑇 for any 𝑇 ∈ N and output a parameter 𝜃 ,
Revised: ( 𝑋 𝑖 ,𝑌 𝑖 ) 𝑖 ≤ 𝑇 ∼ 𝜌 ⊗ 𝑇 for any 𝑇 ∈ N and output a parameter 𝜃 ,

Original: [Equation]
Revised: [Equation]

Original: The supremum over 𝜌 ∈ M 𝑀 has to be understood as the supremum over all distributions 𝜌
Revised: The supremum over 𝜌 ∈ M 𝑀 has to be understood as the supremum over all distributions 𝜌

Original: ∈
Revised: ∈

Original: Δ X × Ysuch that the problem defined through the risk R ( 𝑓 ; 𝜌 ) : =
Revised: Δ X × Y such that the problem defined through the risk R ( 𝑓 ; 𝜌 ) : =

Original: E 𝜌 [ ℓ ( 𝑓 ( 𝑋 ) ,𝑌 )] is minimized for 𝑓 𝜌 thatverifies Assumption 1 with ∥ 𝜃 ∗ ∥ bounded by a constant 𝑀 .
Revised: E 𝜌 [ ℓ ( 𝑓 ( 𝑋 ) ,𝑌 )] is minimized for 𝑓 𝜌 that verifies Assumption 1 with ∥ 𝜃 ∗ ∥ bounded by a constant 𝑀 .

Original: The same theorem applies for least-squares with a different universal constant.
Revised: The same theorem applies for least-squares with a different universal constant.

Original: It should be notedthat minimax lower bounds are in essence quantifying worst cases of a given class of problems.
Revised: It should be noted that minimax lower bounds are in essence quantifying worst cases of a given class of problems.

Original: Inparticular, to prove Theorem 2, we consider distributions that lead to hard problems; more specifically,we assumed the variance of the conditional distribution ( 𝑌 | 𝑋 ) to be high.
Revised: In particular, to prove Theorem 2, we consider distributions that lead to hard problems; more specifically, we assumed the variance of the conditional distribution ( 𝑌 | 𝑋 ) to be high.

Original: The practitioner shouldkeep in mind that it is possible to add additional structure on the solution, leverage active learningor semi-supervised strategy such as uncertainty sampling (Nguyen et al., 2021), or Laplacianregularization (Zhu et al., 2003; Cabannes et al., 2021a), and reduce the optimal rates of convergence.
Revised: The practitioner should keep in mind that it is possible to add additional structure on the solution, leverage active learning or semi-supervised strategy such as uncertainty sampling (Nguyen et al., 2021), or Laplacian regularization (Zhu et al., 2003; Cabannes et al., 2021a), and reduce the optimal rates of convergence.

Original: To conclude this section, let us remark that most of our derivations could easily be refined forpractitioners facing a slightly different cost model for annotation.
Revised: To conclude this section, let us remark that most of our derivations could easily be refined for practitioners facing a slightly different cost model for annotation.

Original: In particular, they might preferto perform batches of annotations before updating 𝜃 rather than modifying the question strategyafter each input annotation.
Revised: In particular, they might prefer to perform batches of annotations before updating 𝜃 rather than modifying the question strategy after each input annotation.

Original: This would be similar to mini-batching in gradient descent.
Revised: This would be similar to mini-batching in gradient descent.

Original: Indeed, thedependency of our result on the annotation cost model and on Assumption 1 should not be seen as alimitation but rather as a proof of concept.
Revised: Indeed, the dependency of our result on the annotation cost model and on Assumption 1 should not be seen as a limitation but rather as a proof of concept.

Original:  Numerical analysis
Revised:  Numerical analysis

Original: In this section, we illustrate the differences between our active method versus a classical passivemethod, for regression and classification problems.
Revised: In this section, we illustrate the differences between our active method versus a classical passive method, for regression and classification problems.

Original: Extensive details are provided in Appendix E.
Revised: Extensive details are provided in Appendix E.

Original: Our code is available online at anonymized-url .
Revised: Our code is available online at https://github.com/VivienCabannes/active-labeling .

Original: Let us begin with the regression problem that consists in estimating the function 𝑓 ∗ that maps𝑥 ∈ [ 0 , 1 ] to sin ( 2 𝜋𝑥 ) ∈ R .
Revised: Let us begin with the regression problem that consists in estimating the function 𝑓 ∗ that maps 𝑥 ∈ [ 0 , 1 ] to sin ( 2 𝜋𝑥 ) ∈ R .

Original: Such a regular function, which belongs to any Hölder or Sobolev classesof functions, can be estimated with the Gaussian kernel, which would ensure Assumption 1, andthat corresponds to a feature map 𝜑
Revised: Such a regular function, which belongs to any Hölder or Sobolev classes of functions, can be estimated with the Gaussian kernel, which would ensure Assumption 1, and that corresponds to a feature map 𝜑

Original: such that 𝑘 ( 𝑥, 𝑥 ′ ) :
Revised: such that 𝑘 ( 𝑥, 𝑥 ′ ) :

Original: = ⟨ 𝜑 ( 𝑥 ) , 𝜑 ( 𝑥 ′ )⟩ = exp (− | 𝑥 − 𝑥 ′ | /( 2 𝜎 2 ))
Revised: = ⟨ 𝜑 ( 𝑥 ) , 𝜑 ( 𝑥 ′ )⟩ = exp (− | 𝑥 − 𝑥 ′ | /( 2 𝜎 2 ))

Original: forany bandwidth parameter 𝜎 > 0.
Revised: for any bandwidth parameter 𝜎 > 0.

Original: On Figure 1, we focus on estimating 𝑓 ∗ given data ( 𝑋 𝑖 )
Revised: On Figure 1, we focus on estimating 𝑓 ∗ given data ( 𝑋 𝑖 )

Original: 𝑖 ∈[ 𝑇 ] thatare uniform on [ 0 , 1 ] in the noiseless setting where 𝑌 𝑖 = 𝑓 ∗ ( 𝑋 𝑖 ) , based on the minimization of theabsolute deviation loss.
Revised: 𝑖 ∈ [ 𝑇 ] that are uniform on [ 0 , 1 ] in the noiseless setting where 𝑌 𝑖 = 𝑓 ∗ ( 𝑋 𝑖 ) , based on the minimization of the absolute deviation loss.

Original: The passive baseline consists in randomly choosing a threshold 𝑈 𝑖 ∼ N ( 0 , 1 ) and acquiring the observations ( 1 𝑌 𝑖 >𝑈 𝑖 )
Revised: The passive baseline consists in randomly choosing a threshold 𝑈 𝑖 ∼ N ( 0 , 1 ) and acquiring the observations ( 1 𝑌 𝑖 >𝑈 𝑖 )

Original: 𝑖 ∈
Revised: 𝑖 ∈

Original: [ 𝑇 ] that can be cast as the observation of the half-space𝑆 𝑖 = (cid:8) 𝑦 ∈ Y (cid:12)(cid:12) 1 𝑦>𝑈 𝑖
Revised: [ 𝑇 ] that can be cast as the observation of the half-space 𝑆 𝑖 = (cid:8) 𝑦 ∈ Y (cid:12)(cid:12) 1 𝑦>𝑈 𝑖

Original: = 1 𝑌 𝑖 >𝑈 𝑖 (cid:9) = :
Revised: = 1 𝑌 𝑖 >𝑈 𝑖 (cid:9) = :

Original: 𝑠 ( 𝑌 𝑖 ,𝑈 𝑖 ) .
Revised: 𝑠 ( 𝑌 𝑖 ,𝑈 𝑖 ) .

Original: In this noiseless setting, a good baseline to learn 𝑓 ∗ fromthe data ( 𝑋 𝑖 , 𝑆 𝑖 ) is provided by the infimum loss characterization (see Cabannes et al., 2020)
Revised: In this noiseless setting, a good baseline to learn 𝑓 ∗ from the data ( 𝑋 𝑖 , 𝑆 𝑖 ) is provided by the infimum loss characterization (see Cabannes et al., 2020)

Original: [Equation]
Revised: [Equation]

Original: arg [Equation] 𝑦 )] , where the distribution over 𝑋 corresponds to the marginal of 𝜌 over X , and the distribution over( 𝑆 | 𝑋 = 𝑥 ) is the pushforward of 𝑈 ∼ N ( 0 , 1 ) under 𝑠 ( 𝑓 ∗ ( 𝑥 ) , ·) .
Revised: where the distribution over 𝑋 corresponds to the marginal of 𝜌 over X , and the distribution over ( 𝑆 | 𝑋 = 𝑥 ) is the pushforward of 𝑈 ∼ N ( 0 , 1 ) under 𝑠 ( 𝑓 ∗ ( 𝑥 ) , ·) .

Original: The left plot on Figure 1 correspondsto an instance of SGD on such an objective based on the data ( 𝑋 𝑖 , 𝑆 𝑖 ) , while the right plot correspondsto Algorithm 1.
Revised: The left plot on Figure 1 corresponds to an instance of SGD on such an objective based on the data ( 𝑋 𝑖 , 𝑆 𝑖 ) , while the right plot corresponds to Algorithm 1.

Original: We take the same hyperparameters for both plots, a bandwidth 𝜎 = 0 .
Revised: We take the same hyperparameters for both plots, a bandwidth 𝜎 = 0 .

Original: 2 and an SGDstep size 𝛾 = 0 .
Revised: 2 and an SGD step size 𝛾 = 0 .

Original: We refer the curious reader to Figure 7 in Appendix E for plots illustrating thestreaming history, and to Figure 10 for “real-world” experiments.[Figure]
Revised: We refer the curious reader to Figure 7 in Appendix E for plots illustrating the streaming history, and to Figure 10 for “real-world” experiments.

Original: To illustrate the versatility of our method, we approach a classification problem through the mediansurrogate technique presented in Proposition 3.
Revised: To illustrate the versatility of our method, we approach a classification problem through the median surrogate technique presented in Proposition 3.

Original: To do so, we consider the classification problemwith 𝑚 ∈ N classes, X
Revised: To do so, we consider the classification problem [Figure] Above Below

Original: =
Revised: [Figure]

Original:  [ 0 , 1 ] and the conditional distribution ( 𝑌 | 𝑋 ) linearly interpolating between
Revised: with 𝑚 ∈ N classes, X = [ 0 , 1 ] and the conditional distribution ( 𝑌 | 𝑋 ) linearly interpolating between

Original: Dirac in 𝑦 1 , 𝑦 2 and 𝑦 3 respectively for 𝑥 = 0, 𝑥 = 1 / 2 and 𝑥 = 1 and the uniform distribution for𝑥 = 1 / 4 and 𝑥 = 3 / 4; and 𝑋 uniform on X \ ([ 1 / 4 − 𝜀, 1 / 4 + 𝜀 ] ∪ [ 3 / 4 − 𝜀, 3 / 4 + 𝜀 ]) . 
Revised: Dirac in 𝑦 1 , 𝑦 2 and 𝑦 3 respectively for 𝑥 = 0, 𝑥 = 1 / 2 and 𝑥 = 1 and the uniform distribution for 𝑥 = 1 / 4 and 𝑥 = 3 / 4; and 𝑋 uniform on X \ ([ 1 / 4 − 𝜀, 1 / 4 + 𝜀 ] ∪ [ 3 / 4 − 𝜀, 3 / 4 + 𝜀 ]) . [Figure]

Original:  Discussion 
Revised:  Discussion 7.1 Discrete output problems

Original: 7.1 Discrete output problems

Revised: In this section, we discuss casting Algorithm 1 into a procedure to tackle discrete-output problems, by leveraging surrogate regression tasks.

Original: Learning problems with discrete output spaces are not as well understood as regression problems.
Revised: Learning problems with discrete output spaces are not as well understood as regression problems.

Original: This is a consequence of the complexity of dealing with combinatorial structures in contrast withcontinuous metric spaces.
Revised: This is a consequence of the complexity of dealing with combinatorial structures in contrast with continuous metric spaces.

Original: In particular, gradients are not defined for discrete output models.
Revised: In particular, gradients are not defined for discrete output models.

Original: Thecurrent state-of-the-art framework to deal with discrete output problems is to introduce a continuoussurrogate problem whose solution can be decoded as a solution on the original problem (Bartlett et al.
Revised: The current state-of-the-art framework to deal with discrete output problems is to introduce a continuous surrogate problem whose solution can be decoded as a solution on the original problem (Bartlett et al.,

Original: For example, one could solve a classification task with a median regression surrogate problem,which is the object of the next proposition, proven in Appendix C.
Revised: For example, one could solve a classification task with a median regression surrogate problem, which is the object of the next proposition, proven in Appendix C.

Original: Proposition 3 (Consistency of median surrogate) .
Revised: Proposition 3 (Consistency of median surrogate) .

Original: The classification setting where Y is a finite space,and ℓ : Y × Y → R is the zero-one loss ℓ ( 𝑦, 𝑧 )
Revised: The classification setting where Y is a finite space, and ℓ : Y × Y → R is the zero-one loss ℓ ( 𝑦, 𝑧 )

Original: = 1 𝑦 ≠ 𝑧 can be solved as a regression task through thesimplex embedding of Y in R Y with the orthonormal basis ( 𝑒 𝑦 ) 𝑦 ∈ Y .
Revised: = 1 𝑦 ≠ 𝑧 can be solved as a regression task through the simplex embedding of Y in R Y with the orthonormal basis ( 𝑒 𝑦 ) 𝑦 ∈ Y .

Original: More precisely, if 𝑔 ∗ : X → R Yis the minimizer of the median surrogate risk R 𝑆 ( 𝑔 ) =
Revised: More precisely, if 𝑔 ∗ : X → R Y is the minimizer of the median surrogate risk R 𝑆 ( 𝑔 ) =

Original: E [∥ 𝑔 ( 𝑋 ) − 𝑒 𝑌 ∥] , then 𝑓 ∗ : X → Y definedas 𝑓 ∗ ( 𝑥 ) =
Revised: E [∥ 𝑔 ( 𝑋 ) − 𝑒 𝑌 ∥] , then 𝑓 ∗ : X → Y defined as 𝑓 ∗ ( 𝑥 ) =

Original: arg max 𝑦 ∈ Y 𝑔 ∗ 𝑦 ( 𝑥 ) minimizes the original risk R ( 𝑓 ) =
Revised: arg max 𝑦 ∈ Y 𝑔 ∗ 𝑦 ( 𝑥 ) minimizes the original risk R ( 𝑓 ) =

Original: E [ ℓ ( 𝑓 ( 𝑋 ) ,𝑌 )] .
Revised: E [ ℓ ( 𝑓 ( 𝑋 ) ,𝑌 )] .

Original: More generally, any discrete output problem can be solved by reusing the consistent least-squaressurrogate of Ciliberto et al.
Revised: More generally, any discrete output problem can be solved by reusing the consistent least-squares surrogate of Ciliberto et al.

Original: Algorithm 1 can be adapted to the least-squares problem based on[Figure] specifications at the beginning of Section 4.
Revised: Algorithm 1 can be adapted to the least-squares problem based on specifications at the beginning of Section 4.

Original: This allows using our method in an off-the-shelve fashionfor all discrete output problems.
Revised: This allows using our method in an off-the-shelve fashion for all discrete output problems.

Original:  In this setting, Theorem 1 can be refined under margin conditionswhere our approach would exhibit exponential convergence rates as illustrated on Figure 2.
Revised: For example, a problem consisting in ranking preferences over 𝑚 items can be approached with the Kendall correlation loss ℓ ( 𝑦, 𝑧 ) = − 𝜑 ( 𝑦 ) ⊤ 𝜑 ( 𝑧 ) with 𝜑 ( 𝑦 ) = ( 1 𝑦 ( 𝑖 ) >𝑦 ( 𝑗 ) ) for 𝑖 < 𝑗 ≤ 𝑚 where 𝑦 and 𝑧 are permutation over [ 𝑚 ] that encodes the rank of each element in terms of user preferences. In this setting, the surrogate task introduced by (Ciliberto et al., 2020) consists in learning 𝑔 ( 𝑥 ) =

Revised: E [ 𝜑 ( 𝑌 )| 𝑋 = 𝑥 ] as a least-squares problem.

Revised: The half-space surrogate queries translate directly into the questions (cid:205)

Revised: 𝑖<𝑗 ≤ 𝑚 𝑤 ( 𝑖, 𝑗 )

Revised: 1 𝑦 ( 𝑖 ) >𝑦 ( 𝑗 ) > 𝑐 for some ( 𝑤 ( 𝑖, 𝑗 )) , 𝑐 in R .

Revised: In particular, if 𝑈 is chosen to be uniform on the canonical basis (rather than on the sphere), those questions translate into pairwise orderings ( e.g. does user 𝑥 prefer movie 𝑖 or movie 𝑗 ?).

Revised: In terms of guarantee akin to Theorem 1, retaking the calibration inequality of (Ciliberto et al., 2020), we get convergence rates of the form 𝑚 3 / 2 𝑇 − 1 / 4 .

Original: As a sidenote, while we are not aware of any generic theory encompassing the absolute-deviation surrogate  of 
Revised: In terms of guarantee akin to Theorem 2, since we need as least log 2 ( 𝑚 ! ) ≃ 𝑚 log ( 𝑚 ) binary queries to discriminate between 𝑚 ! permutations, we can expect a lower bound in 𝑚 1 / 2 log ( 𝑚 ) 1 / 2 𝑇 − 1 / 2 . More generally, many ranking problems can be approached with correlation losses and tackled through surrogate regression problems on the convex hulls of some well-known polytopes such as the Birkhoff polytope or the permutohedron ( e.g.

Revised: Ailon, 2014).

Revised: Although their descriptions is out-of-scope of this paper, linear cuts of those polytopes form well-structured queries sets.

Revised: For example, the faces of all dimensions of the permutohedron correspond, in a one-to-one fashion, to strict weak orderings (Ziegler, 1995).

Original: Proposition 3, we showcase its superiority over least-squares on at least two types of problems on 
Revised: In those discrete settings, Theorem 1 can be refined under low noise conditions. In particular, under generalization of the Massart noise condition, our approach could even exhibit exponential convergence rates as illustrated on Figure 2. For classification problems, this condition can be expressed as the existence of a threshold 𝛿 > 0 such that for almost all 𝑥 ∈ X and 𝑧 ∈ Y , we have P ( 𝑌 = 𝑓 ( 𝑥 )| 𝑋 = 𝑥 ) − P ( 𝑌 = 𝑧 | 𝑋 = 𝑥 ) ∉ ( 0 , 𝛿 ) .

Original: Figures 3 and 4 in Appendix C.
Revised: Arguably, this assumption is met on well-curated images dataset such as ImageNet or CIFAR10, where for each input 𝑋 the most probable class has always more than e.g. 60% of chance to be the target 𝑌 . When this assumption hold together with Assumption 1 (when the surrogate target 𝑔 ∗ belongs to the RKHS and the kernel is bounded), then the right hand-side of equation (6) can be replaced by exp (− 𝑐𝑇 ) for some constant 𝑐 . The proof would be a simple adaptation of the works of (Pillaud-Vivien et al., 2018a; Cabannes et al., 2021b) to our case.

Original: 7.2 Supervised learning baseline with resampling
Revised: 7.2 Supervised learning baseline with resampling

Revised: In this section, we discuss simple supervised learning baselines that compete with Algorithm 1 when resampling is allowed.

Original: When resampling is allowed a simple baseline for the active labeling problem is provided by supervisedlearning.
Revised: When resampling is allowed a simple baseline for the active labeling problem is provided by supervised learning.

Original: In regression problems with the query of any half-space, a method that consists in annotatingeach ( 𝑌 𝑖 ) 𝑖 ≤ 𝑛 ( 𝑇,𝜀 ) up to precision 𝜀 , before using any supervised learning method to learn 𝑓 from( 𝑋 𝑖 ,𝑌 𝑖 ) 𝑖 ≤ 𝑛 ( 𝑇,𝜀 ) could acquire 𝑛 ( 𝑇, 𝜀 ) ≃ 𝑇 / 𝑚 log 2 ( 𝜀 − 1 ) data points with a dichotomic search along alldirections, assuming 𝑌 𝑖 bounded or sub-Gaussian.
Revised: In regression problems with the query of any half-space, a method that consists in annotating each ( 𝑌 𝑖 ) 𝑖 ≤ 𝑛 ( 𝑇,𝜀 ) up to precision 𝜀 , before using any supervised learning method to learn 𝑓 from ( 𝑋 𝑖 ,𝑌 𝑖 ) 𝑖 ≤ 𝑛 ( 𝑇,𝜀 ) could acquire 𝑛 ( 𝑇, 𝜀 ) ≃ 𝑇 / 𝑚 log 2 ( 𝜀 − 1 ) data points with a dichotomic search along all directions, assuming 𝑌 𝑖 bounded or sub-Gaussian.

Original: In terms of minimax rates, such a procedure cannotperform better than in 𝑛 ( 𝑇, 𝜀 ) − 1 / 2 + 𝜀 , the first term being due to the statistical limit in Theorem 2,the second due to the incertitude 𝜀 on each 𝑌 𝑖 that transfers to the same level of incertitude on 𝑓 .
Revised: In terms of minimax rates, such a procedure cannot perform better than in 𝑛 ( 𝑇, 𝜀 ) − 1 / 2 + 𝜀 , the first term being due to the statistical limit in Theorem 2, the second due to the incertitude 𝜀 on each 𝑌 𝑖 that transfers to the same level of incertitude on 𝑓 .

Original: Optimizing with respect to 𝜀 yields a bound in 𝑂 ( 𝑇 − 1 / 2 log ( 𝑇 ) 1 / 2 ) .
Revised: Optimizing with respect to 𝜀 yields a bound in 𝑂 ( 𝑇 − 1 / 2 log ( 𝑇 ) 1 / 2 ) .

Original: Therefore, this not-so-naivebaseline is only suboptimal by a factor log ( 𝑇 ) 1 / 2 .
Revised: Therefore, this not-so-naive baseline is only suboptimal by a factor log ( 𝑇 ) 1 / 2 .

Original: In the meanwhile, Algorithm 1 can be rewrittenwith resampling, as well as Theorem 1, which we prove in Appendix A.2.
Revised: In the meanwhile, Algorithm 1 can be rewritten with resampling, as well as Theorem 1, which we prove in Appendix A.2.

Original: Hence, our technique willstill achieve minimax optimality for the problem “with resampling”.
Revised: Hence, our technique will still achieve minimax optimality for the problem “with resampling”.

Original: In other terms, by decidingto acquire more imprecise information, our algorithm reduces annotation cost for a given level ofgeneralization error (or equivalently reduces generalization error for a given annotation budget) by afactor log ( 𝑇 ) 1 / 2 when compared to this baseline.
Revised: In other terms, by deciding to acquire more imprecise information, our algorithm reduces annotation cost for a given level of generalization error (or equivalently reduces generalization error for a given annotation budget) by a factor log ( 𝑇 ) 1 / 2 when compared to this baseline.

Original: The picture is slightly different for discrete-output problems.
Revised: The picture is slightly different for discrete-output problems.

Original: If one can ask any question 𝑠 ∈ 2 Y thenwith a dichotomic search, one can retrieve any label with log 2 ( 𝑚 ) questions.
Revised: If one can ask any question 𝑠 ∈ 2 Y then with a dichotomic search, one can retrieve any label with log 2 ( 𝑚 ) questions.

Original: Hence, to theoreticallybeat the fully supervised baseline with the SGD method described in Section 3, one would have toderive a gradient strategy (2) with a small enough second moment ( e.g. , for convex losses that arenon-smooth nor strongly convex, the increase in the second moment compared to the usual stochasticgradients should be no greater than log 2 ( 𝑚 ) 1 / 2 ).
Revised: Hence, to theoretically beat the fully supervised baseline with the SGD method described in Section 3, one would have to derive a gradient strategy (2) with a small enough second moment ( e.g. , for convex losses that are non-smooth nor strongly convex, the increase in the second moment compared to the usual stochastic gradients should be no greater than log 2 ( 𝑚 ) 1 / 2 ).

Original: How to best refine our technique to better take intoaccount the discrete structure of the output space is an open question.
Revised: How to best refine our technique to better take into account the discrete structure of the output space is an open question.

Original: Introducing bias that does notmodify convergence properties while reducing variance eventually thanks to importance sampling isa potential way to approach this problem.
Revised: Introducing bias that does not modify convergence properties while reducing variance eventually thanks to importance sampling is a potential way to approach this problem.

Original: A simpler idea would be to remember information of the type 𝑌 𝑖 ∈ 𝑠 to restrict the questions asked in order to locate 𝑓 𝜃 𝑡 ( 𝑋 𝑖 ) − 𝑌 𝑖 when performing stochasticgradient descent with resampling.
Revised: A simpler idea would be to remember information of the type 𝑌 𝑖 ∈ 𝑠 to restrict the questions asked in order to locate 𝑓 𝜃 𝑡 ( 𝑋 𝑖 ) − 𝑌 𝑖 when performing stochastic gradient descent with resampling.

Original: Combinatorial bandits might also provide helpful insights on thematter.
Revised: Combinatorial bandits might also provide helpful insights on the matter.

Original: Ultimately, we would like to build an understanding of the whole distribution ( 𝑌 | 𝑋 ) and notonly of 𝑓 ∗ ( 𝑋 ) as we explore labels in order to refine this exploration.
Revised: Ultimately, we would like to build an understanding of the whole distribution ( 𝑌 | 𝑋 ) and not only of 𝑓 ∗ ( 𝑋 ) as we explore labels in order to refine this exploration.

Original: 7.3 Min-max approaches
Revised: 7.3 Min-max approaches

Revised: In this section, we discuss potential extensions of our SGD procedure, based on min-max variational objectives.

Original: Min-max approaches have been popularized for searching games and active learning, where onesearches for the question that minimizes the size of the space where a potential guess could lie underthe worst possible answer to that question.
Revised: Min-max approaches have been popularized for searching games and active learning, where one searches for the question that minimizes the size of the space where a potential guess could lie under the worst possible answer to that question.

Original: A particularly well illustrative example is the solution ofthe Mastermind game proposed by Knuth (1977).
Revised: A particularly well illustrative example is the solution of the Mastermind game proposed by Knuth (1977).

Original: While our work leverages plain SGD, one couldbuild on the vector field point-of-view of gradient descent (see, e.g. , Bubeck, 2015) to tackle min-maxconvex concave problems with similar guarantees.
Revised: While our work leverages plain SGD, one could build on the vector field point-of-view of gradient descent (see, e.g. , Bubeck, 2015) to tackle min-max convex concave problems with similar guarantees.

Original: In particular, we could design weakly supervisedlosses 𝐿 ( 𝑓 ( 𝑥 ) , 𝑠 ; 1 𝑦 ∈ 𝑠 ) and min-max games where a prediction player aims at minimizing such a losswith respect to the prediction 𝑓 , while the query player aims at maximizing it with respect to thequestion 𝑠 , that is querying information that best elicit mistakes made by the prediction player.
Revised: In particular, we could design weakly supervised losses 𝐿 ( 𝑓 ( 𝑥 ) , 𝑠 ; 1 𝑦 ∈ 𝑠 ) and min-max games where a prediction player aims at minimizing such a loss with respect to the prediction 𝑓 , while the query player aims at maximizing it with respect to the question 𝑠 , that is querying information that best elicit mistakes made by the prediction player.

Original: Forexample, the dual norm characterization of the norm leads to the following min-max approach to themedian regression
Revised: For example, the dual norm characterization of the norm leads to the following min-max approach to the median regression

Original: [Equation]
Revised: [Equation]

Original: Such min-max formulations would be of interest if they lead to improvement of computational andstatistical efficiencies, similarly to the work of Babichev et al.
Revised: Such min-max formulations would be of interest if they lead to improvement of computational and statistical efficiencies, similarly to the work of Babichev et al.

Original: For classification problems, thefollowing proposition introduces such a game and suggests its suitability.
Revised: For classification problems, the following proposition introduces such a game and suggests its suitability.

Original: Its proof can be found in
Revised: Its proof can be found in

Original: Appendix D.
Revised: Appendix D.

Original: Proposition 4.
Revised: Proposition 4.

Original: Consider the classification problem of learning 𝑓 ∗ : X → Y where Y is of finitecardinality, with the 0-1 loss ℓ ( 𝑧, 𝑦 )
Revised: Consider the classification problem of learning 𝑓 ∗ : X → Y where Y is of finite cardinality, with the 0-1 loss ℓ ( 𝑧, 𝑦 )

Original: = 1 𝑧 ≠ 𝑦 , minimizing the risk (1) under a distribution 𝜌 on X × Y .
Revised: = 1 𝑧 ≠ 𝑦 , minimizing the risk (1) under a distribution 𝜌 on X × Y .

Original: Introduce the surrogate score functions 𝑔 : X → Δ Y ; 𝑥
Revised: Introduce the surrogate score functions 𝑔 : X → Δ Y ; 𝑥

Original: → 𝑣 where 𝑣 = ( 𝑣 𝑦 ) 𝑦
Revised: → 𝑣 where 𝑣 = ( 𝑣 𝑦 ) 𝑦

Original: ∈ Y is a family ofnon-negative weights that sum to one, as well as the surrogate loss function 𝐿 : Δ Y × S × {− 1 , 1 } →323 R ; ( 𝑣, 𝑆, 𝜀 ) =
Revised: ∈ Y is a family of non-negative weights that sum to one, as well as the surrogate loss function 𝐿 : Δ Y × S × {− 1 , 1 } → R ; ( 𝑣, 𝑆, 𝜀 ) =

Original: 𝜀 ( 1 − 2 (cid:205) 𝑦 ∈ 𝑆 𝑣 𝑦 ) , and the min-max gamemin 𝑔
Revised: 𝜀 ( 1 − 2 (cid:205) 𝑦 ∈ 𝑆 𝑣 𝑦 ) , and the min-max game

Original: [Equation]
Revised: [Equation]

Original: When S contains the singletons and with the low-noise condition that P ( 𝑌 ≠ 𝑓 ∗ ( 𝑥 ) | 𝑋 = 𝑥 ) < 1 / 2almost everywhere, then 𝑓 ∗ can be learned through the relation 𝑓 ∗ ( 𝑥 ) =
Revised: When S contains the singletons and with the low-noise condition that P ( 𝑌 ≠ 𝑓 ∗ ( 𝑥 ) | 𝑋 = 𝑥 ) < 1 /almost everywhere, then 𝑓 ∗ can be learned through the relation 𝑓 ∗ ( 𝑥 ) =

Original: arg min 𝑦 ∈ Y 𝑔 ∗ ( 𝑥 ) 𝑦 for theunique minimizer 𝑔 ∗ of (8) .
Revised: arg min 𝑦 ∈ Y 𝑔 ∗ ( 𝑥 ) 𝑦 for the unique minimizer 𝑔 ∗ of (8) .

Original: Moreover, the minimization of the empirical version of this objectivewith the stochastic gradient updates for saddle point problems provides a natural “active labeling”scheme to find this 𝑔 ∗ .
Revised: Moreover, the minimization of the empirical version of this objective with the stochastic gradient updates for saddle point problems provides a natural “active labeling” scheme to find this 𝑔 ∗ .

Revised: On the one hand, this min-max formulation could help to easily incorporate restrictions on the sets to query.

Revised: On the other hand, it is not completely clear how to best update (or derive an unbiased stochastic gradient strategy for) the adversarial query strategy 𝜇 based on partial information.

Original:  Conclusion
Revised:  Conclusion

Original: We have introduced the “active labeling” problem, which corresponds to “active partially supervisedlearning”.
Revised: We have introduced the “active labeling” problem, which corresponds to “active partially supervised learning”.

Original: We provided a solution to this problem based on stochastic gradient descent.
Revised: We provided a solution to this problem based on stochastic gradient descent.

Original: Although ourmethod can be used for any discrete output problem, we detailed how it works for median regression,where we show that it optimizes the generalization error for a given annotation budget.
Revised: Although our method can be used for any discrete output problem, we detailed how it works for median regression, where we show that it optimizes the generalization error for a given annotation budget.

Original: In a nearfuture, we would like to focus on better exploiting the discrete structure of classification problems,eventually with resampling strategies.
Revised: In a near future, we would like to focus on better exploiting the discrete structure of classification problems, eventually with resampling strategies.

Original: Understanding more precisely the key issues in applications concerned with privacy, and studyinghow weak gradients might provide a good trade-off between learning efficiently and revealing toomuch information also provide interesting follow-ups.
Revised: Understanding more precisely the key issues in applications concerned with privacy, and studying how weak gradients might provide a good trade-off between learning efficiently and revealing too much information also provide interesting follow-ups.

Original: Finally, regarding dataset annotation, exploringdifferent paradigms of weakly supervised learning would lead to different active weakly supervisedlearning frameworks.
Revised: Finally, regarding dataset annotation, exploring different paradigms of weakly supervised learning would lead to different active weakly supervised learning frameworks.

Original: While this work is based on partial labeling, similar formalization could bemade based on other weak supervision models, such as aggregation ( e.g. , Ratner et al., 2020), orgroup statistics (Dietterich et al., 1997).
Revised: While this work is based on partial labeling, similar formalization could be made based on other weak supervision models, such as aggregation ( e.g. , Ratner et al., 2020), or group statistics (Dietterich et al., 1997).

Original: In particular, annotating a huge dataset is often done by bagging inputs according to predicted labels and correcting errors that can be spotted on those bagsof inputs (Deng et al., 2009).
Revised: In particular, annotating a huge dataset is often done by bagging inputs according to predicted labels and correcting errors that can be spotted on those bags of inputs (Deng et al., 2009).

Original: We left for future work the study of variants of the “active labeling”problem that model those settings.Checklist
Revised: We left for future work the study of variants of the “active labeling” problem that model those settings.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? [Yes]
Revised: Acknowledgments and Disclosure of Funding

Original: (b) Did you describe the limitations of your work?

Original: [Yes] See discussion section.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]

Original: This work

Original: aims at developping advanced techniques to learn without too much supervision.

Original: Such

Original: a quest of increasing AI systems capability at a reduced human labor cost is associated

Original: with broad societal issues.

Original: Those questions being really generic, we did not mentionthem in the main text.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main

Original: experimental results (either in the supplemental material or as a URL)?

Original: [Yes](b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes]

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments

Original: multiple times)?

Original: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e

Original: g., typeof GPUs, internal cluster, or cloud provider)?

Original: [N/A]

Original: The experiments were run on a

Original: personal laptop and did not require many charges.

Original: Indeed, the amount ofcompute for experiments were similar to the amount used to write this paper. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: This work was funded in part by the French government under management of Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR-19-P3IA- (PRAIRIE 3IA Institute).

Original: [Yes] Although we have not cited the creators of some L A TEX libraries we used such as
Revised: We also acknowledge support of the European Research Council (grants

Original: Michael

Original: Sharpe and the

Original: newtx package which we used for fonts in our text

Original: (b) Did you mention the license of the assets? [N/A] Numpy and LIBSVM are under Berkeley Software Distribution licenses (respectively the liberal and revised ones), Python and matplotlib are under the Python Software Foundation license.
Revised: SEQUOIA 724063 and REAL 94790).

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?

Original: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?

Original: [N/A]

Original: If you

Original: used

Original: crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?

Original: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?

Original: [N/A]