Venue: RBCDSAI DAI 2023 Conference

Reviews:

final_score: rating: 3: Accept 
review_content: summary: The authors propose an approach to address the labeling bias in active learning algorithms
strengths: - The authors propose an approach to address the labeling bias. This problem is increasingly becoming important and the preliminary work of the authors is timely. 
- The authors evaluate their work by using 5 different datasets from different domains. 


weaknesses: - The authors do not cite the review that shows "about 63% of the works involved the usage of human-annotated labels"
- It is not clear how is the distance from the mean computed. Specifically, what are the thresholds used to bin/label the data points as farther from the mean or closer to the mean.
-  the terms in equation (1) need to be elaborated. It is not clear what is $p_k$, $x^u$ in $sim(x, x^u)$, and do the authors imply that the entropy of each $x$ needs to be computed after every query?
- It is also not clear how is the mean being computed in the various datasets. For instance, what is the mean in the wine data set? Is it the mean of the amount of alcohol content, the color, the amount of sulfates etc? 
- It is not clear how reference [12] is the wine dataset. The same goes for the other datasets as well. If the datasets used in this work are publicly available datasets, then the reference should point to the dataset.  


Nitpicks.

- There are spaces missing in multiple places. For instance "offer biased labels.AL" and "this class[4, 5] considered the presence of human bias as random or a uniformly distributed error while proposing novel algorithms.J.Du et.al.[6]" and "including simple decision heuristics[7]"
- Please rephrase the references based on the following guidelines: https://www.read.seas.harvard.edu/~kohler/latex.html#refcap
- A couple of paragraphs would need to be rewritten because of grammatical mistakes. An example is as follows. "This hypothesis can be illustrated with a case where the task is to classify a car’s condition based on its usage period (Let the average usage be five years). Intuitively, the human oracle would find it easier to classify cars that are 2/10 years old than a car that has been used for five years. i.e., Cars with attribute values closer to their mean."
-
review_title: null

Meta-review: Thank you for submitting your work to the Deployable AI conference, which is jointly organized by the Center for Responsible AI (CeRAI) & Robert Bosch Center for Deployable AI (RBC-DSAI), IIT Madras, Aalto University, University of Helsinki, University of Jyväskylä, and Oulu University.

We are pleased to inform you that your article is accepted. We invite you to present poster in GatherTown on Day-3 of the conference, on Wednesday June 7th. The poster should be a PDF in landscape mode in A1 page size. Please send your posters into smevents@rbcdsai.org, and projectofficer@rbcdsai.org by 10 am IST Monday 5.6.23. You will receive the Zoom links with the calendar invites for your presentations by Friday 2.6.23

All presenting authors must register for the conference – please do so ASAP if you haven’t already. For more information and registration, please visit the conference page: https://rbcdsai.iitm.ac.in/DAI-2023/index.html.

Please also ensure that all authors have their profiles on openreview. 
Decision: Accept (Poster)