Label distribution‐based noise correction for multiclass crowdsourcing

International Journal of Intelligent SystemsVolume 37, Issue 9 p. 5752-5767 RESEARCH ARTICLE Label distribution-based noise correction for multiclass crowdsourcing Ziqi Chen, Ziqi Chen orcid.org/0000-0002-3126-7555 School of Computer Science, China University of Geosciences, Wuhan, ChinaSearch for more papers by this authorLiangxiao Jiang, Corresponding Author Liangxiao Jiang ljiang@cug.edu.cn orcid.org/0000-0003-2201-3526 School of Computer Science, China University of Geosciences, Wuhan, China Key Laboratory of Artificial Intelligence, Ministry of Education, Shanghai, China Correspondence Liangxiao Jiang, School of Computer Science, China University of Geosciences, 430074 Wuhan, China. Email: ljiang@cug.edu.cnSearch for more papers by this authorChaoqun Li, Chaoqun Li orcid.org/0000-0003-0620-6344 School of Mathematics and Physics, China University of Geosciences, Wuhan, ChinaSearch for more papers by this author Ziqi Chen, Ziqi Chen orcid.org/0000-0002-3126-7555 School of Computer Science, China University of Geosciences, Wuhan, ChinaSearch for more papers by this authorLiangxiao Jiang, Corresponding Author Liangxiao Jiang ljiang@cug.edu.cn orcid.org/0000-0003-2201-3526 School of Computer Science, China University of Geosciences, Wuhan, China Key Laboratory of Artificial Intelligence, Ministry of Education, Shanghai, China Correspondence Liangxiao Jiang, School of Computer Science, China University of Geosciences, 430074 Wuhan, China. Email: ljiang@cug.edu.cnSearch for more papers by this authorChaoqun Li, Chaoqun Li orcid.org/0000-0003-0620-6344 School of Mathematics and Physics, China University of Geosciences, Wuhan, ChinaSearch for more papers by this author First published: 09 January 2022 https://doi.org/10.1002/int.22812Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract In crowdsourcing scenarios, we can often obtain each instance's multiple noisy labels from different crowd workers and then use a label integration method to infer its integrated label. In spite of the effectiveness of label integration methods, a certain level of label noise still exists in integrated labels. To reduce the impact of label noise, noise correction has attracted much attention from researchers, and therefore a certain number of noise correction methods were proposed in recent years. Among them, Between-class Margin-based Noise Correction (BMNC) has demonstrated remarkable denoising performance. However, BMNC can only handle binary classification tasks. For multiclass classification tasks, we propose an effective but very simple noise correction method in this paper. We refer to our proposed method as Label Distribution-based Noise Correction (LDNC). At first, LDNC transforms each instance's multiple noisy labels into a label distribution. Then, LDNC uses the margin between the first- and second-largest label probabilities in the label distribution to identify and filter each possible noise instance and thus obtains a clean set and a noise set. Finally, LDNC builds a classifier on the clean set to relabel all instances in the noise set. The experimental results on 16 simulated and one real-world multiclass crowdsourced data sets show that LDNC significantly outperforms all the other existing state-of-the-art noise correction methods. Volume37, Issue9September 2022Pages 5752-5767 RelatedInformation