Structured prediction from bandit feedback describes a learning scenario where instead of having access to a gold standard structure, a learner only receives partial feedback in form of the loss value of a predicted structure. We present new learning objectives and algorithms for this interactive scenario, focusing on convergence speed and ease of elicitability of feedback. We present supervised-to-bandit simulation experiments for several NLP tasks (machine translation, sequence labeling, text classification), showing that bandit learning from relative preferences eases feedback strength and yields improved empirical convergence.
Learning NLP tasks ((machine translation, sequence labeling, text classification) from bandit feedback has been studied previously REF , and has been extended to neural sequence-to-sequence models .
Learning Structured Predictors from Bandit Feedback for Interactive NLP
ACL
Computer Science