  Despite deep learning (DL) success in classification problems, DL classifiers
do not provide a sound mechanism to decide when to refrain from predicting.
Recent works tried to control the overall prediction risk with classification
with rejection options. However, existing works overlook the different
significance of different classes. We introduce Set-classifier with
Class-specific RIsk Bounds (SCRIB) to tackle this problem, assigning multiple
labels to each example. Given the output of a black-box model on the validation
set, SCRIB constructs a set-classifier that controls the class-specific
prediction risks with a theoretical guarantee. The key idea is to reject when
the set classifier returns more than one label. We validated SCRIB on several
medical applications, including sleep staging on electroencephalogram (EEG)
data, X-ray COVID image classification, and atrial fibrillation detection based
on electrocardiogram (ECG) data. SCRIB obtained desirable class-specific risks,
which are 35\%-88\% closer to the target risks than baseline methods.
