A computer-implemented method for processing an image of an object, comprising: receiving an image classification from a trained convolutional neural network for an input image (101, 501) of the object or for each patch of the input image (101, 501), wherein the image classification is either a positive classification that corresponds to detection of an anomaly in the input image (101, 501) or in one of the patches or a negative classification that corresponds to absence of an anomaly in the input image (101, 501) or in one of the patches; performing a backward propagation analysis of the input image (101, 501) or of one of the patches in response of detecting a positive classification for this patch for each layer of the CNN to generate an attention mapping (103, 505), wherein the attention mapping (103, 505) includes a positive attention map generated from a positive signal in the backward propagation and a negative attention map generated from a negative signal in the backward propagation, the positive and negative signal in the backward propagation being set by setting positive and negative nodes in a fully connected layer in the CNN; generating a positive mask based on intensity thresholds (T1, T2) of the positive attention map and a negative mask based on intensity thresholds (T1', T2') of the negative attention map, both maps including pixels considered to be uncertain (310K, 320K); generating an aggregated mask (410, 507) by aggregating the positive mask and the negative mask; and determining an image of segmented anomalous pixels (225, 508) based on the aggregated mask (410, 507) using a graph-cut color segmentation technique. receiving an image classification from a trained convolutional neural network for an input image (101, 501) of the object or for each patch of the input image (101, 501), wherein the image classification is either a positive classification that corresponds to detection of an anomaly in the input image (101, 501) or in one of the patches or a negative classification that corresponds to absence of an anomaly in the input image (101, 501) or in one of the patches; performing a backward propagation analysis of the input image (101, 501) or of one of the patches in response of detecting a positive classification for this patch for each layer of the CNN to generate an attention mapping (103, 505), wherein the attention mapping (103, 505) includes a positive attention map generated from a positive signal in the backward propagation and a negative attention map generated from a negative signal in the backward propagation, the positive and negative signal in the backward propagation being set by setting positive and negative nodes in a fully connected layer in the CNN; generating a positive mask based on intensity thresholds (T1, T2) of the positive attention map and a negative mask based on intensity thresholds (T1', T2') of the negative attention map, both maps including pixels considered to be uncertain (310K, 320K); generating an aggregated mask (410, 507) by aggregating the positive mask and the negative mask; and determining an image of segmented anomalous pixels (225, 508) based on the aggregated mask (410, 507) using a graph-cut color segmentation technique.