Package: ece-loss
APyTorchimplementation of the Exclusive Cross Entropy Loss.Free software:Apache 2.0 license(please cite our work if you use it)FeaturesPerform sparse-shot learning from non-exhaustively annotated datasetsPlug-n-play components of Binary Exclusive Cross-Entropy and Exclusive Cross-entropy as substitutes for the original Cross-entropy pytorch functions1-2 changes in lines of code for Exclusive cross-entropy loss compared to native pytorch cross-entropySimple and modular loss class for problem personalisation if neededExample training code provided for a simple segmentation caseExamplesSee here for the simplest example in a converged training stateimportExclusiveCrossEntropyLossloss=ExclusiveCrossEntropyLoss()input=torch.randn(2,3,requires_grad=True)target=torch.empty(2,dtype=torch.long).random_(3)output=loss(input,target)For setting the epoch state during training theset_epochmust be used for the annealing functions asimportExclusiveCrossEntropyLoss# Epoch `loss.epoch` is initialised at zero. Can be changed by `ExclusiveCrossEntropyLoss(epoch=epoch)`loss=ExclusiveCrossEntropyLoss()# Updates the current epoch. The loss value for the unallabelled samples depends heavily on the current state#     because the background sampling and threshold annealing function decide how much of the background class#     to incorporate into the loss and how strict the exclusivity condition should be.loss.set_epoch(100)input=torch.randn(2,3,requires_grad=True)target=torch.empty(2,dtype=torch.long).random_(3)output=loss(input,target)# Alternatively, epoch can be given as an optional argument as `loss(input, target, epoch=epoch)`Be default, sigmoid annealing functions are used for both the (negative) background sampling and the exclusivity condition. If a different annealing is required then this can be changed byimportExclusiveCrossEntropyLossannealing_funct=lambdaepoch,threshold:threshold# For no annealingloss=ExclusiveCrossEntropyLoss(exclusivity_threshold_annealing=annealing_funct,background_sampling_annealing=annealing_funct)For switching off the exclusivity condition, or adjusting the other parameters of the exclusive loss this can be done byimportExclusiveCrossEntropyLossloss=ExclusiveCrossEntropyLoss(exclusivity_threshold=1,background_sampling_threshold=1,exclusivity_threshold_annealing=lambdaepoch,threshold:threshold,background_sampling_annealing=lambdaepoch,threshold:threshold,focal_loss_parameters=(1,0),)Lastly and very importantly, the samples that are considered as unlabelled must be defined. By default the multiclass exlusive cross entropy (ExclusiveCrossEntropyLoss) defines unlabelled samples as the ones with targets equal to zero; as defined by thezeroth_label_as_unannotatedfunction.For the binary loss (BCELoss, BCEWithLogitsLoss), this is switched off and assumes all input as unlabelled, as defined by the_identity_mapping_single_input_torch_onesinner function.If custom unlabelled sample mapping is required this can be adjusted by setting the unannotated_mapping variable asimportExclusiveCrossEntropyLossloss=ExclusiveCrossEntropyLoss()loss.set_unannotated_mapping(lambdatargets:targets==1)# For the background class being assigned integer 1A proof of concept is provided for the TNBC dataset in the examples directory with the necessary code to use the exclusive cross-entropy loss in a segmentation task.Installpip install -e .Use the lossimportExclusiveCrossEntropyLossloss=ExclusiveCrossEntropyLoss()output=loss(input,target)# just as in the ordinary CrossEntropyLossFor more specific usages the exclusive configuration can be adjusted by:loss=ExclusiveCrossEntropyLoss(exclusivity_threshold=0.5,background_sampling_threshold=0.5,exclusivity_threshold_annealing=annealing_function,background_sampling_annealing=annealing_function,focal_loss_parameters=(0.2,0.1))# indicating the default values and a general annealing_functionRun PyTorch ExperimentsAfter installing ECE run:python train_tnbc [--seed] [--lr] [--loss] [--train_path] [--train_path] [--eval_path] [--test_path] [--epochs] [--batch_size] [--device]Available values for--lossareeceandcefor exclusive cross-entropy and cross-entropy respectively.Use the--deviceflag to set device eithercudato train on the GPU orcputo train on the CPU.The simple segmentation task on TNBC on the lisa surf sara cluster, using a GTX1080-ti GPU the results are:DICECross-entropyExclusive Cross-EntropyTNBC @30%0.780.78TNBC @30%0.080.41CitationPanteli, A., Teuwen, J., Horlings, H. and Gavves, E.; Sparse-shot Learning with Exclusive Cross-Entropy for ExtremelyMany Localisations; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 2813-2823If you use our code, please cite:@InProceedings{Panteli_2021_ICCV,
    author    = {Panteli, Andreas and Teuwen, Jonas and Horlings, Hugo and Gavves, Efstratios},
    title     = {Sparse-Shot Learning With Exclusive Cross-Entropy for Extremely Many Localisations},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {2813-2823}
}