Before revision:  However, fundamental pretraining method capabilities like few to zero-shot learningor preserving minority concept ( long-tail ) prediction performance along with accordingly designed evaluation scenarios remain open challenges . We thus propose Contrastive Label-Embedding Self-Supervision (CLESS) pretraining , which enables pretraining from multiple magnitudes smaller, 'task internal' data only, while still strongly improving fully supervised, long-tail , few-shot and self-supervised zero-shot learning abilities.
After revision:  However, fundamental pretraining method capabilities like few to zero-shot learningor preserving minority concept ( long-tail generalization, since long-tail , few-shot and self-supervised zero-shot learning abilities.