To overcome this and further improve the ability of the model to recognize under-represented data, this paper considers training the model with a self-supervised or semi-supervised learning-based approach to facilitate learning. Both semi-supervised and self-supervised learning are effective ways to leverage information from unlabeled data, avoiding the expensive cost of collecting and annotating large datasets. Self-supervised learning, which is a subset of unsupervised learning methods, has been proposed to learn features from the images themselves without any annotation [33]. Contrastive self-supervised learning is a technique for learning representations by comparing among multiple input samples. Its emergence has significantly narrowed the gap between unsupervised learning and supervised learning. In recent years, the promising performance of contrastive self-supervised learning both in computer vision (CV) and natural language processing (NLP) has shown that the underlying latent representations can be learned from unlabeled data by contrastive self-supervised learning [34]. Semi-supervised learning is a learning paradigm that uses a combination of labeled and unlabeled data to train a model. Adding unlabeled samples to the training dataset changes the distribution of the original dataset, which consequently affects the model in terms of making decisions. If two points, x1 and x2, are close in a high-density region, then the corresponding output y1 should be close to y2. Under such a smoothness assumption, the additional unlabeled data help the model to find a more accurate decision boundary [33]. According to recent advances in deep learning, semi-supervised learning outperforms supervised learning that uses only labeled data [35]. There are few studies that have applied either self-supervised or semi-supervised learning in DR detection. Common approaches used to apply self-supervised or semi-supervised learning in other fields usually employ these learning methods to perform the target tasks directly. Instead, self-supervised and semi-supervised learning are both used as a wrapper algorithm in this study. Specifically, the model is first pre-trained on unlabeled retinal fundus data with self-supervised or semi-supervised learning; then, the learned presentations are transferred to a model that will be fine-tuned for DR detection with supervised learning using the labeled data, so that the useful knowledge learned from unlabeled data in the same domain can help the target model to move in the right direction of finding appropriate parameters from the beginning. Adopting a wrapper algorithm that utilizes unlabeled DR data to the basic supervised learning baseline shows better performance than using supervised learning alone. Furthermore, the combination of self-supervised and semi-supervised learning proposed in this study also results in improved accuracy and significant reduction in training time. This novel approach is very useful when labeled data are too sparse to train a model for annotating unlabeled data. Regarding imbalanced learning, the biased model caused by imbalanced labels can be re-balanced to some extent under the influence of unlabeled data. To further deal with the imbalanced learning problem, the classifier of the model is additionally fine-tuned on the re-balanced training dataset obtained by re-sampling. The results of the experiment demonstrate that the proposed methods enhance the performance of DR detection from a supervised learning baseline by improving the performance on the imbalanced data. Accuracy (ACC), sensitivity (TPR), and specificity (TNR) were used as evaluation metrics. In the case of evaluating the model using the balanced EyePaCS [36] test dataset, the false-negative error rate, which was 100%, was reduced to 14.8%, and the accuracy rate, which was only 50%, improved to 86.4%. The best performance was obtained on the balanced DDR [37] test dataset, with an ACC of 89.62%, TPR of 86.39%, and TNR of 92.84%; this model was trained on the smaller balanced training set.