ADKU/ResearchGPT_model_dataset huggingface dataset:
Deep learning solutions for smart city challenges in urban development
Authors: Pengjun Wu, Zhanzhi Zhang, Xueyi Peng, Ran Wang

In the realm of urban planning, the integration of deep learning technologies has emerged as a transformative force, promising to revolutionize the way cities are designed, managed, and optimized. This research embarks on a multifaceted exploration that combines the power of deep learning with Bayesian regularization techniques to enhance the performance and reliability of neural networks tailored for urban planning applications. Deep learning, characterized by its ability to extract complex patterns from vast urban datasets, has the potential to offer unprecedented insights into urban dynamics, transportation networks, and environmental sustainability. However, the complexity of these models often leads to challenges such as overfitting and limited interpretability. To address these issues, Bayesian regularization methods are employed to imbue neural networks with a principled framework that enhances generalization while quantifying predictive uncertainty. This research unfolds with the practical implementation of Bayesian regularization within neural networks, focusing on applications ranging from traffic prediction, urban infrastructure, data privacy, safety and security. By integrating Bayesian regularization, the aim is to, not only improve model performance in terms of accuracy and reliability but also to provide planners and decision-makers with probabilistic insights into the outcomes of various urban interventions. In tandem with quantitative assessments, graphical analysis is wielded as a crucial tool to visualize the inner workings of deep learning models in the context of urban planning. Through graphical representations, network visualizations, and decision boundary analysis, we uncover how Bayesian regularization influences neural network architecture and enhances interpretability.
Age transformation based on deep learning: a survey
Authors: Yingchun Guo, Xin Su, Gang Yan, Ye Zhu, Xueqi Lv

Age transformation aims to preserve personalized facial information while altering a given face to appear at a target age. This technique finds extensive applications in fields such as face recognition, movie special effects, and social entertainment, among others. With the advancement of deep learning, particularly Generative Adversarial Networks (GANs), research on age transformation has made significant progress, leading to the emergence of a diverse range of deep learning-based methods. However, a comprehensive and systematic literature review of these methods is currently lacking. In this survey, we provide an all-encompassing review of deep learning methods for facial aging. Firstly, we summarize the key aspects of feature preservation during the age transformation process. Subsequently, we present a comprehensive overview of facial age transformation techniques, categorized according to various deep learning network architectures. Additionally, we conduct an analysis and comparison of commonly used face image datasets, offering recommendations for dataset selection. Furthermore, we consolidate the qualitative and quantitative evaluation metrics commonly employed in age transformation methodologies through experimental assessment. Finally, we address potential areas of future research in age transformation methods, based on the current challenges and limitations.
Advances of Pipeline Model Parallelism for Deep Learning Training: An Overview
Authors: Lei Guan 
            (关　磊), Dong-Sheng Li 
            (李东升), Ji-Ye Liang 
            (梁吉业), Wen-Jian Wang 
            (王文剑), Ke-Shi Ge 
            (葛可适), Xi-Cheng Lu 
            (卢锡城)

Deep learning has become the cornerstone of artificial intelligence, playing an increasingly important role in human production and lifestyle. However, as the complexity of problem-solving increases, deep learning models become increasingly intricate, resulting in a proliferation of large language models with an astonishing number of parameters. Pipeline model parallelism (PMP) has emerged as one of the mainstream approaches to addressing the significant challenge of training “big models”. This paper presents a comprehensive review of PMP. It covers the basic concepts and main challenges of PMP. It also comprehensively compares synchronous and asynchronous pipeline schedules for PMP approaches, and discusses the main techniques to achieve load balance for both intra-node and inter-node training. Furthermore, the main techniques to optimize computation, storage, and communication are presented, with potential research directions being discussed.
Deep learning-based channel estimation for wireless ultraviolet MIMO communication systems
Authors: Taifei Zhao, Yuxin Sun, Xinzhe Lü, Shuang Zhang

To solve the problems of pulse broadening and channel fading caused by atmospheric scattering and turbulence, multiple-input multiple-output (MIMO) technology is a valid way. A wireless ultraviolet (UV) MIMO channel estimation approach based on deep learning is provided in this paper. The deep learning is used to convert the channel estimation into the image processing. By combining convolutional neural network (CNN) and attention mechanism (AM), the learning model is designed to extract the depth features of channel state information (CSI). The simulation results show that the approach proposed in this paper can perform channel estimation effectively for UV MIMO communication and can better suppress the fading caused by scattering and turbulence in the MIMO scattering channel.
Imbalcbl: addressing deep learning challenges with small and imbalanced datasets
Authors: Saqib ul Sabha, Assif Assad, Sadaf Shafi, Nusrat Mohi Ud Din, Rayees Ahmad Dar, Muzafar Rasool Bhat

Deep learning, while transformative for computer vision, frequently falters when confronted with small and imbalanced datasets. Despite substantial progress in this domain, prevailing models often underachieve under these constraints. Addressing this, we introduce an innovative contrast-based learning strategy for small and imbalanced data that significantly bolsters the proficiency of deep learning architectures on these challenging datasets. By ingeniously concatenating training images, the effective training dataset expands from n to \(n^2\), affording richer data for model training, even when n is very small. Remarkably, our solution remains indifferent to specific loss functions or network architectures, endorsing its adaptability for diverse classification scenarios. Rigorously benchmarked against four benchmark datasets, our approach was juxtaposed with state-of-the-art oversampling paradigms. The empirical evidence underscores our method’s superior efficacy, outshining contemporaries across metrics like Balanced accuracy, F1 score, and Geometric mean. Noteworthy increments include 7–16% on the Covid-19 dataset, 4–20% for Honey bees, 1–6% on CIFAR-10, and 1–9% on FashionMNIST. In essence, our proposed method offers a potent remedy for the perennial issues stemming from scanty and skewed data in deep learning.
ConvDepthTransEnsembleNet: An Improved Deep Learning Approach for Rice Crop Leaf Disease Classification
Authors: Kavita Bathe, Nita Patil, Sanjay Patil, Devanand Bathe, Kuldeep Kumar

Agricultural sector, in association with its allied sectors, plays pivotal role in the progress of a country. In spite of notable contribution, Agricultural sector suffers from several challenges. Crop loss due to diseases is considered as one of the major challenges. Rice is one of the major cereal crops in India. Its production is highly impacted due to crop diseases. Early and accurate detection of rice crop leaf diseases is essential for aforementioned issue. Over the years, conventional methods of crop leaf disease detection are used but with its own limitations. Recent technological advancement in computer vision, deep learning has created new pathways in agricultural sector. Deep learning models require huge data which is quite a great challenge. In such case, building up of a robust deep learning model could work with limited and unbalanced dataset, with good generalization performances. In this study, weighted deep ensemble learning approach is used for performance improvement of rice crop leaf disease classification task. The ensemble method ConvDepthTransEnsembleNet is proposed. To show the effectiveness of proposed work, experiments are conducted on diverse datasets. ConvDepthTransEnsembleNet is a lightweight model which has achieved the accuracy of 96.88% on limited and unbalanced dataset. The experimental results show that proposed model outperforms the individual classifiers based on conventional methods and transfer learning approach in terms of significant reduction in parameters and improved upon generalization performance. The proposed model is highly useful for implementing deep learning models with resources constrained devices.
Transfer learning based cascaded deep learning network and mask recognition for COVID-19
Authors: Fengyin Li, Xiaojiao Wang, Yuhong Sun, Tao Li, Junrong Ge

The COVID-19 is still spreading today, and it has caused great harm to human beings. The system at the entrance of public places such as shopping malls and stations should check whether pedestrians are wearing masks. However, pedestrians often pass the system inspection by wearing cotton masks, scarves, etc. Therefore, the detection system not only needs to check whether pedestrians are wearing masks, but also needs to detect the type of masks. Based on the lightweight network architecture MobilenetV3, this paper proposes a cascaded deep learning network based on transfer learning, and then designs a mask recognition system based on the cascaded deep learning network. By modifying the activation function of the MobilenetV3 output layer and the structure of the model, two MobilenetV3 networks suitable for cascading are obtained. By introducing transfer learning into the training process of two modified MobilenetV3 networks and a multi-task convolutional neural network, the ImagNet underlying parameters of the network models are obtained in advance, which reduces the computational load of the models. The cascaded deep learning network consists of a multi-task convolutional neural network cascaded with these two modified MobilenetV3 networks. A multi-task convolutional neural network is used to detect faces in images, and two modified MobilenetV3 networks are used as the backbone network to extract the features of masks. After comparing with the classification results of the modified MobilenetV3 neural network before cascading, the classification accuracy of the cascading learning network is improved by 7%, and the excellent performance of the cascading network can be seen.
Efficient screening framework for organic solar cells with deep learning and ensemble learning
Authors: Hongshuai Wang, Jie Feng, Zhihao Dong, Lujie Jin, Miaomiao Li, Jianyu Yuan, Youyong Li

Organic photovoltaics have attracted worldwide interest due to their unique advantages in developing low-cost, lightweight, and flexible power sources. Functional molecular design and synthesis have been put forward to accelerate the discovery of ideal organic semiconductors. However, it is extremely expensive to conduct experimental screening of the wide organic compound space. Here we develop a framework by combining a deep learning model (graph neural network) and an ensemble learning model (Light Gradient Boosting Machine), which enables rapid and accurate screening of organic photovoltaic molecules. This framework establishes the relationship between molecular structure, molecular properties, and device efficiency. Our framework evaluates the chemical structure of the organic photovoltaic molecules directly and accurately. Since it does not involve density functional theory calculations, it makes fast predictions. The reliability of our framework is verified with data from previous reports and our newly synthesized organic molecules. Our work provides an efficient method for developing new organic optoelectronic materials.
Identifying strawberry appearance quality based on unsupervised deep learning
Authors: Hongfei Zhu, Xingyu Liu, Hao Zheng, Lianhe Yang, Xuchen Li, Zhongzhi Han

The strawberry appearance is an essential standard for judging the quality, so it is crucial to accurately identify the strawberry appearance quality for intelligent picking. This study proposed a new strawberry appearance quality detection based on unsupervised deep learning. Firstly, using deep learning (Resnet18, Resnet50, and Resnet101) to extract the strawberry image feature information. And using the t-SNE (t-distribution stochastic neighbor embedding) to reduce the feature vectors’ dimension. Finally, the unsupervised learning method (Gaussian Mixture Model) was used to cluster strawberries’ feature points. The results showed that: (1) the clustering performance based on Resnet101 was effective in 2-dimensional space, the cluster accuracy was 94.89%, and the validation accuracy was 91.79%. (2) The clustering method based on Resnet50 had good performance in the 3-dimensional space, the cluster accuracy was 96.10%, and the validation accuracy was 93.08%. (3) The accuracy of deep features plus RF (random forest) was 95.00% under limited data. Thus this method will promote intelligent picking strawberry equipment and it will overcome the supervised learning drawback that divides image datasets according to prior knowledge.
A survey on advanced machine learning and deep learning techniques assisting in renewable energy generation
Authors: Sri Revathi B.

The sustainability of the earth depends on renewable energy. Forecasting the output of renewable energy has a big impact on how we operate and manage our power networks. Accurate forecasting of renewable energy generation is crucial to ensuring grid dependability and permanence and reducing the risk and cost of the energy market and infrastructure. Although there are several approaches to forecasting solar radiation on a global scale, the two most common ones are machine learning algorithms and cloud pictures combined with physical models. The objective is to present a summary of machine learning-based techniques for solar irradiation forecasting in this context. Renewable energy is being used more and more in the world’s energy grid. Numerous strategies, including hybrids, physical models, statistical approaches, and artificial intelligence techniques, have been developed to anticipate the use of renewable energy. This paper examines methods for forecasting renewable energy based on deep learning and machine learning. Review and analysis of deep learning and machine learning forecasts for renewable energy come first. The second paragraph describes metaheuristic optimization techniques for renewable energy. The third topic was the open issue of projecting renewable energy. I will wrap up with a few potential future job objectives.
Varicocele detection in ultrasound images using deep learning
Authors: Omar AlZoubi, Mohammad Abu Awad, Ayman M. Abdalla, Laaly Samrraie

Varicocele is a disease exhibited by an abnormal dilation of the scrotal venous pampiniform plexus. It is a common cause of infertility and in some cases may cause pain or discomfort. This paper presents a new approach to automatic varicocele classification by employing convolutional neural networks with deep learning. The available dataset consists of images converted into different color modes. Each color mode dataset is partitioned, augmented, and trained by employing a deep learning network. Experiments were run with all possible combinations of four different pre-trained models and over three color modes to determine the best combination that achieves the highest performance with and without augmentation. The implementation results of training and testing were evaluated with several metrics. The analysis of the results demonstrated the efficacy of the proposed system as it identified and classified varicocele with a relatively high accuracy that surpassed the previous works with a significant improvement.
Enhancing image retrieval through entropy-based deep metric learning
Authors: Kambiz Rahbar, Fatemeh Taheri

The increasing demand for effective retrieval from image datasets has been driven by the rapid growth of digital images. Image retrieval is a method for creating a structured database based on searching for similar images to a user's query image. Effective image representation and extraction of distinctive features are among the challenges faced by image retrieval systems. The proposed approach utilizes a triplet loss function based on binary cross-entropy to train a Siamese network, allowing for deep metric learning and creating a discriminative feature space with maximum discrimination between classes and minimum intra-class distance. In this approach, a pre-trained neural network is serialized with Siamese network. Initially, image features are extracted using a pre-trained convolutional neural network. Then, a Siamese network is trained to create a discriminative feature space using deep metric learning. Learning in this method is controlled by a triplet loss function based on binary cross-entropy with anchor, positive, and negative samples. The transformation from the initial feature space to the discriminative feature space is achieved through deep metric learning in the Siamese network. The proposed approach proves to effective in discriminating features of diverse classes in the dataset. Visualization of the feature space is also demonstrated using the t-SNE statistical technique. Furthermore, the explainability of the proposed approach is presented by examining the Shapley value. The proposed approach was examined on the Corel10K, Caltech101, and Caltech256 datasets. The best reported results with the precision metric were 0.988 for the Corel10K dataset and 0.951 and 0.902 for the Caltech101 and Caltech256 datasets, respectively. Experimental results show that the proposed approach effectively improves the retrieval results of similar samples to the query image creating discrimination in the feature vectors, even with dimensionality reduction.
Legal sentence boundary detection using hybrid deep learning and statistical models
Authors: Reshma Sheik, Sneha Rao Ganta, S. Jaya Nirmala

Sentence boundary detection (SBD) represents an important first step in natural language processing since accurately identifying sentence boundaries significantly impacts downstream applications. Nevertheless, detecting sentence boundaries within legal texts poses a unique and challenging problem due to their distinct structural and linguistic features. Our approach utilizes deep learning models to leverage delimiter and surrounding context information as input, enabling precise detection of sentence boundaries in English legal texts. We evaluate various deep learning models, including domain-specific transformer models like LegalBERT and CaseLawBERT. To assess the efficacy of our deep learning models, we compare them with a state-of-the-art domain-specific statistical conditional random field (CRF) model. After considering model size, F1-score, and inference time, we identify the Convolutional Neural Network Model (CNN) as the top-performing deep learning model. To further enhance performance, we integrate the features of the CNN model into the subsequent CRF model, creating a hybrid architecture that combines the strengths of both models. Our experiments demonstrate that the hybrid model outperforms the baseline model, achieving a 4% improvement in the F1-score. Additional experiments showcase the superiority of the hybrid model over SBD open-source libraries when confronted with an out-of-domain test set. These findings underscore the importance of efficient SBD in legal texts and emphasize the advantages of employing deep learning models and hybrid architectures to achieve optimal performance.
Deep Learning Methods for Vibration-Based Structural Health Monitoring: A Review
Authors: Hao Wang, Baoli Wang, Caixia Cui

The vibration signal is an effective diagnostic tool in structural health monitoring (SHM) fields that is closely related to abnormal states. Deep learning methods have got remarkable success in utilizing vibration signals for damage detection. This paper presents a systematic review of deep learning methods for SHM, focusing on the utilization of vibration signal data from different model perspectives. In recent years, there has been a significant increase in research on deep learning for vibration-based SHM. The accuracy of such works is equivalent to that of traditional machine learning approaches, and better results could be achieved by integrating multiple approaches. Furthermore, we found that transfer learning methods yield promising results when limited data are available to train the model. This paper aims to comprehensively review deep learning research on health monitoring using vibration signal data from multiple perspectives, with a particular emphasis on transfer learning methods for SHM. It fills the gap that existing reviews lack in the discussion of transfer learning for SHM. Finally, we analyze the challenges faced by current research and provide recommendations for future work.
Hyperbolic Deep Learning in Computer Vision: A Survey
Authors: Pascal Mettes, Mina Ghadimi Atigh, Martin Keller-Ressel, Jeffrey Gu, Serena Yeung

Deep representation learning is a ubiquitous part of modern computer vision. While Euclidean space has been the de facto standard manifold for learning visual representations, hyperbolic space has recently gained rapid traction for learning in computer vision. Specifically, hyperbolic learning has shown a strong potential to embed hierarchical structures, learn from limited samples, quantify uncertainty, add robustness, limit error severity, and more. In this paper, we provide a categorization and in-depth overview of current literature on hyperbolic learning for computer vision. We research both supervised and unsupervised literature and identify three main research themes in each direction. We outline how hyperbolic learning is performed in all themes and discuss the main research problems that benefit from current advances in hyperbolic learning for computer vision. Moreover, we provide a high-level intuition behind hyperbolic geometry and outline open research questions to further advance research in this direction.
From distributed machine to distributed deep learning: a comprehensive survey
Authors: Mohammad Dehghani, Zahra Yazdanparast

Artificial intelligence has made remarkable progress in handling complex tasks, thanks to advances in hardware acceleration and machine learning algorithms. However, to acquire more accurate outcomes and solve more complex issues, algorithms should be trained with more data. Processing this huge amount of data could be time-consuming and require a great deal of computation. To address these issues, distributed machine learning has been proposed, which involves distributing the data and algorithm across several machines. There has been considerable effort put into developing distributed machine learning algorithms, and different methods have been proposed so far. We divide these algorithms in classification and clustering (traditional machine learning), deep learning and deep reinforcement learning groups. Distributed deep learning has gained more attention in recent years and most of the studies have focused on this approach. Therefore, we mostly concentrate on this category. Based on the investigation of the mentioned algorithms, we highlighted the limitations that should be addressed in future research.