ADKU/ResearchGPT_model_dataset huggingface dataset:
Machine Learning and Deep Learning-Based Students’ Grade Prediction
Authors: Adil Korchi, Fayçal Messaoudi, Ahmed Abatal, Youness Manzali

Predicting student performance in a curriculum or program offers the prospect of improving academic outcomes. By using effective performance prediction methods, instructional leaders can allocate adequate resources and instruction more accurately. This paper aims to identify machine learning algorithm features for predicting student grades as an early intervention. Predictive models spot at-risk students early, allowing educators to provide timely support. Educators can customize teaching methods, and these models assess program success, helping institutions refine or expand them through data-driven decisions. But the problem definition of student grade prediction is to develop predictive models or algorithms that can forecast or estimate the future academic performance or grades of students based on various input features and historical data, and to do so, we utilized a student dataset comprising personal information and grades, employing various regression algorithms, including decision tree, random forest, linear regression, k-nearest neighbor, XGBoost, and deep neural network. We chose these algorithms for their suitability and distinct strengths. We assessed their performance using determination coefficient, mean average error, mean squared error, and root mean squared error. The results showed that the deep neural network outperformed others with a determination coefficient of 99.97%, confirming its reliability in predicting student grades and assessing performance, and this will certainly help to develop predictive models that can accurately forecast or estimate students’ academic performance based on various input features and enable teaching staff to provide timely assistance in addressing these issues.
Rf-based fingerprinting for indoor localization: deep transfer learning approach
Authors: Rokaya Safwat, Eman Shaaban, Shahinaz. M. Al-Tabbakh, Karim Emara

Transfer Learning (TL) has emerged as a powerful approach for improving the performance of Deep Learning systems in various domains by leveraging pre-trained models. It was proven that features learned by deep learning can smoothly be reused across similar domains. Deep transfer learning schemes compensate for limited training data via transfer learning of a rich data environment. This paper investigates the effectiveness of applying TL schemes in indoor localization. It proposes four deep TL models where the knowledge is transferred from the rich-measurement data source domain to multiple target domains with limited data measurements. The architecture of the source domain is based on Convolutional Neural Network (CNN), where the four deep TL models for the target domain are: standalone feature extractor, integrated feature extractor, selective fine-tuning, and weight initialization. We employed a dataset of RF fingerprinting measurement signals representing common interior conditions, including extremely crowded, medium cluttered, low cluttered, and open environments, to test the effectiveness of the proposed TL models. We measured the accuracy and computation time of target-domain models trained, with varied percentages of restricted data sizes: 40%, 30%, 20%, 15%, 10%, 5%, and 2.5%. The experimental results show that all TL models are effective in achieving significant improvement in accuracy when compared to non-transferred models, even with minimal training data size. However, the proper determination of the TL model and the amount of training data profoundly influence the performance results.
Satellite image classification using deep learning approach
Authors: Divakar Yadav, Kritarth Kapoor, Arun Kumar Yadav, Mohit Kumar, Arti Jain, Jorge Morato

Our planet Earth comprises distinguished topologies based on temperature, location, latitude, longitude, and altitude, which can be captured using Remote Sensing Satellites. In this paper, the classification of satellite images is performed based on their topologies and geographical features. Researchers have worked on several machine learning and deep learning methods like support vector machine, k-nearest neighbor, maximum likelihood, deep belief network, etc. that can be used to solve satellite image classification tasks. All strategies give promising results. Recent trends show that a Convolutional Neural Network (CNN) is an excellent deep learning model for classification purposes, which is used in this paper. The open-source EuroSAT dataset is used for classifying the remote images which contain 27,000 images distributed among ten classes. The 3 baseline CNN models are pre-trained, namely- ResNet50, ResNet101, and GoogleNet models. They have other sequence layers added to them with respect to CNN, and data is pre-processed using LAB channel operations. The highest accuracy of 99.68%, precision of 99.42%, recall of 99.51%, and F- Score of 99.45% are achieved using GoogleNet over the pre-processed dataset. The proposed work is compared with the state-of-art methods and it is observed that more layers in CNN do not necessarily provide a better outcome for a medium-sized dataset. The GoogleNet, a 22-layer CNN, performs faster and better than the 50 layers CNN- ResNet50, and 101 layers CNN- ResNet101.
Image steganalysis using deep learning models
Authors: Alexandr Kuznetsov, Nicolas Luhanko, Emanuele Frontoni, Luca Romeo, Riccardo Rosati

In the domain of digital steganography, the problem of efficient and accurate steganalysis is of utmost importance. Steganalysis seeks to detect the presence of hidden data within digital media, a task that is continually evolving due to advancements in steganographic techniques. This study undertakes a detailed exploration of the SRNet model, a prominent deep learning model for steganalysis. We aim to evaluate the impact of various factors, including choice of deep learning framework, model initialization and optimization parameters, and architectural modifications on the model's steganalysis performance. Three separate implementations of the SRNet model are examined in this study: our custom implementation, an implementation using TensorFlow, and another utilizing PyTorch. Each model is evaluated on its ability to detect different payloads, or bytes of hidden data per pixel, in digital images. This investigation includes a thorough comparative analysis of different performance metrics including accuracy, recall, precision, and F1-score. Our findings indicate that the choice of deep learning framework and the parameters utilized for model initialization and optimization play significant roles in influencing the model's steganalysis effectiveness. Notably, the TensorFlow implementation, enhanced with an additional dense layer, outperforms all other models. In contrast, our custom SRNet implementation, trained with fewer epochs, offers a balance between computational cost and steganalysis performance. This study thus provides valuable insights into the adaptability and potential of the SRNet model for steganalysis, illustrating the model's performance under different configurations and implementations. It underscores the importance of continued exploration and optimization in the field of steganalysis, offering guidance for future research in this evolving domain.
Control learning rate for autism facial detection via deep transfer learning
Authors: Abdelkrim El Mouatasim, Mohamed Ikermane

Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder that affects social interaction and communication. Early detection of ASD can significantly improve outcomes for individuals with the disorder, and there has been increasing interest in using machine learning techniques to aid in the diagnosis of ASD. One promising approach is the use of deep learning techniques, particularly convolutional neural networks (CNNs), to classify facial images as indicative of ASD or not. However, choosing a learning rate for optimizing the performance of these deep CNNs can be tedious and may not always result in optimal convergence. In this paper, we propose a novel approach called the control subgradient algorithm (CSA) for tackling ASD diagnosis based on facial images using deep CNNs. CSA is a variation of the subgradient method in which the learning rate is updated by a control step in each iteration of each epoch. We apply CSA to the popular DensNet-121 CNN model and evaluate its performance on a publicly available facial ASD dataset. Our results show that CSA is faster than the baseline method and improves the classification accuracy and loss compared to the baseline. We also demonstrate the effectiveness of using CSA with \(L_1\)-regularization to further improve the performance of our deep CNN model.
Virtual reality-empowered deep-learning analysis of brain cells
Authors: Doris Kaltenecker, Rami Al-Maskari, Moritz Negwer, Luciano Hoeher, Florian Kofler, Shan Zhao, Mihail Todorov, Zhouyi Rong, Johannes Christian Paetzold, Benedikt Wiestler, Marie Piraud, Daniel Rueckert, Julia Geppert, Pauline Morigny, Maria Rohm, Bjoern H. Menze, Stephan Herzig, Mauricio Berriel Diaz, Ali Ertürk

Automated detection of specific cells in three-dimensional datasets such as whole-brain light-sheet image stacks is challenging. Here, we present DELiVR, a virtual reality-trained deep-learning pipeline for detecting c-Fos+ cells as markers for neuronal activity in cleared mouse brains. Virtual reality annotation substantially accelerated training data generation, enabling DELiVR to outperform state-of-the-art cell-segmenting approaches. Our pipeline is available in a user-friendly Docker container that runs with a standalone Fiji plugin. DELiVR features a comprehensive toolkit for data visualization and can be customized to other cell types of interest, as we did here for microglia somata, using Fiji for dataset-specific training. We applied DELiVR to investigate cancer-related brain activity, unveiling an activation pattern that distinguishes weight-stable cancer from cancers associated with weight loss. Overall, DELiVR is a robust deep-learning tool that does not require advanced coding skills to analyze whole-brain imaging data in health and disease.
Fast flow field prediction of pollutant leakage diffusion based on deep learning
Authors: Wan YunBo, Zhao Zhong, Liu Jie, Zuo KuiJun, Zhang Yong

AbstractPredicting pollutant leakage and diffusion processes is crucial for ensuring people’s safety. While the deep learning method offers high simulation efficiency and superior generalization, there is currently a lack of research on predicting pollutant leakage and diffusion flow field using deep learning. Therefore, it is necessary to conduct further studies in this area. This paper introduces a two-level network method to model the flow characteristics of pollutant diffusion. The proposed method in this study demonstrates a significant enhancement in flow field prediction accuracy compared to traditional deep learning methods. Moreover, it improves computational efficiency by over 800 times compared to traditional computational fluid dynamics (CFD) methods. Unlike conventional CFD methods that require grid expansion to calculate all operation conditions, the deep learning method is not confined by grid limitations. While deep learning methods may not entirely replace CFD methods, they can serve as a valuable supplementary tool, expanding the versatility of CFD methods. The findings of this research establish a robust foundation for incorporating deep learning methods in addressing pollutant leakage and diffusion challenges.Graphical abstractA deep learning method for simulating pollutant leakage diffusion.
Image classification of intracranial tumor using deep residual learning technique
Authors: G. Vidya Sagar, M. Ravi Kumar, Sk. Hasane Ahammad, Chella Santhosh

Classifying brain tumours is essential for diagnosing tumour progression and planning effective treatments. Different imaging modalities are used to diagnose brain tumours. The opposite is true for magnetic resonance imaging (MRI), which has gained widespread use due to its superior image quality and the fact that it does not require ionizing radiation. Image classification of intracranial tumors using deep residual learning technique is an application of deep learning in the field of medical imaging analysis. It involves using convolutional neural networks (CNNs) to automatically classify brain images into different categories based on the presence or absence of tumors. ResNet is a deep neural network that addresses the problem of vanishing gradients during training of very deep networks.The deep learning subfield of machine learning has recently shown remarkable success, especially in classification and segmentation. We trained a deep residual network using picture datasets to distinguish between several brain cancers. The information generated by MRI scans is extensive. A radiologist analyses these images. The three most common brain tumours are meningioma, glioma, and pituitary tumour. Brain tumours are complex diseases, and a manual examination may be fraught with error. Experimental outcomes based on various techniques under augmentation with image-based datasets are presented. The accuracy with no augmentation is about 98% and under augmentation is approximately 99.08%.By leveraging deep residual learning techniques, image classification of intracranial tumors can benefit from the ability of deep neural networks to automatically learn complex representations from raw image data.Classification methods that use machine learning to automate the process have proven superior to human curation. Thus, we present a system that can identify and classify utilizing deep CNN-based residual networks.
Attractor Inspired Deep Learning for Modelling Chaotic Systems
Authors: Anurag Dutta, John Harshith, A. Ramamoorthy, K. Lakshmanan

Predicting and understanding the behavior of dynamic systems have driven advancements in various approaches, including physics-based models and data-driven techniques like deep neural networks. Chaotic systems, with their stochastic nature and unpredictable behavior, pose challenges for accurate modeling and forecasting, especially during extreme events. In this paper, we propose a novel deep learning framework called Attractor-Inspired Deep Learning (AiDL), which seamlessly integrates actual statistics and mathematical models of system kinetics. AiDL combines the strengths of physics-informed machine learning and data-driven methods, offering a promising solution for modeling nonlinear systems. By leveraging the intricate dynamics of attractors, AiDL bridges the gap between physics-based models and deep neural networks. We demonstrate the effectiveness of AiDL using real-world data from various domains, including catastrophic weather mechanics, El Niño cycles, and disease transmission. Our empirical results showcase AiDL’s ability to substantially enhance the modeling of extreme events. The proposed AiDL paradigm holds promise for advancing research in Time Series Prediction of Extreme Events and has applications in real-world chaotic system transformations.
Deep learning for high-resolution seismic imaging
Authors: Liyun Ma, Liguo Han, Qiang Feng

Seismic imaging techniques play a crucial role in interpreting subsurface geological structures by analyzing the propagation and reflection of seismic waves. However, traditional methods face challenges in achieving high resolution due to theoretical constraints and computational costs. Leveraging recent advancements in deep learning, this study introduces a neural network framework that integrates Transformer and Convolutional Neural Network (CNN) architectures, enhanced through Adaptive Spatial Feature Fusion (ASFF), to achieve high-resolution seismic imaging. Our approach directly maps seismic data to reflection models, eliminating the need for post-processing low-resolution results. Through extensive numerical experiments, we demonstrate the outstanding ability of this method to accurately infer subsurface structures. Evaluation metrics including Root Mean Square Error (RMSE), Correlation Coefficient (CC), and Structural Similarity Index (SSIM) emphasize the model's capacity to faithfully reconstruct subsurface features. Furthermore, noise injection experiments showcase the reliability of this efficient seismic imaging method, further underscoring the potential of deep learning in seismic imaging.
Predicting equilibrium distributions for molecular systems with deep learning
Authors: Shuxin Zheng, Jiyan He, Chang Liu, Yu Shi, Ziheng Lu, Weitao Feng, Fusong Ju, Jiaxi Wang, Jianwei Zhu, Yaosen Min, He Zhang, Shidi Tang, Hongxia Hao, Peiran Jin, Chi Chen, Frank Noé, Haiguang Liu, Tie-Yan Liu

Advances in deep learning have greatly improved structure prediction of molecules. However, many macroscopic observations that are important for real-world applications are not functions of a single molecular structure but rather determined from the equilibrium distribution of structures. Conventional methods for obtaining these distributions, such as molecular dynamics simulation, are computationally expensive and often intractable. Here we introduce a deep learning framework, called Distributional Graphormer (DiG), in an attempt to predict the equilibrium distribution of molecular systems. Inspired by the annealing process in thermodynamics, DiG uses deep neural networks to transform a simple distribution towards the equilibrium distribution, conditioned on a descriptor of a molecular system such as a chemical graph or a protein sequence. This framework enables the efficient generation of diverse conformations and provides estimations of state densities, orders of magnitude faster than conventional methods. We demonstrate applications of DiG on several molecular tasks, including protein conformation sampling, ligand structure sampling, catalyst–adsorbate sampling and property-guided structure generation. DiG presents a substantial advancement in methodology for statistically understanding molecular systems, opening up new research opportunities in the molecular sciences.
Privacy-Preserving Classification on Deep Learning with Exponential Mechanism
Authors: Quan Ju, Rongqing Xia, Shuhong Li, Xiaojian Zhang

How to protect the privacy of training data in deep learning has been the subject of increasing amounts of related research in recent years. Private Aggregation of Teacher Ensembles (PATE) uses transfer learning and differential privacy methods to provide a broadly applicable data privacy framework in deep learning. PATE combines the Laplacian mechanism and the voting method to achieve deep learning privacy classification. However, the Laplacian mechanism may greatly distort the histogram vote counts of each class. This paper proposes a novel exponential mechanism with PATE to ensure the privacy protection. This proposed method improves the protection effect and accuracy through the screening algorithm and uses the differential privacy combination theorems to reduce the total privacy budget. The data-dependent analysis demonstrates that the exponential mechanism outperforms the original Laplace mechanism. Experimental results show that the proposed method can train models with improved accuracy while requiring a smaller privacy budget when compared to the original Pate framework.
Application of deep learning in isolated tooth identification
Authors: Meng-Xun Li, Zhi-Wei Wang, Xin-Ran Chen, Gui-Song Xia, Yong Zheng, Cui Huang, Zhi Li

BackgroundTeeth identification has a pivotal role in the dental curriculum and provides one of the important foundations of clinical practice. Accurately identifying teeth is a vital aspect of dental education and clinical practice, but can be challenging due to the anatomical similarities between categories. In this study, we aim to explore the possibility of using a deep learning model to classify isolated tooth by a set of photographs.MethodsA collection of 5,100 photographs from 850 isolated human tooth specimens were assembled to serve as the dataset for this study. Each tooth was carefully labeled during the data collection phase through direct observation. We developed a deep learning model that incorporates the state-of-the-art feature extractor and attention mechanism to classify each tooth based on a set of 6 photographs captured from multiple angles. To increase the validity of model evaluation, a voting-based strategy was applied to refine the test set to generate a more reliable label, and the model was evaluated under different types of classification granularities.ResultsThis deep learning model achieved top-3 accuracies of over 90% in all classification types, with an average AUC of 0.95. The Cohen’s Kappa demonstrated good agreement between model prediction and the test set.ConclusionsThis deep learning model can achieve performance comparable to that of human experts and has the potential to become a valuable tool for dental education and various applications in accurately identifying isolated tooth.
State of the Art on Deep Learning-enhanced Rendering Methods
Authors: Qi Wang, Zhihua Zhong, Yuchi Huo, Hujun Bao, Rui Wang

Photorealistic rendering of the virtual world is an important and classic problem in the field of computer graphics. With the development of GPU hardware and continuous research on computer graphics, representing and rendering virtual scenes has become easier and more efficient. However, there are still unresolved challenges in efficiently rendering global illumination effects. At the same time, machine learning and computer vision provide real-world image analysis and synthesis methods, which can be exploited by computer graphics rendering pipelines. Deep learning-enhanced rendering combines techniques from deep learning and computer vision into the traditional graphics rendering pipeline to enhance existing rasterization or Monte Carlo integration renderers. This state-of-the-art report summarizes recent studies of deep learning-enhanced rendering in the computer graphics community. Specifically, we focus on works of renderers represented using neural networks, whether the scene is represented by neural networks or traditional scene files. These works are either for general scenes or specific scenes, which are differentiated by the need to retrain the network for new scenes.
Deep neural networks watermark via universal deep hiding and metric learning
Authors: Zhicheng Ye, Xinpeng Zhang, Guorui Feng

With the rising costs of model training, it is urgent to safeguard the intellectual property of deep neural networks. To achieve this, researchers have proposed various model watermarking techniques. Existing methods utilize visible trigger patterns, which are vulnerable to being detected by humans or detectors. Moreover, these approaches fail to establish active protection mechanisms that link the model with the user’s identity. In this study, we present an innovative imperceptible model watermarking approach that utilizes deep hiding to encode the user’s copyright verification information. This process superimposes a trigger pattern onto clean images, resulting in watermark trigger images. These watermark trigger images closely mimic the original images, achieving excellent stealthiness while enabling the retrieval of the user’s copyright verification information, thus definitively asserting ownership rights. Slight alterations made to the images to maintain stealthiness can weaken the triggering of the watermark pattern. We first leverage the triple loss in metric learning to tackle this challenge of training watermark samples. Using watermark trigger images as anchor samples and selecting appropriate positive and negative samples, we enhance the model’s capability to discern the watermark trigger. Experimental results on CIFAR-10, GTSRB, and Tiny-ImageNet confirm the defender’s capability to embed watermark successfully. The average watermark accuracy exceeds 90%, while the average performance loss is less than 0.05% points. It is also robust to existing watermark removal attacks and backdoor detection methods.