ADKU/ResearchGPT_model_dataset huggingface dataset:
Deep Metric Learning: Loss Functions Comparison
Authors: R. L. Vasilev, A. G. D’yakonov

AbstractAn overview of deep metric learning methods is presented. Although they have appeared in recent years, these methods were compared only with their predecessors, with neural networks of outdated architectures used for representation learning (representations on which the metric is calculated). The described methods were compared on different datasets from several domains, using pre-trained neural networks comparable in performance to SotA (state of the art): ConvNeXt for images and DistilBERT for texts. Labeled datasets were used, divided into two parts (train and test) so that the classes did not overlap (i.e., for each class its objects are fully in train or fully in test). Such a large-scale honest comparison was made for the first time and led to unexpected conclusions, viz. some “old” methods, for example, Tuplet Margin Loss, are superior in performance to their modern modifications and methods proposed in very recent works.
Comparing Machine Learning and Deep Learning Techniques for Text Analytics: Detecting the Severity of Hate Comments Online
Authors: Alaa Marshan, Farah Nasreen Mohamed Nizar, Athina Ioannou, Konstantina Spanaki

Social media platforms have become an increasingly popular tool for individuals to share their thoughts and opinions with other people. However, very often people tend to misuse social media posting abusive comments. Abusive and harassing behaviours can have adverse effects on people's lives. This study takes a novel approach to combat harassment in online platforms by detecting the severity of abusive comments, that has not been investigated before. The study compares the performance of machine learning models such as Naïve Bayes, Random Forest, and Support Vector Machine, with deep learning models such as Convolutional Neural Network (CNN) and Bi-directional Long Short-Term Memory (Bi-LSTM). Moreover, in this work we investigate the effect of text pre-processing on the performance of the machine and deep learning models, the feature set for the abusive comments was made using unigrams and bigrams for the machine learning models and word embeddings for the deep learning models. The comparison of the models’ performances showed that the Random Forest with bigrams achieved the best overall performance with an accuracy of (0.94), a precision of (0.91), a recall of (0.94), and an F1 score of (0.92). The study develops an efficient model to detect severity of abusive language in online platforms, offering important implications both to theory and practice.
Interpretable predictions of chaotic dynamical systems using dynamical system deep learning
Authors: Mingyu Wang, Jianping Li

Making accurate predictions of chaotic dynamical systems is an essential but challenging task with many practical applications in various disciplines. However, the current dynamical methods can only provide short-term precise predictions, while prevailing deep learning techniques with better performances always suffer from model complexity and interpretability. Here, we propose a new dynamic-based deep learning method, namely the dynamical system deep learning (DSDL), to achieve interpretable long-term precise predictions by the combination of nonlinear dynamics theory and deep learning methods. As validated by four chaotic dynamical systems with different complexities, the DSDL framework significantly outperforms other dynamical and deep learning methods. Furthermore, the DSDL also reduces the model complexity and realizes the model transparency to make it more interpretable. We firmly believe that the DSDL framework is a promising and effective method for comprehending and predicting chaotic dynamical systems.
A comprehensive review of COVID-19 detection with machine learning and deep learning techniques
Authors: Sreeparna Das, Ishan Ayus, Deepak Gupta

PurposeThe first transmission of coronavirus to humans started in Wuhan city of China, took the shape of a pandemic called Corona Virus Disease 2019 (COVID-19), and posed a principal threat to the entire world. The researchers are trying to inculcate artificial intelligence (Machine learning or deep learning models) for the efficient detection of COVID-19. This research explores all the existing machine learning (ML) or deep learning (DL) models, used for COVID-19 detection which may help the researcher to explore in different directions. The main purpose of this review article is to present a compact overview of the application of artificial intelligence to the research experts, helping them to explore the future scopes of improvement.MethodsThe researchers have used various machine learning, deep learning, and a combination of machine and deep learning models for extracting significant features and classifying various health conditions in COVID-19 patients. For this purpose, the researchers have utilized different image modalities such as CT-Scan, X-Ray, etc. This study has collected over 200 research papers from various repositories like Google Scholar, PubMed, Web of Science, etc. These research papers were passed through various levels of scrutiny and finally, 50 research articles were selected.ResultsIn those listed articles, the ML / DL models showed an accuracy of 99% and above while performing the classification of COVID-19. This study has also presented various clinical applications of various research. This study specifies the importance of various machine and deep learning models in the field of medical diagnosis and research.ConclusionIn conclusion, it is evident that ML/DL models have made significant progress in recent years, but there are still limitations that need to be addressed. Overfitting is one such limitation that can lead to incorrect predictions and overburdening of the models. The research community must continue to work towards finding ways to overcome these limitations and make machine and deep learning models even more effective and efficient. Through this ongoing research and development, we can expect even greater advances in the future.
Continual learning, deep reinforcement learning, and microcircuits: a novel method for clever game playing
Authors: Oscar Chang, Leo Ramos, Manuel Eugenio Morocho-Cayamcela, Rolando Armas, Luis Zhinin-Vera

Contemporary neural networks frequently encounter the challenge of catastrophic forgetting, wherein newly acquired learning can overwrite and erase previously learned information. The paradigm of continual learning offers a promising solution by enabling intelligent systems to retain and build upon their acquired knowledge over time. This paper introduces a novel approach within the continual learning framework, employing deep reinforcement learning agents that process unprocessed pixel data and interact with microcircuit-like components. These agents autonomously advance through a series of learning stages, culminating in the development of a sophisticated neural network system optimized for predictive performance in the game of tic-tac-toe. Structured to operate in sequential order, each agent is tasked with achieving forward-looking objectives based on Bellman’s principles of reinforcement learning. Knowledge retention is facilitated through the integration of specific microcircuits, which securely store the insights gained by each agent. During the training phase, these microcircuits work in concert, employing high-energy, sparse encoding techniques to enhance learning efficiency and effectiveness. The core contribution of this paper is the establishment of an artificial neural network system capable of accurately predicting tic-tac-toe moves, akin to the observational strategies employed by humans. Our experimental results demonstrate that after approximately 5000 cycles of backpropagation, the system significantly reduced the training loss to \(L_{DQN}<0.1\), thereby increasing the expected cumulative reward. This advancement in training efficiency translates into superior predictive capabilities, enabling the system to secure consistent victories by anticipating up to four moves ahead.
Application of deep learning for characterizing microstructures in SBS modified asphalt
Authors: Enhao Zhang, Liyan Shan, Yapeng Guo, Shuang Liu

Microstructures in asphalt, often resembling bee structures, are pivotal in influencing asphalt performance and, by extension, sustainable fuel production. This study employs deep learning techniques to investigate the impact of different Styrene–Butadiene–Styrene (SBS) modifiers on asphalt microstructures, akin to bee structures. The employed deep learning model was trained on a diverse dataset comprising 200 of images sourced from testing. The dataset was carefully curated to address specific challenges in data labeling precision. This involved individualized labeling sessions and adjustments in the number of targets per image, contributing to enhanced precision and increased dataset size. The research begins with the development of a deep learning model trained on a dataset comprising images featuring bee-like structures within asphalt. The model excels in accurately identifying and segmenting these structures. Subsequently, the deep learning approach is compared with existing methods for bee structure segmentation to establish its precision and superiority. Employing frequency distribution histograms, the distribution patterns of bee structures within various types of SBS-modified asphalt is analyzed, quantitatively assessing the influence of diverse modifier types on these microstructural attributes. The findings in this study underscore the deep learning model's efficacy in recognizing and segmenting bee structures with introduced metrics effectively capturing the distinctive characteristics of various asphalt microstructures. This study paves the way for comprehensive analyses of microstructural metrics, including parameters such as perimeter, area, quantity, and related indicators, thus contributing to the development of fundamental asphalt structural units suitable for processes like molecular simulation and finite element analysis. Moreover, it propels the application of deep learning methodologies in the realm of road materials research, opening new avenues for innovative explorations that can ultimately benefit sustainable fuel production.
Deep Machine Learning in Optimization of Scientific Research Activities
Authors: E. V. Melnikova

Abstract—This article provides a general overview of machine learning, a subdomain of artificial intelligence. The substance of the deep learning process is explained, and key features of deep learning as a high-level artificial intelligence technology are outlined. Differences between deep and conventional machine learning are analyzed. The architecture of deep learning models is considered. Issues with using deep learning in neural networks are outlined, and key processes of the functioning of neural networks are described. The importance of deep learning neural networks for processing big data is noted. Specific examples of application of deep learning algorithms in various research fields, specifically, scientometrics, bibliometrics, medicine, geoseismic research, and others, are provided. It is shown that deep learning plays an important role in optimizing research activities and improving research productivity.
Detecting and Mitigating Encoded Bias in Deep Learning-Based Stealth Assessment Models for Reflection-Enriched Game-Based Learning Environments
Authors: Anisha Gupta, Dan Carpenter, Wookhee Min, Jonathan Rowe, Roger Azevedo, James Lester

Reflection plays a critical role in learning. Game-based learning environments have significant potential to elicit and support student reflection by prompting learners to think critically about their own learning processes and performance. Stealth assessment models, used for unobtrusively assessing student competencies from evidence of game interaction data and facilitating learning through adaptive feedback, can be enhanced by incorporating evidence from students’ written reflections. We present a deep learning-based stealth assessment framework that predicts depth of student reflections and science content post-test scores during game-based learning. With the increasing adoption of AI techniques in decision-making processes, it is important to evaluate the fairness of these models. To address this concern, we investigate encoded bias in our stealth assessment model with respect to student gender and prior game-playing experience in deep learning-based stealth assessment models and examine the impact of debiasing on the models’ predictive performance. We evaluate the predictive performance of the deep learning-based stealth assessment models and measure encoded bias with the Absolute Between-ROC Area (ABROCA) statistic using gameplay data from 119 students collected in a series of classroom studies with a reflection-enriched game-based learning environment for middle school microbiology, Crystal Island. The results demonstrate the effectiveness of deep learning-based stealth assessment models and multiple debiasing techniques for deriving algorithmically fair stealth assessment models.
Reading Between the Lines: Machine Learning Ensemble and Deep Learning for Implied Threat Detection in Textual Data
Authors: Muhammad Owais Raza, Areej Fatemah Meghji, Naeem Ahmed Mahoto, Mana Saleh Al Reshan, Hamad Ali Abosaq, Adel Sulaiman, Asadullah Shaikh

With the increase in the generation and spread of textual content on social media, natural language processing (NLP) has become an important area of research for detecting underlying threats, racial abuse, violence, and implied warnings in the content. The subtlety and ambiguity of language make the development of effective models for detecting threats in text a challenging task. This task is further complicated when the threat is not explicitly conveyed. This study focuses on the task of implied threat detection using an explicitly designed machine-generated dataset with both linguistic and lexical features. We evaluated the performance of different machine learning algorithms on these features including Support Vector Machines, Logistic Regression, Naive Bayes, Decision Tree, and K-nearest neighbors. The ensembling approaches of Adaboost, Random Forest, and Gradient Boosting were also explored. Deep learning modeling was performed using Long Short-Term Memory, Deep Neural Networks (DNN), and Bidirectional Long Short-Term Memory (BiLSTM). Based on the evaluation, it was observed that classical and ensemble models overfit while working with linguistic features. The performance of these models improved when working with lexical features. The model based on logistic regression exhibited superior performance with an F1 score of 77.13%. While experimenting with deep learning models, DNN achieved an F1 score of 91.49% while the BiLSTM achieved an F1 score of 91.61% while working with lexical features. The current study provides a baseline for future research in the domain of implied threat detection.
Deep learning-based phenotyping reclassifies combined hepatocellular-cholangiocarcinoma
Authors: Julien Calderaro, Narmin Ghaffari Laleh, Qinghe Zeng, Pascale Maille, Loetitia Favre, Anaïs Pujals, Christophe Klein, Céline Bazille, Lara R. Heij, Arnaud Uguen, Tom Luedde, Luca Di Tommaso, Aurélie Beaufrère, Augustin Chatain, Delphine Gastineau, Cong Trung Nguyen, Hiep Nguyen-Canh, Khuyen Nguyen Thi, Viviane Gnemmi, Rondell P. Graham, Frédéric Charlotte, Dominique Wendum, Mukul Vij, Daniela S. Allende, Federico Aucejo, Alba Diaz, Benjamin Rivière, Astrid Herrero, Katja Evert, Diego Francesco Calvisi, Jérémy Augustin, Wei Qiang Leow, Howard Ho Wai Leung, Emmanuel Boleslawski, Mohamed Rela, Arnaud François, Anthony Wing-Hung Cha, Alejandro Forner, Maria Reig, Manon Allaire, Olivier Scatton, Denis Chatelain, Camille Boulagnon-Rombi, Nathalie Sturm, Benjamin Menahem, Eric Frouin, David Tougeron, Christophe Tournigand, Emmanuelle Kempf, Haeryoung Kim, Massih Ningarhari, Sophie Michalak-Provost, Purva Gopal, Raffaele Brustia, Eric Vibert, Kornelius Schulze, Darius F. Rüther, Sören A. Weidemann, Rami Rhaiem, Jean-Michel Pawlotsky, Xuchen Zhang, Alain Luciani, Sébastien Mulé, Alexis Laurent, Giuliana Amaddeo, Hélène Regnault, Eleonora De Martin, Christine Sempoux, Pooja Navale, Maria Westerhoff, Regina Cheuk-Lam Lo, Jan Bednarsch, Annette Gouw, Catherine Guettier, Marie Lequoy, Kenichi Harada, Pimsiri Sripongpun, Poowadon Wetwittayaklang, Nicolas Loménie, Jarukit Tantipisit, Apichat Kaewdech, Jeanne Shen, Valérie Paradis, Stefano Caruso, Jakob Nikolas Kather

Primary liver cancer arises either from hepatocytic or biliary lineage cells, giving rise to hepatocellular carcinoma (HCC) or intrahepatic cholangiocarcinoma (ICCA). Combined hepatocellular- cholangiocarcinomas (cHCC-CCA) exhibit equivocal or mixed features of both, causing diagnostic uncertainty and difficulty in determining proper management. Here, we perform a comprehensive deep learning-based phenotyping of multiple cohorts of patients. We show that deep learning can reproduce the diagnosis of HCC vs. CCA with a high performance. We analyze a series of 405 cHCC-CCA patients and demonstrate that the model can reclassify the tumors as HCC or ICCA, and that the predictions are consistent with clinical outcomes, genetic alterations and in situ spatial gene expression profiling. This type of approach could improve treatment decisions and ultimately clinical outcome for patients with rare and biphenotypic cancers such as cHCC-CCA.
Advancing Naturalistic Affective Science with Deep Learning
Authors: Chujun Lin, Landry S. Bulls, Lindsey J. Tepfer, Amisha D. Vyas, Mark A. Thornton

People express their own emotions and perceive others’ emotions via a variety of channels, including facial movements, body gestures, vocal prosody, and language. Studying these channels of affective behavior offers insight into both the experience and perception of emotion. Prior research has predominantly focused on studying individual channels of affective behavior in isolation using tightly controlled, non-naturalistic experiments. This approach limits our understanding of emotion in more naturalistic contexts where different channels of information tend to interact. Traditional methods struggle to address this limitation: manually annotating behavior is time-consuming, making it infeasible to do at large scale; manually selecting and manipulating stimuli based on hypotheses may neglect unanticipated features, potentially generating biased conclusions; and common linear modeling approaches cannot fully capture the complex, nonlinear, and interactive nature of real-life affective processes. In this methodology review, we describe how deep learning can be applied to address these challenges to advance a more naturalistic affective science. First, we describe current practices in affective research and explain why existing methods face challenges in revealing a more naturalistic understanding of emotion. Second, we introduce deep learning approaches and explain how they can be applied to tackle three main challenges: quantifying naturalistic behaviors, selecting and manipulating naturalistic stimuli, and modeling naturalistic affective processes. Finally, we describe the limitations of these deep learning methods, and how these limitations might be avoided or mitigated. By detailing the promise and the peril of deep learning, this review aims to pave the way for a more naturalistic affective science.
Bearing Fault Diagnosis Based on Artificial Intelligence Methods: Machine Learning and Deep Learning
Authors: Ahmed Ghorbel, Sarra Eddai, Bouthayna Limam, Nabih Feki, Mohamed Haddar

This paper presents a comprehensive study on the application of Artificial Intelligence (AI) methods, specifically machine learning and deep learning, for the diagnosis of bearing faults. The study explores both data preprocessing-dependent methods (Support Vector Machine, Nearest Neighbor, and Decision Tree) and a preprocessing-independent method (1D Convolutional Neural Network). The experiment setup utilizes the Case Western Reserve University dataset for signal acquisition. A detailed strategy for data processing is developed, encompassing initialization, data loading, signal filtration, decomposition, feature extraction in both time- and frequency-domains, and feature selection. Indeed, the study involves working with four datasets, selected based on the distribution curves of the indicators as a function of the number of observations. The results demonstrate remarkable performance of the AI methods in bearing fault diagnosis. The 1D-CNN model, in particular, shows high robustness and accuracy, even in the presence of load variations. The findings of this study shed light on the significant potential of AI methods in improving the accuracy and efficiency of bearing fault diagnosis.
Exploring the effects of digital technology on deep learning: a meta-analysis
Authors: Xiu-Yi Wu

The impact of digital technology on learning outcomes, specifically deep learning, has been a subject of considerable debate and scrutiny in educational settings. This study aims to provide clarity by conducting a meta-analysis of empirical publications that examine students' deep learning outcomes in relation to digital technology. A comprehensive search of databases and a thorough literature review yielded 60 high-quality, peer-reviewed journal articles that met the inclusion criteria. Using Review Manager 5.4.1 software, a meta-analysis was conducted to assess the overall effectiveness of digital technology. The calculated effect size indicates a positive influence of digital technology on students' deep learning outcomes. Furthermore, a moderator variable analysis revealed several significant findings: 1. Different categories of digital technology tools have a favorable impact on deep learning outcomes; 2. The duration of digital technology treatment does not significantly affect deep learning outcomes; 3. Digital technology demonstrates a highly positive influence on deep learning within the humanities and social sciences disciplines; 4. Combining online and offline utilization of digital technology in education leads to a substantially greater enhancement in deep learning compared to relying solely on online methods; 5. The effectiveness of digital technology on deep learning is enhanced when accompanied by appropriate instructional guidance; 6. Utilizing digital technology in a systematic manner produces different outcomes compared to fragmented approaches, highlighting the importance of a cohesive implementation; 7. Integrating digital technology with collaborative learning has a more pronounced effect on deep learning compared to independent learning. These findings contribute to our understanding of the impact of digital technology on deep learning outcomes and underscore the importance of thoughtful integration and instructional support in educational contexts.