Reviewerly/exHarmony huggingface dataset:
On Faster Convergence of Scaled Sign Gradient Descent

Authors:
raw_author_name: Xiuxian Li
raw_affiliation_string: Department of Control Science and Engineering, College of Electronics and Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, and Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai, China
raw_author_name: Li Li
raw_affiliation_string: Department of Control Science and Engineering, College of Electronics and Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, and Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai, China
raw_author_name: Yiguang Hong
raw_affiliation_string: Department of Control Science and Engineering, College of Electronics and Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, and Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai, China
raw_author_name: Jie Chen
raw_affiliation_string: Department of Control Science and Engineering, College of Electronics and Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, and Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai, China

Abstract: Communication has been seen as a significant bottleneck in industrial applications over large-scale networks. To alleviate the communication burden, sign-based optimization algorithms have gained popularity recently in both industrial and academic communities, which is shown to be closely related to adaptive gradient methods, such as Adam. Along this line, this paper investigates faster convergence for a variant of sign-based gradient descent, called scaled SIGNGD, in three cases: 1) the objective function is strongly convex; 2) the objective function is nonconvex but satisfies the Polyak-Łojasiewicz (PL) inequality; 3) the gradient is stochastic, called scaled SIGNSGD in this case. For the first two cases, it can be shown that the scaled SIGNGD converges at a linear rate. For case 3), the algorithm is shown to converge linearly to a neighborhood of the optimal value when a constant learning rate is employed, and the algorithm converges at a rate of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$O(1/k+1/k^{2}+1/k^{3})$</tex-math></inline-formula> when using a diminishing learning rate, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> is the iteration number. The results are also extended to the distributed setting by majority vote in a parameter-server framework. Finally, numerical experiments are performed to corroborate the theoretical findings.
The Analysis of Educational Needs for Secondary School Teacher’s Emotional Competencies: Focusing on General Education Teachers

Authors:
raw_author_name: Jin-Sun KIM

Abstract: The purpose of this study was to examine the gap between the importance level and their current performance level of secondary school teachers' emotional competencies and to identify the priority of education to improve emotional competencies. Using the emotional competency scale for secondary school teacher, a survey was conducted involving 250 teachers. The collected data were examined using a t-test, Borich's need analysis and Hershkowitz criticality function to determine the priorities of the educational needs regarding emotional competencies. The results of the study are as follows. First, the performance levels of secondary school teachers were lower than the importance levels in all of competencies, with statistically significant differences in all areas. Second, the highest priority educational competencies include emotional regulation, emotional understanding and appraisal, communication, and positivity, primarily focusing on individual emotional competencies. The results of this study indicate that, in the development of emotional competency enhancement programs for secondary school teachers, it is important to prioritize the emotional competencies that enable teachers to regulate, manage, and accurately understand their own emotions.
Residual Q-Networks for Value Function Factorizing in Multiagent Reinforcement Learning

Authors:
raw_author_name: Varuna De Silva
raw_affiliation_string: Institute for Digital Technologies, Loughborough University London, London, U.K.
raw_author_name: Joosep Hook
raw_affiliation_string: Institute for Digital Technologies, Loughborough University London, London, U.K.
raw_author_name: Ahmet Kondoz
raw_affiliation_string: Institute for Digital Technologies, Loughborough University London, London, U.K.

Abstract: Multiagent reinforcement learning (MARL) is useful in many problems that require the cooperation and coordination of multiple agents. Learning optimal policies using reinforcement learning in a multiagent setting can be very difficult as the number of agents increases. Recent solutions such as value decomposition networks (VDNs), QMIX, QTRAN, and QPLEX adhere to the centralized training and decentralized execution (CTDE) scheme and perform factorization of the joint action-value functions. However, these methods still suffer from increased environmental complexity, and at times fail to converge in a stable manner. We propose a novel concept of residual Q-networks (RQNs) for MARL, which learns to transform the individual Q -value trajectories in a way that preserves the individual-global-max (IGM) criteria, but is more robust in factorizing action-value functions. The RQN acts as an auxiliary network that accelerates convergence and will become obsolete as the agents reach the training objectives. The performance of the proposed method is compared against several state-of-the-art techniques such as QPLEX, QMIX, QTRAN, and VDN, in a range of multiagent cooperative tasks. The results illustrate that the proposed method, in general, converges faster, with increased stability, and shows robust performance in a wider family of environments. The improvements in results are more prominent in environments with severe punishments for noncooperative behaviors and especially in the absence of complete state information during training time.
Transforming Education

Authors:
raw_author_name: Andreia Bem Machado
raw_affiliation_string: Universidade Federal de Santa Catarina, Brazil
raw_author_name: Ramesh Sharma
raw_affiliation_string: Ambedkar University, Delhi, India
raw_author_name: Rui Nunes Cruz
raw_affiliation_string: Universidade Europeia, Portugal

Abstract: Generative AI systems are increasingly present in our daily lives, helping us make crucial decisions. They use machine learning algorithms and tools, fed with millions of data collected from the web, producing entirely new information and generating variations. And this is not just limited to texts — it can produce images, audio, videos, even code, or new programming languages. There are several fields where generative AI can have a considerable impact in the coming years. In this context, the issues proposed in this chapter are: What is generative AI? What is prompt engineering? How to transform education using generative AI and prompt engineering in creating synthetic content? To respond to the research problem, the following objective will be achieved: Investigate how to transform education using generative AI and prompt engineering in the creation of synthetic content. It is concluded that generative AI tools can also help create more efficient exercises. Teachers and educators can use technology to create instructional materials and present summaries of concepts.
A multilayer perceptron neural network approach for optimizing solar irradiance forecasting in Central Africa with meteorological insights

Authors:
raw_author_name: Jean Gaston Tamba
raw_affiliation_string: Technologies and Applied Sciences Laboratory, University Institute of Technology of Douala, University of Douala, P.O. Box: 8689, Douala, Cameroon
raw_author_name: Mohit Bajaj
raw_affiliation_string: Department of Electrical Engineering, Graphic Era (Deemed to be University), Dehradun, 248002, India

Abstract: Promoting renewable energy sources, particularly in the solar industry, has the potential to address the energy shortfall in Central Africa. Nevertheless, a difficulty occurs due to the erratic characteristics of solar irradiance data, which is influenced by climatic fluctuations and challenging to regulate. The current investigation focuses on predicting solar irradiance on an inclined surface, taking into consideration the impact of climatic variables such as temperature, wind speed, humidity, and air pressure. The used methodology for this objective is Artificial Neural Network (ANN), and the inquiry is carried out in the metropolitan region of Douala. The data collection device used in this research is the meteorological station located at the IUT of Douala. This station was built as a component of the Douala sustainable city effort, in partnership with the CUD and the IRD. Data was collected at 30-min intervals for a duration of around 2 years, namely from January 17, 2019, to October 30, 2020. The aforementioned data has been saved in a database that underwent pre-processing in Excel and later employed MATLAB for the creation of the artificial neural network model. 80% of the available data was utilized for training the network, 15% was allotted for validation, and the remaining 5% was used for testing. Different combinations of input data were evaluated to ascertain their individual degrees of accuracy. The logistic Sigmoid function, with 50 hidden layer neurons, yielded a correlation coefficient of 98.883% between the observed and estimated sun irradiation. This function is suggested for evaluating the intensities of solar radiation at the place being researched and at other sites that have similar climatic conditions.
OPAL: Occlusion Pattern Aware Loss for Unsupervised Light Field Disparity Estimation

Authors:
raw_author_name: Peng Li
raw_affiliation_string: Tsinghua University, Beijing, China
raw_author_name: Jiayin Zhao
raw_affiliation_string: Tsinghua University, Beijing, China
raw_author_name: Jingyao Wu
raw_affiliation_string: Tsinghua University, Beijing, China
raw_author_name: Chao Deng
raw_affiliation_string: Tsinghua University, Beijing, China
raw_author_name: Yuqi Han
raw_affiliation_string: Tsinghua University, Beijing, China
raw_author_name: Haoqian Wang
raw_affiliation_string: Tsinghua University, Beijing, China
raw_author_name: Tao Yu
raw_affiliation_string: Tsinghua University, Beijing, China

Abstract: Light field disparity estimation is an essential task in computer vision. Currently, supervised learning-based methods have achieved better performance than both unsupervised and optimization-based methods. However, the generalization capacity of supervised methods on real-world data, where no ground truth is available for training, remains limited. In this paper, we argue that unsupervised methods can achieve not only much stronger generalization capacity on real-world data but also more accurate disparity estimation results on synthetic datasets. To fulfill this goal, we present the Occlusion Pattern Aware Loss, named OPAL, which successfully extracts and encodes general occlusion patterns inherent in the light field for calculating the disparity loss. OPAL enables: i) accurate and robust disparity estimation by teaching the network how to handle occlusions effectively and ii) significantly reduced network parameters required for accurate and efficient estimation. We further propose an EPI transformer and a gradient-based refinement module for achieving more accurate and pixel-aligned disparity estimation results. Extensive experiments demonstrate our method not only significantly improves the accuracy compared with SOTA unsupervised methods, but also possesses stronger generalization capacity on real-world data compared with SOTA supervised methods. Last but not least, the network training and inference efficiency are much higher than existing learning-based methods. Our code will be made publicly available.
Robust human locomotion and localization activity recognition over multisensory

Authors:
raw_author_name: Mohammed Alonazi
raw_affiliation_string: Department of Information Systems, Saudi Arabia
raw_author_name: Maha Abdelhaq
raw_affiliation_string: Department of Information Technology, Saudi Arabia
raw_author_name: Naif Al Mudawi
raw_author_name: Asaad Algarni
raw_affiliation_string: Department of Computer Sciences, Faculty of Computing and Information Technology, Northern Border University, Saudi Arabia
raw_author_name: Ahmad Jalal
raw_affiliation_string: Department of Computer Science, Pakistan
raw_author_name: Hui Liu
raw_affiliation_string: Cognitive Systems Lab, Germany

Abstract: Human activity recognition (HAR) plays a pivotal role in various domains, including healthcare, sports, robotics, and security. With the growing popularity of wearable devices, particularly Inertial Measurement Units (IMUs) and Ambient sensors, researchers and engineers have sought to take advantage of these advances to accurately and efficiently detect and classify human activities. This research paper presents an advanced methodology for human activity and localization recognition, utilizing smartphone IMU, Ambient, GPS, and Audio sensor data from two public benchmark datasets: the Opportunity dataset and the Extrasensory dataset. The Opportunity dataset was collected from 12 subjects participating in a range of daily activities, and it captures data from various body-worn and object-associated sensors. The Extrasensory dataset features data from 60 participants, including thousands of data samples from smartphone and smartwatch sensors, labeled with a wide array of human activities. Our study incorporates novel feature extraction techniques for signal, GPS, and audio sensor data. Specifically, for localization, GPS, audio, and IMU sensors are utilized, while IMU and Ambient sensors are employed for locomotion activity recognition. To achieve accurate activity classification, state-of-the-art deep learning techniques, such as convolutional neural networks (CNN) and long short-term memory (LSTM), have been explored. For indoor/outdoor activities, CNNs are applied, while LSTMs are utilized for locomotion activity recognition. The proposed system has been evaluated using the k-fold cross-validation method, achieving accuracy rates of 97% and 89% for locomotion activity over the Opportunity and Extrasensory datasets, respectively, and 96% for indoor/outdoor activity over the Extrasensory dataset. These results highlight the efficiency of our methodology in accurately detecting various human activities, showing its potential for real-world applications. Moreover, the research paper introduces a hybrid system that combines machine learning and deep learning features, enhancing activity recognition performance by leveraging the strengths of both approaches.
Strategic Analysis of the Parameter Servers and Participants in Federated Learning: An Evolutionary Game Perspective

Authors:
raw_author_name: Xinggang Luo
raw_affiliation_string: School of Management, Hangzhou Dianzi University, Hangzhou, China
raw_author_name: Zhongliang Zhang
raw_affiliation_string: School of Management, Hangzhou Dianzi University, Hangzhou, China
raw_author_name: Shengnan Hu
raw_affiliation_string: School of Management, Hangzhou Dianzi University, Hangzhou, China

Abstract: Federated learning (FL) is a new decentralized deep learning paradigm developed for collaborative model training and solving the problem of data privacy and has received extensive attention from both the academic and business worlds. However, FL still faces challenges in encouraging participants to contribute private data and computational resources. Although many studies have applied game theory models to improve the incentive mechanism design of FL, they assume that the players are absolutely rational and that the game models are static. In this study, a mathematical model based on evolutionary game theory (EGT) is established to analyze the interaction between parameter servers and participants, considering that the participants are not completely rational in the long-term dynamic decision-making process. The evolutionarily stable status of the FL system and the strategies of the parameter servers and participants were analyzed under eight different scenarios. Based on the model analysis and results of the numerical experiments, managerial insights for maintaining a sustainable FL system are summarized.
Quantum Vision Transformers

Authors:
raw_author_name: El Amine Cherrat
raw_affiliation_string: IRIF, CNRS - Université Paris Cité, France
raw_author_name: Natansh Mathur
raw_affiliation_string: IRIF, CNRS - Université Paris Cité, France; QC Ware, Palo Alto, USA and Paris, France
raw_author_name: Jonas Landman
raw_affiliation_string: QC Ware, Palo Alto, USA and Paris, France; School of Informatics, University of Edinburgh, Scotland, UK
raw_author_name: Martin Strahm
raw_affiliation_string: F. Hoffmann La Roche AG

Abstract: In this work, quantum transformers are designed and analysed in detail by extending the state-of-the-art classical transformer neural network architectures known to be very performant in natural language processing and image analysis. Building upon the previous work, which uses parametrised quantum circuits for data loading and orthogonal neural layers, we introduce three types of quantum transformers for training and inference, including a quantum transformer based on compound matrices, which guarantees a theoretical advantage of the quantum attention mechanism compared to their classical counterpart both in terms of asymptotic run time and the number of model parameters. These quantum architectures can be built using shallow quantum circuits and produce qualitatively different classification models. The three proposed quantum attention layers vary on the spectrum between closely following the classical transformers and exhibiting more quantum characteristics. As building blocks of the quantum transformer, we propose a novel method for loading a matrix as quantum states as well as two new trainable quantum orthogonal layers adaptable to different levels of connectivity and quality of quantum computers. We performed extensive simulations of the quantum transformers on standard medical image datasets that showed competitively, and at times better performance compared to the classical benchmarks, including the best-in-class classical vision transformers. The quantum transformers we trained on these small-scale datasets require fewer parameters compared to standard classical benchmarks. Finally, we implemented our quantum transformers on superconducting quantum computers and obtained encouraging results for up to six qubit experiments.
LEAF: A Federated Learning-Aware Privacy-Preserving Framework for Healthcare Ecosystem

Authors:
raw_author_name: Nisarg P. Patel
raw_affiliation_string: Department of Computer Science and Engineering, Institute of Technology, Nirma University, Ahmedabad, India
raw_author_name: Raj Parekh
raw_affiliation_string: Department of Computer Science and Engineering, Institute of Technology, Nirma University, Ahmedabad, India
raw_author_name: Saad Ali Amin
raw_affiliation_string: College of Engineering and Information Technology, University of Dubai, Dubai, UAE
raw_author_name: Rajesh Gupta
raw_affiliation_string: Department of Computer Science and Engineering, Institute of Technology, Nirma University, Ahmedabad, India
raw_author_name: Sudeep Tanwar
raw_affiliation_string: Department of Computer Science and Engineering, Institute of Technology, Nirma University, Ahmedabad, India
raw_author_name: Neeraj Kumar
raw_affiliation_string: Department of CSED, Thapar Institute of Engineering &#x0026; Technology, Patiala, India
raw_author_name: Rahat Iqbal
raw_affiliation_string: College of Engineering and Information Technology, University of Dubai, Dubai, UAE
raw_author_name: Ravi Sharma
raw_affiliation_string: Centre for Inter-Disciplinary Research and Innovation, University of Petroleum and Energy Studies, Dehradun, India

Abstract: Over the last decades, the healthcare industry has been revolutionized like anything, especially after the Covid-19 surge. Various artificial intelligence approaches have also been explored during this era for their applicability in healthcare. However, traditional AI techniques and algorithms are prone to overfitting with minimum robustness to unseen or untrained data. So, there is a requirement for new techniques which can counterfeit the issues mentioned earlier. Federated learning (FL) can help to make specific AI services for the network of hospitals with less overfitting and more robust modules. However, with the inclusion of FL, the problem related to user privacy is the biggest challenge, making using FL in the real world a grand challenge. Most solutions presented in the literature used blockchain technology to mitigate the issues mentioned earlier. However, it prevents third-party systems from penetrating the decision process, but the network devices can access shared data. Moreover, blockchain implementation requires new paradigms and infrastructure with an additional overhead cost. Motivated by these facts, the paper presents a limited access encryption algorithm incorporating FL (LEAF) framework, i.e., an encryption technique that solves privacy issues with the help of edge-enabled AI models. The proposed LEAF framework preserves user privacy and minimizes overhead costs. The authors have evaluated the performance of the LEAF framework using extensive simulations and achieved superior results. The achieved accuracy of the proposed LEAF framework is 3% more than that of the traditional centralized and FL-based systems advantages without compromising user privacy. In the best scenario, the proposed framework’s encryption process also compresses the data size by 4-5 times.