Reviewerly/exHarmony huggingface dataset:
Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

Multimodality Representation Learning, as a technique of learning to embed information from different modalities and their correlations, has achieved remarkable success on a variety of applications, such as Visual Question Answering (VQA), Natural Language for Visual Reasoning (NLVR), and Vision Language Retrieval (VLR). Among these applications, cross-modal interaction and complementary information from different modalities are crucial for advanced models to perform any multimodal task, e.g., understand, recognize, retrieve, or generate optimally. Researchers have proposed diverse methods to address these tasks. The different variants of transformer-based architectures performed extraordinarily on multiple modalities. This survey presents the comprehensive literature on the evolution and enhancement of deep learning multimodal architectures to deal with textual, visual and audio features for diverse cross-modal and modern multimodal tasks. This study summarizes the ( i ) recent task-specific deep learning methodologies, ( ii ) the pretraining types and multimodal pretraining objectives, ( iii ) from state-of-the-art pretrained multimodal approaches to unifying architectures, and ( iv ) multimodal task categories and possible future improvements that can be devised for better multimodal learning. Moreover, we prepare a dataset section for new researchers that covers most of the benchmarks for pretraining and finetuning. Finally, major challenges, gaps, and potential research topics are explored. A constantly-updated paperlist related to our survey is maintained at https://github.com/marslanm/multimodality-representation-learning .
All optical operation of a superconducting photonic interface

Quantum photonic processing via electro-optic components typically requires electronic links across different operation environments, especially when interfacing cryogenic components such as superconducting single photon detectors with room-temperature control and readout electronics. However, readout and driving electronics can introduce detrimental parasitic effects. Here we show an all-optical control and readout of a superconducting nanowire single photon detector (SNSPD), completely electrically decoupled from room temperature electronics. We provide the operation power for the superconducting detector via a cryogenic photodiode, and readout single photon detection signals via a cryogenic electro-optic modulator in the same cryostat. This method opens the possibility for control and readout of superconducting circuits, and feedforward for photonic quantum computing.
Teaching Building Information Modeling in the Metaverse—An Approach Based on Quantitative and Qualitative Evaluation of the Students Perspective

The teaching of civil engineering consists of different didactic approaches, such as lectures, group work or research-based teaching, depending on the respective courses. Currently, the metaverse is gaining importance in teaching and offers the possibility of a new teaching approach for civil engineering and especially for the teaching of courses from the areas of “Digital Design and Construction”. Although the advantages of teaching in the metaverse, such as location and time independence or a higher learning outcome, are mentioned in the literature, there are also challenges that must be considered when teaching in the metaverse. Against this background, this paper examines the implications of using the metaverse as a teaching tool in teaching “Digital Design and Construction”. The impact of teaching BIM in the metaverse is evaluated by (1) a literature review and workshops to evaluate use cases and demands for extended reality (XR) and the metaverse, (2) integrating XR and the metaverse in the courses and valuation by quantitative evaluations and (3) analyzing student papers of the courses and outcomes of a World Café. Due to these steps, this paper presents a novel approach by reflecting the students’ perspective. Furthermore, this paper presents a validated approach for integrating BIM and the metaverse in teaching.
Refined finite-size analysis of binary-modulation continuous-variable quantum key distribution

Recent studies showed the finite-size security of binary-modulation CV-QKD protocols against general attacks. However, they gave poor key-rate scaling against transmission distance. Here, we extend the security proof based on complementarity, which is used in the discrete-variable QKD, to the previously developed binary-modulation CV-QKD protocols with the reverse reconciliation under the finite-size regime and obtain large improvements in the key rates. Notably, the key rate in the asymptotic limit scales linearly against the attenuation rate, which is known to be optimal scaling but is not achieved in previous finite-size analyses. This refined security approach may offer full-fledged security proofs for other discrete-modulation CV-QKD protocols.
Considerations, Advances, and Challenges Associated with the Use of Specific Emitter Identification in the Security of Internet of Things Deployments: A Survey

Initially introduced almost thirty years ago for the express purpose of providing electronic warfare systems the capabilities to detect, characterize, and identify radar emitters, Specific Emitter Identification (SEI) has recently received a lot of attention within the research community as a physical layer technique for securing Internet of Things (IoT) deployments. This attention is largely due to SEI’s demonstrated success in passively and uniquely identifying wireless emitters using traditional machine learning and the success of Deep Learning (DL) within the natural language processing and computer vision areas. SEI exploits distinct and unintentional features present within an emitter’s transmitted signals. These distinctive and unintentional features are attributed to slight manufacturing and assembly variations within and between the components, sub-systems, and systems comprising an emitter’s Radio Frequency (RF) front end. Although sufficient to facilitate SEI, these features do not hinder normal operations such as detection, channel estimation, timing, and demodulation. However, despite the plethora of SEI publications, it has remained largely a focus of academic endeavors, primarily focusing on proof-of-concept demonstration and little to no use in operational networks for various reasons. The focus of this survey is a review of SEI publications from the perspective of its use as a practical, effective, and usable IoT security mechanism; thus, we use IoT requirements and constraints (e.g., wireless standard, nature of their deployment) as a lens through which each reviewed paper is analyzed. Previous surveys have not taken such an approach and have only used IoT as motivation, a setting, or a context. In this survey, we consider operating conditions, SEI threats, SEI at scale, publicly available data sets, and SEI considerations that are dictated by the fact that it is to be employed by IoT devices or IoT infrastructure.
Requirements Elicitation and Stakeholder Communications for Explainable Machine Learning Systems: State of the Art

Machine Learning (ML) systems are currently used widely. While the requirements elicitation and management processes are well understood and standardized for software systems, they have not been adopted for ML systems. These processes pose additional complexity due to the multitude of stakeholders, their understanding of ML concepts, and their varied requirement orientations. Communication among the stakeholders is a must to generate an appropriate requirements document to guide the development of the system. Another aspect of ML systems is their black box mode of operation. It is essential to know the reasoning behind the predictions of these systems, especially for critical applications. These explainability requirements also must be taken into consideration. This paper reviews state of the art in these two areas to form a basis for developing a framework for stakeholder interaction to facilitate the requirement management of ML systems.
Facilitating Metaverse Integration in Education via Extended Reality and the Internet of Things: AN OVERVIEW

The metaverse, a virtual space that combines elements of the physical and virtual worlds, has gained significant attention in recent years. Its potential to transform education is considerable, with extended reality (XR) technologies, such as virtual and augmented reality, and the Internet of Everything (IoE) enabling the creation of immersive, interactive learning experiences that can enhance learning outcomes and promote educational accessibility. However, implementing a metaverse in education also poses challenges, including the need for standardized XR content, the expense of hardware and software, and concerns regarding privacy and security. This paper aims to investigate the opportunities and challenges associated with using XR and IoE to facilitate a metaverse in education. Moreover, this study offers a comparative analysis of different approaches to implementing a metaverse in education. Our findings suggest that the benefits of a metaverse in education are substantial and that careful planning, research, and collaboration can help overcome the challenges and realize the potential of this technology in education.
Arabic User Requirements Classification Using Machine Learning

Requirement engineering is a crucial step in software engineering as it forms the foundation for all subsequent stages and significantly affects whether software development is successful or unsuccessful. The same software requirement document generally includes both functional and non-functional requirements are essential. However, expressing these requirements in natural language necessitates considerable human effort to classify them. Manual classification can be laborious, time-consuming, expensive, and prone to inaccuracies. Inaccurate classification can result in misunderstandings or ambiguities in requirements, leading to incomplete products that fail to meet customer demands. While many studies have proposed English language requirements classification techniques, there is an absence of research in Arabic requirements classification. Moreover, there aren't many Arabic datasets that are openly accessible in this sector. In this paper, we offer a method for gathering Arabic data sets for needs and utilizing machine learning techniques to automatically categorize software requirements written in Arabic into functional and non-functional requirements. This study intends to assist software developers in time savings, lower the cost and effort associated with the manual classification process, and improve the effectiveness of the requirements engineering phase. Further gives scholars in this subject a new topic to explore.
A Study of Human Fitness Pose Classification Using Artificial Neural Networks

Motivated by the increasing demand for detecting and classifying human poses in the realm of personalized fitness training by AI technologies, which provide feedback on the form to help users exercise more accurately, and the proven effectiveness of some deep learning models in achieving that, this study aims to investigate three different ensemble approaches for artificial neural network models for detecting and classifying human poses. The pre-trained MoveNet model was employed to extract the positions of 17 body keypoints, which were used as input data for the subsequent three classification models - a Feedforward Neural Network (FNN), LSTM, and GRU. The LSTM and GRU models have the ability to process time series data as input, resulting in improved accuracy compared to the FNN model. Specifically, the LSTM model achieved an accuracy of nearly 95%, while the GRU model outperformed with an accuracy exceeding 95% and the potential to reach 97.27 %.
Human Body Action Recognition based on 3D Feature Fusion over RGB Scenes

Human body action recognition (HBAR) is an important area of research in machine learning and image processing due to its vast range of applications. Similarly, estimating various components of human anatomy from RGB scenes are essential to human action tracking. The present study involves the implementation of human key body point detection for pose estimation. During abstracted silhouettes and vital human points, the proposed system has extracted two main features, such as 3D features and distance features. Once the relevant attributes have been incorporated, executing precise tasks and strategies is critical to achieving maximum features. Therefore, we used the t- SNE-based data refinement technique for optimal feature selection. Finally, for training, a classification model random forest is utilized. The presented system is verified on a recognized benchmark dataset, i.e., the UCF sports database. The experimental settings have revealed that our proposed system has attained better performance and outperformed other present state-of-the-art methods regarding mean recognition rate.
Blockchain Based Voting Framework With Non-Transferable Non-Fungible Tokens

Ensuring the integrity of voting systems is essential for the functioning of modern democratic societies. However, election interference and fraud remain persistent issues that threaten to undermine public trust in the democratic process. While blockchain technology has the potential to improve traceability and transparency in voting systems, the current use of fungible tokens in blockchain-based voting systems leaves them vulnerable to manipulation through vote buying and selling. To address this problem, this research proposes the use of non-fungible tokens (NFTs) with smart contract functions for voting systems. This novel approach effectively represents complicated operations and ensures integrity, security, and privacy. in addition, it prevents fraud, corruption, tampering, and counterfeiting. The research will develop a framework and analyse the use of NFTs in voting systems to improve the integrity of the democratic process.
Arabic Automatic Speech Recognition for Speakers With Speech Disorders: A Comprehensive Review

Automatic speech recognition (ASR) aims to convert human speech signal into text, where the quality of the speech signal is essential to assure correct recognition. However, some people suffer speech disorders (SD) of various kinds such as distortion and substitution, which may degrade the recognition of the speech signal in ASR domain. Based on our literature investigation, most of the available ASR systems handle normal/regular speech with no consideration for SD and disabilities. This paper provides a comprehensive review on Arabic ASR for speakers suffering SD. In this paper, classifications of SD are provided. In addition, research attempts on Arabic ASR for speakers suffering SD are also highlighted in order to identify the benchmark datasets, tools, algorithms, and research gaps. To the best of our knowledge, all previous research attempts on Arabic ASR for speakers suffering SD focused on the Arabic letter /r/ in limited isolated words mode. This shows that there is a need to consider other types of SD in ASR domain for Arabic language, and consider continuous speech mode. In addition, Arabic speech datasets are very limited for SD research, which can be treated as an important research gap to be considered.
Learning framework based on ER Rule for data streams with generalized feature spaces

Learning with data streams has recently been the focus of extensive research and various solutions have been proposed. However, most such studies assume that the features remain unchanged or change with stringent hypotheses, which cannot always hold in practical applications. Considering the proven superiority of Evidential Reasoning Rule (ER Rule) classifier and its parameters are separately associated with each feature, a learning framework based on the ER Rule for data streams with Generalized Feature Spaces (ER Rule-GFS) is developed. In ER Rule-GFS, the feature spaces of data streams are generalized, which means that at each stage, new features and existing features will occur randomly or not. When a new data stream arrives, a certain effective feature selection algorithm is used to select the informative features first. Then, the selected features and the features stored thus far are divided into three categories, namely, active/new/silent. According to whether active and new are null, three generalized basic learning problems are formulated, and their solutions are given. Afterwards, if there are too many features, follow-on feature selection will be performed mainly in silent based on the activated frequency and weight. The effectiveness of ER Rule-GFS is verified by various experiments on 20 synthetic datasets.
Embedding Clustering via Autoencoder and Projection onto Convex Set

Projection onto Convex Set (POCS) is a powerful signal processing tool for various convex optimization problems. For non-intersecting convex sets, the simultaneous POCS method can result in a minimum mean square error solution. This property of POCS has been applied to clustering analysis and the POCS-based clustering algorithm was proposed earlier. In the POCS-based clustering algorithm, each data point is treated as a convex set, and a parallel projection operation from every cluster prototype to its corresponding data members is carried out in order to minimize the objective function and to update the memberships and prototypes. The algorithm works competitively against conventional clustering methods in terms of execution speed and clustering error on general clustering tasks. In this paper, the performance of the POCS-based clustering algorithm on a more complex task, embedding clustering, is investigated in order to further demonstrate its potential in benefiting other high-level tasks. To this end, an off-the-shelf FaceNet model and an autoencoder network are adopted to synthesize two sets of feature embeddings from the Five Celebrity Faces and MNIST datasets, respectively, for experiments and analyses. The empirical evaluations show that the POCS-based clustering algorithm can yield favorable results when compared with other prevailing clustering schemes such as the K-Means and Fuzzy C-Means algorithms in embedding clustering problems.
Rule-Based Syntactic Analysis for Uzbek Language: An Alternative Approach to Overcome Data Scarcity and Enhance Interpretability

This research paper introduces an innovative rule-based syntactic analysis algorithm specifically tailored for the Uzbek language, designed to address and overcome the challenges associated with insufficient data availability commonly faced in machine learning approaches. By leveraging the unique characteristics of Uzbek grammar, the study establishes a comprehensive rule set for effectively parsing sentences in this low-resource language. It covers various aspects including tokenization, construction of dependency trees, followed by rigorous optimization and testing performed on diverse Uzbek texts. Despite the absence of machine learning, the study's relevance is elevated by providing a solution to data scarcity, offering a transparent, interpretable system that ensures faster development, reduced computational resource requirements, and enhanced resilience to noise and errors in data. The paper provides a thorough examination of Uzbek grammar, syntactic features, and a set of parsing rules. It also reviews related works, outlines the proposed algorithm's development, and presents the future potential for Natural Language Processing techniques in low-resource languages like Uzbek.
Developing a Prototype of an Intelligent System for Analyzing Meteorological Data Using IoT Technologies

Currently, the Internet of Things (IoT) technology elements are increasingly being introduced into day-to-day life. This technology is especially applied in forecasting weather conditions not only city- or region-wide, but also in a specific house as part of the “Smart Home” system. Cost management (gas, heating, water, etc.) requires more and more accurate measuring instruments (meters, sensors). In this case, the choice of the analysis method for the received data and intelligent systems for their processing is fundamental. The aim of the study is to develop a prototype of an intelligent system for analyzing meteorological data using IoT technologies. The present paper described the structural scheme of the weather station and its sensors. The algorithm for acquiring and processing data from the weather station sensors has been developed. As a result of processing, data on air temperature, air humidity, and the sum of active temperatures are sent to the intelligent decision-making system.
Regression modeling of censored data based on compound scale mixtures of normal distributions

In the framework of censored regression models, the distribution of the error term can depart significantly from normality, for instance, due to the presence of multimodality, skewness and/or atypical observations. In this paper we propose a novel censored linear regression model where the random errors follow a finite mixture of scale mixtures of normal (SMN) distribution. The SMN is an attractive class of symmetrical heavy-tailed densities that includes the normal, Student-t, slash and the contaminated normal distribution as special cases. This approach allows us to model data with great flexibility, accommodating simultaneously multimodality, heavy tails and skewness depending on the structure of the mixture components. We develop an analytically tractable and efficient EM-type algorithm for iteratively computing the maximum likelihood estimates of the parameters, with standard errors and prediction of the censored values as a by-products. The proposed algorithm has closed-form expressions at the E-step, that rely on formulas for the mean and variance of the truncated SMN distributions. The efficacy of the method is verified through the analysis of simulated and real datasets. The methodology addressed in this paper is implemented in the R package CensMixReg.
Developing xApps for Rogue Base Station Detection in SDR-Enabled O-RAN

In order to support the diverse requirements of 5G communications, a multitude of RAN components are required. To enable multiple vendor support for 5G, each of whom can independently choose components, Open-RAN (O-RAN) defined a set of standards to which the components must adhere. In addition, O-RAN defines the management elements used to manage each component to secure the 5G networks. While the proposed architecture can manage both 4G and 5G environments, including 5G NSA (Non-Standalone), it inherently suffers from the same vulnerabilities found in 4G LTE. Consequently, an attacker can use unprotected signaling and a low-cost Software Defined Radio (SDR) to launch rogue base station (RBS) attacks on the user equipment (UE), even in O-RAN architectures. In this paper, we consider the stability of signals collected from high-quality operational BSs versus cheap RBSs. Using signal stability features, we develop a machine learning (ML) based RBS detector located on the UE. With the aid of an O-RAN xAPP, ML models can be retrained using the data collected from multiple UEs, and the updated model can be delivered to UEs to enable higher detection accuracy. We conduct extensive experiments by implementing an RBS using a USRP B210, enabling O-RAN using E- Release, and data collected from operational BSs. Moreover, the detector is implemented as an Android APP, which realizes the connection to the O-RAN xAPP. The experimental results show that our detector can achieve more than 99% accuracy, precision “recall” and F1 score.