Reviewerly/exHarmony huggingface dataset:
Self-attention deep ConvLSTM with sparse-learned channel dependencies for wearable sensor-based human activity recognition

In this study, we propose a novel deep-learning architecture with sparse learning for human activity recognition. The proposed model contains 1D CNNs and LSTM layers with a self-attention mechanism to enhance a substantial number of time points in time-series data for human activity recognition systems. Based on the recent success of squeeze-and-excite (SE) networks, the proposed deep learning model utilizes the SE module to enhance channel-wise interdependencies, which in turn leads to a boost in performance. In addition, we utilized sparse learning to retrain only weak nodes and freeze stronger nodes in a fully connected layer prior to classification layer. Furthermore, we utilized an entropy-inspired formula to find sparsely located weaker nodes and validated our model on various datasets, including Opportunity, UCI-HAR, and WISDM. Herein, we present an extensive analysis and survey of state-of-the-art studies, in addition to our proposed research. For a fair comparison, we evaluated our deep learning architecture using various performance metrics and achieved better results; the proposed model outperformed state-of-the-art algorithms for human activity recognition.
Parallel private information retrieval protocol with index anonymity for untrusted databases

Private information retrieval (PIR) enables a client to search data from a single (or multiple) untrusted server, without revealing which entry was queried. PIR can be divided into two categories: information-theoretic PIR (IT-PIR) and computational PIR (CPIR). However, there is a deployment challenge with IT-PIR because the non-collusion assumption is difficult to implement in practice. Meanwhile, due to the fact that CPIR involves performing cryptographic operations on each element, its performance significantly decreases as the size and number of database entries increase. To overcome these problems, based on homomorphic encryption, this paper presents a parallel PIR (ParPIR) with index anonymity for untrusted databases, which uses the batch encoding to compress request size. Furthermore, our ParPIR eliminates computationally expensive ciphertext multiplication operations, which improves the response time. Moreover, our ParPIR packs multiple responses through rotation column operations on ciphertext, thereby reducing the response size. Compared to previous PIR protocols, the offline time in our ParPIR has almost 1∼63× improvement; the online time in our ParPIR decreases by at least 49.9∼98.9%. Meanwhile, although the request size of our ParPIR has a 0∼31× increase, the response size of our ParPIR has a 0∼4095× improvement.
Subfield subcodes of projective Reed-Muller codes

Explicit bases for the subfield subcodes of projective Reed-Muller codes over the projective plane and their duals are obtained. In particular, we provide a formula for the dimension of these codes. For the general case over the projective space, we generalize the necessary tools to deal with this case as well: we obtain a universal Gröbner basis for the vanishing ideal of the set of standard representatives of the projective space and we show how to reduce any monomial with respect to this Gröbner basis. With respect to the parameters of these codes, by considering subfield subcodes of projective Reed-Muller codes we obtain long linear codes with good parameters over a small finite field.
Multi-objective reinforcement learning for autonomous drone navigation in urban areas with wind zones

Drones can be used for tasks such as data collection and logistics in civil engineering. Current research on autonomous drones mainly focuses on planning a safe path and avoiding obstacles in a static environment. However, navigating a drone in complex environments like urban areas involves many dynamic constraints, such as building layout, winds, and signal coverage, which are interdependent. The wind factor is the most important among these environmental factors, which may cause a drone to lose control or even crash. This paper presents a multi-objective navigation reinforcement learning algorithm (MONRL) for the drone to navigate and avoid obstacles in an unknown environment when dynamic wind zones present, with only imagery data about the building layouts. Based on a deep reinforcement learning and memory architecture, the drone develops policies to prioritize navigation decisions, optimizing the path while minimizing negative impact of winds with only sparse sensor data, in our case, camera inputs. By leveraging the advantages of the proposed method in estimating the environmental factors from previous trials, no aerodynamic force sensors are needed for the drone to develop effective strategies to navigate to target while counteracting to milder winds and dodging aways from stronger winds. The proposed method was tested in a virtual environment, and a real model of New York City. The method is expected to contribute to new automation algorithms for urban aerial logistics and future automated civil infrastructure inspection.
Personalized Federated Learning with Layer-Wise Feature Transformation via Meta-Learning

Federated learning enables multiple clients to collaboratively learn machine learning models in a privacy-preserving manner. However, in real-world scenarios, a key challenge encountered in federated learning is the statistical heterogeneity among clients. Existing work mainly focused on a single global model shared across the clients, making it hard to generalize well to all clients due to the large discrepancy in the data distributions. To address this challenge, we propose pFedLT , a novel approach that can adapt the single global model to different data distributions. Specifically, we propose to perform a pluggable layer-wise transformation during the local update phase based on scaling and shifting operations. In particular, these operations are learned with a meta-learning strategy. By doing so, pFedLT can capture the diversity of data distribution among clients, therefore, can generalize well even when the data distributions among clients exhibit high statistical heterogeneity. We conduct extensive experiments on synthetic and real-world datasets (MNIST, Fashion_MNIST, CIFAR-10, and Office+Caltech10) under different Non-IID settings. Experimental results demonstrate that pFedLT significantly improves the model accuracy by up to 11.67% and reduces the communication costs compared with state-of-the-art approaches.
A Novel Neural Ensemble Architecture for On-the-fly Classification of Evolving Text Streams

We study on-the-fly classification of evolving text streams in which the relation between the input data and target labels changes over time—i.e., “concept drift.” These variations decrease the model’s performance, as predictions become less accurate over time and they necessitate a more adaptable system. While most studies focus on concept drift detection and handling with ensemble approaches, the application of neural models in this area is relatively less studied. We introduce Adaptive Neural Ensemble Network ( AdaNEN ), a novel ensemble-based neural approach, capable of handling concept drift in data streams. With our novel architecture, we address some of the problems neural models face when exploited for online adaptive learning environments. Most current studies address concept drift detection and handling in numerical streams, and the evolving text stream classification remains relatively unexplored. We hypothesize that the lack of public and large-scale experimental data could be one reason. To this end, we propose a method based on an existing approach for generating evolving text streams by introducing various types of concept drifts to real-world text datasets. We provide an extensive evaluation of our proposed approach using 12 state-of-the-art baselines and 13 datasets. We first evaluate concept drift handling capability of AdaNEN and the baseline models on evolving numerical streams; this aims to demonstrate the concept drift handling capabilities of our method on a general spectrum and motivate its use in evolving text streams. The models are then evaluated in evolving text stream classification. Our experimental results show that AdaNEN consistently outperforms the existing approaches in terms of predictive performance with conservative efficiency.
Leveraging Dual Gloss Encoders in Chinese Biomedical Entity Linking

Entity linking is the task of assigning a unique identity to named entities mentioned in a text, a sort of word sense disambiguation that focuses on automatically determining a pre-defined sense for a target entity to be disambiguated. This study proposes the DGE (Dual Gloss Encoders) model for Chinese entity linking in the biomedical domain. We separately model a dual encoder architecture, comprising a context-aware gloss encoder and a lexical gloss encoder, for contextualized embedding representations. DGE are then jointly optimized to assign the nearest gloss with the highest score for target entity disambiguation. The experimental datasets consist of a total of 10,218 sentences that were manually annotated with glosses defined in the BabelNet 5.0 across 40 distinct biomedical entities. Experimental results show that the DGE model achieved an F1-score of 97.81, outperforming other existing methods. A series of model analyses indicate that the proposed approach is effective for Chinese biomedical entity linking.
Improving the Detection of Multilingual South African Abusive Language via Skip-gram Using Joint Multilevel Domain Adaptation

The distinctiveness and sparsity of low-resource multilingual South African abusive language necessitate the development of a novel solution to automatically detect different classes of abusive language instances using machine learning. Skip-gram has been used to address sparsity in machine learning classification problems but is inadequate in detecting South African abusive language due to the considerable amount of rare features and class imbalance. Joint Domain Adaptation has been used to enlarge features of a low-resource target domain for improved classification outcomes by jointly learning from the target domain and large-resource source domain. This article, therefore, builds a Skip-gram model based on Joint Domain Adaptation to improve the detection of multilingual South African abusive language. Contrary to the existing Joint Domain Adaptation approaches, a Joint Multilevel Domain Adaptation model involving adaptation of monolingual source domain instances and multilingual target domain instances with high frequency of rare features was executed at the first level and adaptation of target-domain features and first-level features at the next level. Both surface-level and embedding word features were used to evaluate the proposed model. In the evaluation of surface-level features, the Joint Multilevel Domain Adaptation model outperformed the state-of-the-art models with accuracy of 0.92 and F1-score of 0.68. In the evaluation of embedding features, the proposed model outperformed the state-of-the-art models with accuracy of 0.88 and F1-score of 0.64. The Joint Multilevel Domain Adaptation model significantly improved the average information gain of the rare features in different language categories and reduced class imbalance.
Optimal Hadamard Gate Count for Clifford+ <i>T</i> Synthesis of Pauli Rotations Sequences

The Clifford+ T gate set is commonly used to perform universal quantum computation. In such setup the T gate is typically much more expensive to implement in a fault-tolerant way than Clifford gates. To improve the feasibility of fault-tolerant quantum computing it is then crucial to minimize the number of T gates. Many algorithms, yielding effective results, have been designed to address this problem. It has been demonstrated that performing a pre-processing step consisting of reducing the number of Hadamard gates in the circuit can help to exploit the full potential of these algorithms and thereby lead to a substantial T -count reduction. Moreover, minimizing the number of Hadamard gates also restrains the number of additional qubits and operations resulting from the gadgetization of Hadamard gates, a procedure used by some compilers to further reduce the number of T gates. In this work we tackle the Hadamard gate reduction problem, and propose an algorithm for synthesizing a sequence of π /4 Pauli rotations with a minimal number of Hadamard gates. Based on this result, we present an algorithm which optimally minimizes the number of Hadamard gates lying between the first and the last T gate of the circuit.
NNC-GCN: Neighbours-to-Neighbours Contrastive Graph Convolutional Network for Semi-Supervised Classification

Contrastive learning (CL) is a popular learning paradigm in deep learning, which uses contrastive principle to learn low-dimensional embeddings, and has been applied in Graph Neural Networks (GNNs) successfully. Existing works of contrastive multi-view GNNs usually focus on point-to-point contrastive learning strategies. However, they neglect the local information in neighbors, which brings isolated positive samples. The quality of selected positive samples is hard to evaluate, and these samples may lead to invalid contrastiveness. Therefore, we propose a simple and efficient neighbors-to-neighbors contrastive graph neural network (NNC-GCN), which constructs a consistent multi-view by using the topologies of original input graphs. Moreover, we raise a new learning problem of unlabeled data base on these constructed multi-view topologies and propose a loss function NNC-InfoNCE to guide its learning process. The NNC-InfoNCE is an improved version of InfoNCE, which can be adapted to neighborhood-level contrast learning. Specifically, the neighborhoods and the remaining nodes of the selected anchor are weighted and treated as positive and negative sample sets. The experimental results show that our method is effective on public benchmark datasets.
Ibn-Ginni: An Improved Morphological Analyzer for Arabic

Arabic is a morphologically rich language, which means that the Arabic language has a complicated system of word formation and structure. The affixes in the Arabic language (i.e., prefixes and suffixes) can be added to root words to generate different meanings and grammatical functions. These affixes can indicate aspects such as tense, gender, number, case, person, and more. In addition, the meaning and function of words can be modified in Arabic using an internal structure known as morphological patterns. Computational morphological analyzers of Arabic are vital to developing Arabic language processing toolkits. In this article, we introduce a new morphological analyzer (Ibn-Ginni) that inherits the speed and quality of the Buckwalter Arabic Morphological Analyzer (BAMA). The BAMA has poor coverage of the classical Arabic language. Hence, the coverage of classical Arabic is improved by using the Alkhalil analyzer. Although it is slow, it was used to generate a huge number of solutions for 3 million unique Arabic words collected from different resources. These word form-based solutions were converted to stem-based solutions, refined manually, and added to the database of BAMA, resulting in substantial improvements in the quality of the analysis. Hence, Ibn-Ginni is a hybrid system between BAMA and Alkhalil analyzers and may be considered an efficient large-scale analyzer. The Ibn-Ginni analyzer analyzed 0.6 million more words than the BAMA analyzer. Therefore, our analyzer significantly improves the coverage of the Arabic language. Besides, the Ibn-Ginni analyzer is high speed at providing solutions, the average time to analyze a word is 0.3 ms. Using a corpus designed for benchmarking Arabic morphological analyzers, our analyzer was able to find all solutions for 72.72% of the words. Moreover, the analyzer did not provide all possible morphological solutions for 24.24% of the words. The analyzer and its morphological database are publicly available on GitHub. 1
Bridging Formal Methods and Machine Learning with Model Checking and Global Optimisation

Formal methods and machine learning are two research fields with drastically different foundations and philosophies. Formal methods utilise mathematically rigorous techniques for software and hardware systems' specification, development and verification. Machine learning focuses on pragmatic approaches to gradually improve a parameterised model by observing a training data set. While historically, the two fields lack communication, this trend has changed in the past few years with an outburst of research interest in the robustness verification of neural networks. This paper will briefly review these works, and focus on the urgent need for broader and more in-depth communication between the two fields, with the ultimate goal of developing learning-enabled systems with excellent performance and acceptable safety and security. We present a specification language, MLS2, and show that it can express a set of known safety and security properties, including generalisation, uncertainty, robustness, data poisoning, backdoor, model stealing, membership inference, model inversion, interpretability, and fairness. To verify MLS2 properties, we promote the global optimisation-based methods, which have provable guarantees on the convergence to the optimal solution. Many of them have theoretical bounds on the gap between current solutions and the optimal solution.
CLUE: Contrastive language-guided learning for referring video object segmentation

Referring video object segmentation (R-VOS), the task of separating the object described by a natural language query from the video frames, has become increasingly critical with recent advances in multi-modal understanding. Existing approaches are mainly visual-dominant in both representation-learning and decision-making process, and are less sensitive to fine-grained clues in text description. To address this, we propose a language-guided contrastive learning and data augmentation framework to enhance the model sensitivity to the fine-grained textual clues (i.e., color, location, subject) in the text that relate heavily to the video information. By substituting key information of the original sentences and paraphrasing them with a text-based generation model, our approach conducts contrastive learning through automatically building diverse and fluent contrastive samples. We further enhance the multi-modal alignment with a sparse attention mechanism, which can find the most relevant video information by optimal transport. Experiments on a large-scale R-VOS benchmark show that our method significantly improves strong Transformer-based baselines, and further analysis demonstrates the better ability of our model in identifying textual semantics.
Multi-supervision transformer combining bounding box and mask for data-limited pose estimation

In scenarios where acquiring accurate annotated pose data is difficult, obtaining sufficient training samples becomes a challenge. Given these data constraints, it is challenging to provide optimal end-to-end supervision with limited pose information. To address this limitation, we propose a novel approach, named MSPose, which combines diverse data sources from human instances to provide multi-level supervision in human pose estimation. This approach mitigates the impact of scarce training data and enhances pose estimation precision by synergistically incorporating human bounding box and mask information. Our proposed method achieves multi-level model activation for the human body region, structure, and joints. This hierarchical pixel activation is facilitated by integrating human bounding box and mask signals as supplementary supervision during training. Importantly, we exclude auxiliary branches to ensure complete freedom during inference. Our experimental validation on a quarter of the COCO and MPII datasets demonstrates the competitiveness of our method compared to state-of-the-art methods. Compared to TokenPose-T, our MSPose-T achieves a 6.1 points improvement in average precision and a corresponding 3.1 points increase on the full COCO validation set. On the MPII dataset, MSPose achieves the highest Mean score of 1.9 points. Our approach achieves these improvements while maintaining consistent parameters and GFLOPs, highlighting the advantage of our approach.
FAUC-S: Deep AUC maximization by focusing on hard samples

Deep AUC maximization (DAM) is a popular method to deal with complex imbalanced classification problems. It learns a deep neural network by minimizing a surrogate AUC loss. The mostly used surrogates are the pairwise square loss and its variants. However, they are usually used in regression problems and are sensitive to noise samples. The easy samples suffer larger losses than those without correct classification, which is unreasonable in optimization problems. In this paper, we propose a focus AUC loss based on samples (FAUC-S) by constructing a differentiable weight function that identifies hard and easy samples, which makes easy samples have small losses, and hard samples have large losses. It is more reasonable than the traditional AUC loss while having the same advantages on large-scale datasets. In experiments, we use an end-to-end compositional DAM to investigate the effect of different weight functions on the AUC loss and compare it with other methods. Experimental results on several benchmark datasets demonstrate that FAUC-S achieves superior performance in terms of AUC compared to existing state-of-the-art methods.
Three-way evidence theory-based density peak clustering with the principle of justifiable granularity

Clustering by fast search and find of density peaks (DPC) is an effective clustering approach that can find all the cluster centers at once with just one parameter and without iterative processing. However, the cutoff distance, a key parameter of density measurement in the DPC approach, affects the quality of the final clustering results. Its selection relies on experimental experience and lacks of a semantic explanation. Furthermore, the allocation strategy of the traditional DPC approach may cause several points to be assigned incorrectly, leading to subsequent points being assigned incorrectly and ultimately forming continuous allocation errors. To overcome the deficiencies, this paper proposes a novel three-way evidence theory-based density peak clustering with the principle of justifiable granularity (3 W-PEDP). First, the computation of the cutoff distance is converted into the search for nearest neighbors. From the perspective of granular computing, 3 W-PEDP transforms the neighbor selection issue into the construction of justifiable granularity. And the optimal neighbors can be achieved with the construction of coverage and specificity criteria. Second, inspired by three-way clustering, we adopt a two-stage method for sample allocation. On the one hand, for core point allocation, a two-layer nearest neighbor is constructed based on the achieved optimal neighbors. On the other hand, we designed a new evidence mass function to guide us in assigning the remaining points. In this novel evidence mass function, not only the labels of the assigned samples are considered, but also the information of the neighborhoods around the unassigned samples is fused. Finally, we assess the effectiveness of 3 W-PEDP on numerous public synthetic datasets and UCI real-world datasets. Then, detail comparing results with several popular clustering methods are presented. In addition, experimental studies verify the effectiveness of constructing justifiable granularity in selecting the optimal neighbors. The experimental results demonstrate 3 W-PEDP has good adaptability and robustness, which can achieve better clustering performance. Our source code is available at https://github.com/Luyangabc/3W-PEDP.