Reviewerly/exHarmony huggingface dataset:
Dual-branch counting method for dense crowd based on self-attention mechanism

A dense crowd counting method based on self-attention mechanism with dual-branch fusion network is proposed in this paper. Our method aims to address the problems of large variations in head scales and complex backgrounds in dense crowd images. This method combines the CNN and Transformer network frameworks and consists of shallow feature extraction network, dual-branch fusion network, and deep feature extraction network. The VGG16 network is employed by the shallow feature extraction network to extract low-level features. A multi-scale CNN branch and a Transformer branch built on an improved self-attention module make up the dual-branch fusion network, which collects local and global information on crowd areas, respectively. The Transformer network, which is based on a mixed attention module, is employed by the deep feature extraction network to further separate complicated backgrounds and concentrate on crowd areas. Both counting-level weakly supervised and location-level fully supervised methods are employed in the experiments. On four widely used datasets, the results demonstrate that the proposed method outperforms the most recent research. Our method has a higher counting accuracy with low parameter volumes and a counting accuracy of 89.1% under full supervision when compared to existing weakly supervised methods. The results of the experiments demonstrate that the method has excellent crowd counting performance and can accurately count in high-density and high-occlusion scenes.
Development of an automatic grammar checker for Yorùbá word processing using Government and Binding Theory

Grammar checkers have been developed for well-resourced languages such as French, German and English. However, there is a dearth of automatic grammar checkers for African languages including Standard Yorùbá (SY) and consequently no standard software application. This study developed a software system for Yorùbá grammar checking. Requirements for a Yorùbá grammar checking system were investigated by gathering information on Yorùbá syntax from Textbooks and Journals. Based on the information gathered, a computational model was formulated using the theoretical framework of Government and Binding (GB). The formulated system was designed using the Context Free Grammar Tool in Automata Theory. The system was implemented using the Python programming language version 3.5 and Natural Language Tool kit (NLTK). The system was tested for accuracy using a total of three hundred sentences. In detecting grammatically correct and incorrect sentences, an accuracy of 86.67%, 88.89% and 88.89% were achieved for simple, complex and compound sentences respectively. The grammar checker has been able to function as an automatic grammatical error checker of Yorùbá digital text. Its implementation in software applications has the potential to increase the quality of Yorùbá word processing. Also, the dictionary of words created can be used in other areas of human language processing.
ATFA: Adversarial Time–Frequency Attention network for sensor-based multimodal human activity recognition

Human activity recognition (HAR) is a key component of mobile and ubiquitous computing. With the increasing number of sensors embedded in devices such as smartphones and smartwatches, models are enhanced to recognize more complex human behaviors. However, this also leads to increased data heterogeneity, caused by diverse user backgrounds, health conditions, and sensing environments, which makes it difficult to align the multimodal data distributions across individuals. This paper proposes a novel unsupervised domain adaptation framework via Adversarial Time-Frequency Attention (ATFA) to efficiently adapt models to new users. In particular, the proposed attention-based modality fusion module captures and fuses important modalities based on their context to reduce redundant information. Additionally, the network explores frequency domain features to improve performance in recognizing human activities. Extensive experiments are conducted on three publicly available HAR datasets to demonstrate the superiority of our proposed method compared to state-of-the-art baselines.
Exploring the optimality of byte-wise permutations of a piccolo-type block cipher

Piccolo is a lightweight block cipher based on a 16-bit word 4-line generalized Feistel structure. Piccolo adopts byte-wise round permutation (RP) instead of the typical word-based RP to improve diffusion. In this paper, we explore the optimality of byte-based RP from the viewpoint of security. We evaluate the security of differential, linear, impossible differential, and integral attacks for all byte-wise RPs using mixed integer linear programming (MILP). We show that the RP of Piccolo is optimal in terms of the number of rounds required to guarantee security against such attacks. In addition, we introduce new two classes of RPs that require 7 rounds for security against impossible differential attacks, which is one round less than required by Piccolo. These new classes require 7/9 and 8/8 rounds to guarantee security against differential/linear attacks, respectively, which is more rounds than required by Piccolo.
TIEN: Temporal interest-aware evolution model for “Next Item Recommendation”

The “next item recommendation” is used to predict the next item that a user may be interested in and its main challenges is identifying the user's interests. The current state-of-the-art methods mostly adopt time-series-based recommendation methods but ignore the role of time intervals in the interest evolution process. This motivated us to propose a new Time Interest-aware Evolutionary Network model (TIEN) to predict the next item that a user may be interested in. TIEN highlights the role of time intervals in the interest evolution process and divides interests into long-term interests and short-term interests. In TIEN: (1) We devise a time interval-based attention network TAN to capture the user's interests, which employs item embeddings, behavior sequence embeddings, and time intervals to calculate the user's interest attention; (2) We devise an ATT-GRU network based on candidate activation vectors to capture the interest evolution process, which skillfully leverages the reset gate, hidden layer state, item information, time intervals, etc. to participate in interest evolution; (3) We introduce a dual-network collaborative training mechanism, where the obtained interests of users are treated as weights to guide the interest evolution process. Experimental results on multiple benchmark datasets show that TIEN has higher performance than the current state-of-the-art methods.
Fused robust geometric nonparallel hyperplane support vector machine for pattern classification

Recently, introducing nonconvex loss functions in support vector machine (SVM) to improve the robustness against varies noises has been drawing much attention. In this paper, we first construct a new robust capped asymmetric elastic net (CaEN) loss function. Second, we describe a novel robust Huberized kernel-based (HK) loss function and theoretically demonstrate several important properties, such as smoothness, boundness and the trade-off between the standard least squares and the truncated least squares. Finally, we apply the CaEN loss and the HK loss into elastic net nonparallel hyperplane SVM (ENNHSVM) to develop a fused robust geometric nonparallel SVM (FRGNHSVM). The proposed FRGNHSVM not only inherits the advantages of ENNHSVM but also improves the robustness of classification problems. An efficient Pegasos-based DC (difference of convex functions) algorithm is implemented to solve the FRGNHSVM optimization problem. In comparison with four famous SVMs, including Lagrangian SVM, twin SVM, pinball SVM and C-loss twin SVM, experimental results on simulations and twelve UCI datasets show that the proposed FRGNHSVM can often improve more than 5% average prediction accuracy. Moreover, the performance of the high prediction accuracy of FRGHNSVM is more significant along with the ratio of label noise increasing, indicating its superiority in dealing with label-contaminated datasets.
Context-aware Multi-level Question Embedding Fusion for visual question answering

Question model has been widely concerned as the cornerstone of constructing Visual Question Answering (VQA) models. Existing question models attempt to exploit word context to extract multi-level concepts for modeling multi-level questions. However, they still have many defects. For example, most question models utilize simple fusion methods to fuse shallow modules and extract parameter-unshared low-level concepts, leading to poor modeling of multi-level questions; although some question models use deep bidirectional Transformer encoder in external knowledge transfer and BERT for multi-level questions, their complexity is still high. To solve these issues, we propose a novel low-complex contextual question model, termed Context-aware Multi-level Question Embedding Fusion (CMQEF). We formalize its concepts and theories, deduce its modeling process, optimization process and feature extraction process, analyze its low complexity and high expressiveness, and prove that it defines a new way to solve parameter non-sharing for extracting parameter-shared multi-level concepts and optimize the tradeoff between expressiveness and complexity in question models. Extensive experiments on VQAv2 and VQA-CPv2 validate that comparing with the state-of-the-art, our CMQEF outperforms it on SANs and UpDn, reduces the language priors of SANs and UpDn, and has preferable interpretability and applicability. Our code is available at https://github.com/lsdruc/CMQEF-for-VQA.
AEGraph: Node attribute-enhanced graph encoder method

Graph representation learning faces challenges in node classification tasks due to the intricate interplay between node categories, topological structure, significant characteristics, node attributes, and label information. However, prevailing graph convolutional networks (GCNs) exhibit inefficiencies in information propagation and need more robustness when dealing with missing node attributes. To surmount these issues, attribute-based dynamic wandering and attribute enhancement methods are proposed. First, the Node Attribute Random Wandering (NARW) method is proposed to obtain the node sequences. The NARW method fully considers the node-to-node attribute information to determine the wandering direction of the sequences. Subsequently, the node attribute enhanced graph encoder model (AEGraph) is proposed, which aims to improve the node classification performance by combining the graph structure and node attribute information during the information propagation process. Each iteration process includes node label information aggregation and node label information update. Finally, to fully consider the local and global information of the graph, this paper reduces the noise generated by information propagation during the iteration process. The model proposed in this paper shows promising results with a low labeling rate. Experimental results show that the method achieves better results and significant improvements on different datasets.
Graphical Consensus-Based Sharding for Efficient and Secure Sharings in Blockchain-Enabled Internet of Vehicles

Blockchain-enabled Internet of Vehicles provides a reliable collaboration environment for traffic entities and promotes road safety and traffic efficiency through real-time sharings between vehicles and infrastructure. However, a bottleneck is likely to emerge as the Internet of Vehicles scales up. In this paper, we propose a novel Graphical Consensus-based Sharding (GCS) framework, which is underpinned by four important strategies: 1) A graphical consensus is adopted as its intra-shard consensus, where the consensus group is set up according to its maximal connected subgraph, and the leader is elected by its reliability weight. The consensus group is refreshed intermittently by alternating the leader role. 2) Within GCS, the intra-shard data are stored in the local chain, while a block-based directed acyclic graph, rather than a chained structure, is employed as the main chain. The local chain is used to respond to requests within each shard, and the main chain supports the cross-shard sharings. GCS will parallelly optimize the throughput of the blockchain-enabled Internet of Vehicles. 3) GCS further introduces the shard backup and the node scheduling to handle shard failure and overheating by using new backup strategies and a temporal-spatial graph convolutional network prediction model, respectively. 4) An off-chain transmission algorithm is presented for secure sharing between the infrastructure and the vehicles. Simulation results show that the number of Transactions Per Second is 1.69 times higher than that of the non-sharding blockchain, and the pending time is dramatically reduced compared to the mainstreaming sharding approach, which is 1.02s.
Path Planning for the Gantry Welding Robot System Based on Improved RRT*

In the shipbuilding industry, various workpieces are often produced in small batches. A new program must be written for new workpiece to be processed, which leads to the inefficiency of the traditional teaching programming method. In particular, some manufacturers have applied large gantry structures to robots to improve their handling space. Although the external positioning device enhances the robot's flexibility, it also increases the difficulty of path planning. The RRT* algorithm based on sampling is widely used in the path planning of manipulator for its efficient expansibility and probability completeness. However, in the robot system equipped with gantry structure, the increase of freedom makes its efficiency relatively low. Therefore, this article presents an improved RRT* algorithm for autonomous path planning of welding robots with a large gantry structure. This method introduces the sampling pool mechanism, and selects the node nearest to the connection line between the starting node and the target node in the sampling pool, which effectively shortens the length of the search path. In addition, it adopts the strategy of limiting the nearest node to prevent the transitional search of the configuration space. The improved RRT* algorithm proposed in this paper is verified in complex environment, and compared with improved algorithms such as IB-RRT*, the path cost and time cost are increased by 22.2% and 32.5%, respectively, and the success rate is relatively stable.
Imperceptible UAPs for Automatic Modulation Classification Based on Deep Learning

Automatic Modulation Classification (AMC), which based on deep learning has been extensively researched and implemented in wireless communication systems. Universal adversarial perturbation refers to a single perturbation that can cause most samples to be misclassified by deep learning models. In this paper, we aim to achieve imperceptible universal adversarial attacks on AMC models, and thus an imperceptible universal adversarial perturbations (imperceptible UAPs) framework was proposed. Specifically, loss functions were separately designed for target signals and non-target signals, and then a total loss was calculated. This total loss function can be modified by hyperparameters to achieve class-specific universal adversarial attacks (CUAAs), class-discriminative universal adversarial attacks (CD-UAAs), and class-discriminative target universal adversarial attacks (CD-TUAAs). Meanwhile, wavelet reconstruction was applied to the training data, thus further improving the discriminability of the generated UAPs. After CUAAs were implemented, the class dominant in radio signals was visualized and analyzed by confusion matrices. Furthermore, with the confusion matrices of CUAAs, CD-TUAAs were efficiently implemented. The experiments were conducted on two radio signal datasets and models. In most scenarios, CD-UAAs achieved excellent performance with an average ΔAcc of 58. 20% and CD-TUAAs achieved an average ΔK of 73.78%.
CB-DSL: Communication-Efficient and Byzantine-Robust Distributed Swarm Learning on Non-i.i.d. Data

The valuable data collected by IoT devices together with the resurgence of machine learning (ML) stimulate the latest trend of artificial intelligence (AI) at the edge. However, traditional ML and recent federated learning (FL) face major challenges including communication bottleneck, data heterogeneity, and security concerns in edge IoT. Meanwhile, the swarm nature of IoT systems is overlooked by most existing literature, which calls for new designs of distributed learning algorithms. Inspired by the success of biological intelligence (BI) of gregarious organisms, we propose a novel edge learning approach for swarm IoT, called communication-efficient and Byzantine-robust distributed swarm learning (CB-DSL), through a holistic integration of AI-enabled stochastic gradient descent and BI-enabled particle swarm optimization. To deal with the non-i.i.d. data issues and Byzantine attacks, a small amount of global data samples are introduced in CB-DSL and shared among IoT workers, which alleviates the local data heterogeneity effectively and enables to fully utilize the exploration-exploitation mechanism of swarm intelligence. Our convergence analysis theoretically demonstrates that the CB-DSL is superior to the standard FL with better convergence behavior. We also evaluate the model divergence of CB-DSL by deriving its upper bound, which measures the effectiveness of the introduction of the globally shared dataset.
Spoken Language Processing

From the Publisher: New advances in spoken language processing: theory and practice In-depth coverage of speech processing, speech recognition, speech synthesis, spoken language understanding, and speech interface design Many case studies from state-of-the-art systems, including examples from Microsoft’s advanced research labs Spoken Language Processing draws on the latest advances and techniques from multiple fields: computer science, electrical engineering, acoustics, linguistics, mathematics, psychology, and beyond. Starting with the fundamentals, it presents all this and more: Essential background on speech production and perception, probability and information theory, and pattern recognition Extracting information from the speech signal: useful representations and practical compression solutions Modern speech recognition techniques: hidden Markov models, acoustic and language modeling, improving resistance to environmental noises, search algorithms, and large vocabulary speech recognition Text-to-speech: analyzing documents, pitch and duration controls; trainable synthesis, and more Spoken language understanding: dialog management, spoken language applications, and multimodal interfaces To illustrate the book’s methods, the authors present detailed case studies based on state-of-the-art systems, including Microsoft’s Whisper speech recognizer, Whistler text-to-speech system, Dr. Who dialog system, and the MiPad handheld device. Whether you’re planning, designing, building, or purchasing spoken language technology, this is the state of the art—fromalgorithms through business productivity.
CotepRes-Net: An efficient U-Net based deep learning method of liver segmentation from Computed Tomography images

Automatic liver segmentation from CT images is challenging due to the indistinct boundaries between the liver and surrounding organs in the abdominal cavity CT. To address these limitations, we propose a deep learning model called CotepRes-Net, which is based on the U-Net architecture but with modified skip connection using a convolutional block attention module (CBAM). We introduce a modified inception block and a contextual transformer block (CoT) at the bottleneck to capture richer feature representations and leverage global contextual information to constrain the liver shape at the unclear boundary. To accelerate the network convergence in the training stage, we utilize an Efficient channel attention module (ECA) in the decoder layers and replace the convolution operation in each convolution layer with a residual structure. We evaluate the performance of the proposed model using the publicly available dataset LiTS (MICCAI 2017 Liver Tumor Segmentation Challenge) and show the segmentation results on the dataset Sliver07 (Segmentation of the Liver Competition 2007). The results show that our model achieves a dice similarity coefficient (DSC) of 0.9688, Jaccard index of 0.9422, volumetric overlap error (VOE) of 0.0578, relative absolute volume difference (RAVD) of 0.0039, average symmetric surface distance (ASSD) of 1.094 mm and maximum symmetric surface distance (MSSD) of 16.079 mm for liver segmentation. The experimental results show that the proposed method is comparable to or even outperform the state-of-the-art methods in terms of accuracy and computation cost, which can serve as a valuable tool for image-based liver analysis.
An efficient but effective writer: Diffusion-based semi-autoregressive transformer for automated radiology report generation

It is firmly believed that manually diagnosing radiology images is clinically critical but labour-intensive and error-prone. Therefore, an automatic radiology report generation method is highly desired for alleviating the burden imposed on doctors. However, a typical report contains numerous template descriptions and only a few abnormal sentences. This unbalanced distribution makes the generation of template sentences more likely. Additionally, describing an entire report in a word-by-word manner can lead to significant latency during the inference step. Besides, the existing datasets are limited to conventional pneumonia, making them incomplete and one-sided. This work is concerned with forming a better trade-off between generation performance. One key design is an abnormal semantic diffusion module, which progressively absorbs the semantics of abnormal medical terminology and strengthens the linguistic coherence between local tokens. In detail, the generated report is refined by enhancing the incorporation of informative words with limited occurrence frequencies, which alleviates the monotony of template-based generation. Another design is a length-controllable self-attention decoder, which regulates the input length of the sentences used for target word generation. This framework preserves the autoregressive nature of word generation while also maintaining a controllable range, ensuring the efficiency of report generation. Moreover, a novel XRG-COVID-19 clinical dataset is tailored, which includes X-ray scans and professional diagnostic reports of 8676 patients. The experimental results demonstrate the proposed model achieves a better trade-off between performance and speed than those of carefully designed baselines on both the IU X-ray dataset and the proposed XRG-COVID-19 dataset.
Model-Driven IEP-GNN Framework for MIMO Detection With Bayesian Optimization

In this letter, a model-driven detector called IEP-GNN is proposed for massive multiple-input and multiple-output (MIMO) system. Graph neural network (GNN) and improved moment matching (IMM) are integrated into the expectation propagation (EP) algorithm to improve the accuracy of posterior distribution approximation and leverage the self-correction ability of EP algorithm. Moreover, to acquire the training experiences and optimize initial parameters, hotbooting and Bayesian parameter optimization (BPO) are employed respectively, which can further improve the performance of the proposed IEP-GNN. Simulation results show that our proposed IEP-GNN with BPO outperforms other state-of-the-art EP-based detectors while maintaining an acceptable convergence and computational complexity.