Reviewerly/exHarmony huggingface dataset:
Attention-Augmented Memory Network for Image Multi-Label Classification

The purpose of image multi-label classification is to predict all the object categories presented in an image. Some recent works exploit graph convolution network to capture the correlation between labels. Although promising results have been reported, these methods cannot learn salient object features in the images and ignore the correlation between channel feature maps. In addition, the current researches only learn the feature information within individual input image, but fail to mine the contextual information of various categories from the dataset to enhance the input feature representation. To address these issues, we propose an A ttention- A ugmented M emory N etwork ( AAMN ) model for the image multi-label classification task. Specifically, we first propose a novel categorical memory module to excavate the contextual information of various categories from the dataset to augment the current input feature. Secondly, we design a new channel-relation exploration module to capture the inter-channel relationship of features, so as to enhance the correlation between objects in the images. Thirdly, we develop a spatial-relation enhancement module to model second-order statistics of features and capture long-range dependencies between pixels in feature maps, so as to learn salient object features. Experimental results on standard benchmarks, including MS-COCO 2014, PASCAL VOC 2007, and VG-500, demonstrate the effectiveness and superiority of AAMN model, which outperforms current state-of-the-art methods.
3DVerifier: efficient robustness verification for 3D point cloud models

Abstract 3D point cloud models are widely applied in safety-critical scenes, which delivers an urgent need to obtain more solid proofs to verify the robustness of models. Existing verification method for point cloud model is time-expensive and computationally unattainable on large networks. Additionally, they cannot handle the complete PointNet model with joint alignment network that contains multiplication layers, which effectively boosts the performance of 3D models. This motivates us to design a more efficient and general framework to verify various architectures of point cloud models. The key challenges in verifying the large-scale complete PointNet models are addressed as dealing with the cross-non-linearity operations in the multiplication layers and the high computational complexity of high-dimensional point cloud inputs and added layers. Thus, we propose an efficient verification framework, 3DVerifier, to tackle both challenges by adopting a linear relaxation function to bound the multiplication layer and combining forward and backward propagation to compute the certified bounds of the outputs of the point cloud models. Our comprehensive experiments demonstrate that 3DVerifier outperforms existing verification algorithms for 3D models in terms of both efficiency and accuracy. Notably, our approach achieves an orders-of-magnitude improvement in verification efficiency for the large network, and the obtained certified bounds are also significantly tighter than the state-of-the-art verifiers. We release our tool 3DVerifier via https://github.com/TrustAI/3DVerifier for use by the community.
FeaSel-Net: A Recursive Feature Selection Callback in Neural Networks

Selecting only the relevant subsets from all gathered data has never been as challenging as it is in these times of big data and sensor fusion. Multiple complementary methods have emerged for the observation of similar phenomena; oftentimes, many of these techniques are superimposed in order to make the best possible decisions. A pathologist, for example, uses microscopic and spectroscopic techniques to discriminate between healthy and cancerous tissue. Especially in the field of spectroscopy in medicine, an immense number of frequencies are recorded and appropriately sized datasets are rarely acquired due to the time-intensive measurements and the lack of patients. In order to cope with the curse of dimensionality in machine learning, it is necessary to reduce the overhead from irrelevant or redundant features. In this article, we propose a feature selection callback algorithm (FeaSel-Net) that can be embedded in deep neural networks. It recursively prunes the input nodes after the optimizer in the neural network achieves satisfying results. We demonstrate the performance of the feature selection algorithm on different publicly available datasets and compare it to existing feature selection methods. Our algorithm combines the advantages of neural networks’ nonlinear learning ability and the embedding of the feature selection algorithm into the actual classifier optimization.
Progress on the earthquake early warning and shakemaps system using low-cost sensors in Taiwan

Abstract Building an earthquake early warning (EEW) network requires the installation of seismic instruments around the seismogenic zone. Using low-cost sensors to build a seismic network for EEW and to generate shakemaps is a cost-effective way in the field of seismology. The National Taiwan University (NTU) network employing 762 P-Alert low-cost sensors based on micro-electro-mechanical systems (MEMS) technology is operational for almost the last 10 years in Taiwan. This instrumentation is capable of recording the strong ground motions of up to ± 2 g and is dense enough to record the near-field ground motion. The NTU system has shown its importance during various earthquakes that caused damage in Taiwan. Although the system is capable of acting as a regional as well as an onsite warning system, it is particularly useful for onsite warning. Using real-time seismic signals, each P-Alert device provided a 2–8 s warning time for the near-source earthquake regions situated in the blind zone of the Central Weather Bureau (CWB) regional EEW system, during the 2016 $${{{M}}}_{{{w}}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>M</mml:mi> <mml:mi>w</mml:mi> </mml:msub> </mml:math> 6.4 Meinong and 2018 $${{{M}}}_{{{w}}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>M</mml:mi> <mml:mi>w</mml:mi> </mml:msub> </mml:math> 6.4 Hualien earthquakes. The shakemaps plotted by the P-Alert dense network help to assess the damage pattern and act as key features in the risk mitigation process. These shakemaps are delivered to the intended users, including the disaster mitigation authorities, for possible relief purposes. Currently, the P-Alert network can provide peak ground acceleration (PGA), peak ground velocity (PGV), spectral acceleration ( $${S}_{a}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>S</mml:mi> <mml:mi>a</mml:mi> </mml:msub> </mml:math> ) at different periods, and CWB intensity shakemaps. Using shakemaps, it is found that PGV is a better indicator of damage detection than PGA. Encouraged by the performance of the P-Alert network, more instruments are installed in Asia-Pacific countries.
An energy and leakage current monitoring system for abnormality detection in electrical appliances

Unsafe electrical appliances can be hazardous to humans and can cause electrical fires if not monitored, analyzed, and controlled. The purpose of this study is to monitor the system's condition, including the electrical properties of the appliances, and to diagnose fault conditions without deploying sensors on individual appliances and analyzing individual sensor data. Using historical data and an acceptable range of normal and leakage currents, we proposed a hybrid model based on multiclass support vector machines (MSVM) integrated with a rule-based classifier (RBC) to determine the changes in leakage currents caused by installed devices at a certain moment. For this, we developed a sensor-based monitoring device with long-range communication to store real-time data in a cloud database. In the modeling process, RBC algorithm is used to diagnose the constructed device fault and overcurrent fault where MSVM is applied for detecting leakage current fault. To conduct an operational field test, the developed device was integrated into some houses. The results demonstrate the effectiveness of the proposed system in terms of electrical safety monitoring and detection. All the collected data were stored in a structured database that could be remotely accessed through the Internet.
RabbitFX: Efficient Framework for FASTA/Q File Parsing on Modern Multi-Core Platforms

The continuous growth of generated sequencing data leads to the development of a variety of associated bioinformatics tools. However, many of them are not able to fully exploit the resources of modern multi-core systems since they are bottlenecked by parsing files leading to slow execution times. This motivates the design of an efficient method for parsing sequencing data that can exploit the power of modern hardware, especially for modern CPUs with fast storage devices. We have developed RabbitFX, a fast, efficient, and easy-to-use framework for processing biological sequencing data on modern multi-core platforms. It can efficiently read FASTA and FASTQ files by combining a lightweight parsing method by means of an optimized formatting implementation. Furthermore, we provide user-friendly and modularized C++ APIs that can be easily integrated into applications in order to increase their file parsing speed. As proof-of-concept, we have integrated RabbitFX into three I/O-intensive applications: fastp, Ktrim, and Mash. Our evaluation shows that the inclusion of RabbitFX leads to speedups of at least 11.6 (6.6), 2.4 (2.4), and 3.7 (3.2) compared to the original versions on plain (gzip-compressed) files, respectively. These case studies demonstrate that RabbitFX can be easily integrated into a variety of NGS analysis tools to significantly reduce associated runtimes. It is open source software available at https://github.com/RabbitBio/RabbitFX.
Cybersecurity For Defense Economists

Formulae display:?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.
A surrogate-assisted controller for expensive evolutionary reinforcement learning

The integration of Reinforcement Learning (RL) and Evolutionary Algorithms (EAs) aims at simultaneously exploiting the sample efficiency as well as the diversity and robustness of the two paradigms. Recently, hybrid learning frameworks based on this principle have achieved great success in various challenging robot control tasks. However, in these methods, policies from the genetic population are evaluated via interactions with the real environments, limiting their applicability in computationally expensive problems. In this work, we propose Surrogate-assisted Controller (SC), a novel and efficient module that can be integrated into existing frameworks to alleviate the computational burden of EAs by partially replacing the expensive policy evaluation. The key challenge in applying this module is to prevent the optimization process from being misled by the possible false minima introduced by the surrogate. To address this issue, we present two strategies for SC to control the workflow of hybrid frameworks. Experiments on six continuous control tasks from the OpenAI Gym platform show that SC can not only significantly reduce the cost of fitness evaluations, but also boost the performance of the original hybrid frameworks with collaborative learning and evolutionary processes.
Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations

In this paper, we consider privacy aspects of wireless federated learning (FL) with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. OtA FL enables the users to transmit their updates simultaneously with linear processing techniques, which improves resource efficiency. However, this setting is vulnerable to privacy leakage since an adversary node can hear directly the un-coded message. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy due to the reduced signal-to-noise ratio. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server at the same time. More explicitly, spatially correlated perturbations are added to the gradient vectors at the users before transmission. Using the zero-sum property of the correlated perturbations, the side effect of the added perturbation on the aggregated gradients at the edge server can be minimized. In the meanwhile, the added perturbation will not be canceled out at the adversary, which prevents privacy leakage. Theoretical analysis of the perturbation covariance matrix, differential privacy, and model convergence is provided, based on which an optimization problem is formulated to jointly design the covariance matrix and the power scaling factor to balance between privacy protection and convergence performance. Simulation results validate the correlated perturbation approach can provide strong defense ability while guaranteeing high learning accuracy.
Robot Path Planning Based on Multi-strategy Improved RRT* Algorithm

In order to solve the problems of the commonly used RRT* algorithm, such as the large amount of calculation in the uniform random sampling process and the inconsistent results of multiple planned paths in the same environment, a navigation path planning method for underwater robots based on the multi-strategy improved RRT* algorithm was proposed. Firstly, a bidirectional expanding random tree search strategy is used to speed up the path search process, and then the original planned path is used as a path cache to avoid random sampling. At the same time, aiming at the problem of slow convergence of the basic ant colony algorithm in the search process, an improved ant colony algorithm is proposed. By using a new heuristic function and pheromone update rule, the algorithm convergence speed is accelerated. Finally, the improved RTT* algorithm and the improved ant colony algorithm are combined to form an underwater robot path exploration algorithm. Compared with the basic RRT* algorithm, the ant colony algorithm and the particle swarm algorithm, the experimental results show that the improved algorithm has fewer nodes, faster convergence speed and higher accuracy in the path planning of underwater robots.
To Be, or Not to Be Stateful

While research in post-quantum cryptography (PQC) has gained significant momentum, it is only slowly adopted for real-world products. This is largely due to concerns about practicability and maturity. The secure boot process of embedded devices is one scenario where such restraints can result in fundamental security problems. In this work, we present a flexible hardware/software co-design for hash-based signature (HBS) schemes which enables the move to a post-quantum secure boot today. These signature schemes stand out due to their straightforward security proofs and are on the fast track to standardisation. In contrast to previous works, we exploit the performance intensive similarities of the stateful LMS and XMSS schemes as well as the stateless SPHINCS+ scheme. Thus, we enable designers to use a stateful or stateless scheme depending on the constraints of each individual application. To show the feasibility of our approach, we compare our results with hardware accelerated implementations of classical asymmetric algorithms. Further, we lay out the usage of different HBS schemes during the boot process. We compare different schemes, show the importance of parameter choices, and demonstrate the performance gain with different levels of hardware acceleration.
Evaluating Calibration-free Webcam-based Eye Tracking for Gaze-based User Modeling

Eye tracking has been a research tool for decades, providing insights into interactions, usability, and, more recently, gaze-enabled interfaces. Recent work has utilized consumer-grade and webcam-based eye tracking, but is limited by the need to repeatedly calibrate the tracker, which becomes cumbersome for use outside the lab. To address this limitation, we developed an unsupervised algorithm that maps gaze vectors from a webcam to fixation features used for user modeling, bypassing the need for screen-based gaze coordinates, which require a calibration process. We evaluated our approach using three datasets (N=377) encompassing different UIs (computerized reading, an Intelligent Tutoring System), environments (laboratory or the classroom), and a traditional gaze tracker used for comparison. Our research shows that webcam-based gaze features correlate moderately with eye-tracker-based features and can model user engagement and comprehension as accurately as the latter. We discuss applications for research and gaze-enabled user interfaces for long-term use in the wild.
Influence of Passive Haptic and Auditory Feedback on Presence and Mindfulness in Virtual Reality Environments

Virtual Reality (VR) is increasingly being used to promote mindfulness practice. However, the impact of virtual and multimodal interactive spaces on mindfulness practice and presence is still underexplored. To address this gap, we conducted a mixed-method user study (N=12). We explored the impact of various multimodal feedback, in particular tactile feedback (passive haptic feedback by artificial grass) and auditory feedback (footstep sound) at feet level on both mindfulness and presence. We conducted semi-structured interviews and collected quantitative data through three validated questionnaires (SMS, IPQ, WSPQ). We found a significant effect of passive haptic feedback on presence and mindfulness. Tactile feedback improves the focus on the self, facilitates spatial presence and increases involvement. Auditory feedback did not significantly affect presence or mindfulness. While ambient sound was perceived as beneficial for presence, footstep sound subjectively disrupted both presence and mindfulness. Based on our results, we derive design recommendations for multimodal VR applications supporting mindfulness practice.
Assessing Multimodal Dynamics in Multi-Party Collaborative Interactions with Multi-Level Vector Autoregression

Multi-level vector autoregression (mlVAR) is a recently developed dynamic network model for assessing multimodal temporal data streams derived from multiple users over time. Importantly, mlVAR facilitates investigations into highly complex collaborative interactions within a unified framework. In order to demonstrate the utility of mlVAR for understanding the temporal dynamics of multimodal multi-party (MMP) interactions, we apply it to 9 signals measured from 201 users (67 triads) who engaged in a 15-minute collaborative problem solving task. Measured signals reflect participants' affective states (positive valence and negative valence), physiological states (skin conductance and heart rate), attention (gaze fixation duration and gaze dispersion), nonverbal communication (head acceleration and facial expressiveness), and verbal communication (speech rate). Using node-level metrics of in-strength, out-strength, and synchrony, we show that mlVAR is capable of teasing apart complex role-based dynamics (controller, primary contributor, or secondary contributor) between participants. Our findings also provide evidence for a complex feedback system between individuals where internal states (i.e., skin conductance) are influenced by external signals of shared attention and communication (i.e., gaze and speech).
Pose Uncertainty Aware Movement Synchrony Estimation via Spatial-Temporal Graph Transformer

The concept of movement synchrony is derived from the scientific study of interacting dyads in the autism field. Automated movement synchrony estimation has been achieved by utilizing deep learning models applied to other tasks, such as human activity recognition. To better adapt to the movement synchrony estimation task, we proposed a skeleton-based uncertainty-aware graph transformer incorporating joint confidence scores. We uniquely designed a joint position embedding shared between the same joints of interacting individuals and introduced a temporal similarity matrix in temporal attention computation considering the periodic intrinsic of body movements. To further improve the performance, we constructed a dataset for movement synchrony estimation using Human3.6M and pretrained our model on it via contrastive learning. We further applied knowledge distillation to alleviate information loss introduced by pose detector failure in a privacy-preserving way. Our method achieved an overall accuracy of on PT13, a dataset collected from autism therapy interventions, and surpassed its counterpart approaches by a good margin. This work also has implications for synchronous movement activity recognition in group settings, with broad applications in education and sports.
TransGesture: Autoregressive Gesture Generation with RNN-Transducer

This paper presents a gesture generation model based on an RNN-transducer, submitted to the GENEA Challenge 2022. The proposed model consists of three neural networks: Encoder, Prediction Network, and Joint Network, which can be jointly trained in an end-to-end manner. We also introduce new loss functions, namely statistical losses, as the additional term to the standard MSE loss to put motion statistics of generated gestures close to the ground truths’. Finally, we show the subjective evaluation results and discuss the results and takeaways from the challenge.
Exploratory Study on the Perception of Intelligent Virtual Agents With Non-Native Accents Using Synthetic and Natural Speech in German

This paper presents an exploratory study which investigates the impact of different non-native accents and the naturalness of speech on the correct assignment of an Intelligent Virtual Agent's (IVA) mother tongue, as well as its perceived warmth, competence and intelligibility. An online-experiment with a between subjects design was conducted, in which the participants, who were native speakers of German, watched a video of an IVA that spoke German with a non-native accent. The IVA's speech was either synthetically generated or pre-recorded using non-native speakers. The participants experienced an IVA with either a Turkish, Italian or Polish accent, based on the most frequent accents in the German-speaking area. The results revealed that the IVA's accent impacted its perceived warmth, but not its perceived competence and intelligibility. The IVA's naturalness of speech played no role in its classification as a non-native speaker of German but on the correctness of the assigned mother tongue within the Polish accent condition. These results give valuable insight in the perception of non-native speaking IVAs and constitute helpful implications for future research with mixed-cultural IVAs.
Reference-Guided Texture and Structure Inference for Image Inpainting

Existing learning-based image inpainting methods are still in challenge when facing complex semantic environments and diverse hole patterns. The prior information learned from the large scale training data is still insufficient for these situations. Reference images captured covering the same scenes share similar texture and structure priors with the corrupted images, which offers new prospects for the image inpainting tasks. Inspired by this, we first build a benchmark dataset containing 10K pairs of input and reference images for reference-guided inpainting. Then we adopt an encoder-decoder structure to separately infer the texture and structure features of the input image considering their pattern discrepancy of texture and structure during inpainting. A feature alignment module is further designed to refine these features of the input image with the guidance of a reference image. Both quantitative and qualitative evaluations demonstrate the superiority of our method over the state-of-the-art methods in terms of completing complex holes. Code is available at https://github.com/Cameltr/RGTSI.