Reviewerly/exHarmony huggingface dataset:
Recursive distributed fusion estimation for multi-sensor systems with missing measurements, multiple random transmission delays and packet losses

This paper is concerned with the recursive distributed fusion estimation problems for networked multi-sensor systems with missing measurements, multiple random transmission delays, and packet losses. There exist missing measurements in the observation equations due to unpredictable sensor faults. Moreover, there are often random delays and losses during data transmissions from sensors to the fusion center due to limited communication bandwidths of the network. The Kalman-like recursive distributed fusion predictor and filter in the linear unbiased minimum variance (LUMV) sense are, respectively presented based on local estimates, cross-covariance matrices between local estimates, and cross-covariance matrices between the prior fusion estimate and local estimates. The stability and steady-state property of the proposed algorithms are analyzed. Their estimation accuracy is better than that of local estimates and distributed fusion estimates by matrix-weighting local estimates. A simulation example shows the effectiveness of the proposed algorithms.
Contrastive graph neural network-based camouflaged fraud detector

Recently GNN-based graph machine learning methods have been widely used in fraud detection. Prior works have revealed the problems of camouflaged fraudsters on graphs, which dilute the suspicion of aggregated features of fraudsters. The fraudsters’ camouflage lies in two main aspects: 1) fraudsters adjust their attributes to camouflage original hand-craft features against detection. 2) fraudsters connect to benign entities, making the labels of their contexts inconsistent. However, previous neighbor sampling methods based on similarity measures are inadequate in calculating neighbor similarity, resulting in a loss of GNN-based detector performance. In this paper, we enhance the node similarity measure-based neighbor sampling method and the multi-relational aggregator. First, we use supervised contrastive learning to aggregate discriminative node embeddings. After that, the consistent neighbors are sampled according to the contrastive embedding similarity as well as the node feature similarity. Finally, we improve the multi-relational aggregator via contrastive embeddings. Experiments on two real-world datasets demonstrate that our model CACO-GNN achieves the highest scores in F1-macro, AUC, and Recall, with an 18.5% improvement in F1-macro performance, enabling more effective detection of camouflaged fraudsters.
BERT and ELMo in Task of Classifying Social Media Users Posts

This paper considers the problem of automating the previously presented scheme of classification of users’ posts in social media. Different variations of the embeddings obtained from the RuBERT and ELMo language models are used as input features of the classification. Theoretical significance lies in the adaptation of existing language models BERT and ELMo in order to improve the accuracy of solving the applied problem of posts classification. The practical significance is determined by further automation of the classification of users’ text posts, which will form the basis of the system for assessing the expression of their personality traits and, indirectly, vulnerabilities to social engineering attack.
Repeated Knowledge Distillation with Confidence Masking to Mitigate Membership Inference Attacks

Machine learning models are often trained on sensitive data, such as medical records or bank transactions, posing high privacy risks. In fact, membership inference attacks can use the model parameters or predictions to determine whether a given data point was part of the training set. One of the most promising mitigations in literature is Knowledge Distillation (KD). This mitigation consists of first training a teacher model on the sensitive private dataset, and then transferring the teacher knowledge to a student model, by the mean of a surrogate dataset. The student model is then deployed in place of the teacher model. Unfortunately, KD on its own does not provide users much flexibility, meant as the possibility to arbitrarily decide how much utility to sacrifice to get membership-privacy. To address this problem, we propose a novel approach that combines KD with confidence score masking. Concretely, we repeat the distillation procedure multiple times in series and, during each distillation, perturb the teacher predictions using confidence masking techniques. We show that our solution provides more flexibility than standard KD, as it allows users to tune the number of distillation rounds and the strength of the masking function. We implement our approach in a tool, RepKD, and assess our mitigation against white- and black-box attacks on multiple models and datasets. Even when the surrogate dataset is different from the private one (which we believe to be a more realistic setting than is commonly found in literature), our mitigation is able to make the black-box attack completely ineffective and significantly reduce the accuracy of the white-box attack at the cost of only 0.6% test accuracy loss.
The Affiliative Use of Emoji and Hashtags in the Black Lives Matter Movement in Twitter

Protests and counter-protests seek to draw and direct attention and concern with confronting images and slogans. In recent years, as protests and counter-protests have partially migrated to the digital space, such images and slogans have also gone online. Two main ways in which these images and slogans are translated to the online space is through the use of emoji and hashtags. Despite sustained academic interest in online protests, hashtag activism, and the use of emoji across social media platforms, little is known about the specific functional role that emoji and hashtags play in online social movements. In an effort to fill this gap, the current paper studies both hashtags and emoji in the context of the Twitter discourse around the Black Lives Matter movement.
End-to-end three-dimensional designing of complex disordered materials from limited data using machine learning

Precise 3D representation of complex materials, here the lithium-ion batteries, is a critical step toward designing optimized energy storage systems. One requires obtaining several such samples for a more accurate evaluation of uncertainty and variability, which in turn can be costly and time demanding. Using 3D models is crucial when it comes to evaluating the transport and heat capacity of batteries. Further, such models represent the microstructures more precisely where connectivity and heterogeneity can be detected. However, 3D images are hard to access, and the available images are often collected in two dimensions (2D). Such 2D images, on the other hand, are more accessible and often have higher resolution. In this paper, a deep learning method has been applied to take advantage of 2D images and build 3D models of heterogeneous materials through which more accurate characterization and physical evaluations can be achieved. While being trained using only 2D images, the proposed framework can be utilized to generate 3D images. The proposed method is applied to a few realistic 3D images of lithium-ion battery electrodes. The results indicate that the implemented method can reproduce important structural properties while the flow and heat properties are within an acceptable range.
DSD-MatchingNet: Deformable sparse-to-dense feature matching for learning accurate correspondences

Exploring the correspondences across multi-view images is the basis of many computer vision tasks. However, most existing methods are limited on accuracy under challenging conditions. In order to learn more robust and accurate correspondences, we propose the DSD-MatchingNet for local feature matching in this paper. First, we develop a deformable feature extraction module to obtain multi-level feature maps, which harvests contextual information from dynamic receptive fields. The dynamic receptive fields provided by deformable convolution network ensures our method to obtain dense and robust correspondences. Second, we utilize the sparse-to-dense matching with the symmetry of correspondence to implement accurate pixel-level matching, which enables our method to produce more accurate correspondences. Experiments have shown that our proposed DSD-MatchingNet achieves a better performance on image matching benchmark, as well as on visual localization benchmark. Specifically, our method achieves 91.3% mean matching accuracy on HPatches dataset and 99.3% visual localization recalls on Aachen Day-Night dataset.
Galactic swarm optimization with deep transfer learning driven colorectal cancer classification for image guided intervention

• Develop an optimal deep learning based colorectal cancer classifier model. • Present Adam optimizer with MobileNet-based feature extractor. • Employ galactic swarm optimization with LSTM model for classification. • Validate colorectal histopathological images from the Warwick-QU database. In this era of “precision” medicine, image-guided intervention (IGI) enables real-time customized and accurate treatment using imaging phenotype-based approaches. Colorectal cancer (CC) is the third most commonly occurring cancer, resulting in nearly 10% of cases over the globe. Colorectal cancer classification (CCC) of histopathological images by artificial intelligence (AI) approaches not only enhances the accuracy and classifier results but also allows physicians to make prompt decisions. In this view, this article introduces a novel Galactic Swarm Optimization with Deep Transfer Learning Driven Colorectal Cancer Classification (GSODTL-C3M) model for IGI. The primary aim of the GSODTL-C3M model is to appropriately categorize the test images into the existence of CC. To accomplish this, the presented GSODTL-C3M model employs image pre-processing using the bilateral filtering (BF) technique to remove noise. Besides, Adam optimizer with the MobileNet model is applied as a feature extractor. Finally, the GSO methodology with long short-term memory (LSTM) is employed to recognize and classify CC. An extensive range of simulations was taken place, and the results stated the advanced performance of the current state of art classification methodologies.
CATNet: Cross-event attention-based time-aware network for medical event prediction

Medical event prediction (MEP) is a fundamental task in the healthcare domain, which needs to predict medical events, including medications, diagnosis codes, laboratory tests, procedures, outcomes, and so on, according to historical medical records of patients. Many researchers have tried to build MEP models to overcome the challenges caused by the heterogeneous and irregular temporal characteristics of EHR data. However, most of them consider the heterogenous and temporal medical events separately and ignore the correlations among different types of medical events, especially relations between heterogeneous historical medical events and target medical events. In this paper, we propose a novel neural network based on attention mechanism called Cross-event Attention-based Time-aware Network (CATNet) for MEP. It is a time-aware, event-aware and task-adaptive method with the following advantages: 1) modeling heterogeneous information and temporal information in a unified way and considering irregular temporal characteristics locally and globally respectively, 2) taking full advantage of correlations among different types of events via cross-event attention. Experiments on two public datasets (MIMIC-III and eICU) show CATNet outperforms other state-of-the-art methods on various MEP tasks. The source code of CATNet is released at https://github.com/sherry6247/CATNet.git.
Socially Interactive Agent Dialogue

chapter Share on Socially Interactive Agent Dialogue Author: David Traum View Profile Authors Info & Claims The Handbook on Socially Interactive Agents: 20 years of Research on Embodied Conversational Agents, Intelligent Virtual Agents, and Social Robotics Volume 2: Interactivity, Platforms, ApplicationOctober 2022 Pages 45–76https://doi.org/10.1145/3563659.3563663Published:03 November 2022Publication History 5citation11DownloadsMetricsTotal Citations5Total Downloads11Last 12 Months11Last 6 weeks6 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
WMNN: Wearables-Based Multi-Column Neural Network for Human Activity Recognition

In recent years, human activity recognition (HAR) technologies in e-health have triggered broad interest. In literature, mainstream works focus on the body's spatial information (i.e. postures) which lacks the interpretation of key bioinformatics associated with movements, limiting the use in applications requiring comprehensively evaluating motion tasks' correctness. To address the issue, in this article, a Wearables-based Multi-column Neural Network (WMNN) for HAR based on multi-sensor fusion and deep learning is presented. Here, the Tai Chi Eight Methods were utilized as an example as in which both postures and muscle activity strengths are significant. The research work was validated by recruiting 14 subjects in total, and we experimentally show 96.9% and 92.5% accuracy for training and testing, for a total of 144 postures and corresponding muscle activities. The method is then provided with a human-machine interface (HMI), which returns users with motion suggestions (i.e. postures and muscle strength). The report demonstrates that the proposed HAR technique can enhance users' self-training efficiency, potentially promoting the development of the HAR area.
Enhancing Part Features via Contrastive Attention Module for Vehicle Re-identification

Vehicle re-identification’s methods usually exploit the spatial uniform partition strategy via dividing deep feature maps into several parts. Then each of them is further independently processed by the multi-network branch to obtain refined part features. However, the cooperation among those part features is underestimated. This paper proposes a contrastive attention module (CAM) to assess one part feature’s importance based on all parts. Practical cooperation among part features is derived by re-weighting the part feature. Furthermore, a flexible CAM network (CAMNet) compatible with contrastive attention module is proposed to enhance part features for vehicle re-identification. Extensive experiments show that the proposed CAMNet method outperforms many state-of-the-art vehicle re-identification approaches.
Adversarial Label-Poisoning Attacks and Defense for General Multi-Class Models Based on Synthetic Reduced Nearest Neighbor

Machine learning models are vulnerable to data poisoning attacks whose purpose is to undermine the model’s integrity. However, the current literature on data poisoning attacks mainly focuses on ad hoc techniques that are generally limited to either binary classifiers or to gradient-based algorithms. To address these limitations, we propose a novel model-free label-flipping attack based on the multi-modality of the data, in which the adversary targets the clusters of classes while constrained by a label-flipping budget. The complexity of our proposed attack algorithm is linear in time over the size of the dataset. Also, the proposed attack can increase the error up to two times for the same attack budget. Second, a novel defense technique is proposed based on the Synthetic Reduced Nearest Neighbor model. The defense technique can detect and exclude flipped samples on the fly during the training procedure. Our empirical analysis demonstrates that (i) the proposed attack technique can deteriorate the accuracy of several models drastically, and (ii) under the proposed attack, the proposed defense technique significantly outperforms other conventional machine learning models in recovering the accuracy of the targeted model.
RadioML Meets FINN: Enabling Future RF Applications With FPGA Streaming Architectures

Deep neural networks (DNNs) are penetrating into a broad spectrum of applications and replacing manual algorithmic implementations, including the radio frequency communications domain with classical signal processing algorithms. However, the high throughput (gigasamples per second) and low latency requirements of this application domain pose a significant hurdle for adopting computationally demanding DNNs. In this article, we explore highly specialized DNN inference accelerator approaches on field-programmable gate arrays (FPGAs) for RadioML modulation classification. Using an automated end-to-end flow for the generation of the FPGA solution, we can easily explore a spectrum of solutions that optimize for different design targets, including accuracy, power efficiency, resources, throughput, and latency. By leveraging reduced precision arithmetic and customized streaming dataflow, we demonstrate a solution that meets the application requirements and outperforms alternative FPGA efforts by 3.5× in terms of throughput. Against modern embedded graphics processing units (GPUs), we measure $>\mathsf{10}\times$>10× higher throughput and $>\mathsf{100}\times$>100× lower latency under comparable accuracy and power envelopes.
Motion Sickness Detection for Intelligent Vehicles: A Wearable-Device-Based Approach

Motion sickness is known to be an important and common factor affecting passengers' ride experience. With the popularization of intelligent vehicles, the development of multi-modal interaction methods will provide a chance to solve this problem. Therefore, the detection and mitigation of motion sickness will be an important topic for the future development of intelligent vehicles. However, detecting motion sickness using physiological and subjective data obtained in an on-road driving scenario has rarely been studied in related research. This paper proposed a solution framework for the physiological-signal-based motion sickness detection in on-road driving scenarios. Then, according to the framework, an on-road driving experiment is conducted to gather real-time physiological data using wearable devices. 12 participants took part in these experiments and about 120 min of motion sickness data were generated for analysis. The feature extraction was performed to analyze and extract the important physiological features related to motion sickness in real-world scenarios. Then the features were split into several combination groups and the representative machine learning models were trained to compare the results when different combinations were input. These detection models provide reasonable and effective detection when using the limited kind of physiological data. This method proposed in this paper will benefit the application of the motion sickness detection model in real vehicles. Meanwhile, since the visual input signals are not included, the solution is privacy-protected. In the future, with this solution, vehicles will also be able to detect motion sickness levels in real-time through low-cost daily wearable devices, such as smartwatches and glasses.
DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation

Transformer is a deep learning language model widely used for natural language processing (NLP) services in datacenters. Among transformer models, Generative Pretrained Transformer (GPT) has achieved remarkable performance in text generation, or natural language generation (NLG), which needs the processing of a large input context in the summarization stage, followed by the generation stage that produces a single word at a time. The conventional platforms such as GPU are specialized for the parallel processing of large inputs in the summarization stage, but their performance significantly degrades in the generation stage due to its sequential characteristic. Therefore, an efficient hardware platform is required to address the high latency caused by the sequential characteristic of text generation. In this paper, we present DFX, a multi-FPGA acceleration appliance that executes GPT-2 model inference end-to-end with low latency and high throughput in both summarization and generation stages. DFX uses model parallelism and optimized dataflow that is model-and-hardware-aware for fast simultaneous workload execution among devices. Its compute cores operate on custom instructions and provide GPT-2 operations end-to-end. We implement the proposed hardware architecture on four Xilinx Alveo U280 FPGAs and utilize all of the channels of the high bandwidth memory (HBM) and the maximum number of compute resources for high hardware efficiency. DFX achieves 5.58$\times$ speedup and 3.99$\times$ energy efficiency over four NVIDIA V100 GPUs on the modern GPT-2 model. DFX is also 8.21$\times$ more cost-effective than the GPU appliance, suggesting that it is a promising solution for text generation workloads in cloud datacenters.
In-memory Realization of In-situ Few-shot Continual Learning with a Dynamically Evolving Explicit Memory

Continually learning new classes from few training examples without forgetting previous old classes demands a flexible architecture with an inevitably growing portion of storage, in which new examples and classes can be incrementally stored and efficiently retrieved. One viable architectural solution is to tightly couple a stationary deep neural network to a dynamically evolving explicit memory (EM). As the centerpiece of this architecture, we propose an EM unit that leverages energy-efficient in-memory compute (IMC) cores during the course of continual learning operations. We demonstrate for the first time how the EM unit can physically superpose multiple training examples, expand to accommodate unseen classes, and perform similarity search during inference, using operations on an IMC core based on phase-change memory (PCM). Specifically, the physical superposition of few encoded training examples is realized via in-situ progressive crystallization of PCM devices. The classification accuracy achieved on the IMC core remains within a range of 1.28%-2.5% compared to that of the state-of-the-art full-precision baseline software model on both the CIFAR-100 and miniImageNet datasets when continually learning 40 novel classes (from only five examples per class) on top of 60 old classes.
A Novel Framework for Assessment of Deep Face Recognition Systems in Realistic Conditions

Deep convolutional neural networks have shown remarkable results on face recognition (FR). Despite their significant progress, the performance of current face recognition techniques is often assessed in benchmarks under not always realistic conditions. The impact of outdoor environment, post-processing operations, and unexpected human behaviors are not sufficiently studied. This paper proposes a universal methodology that systematically measures the impact of various types of influencing factors on the performance of FR methods. Based on extensive experiments and analysis, the key influencing factors are identified, highlighting the need for suitable precautions on modern FR systems. The robustness of the state-of-the-art deep face recognition techniques is further benchmarked with our assessment framework. The best-performing CNN architecture and discriminative loss function are identified, in order to better guide the deployment of an FR system in real world.