Reviewerly/exHarmony huggingface dataset:
Comparative Study of Adversarial Training Methods for Cold-Start Recommendation

Adversarial training in recommendation is originated to improve the robustness of recommenders to attack signals and has recently shown promising results to alleviate cold-start recommendation. However, existing methods usually should make a trade-off between model robustness and performance, and the underlying reasons why using adversarial samples for training works has not been sufficiently verified. To address this issue, this paper identifies the key components of existing adversarial training methods and presents a taxonomy that defines these methods using three levels of components for perturbation generation, perturbation incorporation, and model optimization. Based on this taxonomy, different variants of existing methods are created, and a comparative study is conducted to verify the influence of each component in cold-start recommendation. Experimental results on two benchmarking datasets show that existing state-of-the-art algorithms can be further improved by a proper pairing of the key components as listed in the taxonomy. Moreover, using case studies and visualization, the influence of the content information of items on cold-start recommendation has been analyzed, and the explanations for the working mechanism of different components as proposed in the taxonomy have been offered. These verify the effectiveness of the proposed taxonomy as a design paradigm for adversarial training.
FADER: Fast adversarial example rejection

Deep neural networks are vulnerable to adversarial examples, i.e., carefully-crafted inputs that mislead classification at test time. Recent defenses have been shown to improve adversarial robustness by detecting anomalous deviations from legitimate training samples at different layer representations - a behavior normally exhibited by adversarial attacks. Despite technical differences, all aforementioned methods share a common backbone structure that we formalize and highlight in this contribution, as it can help in identifying promising research directions and drawbacks of existing methods. The first main contribution of this work is the review of these detection methods in the form of a unifying framework designed to accommodate both existing defenses and newer ones to come. In terms of drawbacks, the overmentioned defenses require comparing input samples against an oversized number of reference prototypes, possibly at different representation layers, dramatically worsening the test-time efficiency. Besides, such defenses are typically based on ensembling classifiers with heuristic methods, rather than optimizing the whole architecture in an end-to-end manner to better perform detection. As a second main contribution of this work, we introduce FADER, a novel technique for speeding up detection-based methods. FADER overcome the issues above by employing RBF networks as detectors: by fixing the number of required prototypes, the runtime complexity of adversarial examples detectors can be controlled. Our experiments outline up to 73 × prototypes reduction compared to analyzed detectors for MNIST dataset, up to 50 × for CIFAR10 dataset, and up to 82 × on ImageNet10 dataset respectively, without sacrificing classification accuracy on both clean and adversarial data.
Energy-Efficient Federated Learning Framework for Digital Twin-Enabled Industrial Internet of Things

The digital twin (DT) bridges the physical world with the digital world in real-time for the Industrial Internet of Things (IIoT) and federated learning (FL) enables edge intelligence services for IIoT under the premise of avoiding privacy leakage. The fusion of two technologies can extremely accelerate the development of Industry 4.0 by enabling instant intelligence services. However, in the resource-constrained IIoT, the energy consumption of performing FL and maintaining the virtual object in the digital space by DT technology become the bottlenecks and can not be ignored. To address these issues, in this paper, we proposed an energy-efficient FL framework for DT-enabled IIoT. In the proposed framework, IIoT devices choose different training methods considering dynamic time-varying environment status to achieve energy-efficient FL, i.e., either train locally or connect to the virtual object by DT in the corresponding server of a small base station (SBS) to train mapped data using computing resources of SBS. Then, we investigate the joint training method selection and resource allocation problem to minimize the energy consumption while satisfying the convergence rate of the training model. Considering the problem is intractable using traditional approaches, we use a deep reinforcement learning (DRL)-based algorithm to solve it. Simulation results show that the proposed framework decreases greatly energy consumption compared with the static framework while satisfying the convergence rate of FL.
ARviz: An Augmented Reality-Enabled Visualization Platform for ROS Applications

Current robot interfaces, such as teach pendants and 2D screen displays, used for task visualization and interaction often seem unintuitive and limited in terms of information flow. This compromises task efficiency as interacting with the interface can distract the user from the task at hand. Augmented reality (AR) technology offers the capability to create visually rich displays and intuitive interaction elements in situ. In recent years, AR has shown promising potential to enable effective human–robot interaction (HRI). We introduce ARviz, a versatile, extendable AR visualization platform built for robot applications developed with the widely used robot operating system (ROS) framework. ARviz aims to provide both a universal visualization platform with the capability of displaying any ROS message data type in AR, as well as a multimodal user interface for interacting with robots over ROS. ARviz is built as a platform incorporating a collection of plugins that provide visualization and/or interaction components. Users can also extend the platform by implementing new plugins to suit their needs. We present three use cases as well as two potential use cases to showcase the capabilities and benefits of the ARviz platform for HRI applications. The open access source code for our ARviz platform is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hri-group/arviz</uri> .
Continual Unsupervised Domain Adaptation for Semantic Segmentation by Online Frequency Domain Style Transfer

When deep neural networks are deployed in a highly automated vehicle for environment perception tasks in an unseen (target) domain that differs from the training (source) domain, the mismatch will result in decreased performance. Domain adaptation methods aim at overcoming this mismatch. Many recently investigated methods for unsupervised domain adaptation train a model using labeled source data and unlabeled target data at the same time. These methods assume that data from the target domain is available during the source domain training, which is not always the case in real applications. In this paper we present a way to perform an online style transfer for continual domain adaptation which improves performance on (multiple) unseen target domains using a given perception model. The approach is based on an image style transfer in the frequency domain and requires neither an adjustment of the given source-trained model parameters to the target domain, nor does it require any considerable amount of memory for storing its frequency domain representation of the source domain style, which is particularly important considering the hardware limitations in an autonomous vehicle.
Review-Aware Neural Recommendation with Cross-Modality Mutual Attention

Two-tower neural networks are popularly used in review-aware recommender systems, in which two encoders are separately employed to learn representations for users and items from reviews. However, such an architecture isolates the information exchange between two encoders, resulting in suboptimal recommendation accuracy. To this end, we propose a novel two-tower style Neural Recommendation with Cross-modality Mutual Attention (NRCMA), which bridges user encoder and item encoder crossing reviews and ratings, in order to select informative words and reviews to learn better representation for users and items. Extensive experiments on three benchmark datasets demonstrate that the cross-modality mutual attention is beneficial to two-tower neural networks, and NRCMA consistently outperforms state-of-the-art review-aware item recommendation techniques.
Language Agnostic Speaker Embedding for Cross-Lingual Personalized Speech Generation

Cross-lingual personalized speech generation seeks to synthesize a target speakers voice from only a few training samples that are in a different language. One popular technique is to condition a speech synthesizer on a speaker embedding, that characterizes the target speaker. Unfortunately, such a speaker embedding is usually affected by the language being spoken, which compromises the speaker similarity in cross-lingual personalized speech generation. In this paper, we propose a novel speaker encoding mechanism that learns a language agnostic speaker embedding to characterize speaker individuality. Specifically, we adopt an encoder-decoder architecture to disentangle the language information from speaker embeddings via multi-task learning. We conduct experiments on both voice conversion and text-to-speech synthesis between English and Mandarin that involve cross-lingual speech generation. All objective and subjective evaluations consistently confirm that the proposed speaker embedding is language agnostic, thus improving cross-lingual personalized speech generation in terms of speaker similarity.
ResSaNet: A Hybrid Backbone of Residual Block and Self-Attention Module for Masked Face Recognition

In recent years, the performances of face recognition have been improved significantly by using convolution neural networks (CNN) as the feature extractors. On the other hands, in order to avoid spreading COVID-19 virus, people would wear mask even when they want to pass the face recognition system. Thus, it is necessary to improve the performance of masked face recognition so that users could utilize face recognition methods more easily. In this paper, we propose a feature extraction backbone named ResSaNet that integrates CNN (especially Residual block) and Self-attention module into the same network. By capturing the local and global information of face area simultaneously, our proposed ResSaNet could achieve promising results on both masked and non-masked testing data.
Clustering of Similar Historical Alarm Subsequences in Industrial Control Systems Using Alarm Series and Characteristic Coactivations

Alarm flood similarity analysis (AFSA) methods are frequently used as a primary step for root-cause analysis, alarm flood pattern mining, and online operator support. AFSA methods have been promoted in several research activities in recent years. However, addressing an often-observed ambiguity of the order of alarms and the annunciation of irrelevant alarms in otherwise similar alarm subsequences remains a challenging task. To address and solve these limitations, this paper presents a novel AFSA method that uses alarm series as input to two extended term frequency-inverse document frequency (TF-IDF)-based clustering approaches, a dimensionality reduction technique, and a novel outlier validation. The method proposed here utilizes both characteristic alarm variables and their coactivations, thus, emphasizing the dynamic properties of alarms to a greater extent. Our method is compared to three relevant methods from the literature. The effectiveness and performance of the examined methods are illustrated by means of an openly accessible dataset based on the “Tennessee-Eastman-Process”. It is shown that the integration of alarm series data improves the overall performance and robustness of the AFSA. Furthermore, the clustering results are less influenced by the ambiguity of the order of alarms and irrelevant alarms, thus overcoming a persistent challenge in alarm management research.
Users, Tasks, and Conversational Agents: A Personality Study

Conversational Agents (CA) have become one of the common user interfaces in many online domains. In this paper, we ask whether users have a preference about the personality of CAs, and whether this preference changes depending on the length and type of the tasks CAs are used for. In an online study (N = 410), we investigated three different CA personalities (introvert, extrovert, and non-personified) in four different tasks with different natures and lengths (teaching, booking, todo, and weather). Most of the participants preferred to interact with a conversational agent (introvert or extrovert) as opposed to a non-personified interface, regardless of their own personality. Results suggested that this preference may be task dependent: when CA’s goal was to provide information, participants preferred an extrovert agent. We did not observe a difference between the preference for introvert and extrovert agents when the task’s goal was to complete an assignment.
Quantum algorithms for approximate function loading

Loading classical data into quantum computers represents an essential stage in many relevant quantum algorithms, especially in the field of quantum machine learning. Therefore, the inefficiency of this loading process means a major bottleneck for the application of these algorithms. Here, we introduce two approximate quantum-state preparation methods for the noisy intermediate-scale quantum era inspired by the Grover-Rudolph algorithm, which partially solve the problem of loading real functions. Indeed, by allowing for an infidelity $\ensuremath{\epsilon}$ and under certain smoothness conditions, we prove that the complexity of the implementation of the Grover-Rudolph algorithm without ancillary qubits, first introduced by M\"ott\"onen et al., results into $\mathcal{O}({2}^{{k}_{0}(\ensuremath{\epsilon})})$, with $n$ the number of qubits and ${k}_{0}(\ensuremath{\epsilon})$ asymptotically independent of $n$. This leads to a dramatic reduction in the number of required two-qubit gates. Aroused by this result, we also propose a variational algorithm capable of loading functions beyond the aforementioned smoothness conditions. Our variational Ansatz is explicitly tailored to the landscape of the function, leading to a quasioptimized number of hyperparameters. This allows us to achieve high fidelity in the loaded state with high speed convergence for the studied examples.
A Study of Improved Methods on Image Inpainting

Inpainting is the ancient art technique of modifying the image when it can’t be detected. This current study discusses the various approaches in image inpainting and compares the methods with their time of detection and accuracy. There are many different approaches to solve the inpainting problem. These approaches such as feature distribution, sparse representation, Markov random field, multiscale graph cuts, neural networks, and GAN-based methods are studied. The limitations that are imposed on the regions of the images to be inpainted are studied in the current work. The applications of such approaches are discussed in brief.
No-Reference Light Field Image Quality Assessment Using Four-Dimensional Sparse Transform

Light field imaging can simultaneously capture the intensity and direction information of light rays in the real world. Light field image (LFI) with four-dimensional (4D) data suffers from quality degradation in the process of compression, reconstruction and processing. How to evaluate the visual quality of LFI is thought-provoking. This paper proposes a no-reference LFI quality assessment metric based on high-dimensional sparse transform. Firstly, LFI's sub-aperture gradient image array (SAGIA), which is still a 4D signal, is generated by high-pass filtering between adjacent SAIs. Then, SAGIA is transformed with 4D discrete cosine transform (4D-DCT). 4D-DCT coefficients of SAGIA can characterize the angular and spatial information of LFI. And the logarithmic amplitudes of the coefficients at the same position of SAGIA?s transformed 4D blocks are averaged as the coefficient energy. Subsequently, the 4D-DCT coefficients of SAGIA are divided into the spatial-angular frequency bands and spatial-angular orientation bands, and the corresponding energy features are extracted by converging the coefficient energy of the same band. In addition, the coefficients' amplitudes at the same position of blocks are fitted by the Weibull distribution. Then, the fitted parameters of each position are concatenated, and cropped with principal component analysis to obtain the compact features. Finally, the extracted features are pooled to predict the visual quality of the distorted LFIs. The experimental results demonstrate that the proposed method is more consistent with the subjective evaluation on three LFI databases, compared with the state-of-the-art image quality assessment methods and LFI quality assessment methods.
Virtual Reality Research: Design Virtual Education System for Epidemic (COVID-19) Knowledge to Public

Advances in information and communication technologies have created a range of new products and services for the well-being of society. Virtual Reality (VR) technology has shown enormous potential in educational, commercial, and medical fields. The recent COVID-19 outbreak highlights a poor global performance in communicating epidemic knowledge to the public. Considering the potential of VR, the research starts from analyzing how to use VR technology to improve public education in COVID-19. The research uses Virtual Storytelling Technology (VST) to promote enthusiasm in user participation. A Plot-based VR education system is proposed in order to provide an immersive, explorative, educational experiences. The system includes three primary modules: the Tutorial Module, the Preparation Module, and the Investigation Module. To remove any potential confusion in the user, the research aims to avoid extremely complicated medical professional content and uses interactive, entertainment methods to improve user participation. In order to evaluate the performance efficiency of the system, we conducted performance evaluations and a user study with 80 participants. Compared with traditional education, the experimental results show that the VR education system can used as an effective educational tool for epidemic (COVID-19) fundamental knowledge. The VR technology can assist government agencies and public organizations to increase public understanding of the spread the epidemic (COVID-19).
Relationships Between Quantum IND-CPA Notions

An encryption scheme is called indistinguishable under chosen plaintext attack (short IND-CPA) if an attacker cannot distinguish the encryptions of two messages of his choice. There are other variants of this definition but they all turn out to be equivalent in the classical case. In this paper, we give a comprehensive overview of these different variants of IND-CPA for symmetric encryption schemes in the quantum setting. We investigate the relationships between these notions and prove various equivalences, implications, non-equivalences, and non-implications between these variants.
Deep Neural Networks for Gun Detection in Public Surveillance

The conventional surveillance and control system of Closed-Circuit Television (CCTV) cameras require human resource supervision. Almost all the criminal activities take place using weapons mostly handheld gun, revolver, or pistol. Automatic gun detection is a vital requirement now-a-days. The use of real-time object detection system for the improvement of surveillance is a promising application of Convolutional Neural Networks (CNN). We are concerned about the real-time detection of weapons for the surveillance cameras, so we focused on the implementation and comparison of faster approaches such as Region (R-CNN) and Region Fully Convolutional Networks (R-FCN) with feature extractor Visual Geometry Group (VGG) and ResNet respectively. Training and testing are done on database that consists of local environment images. These images are taken with different type and high- resolution cameras that minimize the idealism. Some metrics also defined to reduce the false positives which are specific to the solution of problem. This research also contributes to the constitution of a hybrid CNN model of both faster-based R-CNN and R-FCN. Both hybrid and existing models experimented to reduce false positive in weapon detection. Result represented in graph with calculation during and after training with confusion matrix and hybrid model results better than other models.
Scaling Deep-Learning Inference with Chiplet-based Architecture and Photonic Interconnects

Chiplet-based architectures have been proposed to scale computing systems for deep neural networks (DNNs). Prior work has shown that for the chiplet-based DNN accelerators, the electrical network connecting the chiplets poses a major challenge to system performance, energy consumption, and scalability. Some emerging interconnect technologies such as silicon photonics can potentially overcome the challenges facing electrical interconnects as photonic interconnects provide high bandwidth density, superior energy efficiency, and ease of implementing broadcast and multicast operations that are prevalent in DNN inference. In this paper, we propose a chiplet-based architecture named SPRINT for DNN inference. SPRINT uses a global buffer to simplify the data transmission between storage and computation, and includes two novel designs: (1) a reconfigurable photonic network that can support diverse communications in DNN inference with minimal implementation cost, and (2) a customized dataflow that exploits the ease of broadcast and multicast feature of photonic interconnects to support highly parallel DNN computations. Simulation studies using ResNet50 DNN model show that SPRINT achieves 46% and 61% execution time and energy consumption reduction, respectively, as compared to other state-of-the-art chiplet-based architectures with electrical or photonic interconnects.
Lung cancer subtype classification using histopathological images based on weakly supervised multi-instance learning

Objective.Subtype classification plays a guiding role in the clinical diagnosis and treatment of non-small-cell lung cancer (NSCLC). However, due to the gigapixel of whole slide images (WSIs) and the absence of definitive morphological features, most automatic subtype classification methods for NSCLC require manually delineating the regions of interest (ROIs) on WSIs.Approach.In this paper, a weakly supervised framework is proposed for accurate subtype classification while freeing pathologists from pixel-level annotation. With respect to the characteristics of histopathological images, we design a two-stage structure with ROI localization and subtype classification. We first develop a method called multi-resolution expectation-maximization convolutional neural network (MR-EM-CNN) to locate ROIs for subsequent subtype classification. The EM algorithm is introduced to select the discriminative image patches for training a patch-wise network, with only WSI-wise labels available. A multi-resolution mechanism is designed for fine localization, similar to the coarse-to-fine process of manual pathological analysis. In the second stage, we build a novel hierarchical attention multi-scale network (HMS) for subtype classification. HMS can capture multi-scale features flexibly driven by the attention module and implement hierarchical features interaction.Results.Experimental results on the 1002-patient Cancer Genome Atlas dataset achieved an AUC of 0.9602 in the ROI localization and an AUC of 0.9671 for subtype classification.Significance.The proposed method shows superiority compared with other algorithms in the subtype classification of NSCLC. The proposed framework can also be extended to other classification tasks with WSIs.