Reviewerly/exHarmony huggingface dataset:
RetrieveGAN: Image Synthesis via Differentiable Patch Retrieval

Image generation from scene description is a cornerstone technique for the controlled generation, which is beneficial to applications such as content creation and image editing. In this work, we aim to synthesize images from scene description with retrieved patches as reference. We propose a differentiable retrieval module. With the differentiable retrieval module, we can (1) make the entire pipeline end-to-end trainable, enabling the learning of better feature embedding for retrieval; (2) encourage the selection of mutually compatible patches with additional objective functions. We conduct extensive quantitative and qualitative experiments to demonstrate that the proposed method can generate realistic and diverse images, where the retrieved patches are reasonable and mutually compatible.
EM and Power SCA-Resilient AES-256 Through &gt;350× Current-Domain Signature Attenuation and Local Lower Metal Routing

Mathematically secure cryptographic algorithms, when implemented on a physical substrate, leak critical “side-channel” information, leading to power and electromagnetic (EM) analysis attacks. Circuit-level protections involve switched capacitor, buck converter, or series low-dropout (LDO) regulator-based implementations, each of which suffers from significant power, area, or performance tradeoffs and has only achieved a minimum traces to disclosure (MTD) of 10M till date. Utilizing an in-depth white-box model, this work, for the first time, focuses on signature suppression in the current domain, which provides an Attenuation2 enhancement in MTD, leading to orders of magnitude improvement in both power and EM side-channel analysis (SCA) immunities. Using a combination of current-domain “signature attenuation” (CDSA) along with local lower level metal routing, the critical correlated information in the crypto current is significantly suppressed before it reaches the supply pin. Especially, to prevent the EM leakage from its source (metal layers carrying the correlated crypto current acting as antennas), this work embraces lower level metal routing of the CDSA embedding the crypto-IP so that the signature becomes highly suppressed before it passes through the higher metal layers (which radiates significantly) to connect to the external pin. The 65-nm CMOS test chip contains both protected and unprotected parallel AES-256 implementations, running at a clock frequency of 50 MHz. Test vector leakage assessment (TVLA) on the protected CDSA-AES, demonstrated with on-chip measurements for the first time, shows that the higher level metal layers leak significantly more compared with the lower level metal routing. Correlational power and EM analysis (CPA/CEMA) attacks on the unprotected implementation were able to extract the secret key within 8k and 12k traces, respectively, while the protected CDSA-AES could not be broken even after 1B encryptions for both power and EM SCA, evaluated both in the time and frequency domains, showing an improvement of 100× over the prior state-of-the-art countermeasures with comparable power and area overheads.
Multitask Non-Autoregressive Model for Human Motion Prediction

Human motion prediction, which aims at predicting future human skeletons given the past ones, is a typical sequence-to-sequence problem. Therefore, extensive efforts have been devoted to exploring different RNN-based encoder-decoder architectures. However, by generating target poses conditioned on the previously generated ones, these models are prone to bringing issues such as error accumulation problem. In this paper, we argue that such issue is mainly caused by adopting autoregressive manner. Hence, a novel Non-AuToregressive model (NAT) is proposed with a complete non-autoregressive decoding scheme, as well as a context encoder and a positional encoding module. More specifically, the context encoder embeds the given poses from temporal and spatial perspectives. The frame decoder is responsible for predicting each future pose independently. The positional encoding module injects positional signal into the model to indicate the temporal order. Besides, a multitask training paradigm is presented for both low-level human skeleton prediction and high-level human action recognition, resulting in the considerable improvement for the prediction task. Our approach is evaluated on Human3.6M and CMU-Mocap benchmarks and outperforms state-of-the-art autoregressive methods.
Interactive medical image segmentation via a point-based interaction

Due to low tissue contrast, irregular shape, and large location variance, segmenting the objects from different medical imaging modalities (e.g., CT, MR) is considered as an important yet challenging task. In this paper, a novel method is presented for interactive medical image segmentation with the following merits. (1) Its design is fundamentally different from previous pure patch-based and image-based segmentation methods. It is observed that during delineation, the physician repeatedly check the intensity from area inside-object to outside-object to determine the boundary, which indicates that comparison in an inside-out manner is extremely important. Thus, the method innovatively models the segmentation task as learning the representation of bi-directional sequential patches, starting from (or ending in) the given central point of the object. This can be realized by the proposed ConvRNN network embedded with a gated memory propagation unit. (2) Unlike previous interactive methods (requiring bounding box or seed points), the proposed method only asks the physician to merely click on the rough central point of the object before segmentation, which could simultaneously enhance the performance and reduce the segmentation time. (3) The method is utilized in a multi-level framework for better performance. It has been systematically evaluated in three different segmentation tasks, including CT kidney tumor, MR prostate, and PROMISE12 challenge, showing promising results compared with state-of-the-art methods.
Facial Emotion Recognition Using Transfer Learning

Facial Emotion Recognition is considered very significant for Human Computer Interaction (HCI) and they play a vital role in everyday human life. In recent years Convolutional Neural Network (CNNs) has become very popular among researcher for image analysis, because CNNs have generated remarkable results. However, CNNs needs a lot of data to train from scratch. This problem has been addressed by several researchers who have trained CNNs with millions of images, this training knowledge can also be used in a different task which is known as Transfer Learning. AlexNet is one of the best pre-trained CNNs. Our work reflects a brief comparison between modern pre-trained CNNs and using transfer learning with classification approach like Support Vector Machine (SVM), generally known as hybrid classifier. The testing has been done on two very popular expression database Cohn-Kanade+ (CK+) database and Natural Visible and Infrared Expression (NVIE) database. Results clearly depicts that pre-trained CNNs produces better result than handcrafted approaches.
Fisher Information Under Local Differential Privacy

We develop data processing inequalities that describe how Fisher information from statistical samples can scale with the privacy parameter ε under local differential privacy constraints. These bounds are valid under general conditions on the distribution of the score of the statistical model, and they elucidate under which conditions the dependence on ε is linear, quadratic, or exponential. We show how these inequalities imply order-optimal lower bounds for private estimation for both the Gaussian location model and discrete distribution estimation for all levels of privacy ε >0. We further apply these inequalities to sparse Bernoulli models and demonstrate privacy mechanisms and estimators with order-matching squared ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> error.
Exploring the utility of EDA and skin temperature as individual physiological correlates of motion sickness

Motion sickness (MS) is known to be a potentially limiting factor for future self-driving vehicles – specifically in regards to occupant comfort and well-being. With this as a consideration comes the desire to accurately measure, track and even predict MS state in real-time. Previous research has considered physiological measurements to measure MS state, although, this is mainly measured after an MS exposure and not throughout exposure(s) to a MS task. A unique contribution of this paper is in the real-time tracking of subjective MS alongside real-time physiological measurements of Electrodermal Activity (EDA) and skin temperature. Data was collected in both simulator-based (controlled) and on-road (naturalistic) studies. 40 participants provided at total of 61 data sets, providing 1603 min of motion sickness data for analysis. This study is in agreement that these measures are related to MS but evidenced a total lack of reliability for these measures at an individual level for both simulator and on-road experimentation. It is likely that other factors, such as environment and emotional state are more impactful on these physiological measures than MS itself. At a cohort level, the applicability of physiological measures is not considered useful for measuring MS accurately or reliably in real-time. Recommendations for further research include a mixed-measures approach to capture other data types (such as subject activity) and to remove contamination of physiological measures from environmental changes.
A Reversible Data Hiding Algorithm Based on Prediction Error With Large Amounts of Data Hiding in Spatial Domain

In recent years among data hiding technologies, Reversible Data Hiding(RDH) technology has attracted widespread interest and application, which is to hide the secret information in a carrier image and recover the original carrier image losslessly to extract the secret information. Current research on RDH algorithms mainly involving frequency domain, spatial domain, and encryption domain. Based on the Prediction-Error Expansion(PEE) methods, as spatial domain approaches, achieved great progress in the past decade. However, there is a defect in the state-of-the-art methods that with the embedded payload increased, the distortion rate of the cover image increased simultaneously. To solve the problem, we proposed a refined reversible data hiding algorithm based on the PEE method with simple implementation. We improved an effective predictor that all the remaining pixels can be predicted in the embedding process, except for those in the first row, the first column, the last row, and the last column in the original image. The extraction process is the reverse of the embedding process that the embedded information and the original carrier is restored without damage. Our work utilized the correlation between image pixels better to solve the inherent contradiction between payload and distortion rate in the state-of-the-art data hiding algorithms. Proven through experiments, our method achieved a large embedding capacity while keeping the image distortion rate and computing complexity low.
The smooth variable structure filter: A comprehensive review

The smooth variable structure filter (SVSF) is a type of sliding mode filter formulated in a predictor-corrector format and has seen significant development over the last 15 years. In this paper, we provide a comprehensive review of the SVSF and its variants. The developments, applications and improvements of the SVSF in terms of robustness and optimality are investigated. In addition, the combination of the SVSF with different filtering strategies is considered in an effort to improve estimation accuracy while maintaining robustness to model uncertainty. State estimation techniques such as the unscented and cubature Kalman filters (UKF & CKF), SVSF, the combination of SVSF with UKF (UK-SVSF), and the combination of CKF with SVSF (CK-SVSF) are applied on a 4-DOF industrial robotic arm. The SVSF state estimation performance is examined under different operating conditions. The results of these filters have been compared based a number of statistics such as the root mean squared error (RMSE) and mean absolute error (MAE), among others. It is shown that the UK-SVSF and CK-SVSF strategies acquire the best performance in the presence of uncertainties.
Malleable 2.5D Convolution: Learning Receptive Fields Along the Depth-Axis for RGB-D Scene Parsing

Depth data provide geometric information that can bring progress in RGB-D scene parsing tasks. Several recent works propose RGB-D convolution operators that construct receptive fields along the depth-axis to handle 3D neighborhood relations between pixels. However, these methods pre-define depth receptive fields by hyperparameters, making them rely on parameter selection. In this paper, we propose a novel operator called malleable 2.5D convolution to learn the receptive field along the depth-axis. A malleable 2.5D convolution has one or more 2D convolution kernels. Our method assigns each pixel to one of the kernels or none of them according to their relative depth differences, and the assigning process is formulated as a differentiable form so that it can be learnt by gradient descent. The proposed operator runs on standard 2D feature maps and can be seamlessly incorporated into pre-trained CNNs. We conduct extensive experiments on two challenging RGB-D semantic segmentation dataset NYUDv2 and Cityscapes to validate the effectiveness and the generalization ability of our method.
Exclusivity-Consistency Regularized Knowledge Distillation for Face Recognition

Knowledge distillation is an effective tool to compress large pre-trained Convolutional Neural Networks (CNNs) or their ensembles into models applicable to mobile and embedded devices. The success of which mainly comes from two aspects: the designed student network and the exploited knowledge. However, current methods usually suffer from the low-capability of mobile-level student network and the unsatisfactory knowledge for distillation. In this paper, we propose a novel position-aware exclusivity to encourage large diversity among different filters of the same layer to alleviate the low-capability of student network. Moreover, we investigate the effect of several prevailing knowledge for face recognition distillation and conclude that the knowledge of feature consistency is more flexible and preserves much more information than others. Experiments on a variety of face recognition benchmarks have revealed the superiority of our method over the state-of-the-arts.
Secure and Efficient Multi-Signature Schemes for Fabric: An Enterprise Blockchain Platform

Digital signature is a major component of transactions on Blockchain platforms, especially in enterprise Blockchain platforms, where multiple signatures from a set of peers need to be produced to endorse a transaction. However, such process is often complex and time-consuming. Multi-signature, which can improve transaction efficiency by having a set of signers cooperate to produce a joint signature, has attracted extensive attentions. In this work, we propose two multi-signature schemes, GMS and AGMS, which are proved to be more secure and efficient than state-of-the-art multi-signature schemes. Besides, we implement the proposed schemes in a real Enterprise Blockchain platform, Fabric. Experiment results show that the proposed AGMS scheme helps achieve the goal of high transaction efficiency, low storage complexity, as well as high robustness against rogue-key attacks and $k$ -sum problem attacks.
BeCAPTCHA: Behavioral bot detection using touchscreen and mobile sensors benchmarked on HuMIdb

In this paper we study the suitability of a new generation of CAPTCHA methods based on smartphone interactions. The heterogeneous flow of data generated during the interaction with the smartphones can be used to model human behavior when interacting with the technology and improve bot detection algorithms. For this, we propose BeCAPTCHA, a CAPTCHA method based on the analysis of the touchscreen information obtained during a single drag and drop task in combination with the accelerometer data. The goal of BeCAPTCHA is to determine whether the drag and drop task was realized by a human or a bot. We evaluate the method by generating fake samples synthesized with Generative Adversarial Neural Networks and handcrafted methods. Our results suggest the potential of mobile sensors to characterize the human behavior and develop a new generation of CAPTCHAs. The experiments are evaluated with HuMIdb1 (Human Mobile Interaction database), a novel multimodal mobile database that comprises 14 mobile sensors acquired from 600 users. HuMIdb is freely available to the research community.
Scalable and Communication-Efficient Decentralized Federated Edge Learning with Multi-blockchain Framework

The emerging Federated Edge Learning (FEL) technique has drawn considerable attention, which not only ensures good machine learning performance but also solves “data island” problems caused by data privacy concerns. However, large-scale FEL still faces following crucial challenges: (i) there lacks a secure and communication-efficient model training scheme for FEL; (2) there is no scalable and flexible FEL framework for updating local models and global model sharing (trading) management. To bridge the gaps, we first propose a blockchain-empowered secure FEL system with a hierarchical blockchain framework consisting of a main chain and subchains. This framework can achieve scalable and flexible decentralized FEL by individually manage local model updates or model sharing records for performance isolation. A Proof-of-Verifying consensus scheme is then designed to remove low-quality model updates and manage qualified model updates in a decentralized and secure manner, thereby achieving secure FEL. To improve communication efficiency of the blockchain-empowered FEL, a gradient compression scheme is designed to generate sparse but important gradients to reduce communication overhead without compromising accuracy, and also further strengthen privacy preservation of training data. The security analysis and numerical results indicate that the proposed schemes can achieve secure, scalable, and communication-efficient decentralized FEL.
Physical Layer Authentication for 5G Communications: Opportunities and Road Ahead

Resorting to the exploitation of physical attributes, physical-layer authentication (PLA) is a promising technology to supplement and enhance current cryptography-based security mechanisms in wireless communications. in the fifth-generation (5G) communications, many disruptive technologies spring up, such as millimeter-wave communication (mmWave), massive multiple-input and multiple-output (MiMO) and non-orthogonal- multiple-access (NOMA). PLA schemes in 5G networks are facing challenges while exposed to opportunities at the same time. This article seeks to identify the critical technology gaps as well as the feasible enablers in terms of the unique characteristics of 5G networks. The investigation consists of four hierarchical parts. in the first section, existing PLA schemes are reviewed, and the corresponding challenges are discussed in the next section. in the third section, we investigate potential enablers based on the unique characteristics of 5G networks and provide three corresponding PLA case studies. Open problems and research directions on PLA for 5G and beyond are discussed in the last section, including waveform design, feature learning, mobile users, and terahertz communications.
Pairwise Two-Stream ConvNets for Cross-Domain Action Recognition With Small Data

In this work, we target cross-domain action recognition (CDAR) in the video domain and propose a novel end-to-end pairwise two-stream ConvNets (PTC) algorithm for real-life conditions, in which only a few labeled samples are available. To cope with the limited training sample problem, we employ pairwise network architecture that can leverage training samples from a source domain and, thus, requires only a few labeled samples per category from the target domain. In particular, a frame self-attention mechanism and an adaptive weight scheme are embedded into the PTC network to adaptively combine the RGB and flow features. This design can effectively learn domain-invariant features for both the source and target domains. In addition, we propose a sphere boundary sample-selecting scheme that selects the training samples at the boundary of a class (in the feature space) to train the PTC model. In this way, a well-enhanced generalization capability can be achieved. To validate the effectiveness of our PTC model, we construct two CDAR data sets (SDAI Action I and SDAI Action II) that include indoor and outdoor environments; all actions and samples in these data sets were carefully collected from public action data sets. To the best of our knowledge, these are the first data sets specifically designed for the CDAR task. Extensive experiments were conducted on these two data sets. The results show that PTC outperforms state-of-the-art video action recognition methods in terms of both accuracy and training efficiency. It is noteworthy that when only two labeled training samples per category are used in the SDAI Action I data set, PTC achieves 21.9% and 6.8% improvement in accuracy over two-stream and temporal segment networks models, respectively. As an added contribution, the SDAI Action I and SDAI Action II data sets will be released to facilitate future research on the CDAR task.
Exploration of Hands-free Text Entry Techniques For Virtual Reality

Text entry is a common activity in virtual reality (VR) systems. There is a limited number of available hands-free techniques, which allow users to carry out text entry when users’ hands are busy such as holding items or hand-based devices are not available. The most used hands-free text entry technique is DwellType, where a user selects a letter by dwelling over it for a specific period. However, its performance is limited due to the fixed dwell time for each character selection. In this paper, we explore two other hands-free text entry mechanisms in VR: BlinkType and NeckType, which leverage users’ eye blinks and neck’s forward and backward movements to select letters. With a user study, we compare the performance of the two techniques with DwellType. Results show that users can achieve an average text entry rate of 13.47, 11.18 and 11.65 words per minute with BlinkType, NeckType, and DwellType, respectively. Users’ subjective feedback shows BlinkType as the preferred technique for text entry in VR.