Reviewerly/exHarmony huggingface dataset:
Monte Carlo and Reconstruction Membership Inference Attacks against Generative Models

Abstract We present two information leakage attacks that outperform previous work on membership inference against generative models. The first attack allows membership inference without assumptions on the type of the generative model. Contrary to previous evaluation metrics for generative models, like Kernel Density Estimation, it only considers samples of the model which are close to training data records. The second attack specifically targets Variational Autoencoders, achieving high membership inference accuracy. Furthermore, previous work mostly considers membership inference adversaries who perform single record membership inference. We argue for considering regulatory actors who perform set membership inference to identify the use of specific datasets for training. The attacks are evaluated on two generative model architectures, Generative Adversarial Networks (GANs) and Variational Autoen-coders (VAEs), trained on standard image datasets. Our results show that the two attacks yield success rates superior to previous work on most data sets while at the same time having only very mild assumptions. We envision the two attacks in combination with the membership inference attack type formalization as especially useful. For example, to enforce data privacy standards and automatically assessing model quality in machine learning as a service setups. In practice, our work motivates the use of GANs since they prove less vulnerable against information leakage attacks while producing detailed samples.
Assessing interoperability solutions for distributed ledgers

Distributed ledgers (DLs), including blockchains, are gradually becoming an accepted technology. Almost all of these DLs are currently siloed and there is an increasing need for interaction. To fulfill this need, interoperability solutions have been proposed. However, DL interoperability has been scarcely researched and it is not always clear which properties these solutions have, nor which particular issues exist in such solutions. This paper describes a set of key properties with which interoperability solutions can be evaluated. Using these key properties, we systematically compare the two most common solutions and identify several issues, some of which cannot be solved by the solutions themselves. Finally, these properties can be used in practice to assess and compare interoperability solutions, and can be used to determine which kind of solution fits best.
Temporal Convolutional Network with Complementary Inner Bag Loss for Weakly Supervised Anomaly Detection

Weakly supervised anomaly detection (WSAD) is a challenging task with only normal and anomaly video label supervision but required to localize intervals where anomalies take place. We employ multiple instance learning (MIL) for weakly supervised anomaly detection and define a novel inner bag loss (IBL) for MIL to constrain the function space of weakly supervised problem, which consider the lowest anomaly instance score and highest score in each bag. More specifically, the gap between the lowest score and highest score in a positive bag should be large, and that of a negative bag should be small. In order to model the temporal structure of video, we encode preceding adjacent instance by temporal convolutional network (TCN) explicitly. Experimental results show that temporal convolutional network with complementary inner bag loss outperforms the state-of-the-art on the Crime dataset.
Detection of Face Morphing Attacks Based on PRNU Analysis

Recent research found that attacks based on morphed face images, i.e., morphing attacks, pose a severe security risk to face recognition systems. A reliable morphing attack detection from a single face image remains a research challenge since cameras and morphing techniques used by an attacker are unknown at the time of classification. These issues are commonly overseen while many researchers report encouraging detection performance for training and testing morphing attack detection schemes on images obtained from a single face database employing a single morphing algorithm. In this work, a morphing attack detection system based on the analysis of Photo Response Non-Uniformity (PRNU) is presented. More specifically, spatial and spectral features extracted from PRNU patterns across image cells are analyzed. Differences of these features for bona fide and morphed images are estimated during a threshold-selection stage using the Dresden image database which is specifically built for PRNU analysis in digital image forensics. Cross-database evaluations are then conducted employing an ICAO compliant subset of the FRGCv2 database and a Print-Scan database which is a printed and scanned version of said FRGCv2 subset. Bona fide and morphed face images are automatically generated employing four different morphing algorithms. The proposed PRNU-based morphing attack detector is shown to robustly distinguish bona fide and morphed face images achieving an average D-EER of 11.2% in the best configuration. In scenarios where image sources and morphing techniques are unknown, it is shown to significantly outperform other previously established morphing attack detectors. Finally, the limitations and potential of the approach are demonstrated on a dataset of printed and scanned bona fide and morphed face images.
AnonymousNet: Natural Face De-Identification With Measurable Privacy

With billions of personal images being generated from social media and cameras of all sorts on a daily basis, security and privacy are unprecedentedly challenged. Although extensive attempts have been made, existing face image de-identification techniques are either insufficient in photo-reality or incapable of balancing privacy and usability qualitatively and quantitatively, i.e., they fail to answer counterfactual questions such as "is it private now?", "how private is it?", and "can it be more private?" In this paper, we propose a novel framework called AnonymousNet, with an effort to address these issues systematically, balance usability, and enhance privacy in a natural and measurable manner. The framework encompasses four stages: facial attribute estimation, privacy-metric-oriented face obfuscation, directed natural image synthesis, and adversarial perturbation. Not only do we achieve the state-of-the-arts in terms of image quality and attribute prediction accuracy, we are also the first to show that facial privacy is measurable, can be factorized, and accordingly be manipulated in a photo-realistic fashion to fulfill different requirements and application scenarios. Experiments further demonstrate the effectiveness of the proposed framework.
Efficient Featurized Image Pyramid Network for Single Shot Detector

Single-stage object detectors have recently gained popularity due to their combined advantage of high detection accuracy and real-time speed. However, while promising results have been achieved by these detectors on standard-sized objects, their performance on small objects is far from satisfactory. To detect very small/large objects, classical pyramid representation can be exploited, where an image pyramid is used to build a feature pyramid (featurized image pyramid), enabling detection across a range of scales. Existing single-stage detectors avoid such a featurized image pyramid representation due to its memory and time complexity. In this paper, we introduce a light-weight architecture to efficiently produce featurized image pyramid in a single-stage detection framework. The resulting multi-scale features are then injected into the prediction layers of the detector using an attention module. The performance of our detector is validated on two benchmarks: PASCAL VOC and MS COCO. For a 300×300 input, our detector operates at 111 frames per second (FPS) on a Titan X GPU, providing state-of-the-art detection accuracy on PASCAL VOC 2007 testset. On the MS COCO testset, our detector achieves state-of-the-art results surpassing all existing single-stage methods in the case of single-scale inference.
Toward Secure Data Sharing for the IoV: A Quality-Driven Incentive Mechanism With On-Chain and Off-Chain Guarantees

Currently, data sharing for the Internet of Vehicles (IoV) applications has drawn much attention in the framework of developing smart cities and smart transportation. A critical challenge for data sharing is to incentivize users to participate in collecting and sharing data. The traditional incentive mechanism of crowdsourcing is not practical for IoV because of its trust issues. Although blockchain technology has been introduced to address trust issues and security challenges, ensuring trust in off-chain data for the blockchain-based approaches is still an open issue. In this article, we propose a quality-driven auction-based incentive mechanism based on a consortium blockchain that guarantees trust in both on-chain data and off-chain data. We first introduce a consortium blockchain that is used as an open and distributed hyperledger to address the security issue of on-chain data. Then, we formulate the problem as a reverse auction in which the platform acts as an auctioneer that purchases data from users. By utilizing a data quality-driven auction model, the evaluated data quality via expectation maximization is used to ensure the trust in off-chain data. The quality-driven, auction-based incentive mechanism can obtain the high-quality data and optimal social welfare with low social cost. Otherwise, we design a smart contract to perform the data sharing automatically. Finally, the extensive simulations show that our proposed algorithm achieves maximum social welfare, outperforms other solutions, and scales well when the number of users or tasks increase. Moreover, the performance of the smart contract shows its low computing cost.
Making the Invisible Visible: Action Recognition Through Walls and Occlusions

Understanding people's actions and interactions typically depends on seeing them. Automating the process of action recognition from visual data has been the topic of much research in the computer vision community. But what if it is too dark, or if the person is occluded or behind a wall? In this paper, we introduce a neural network model that can detect human actions through walls and occlusions, and in poor lighting conditions. Our model takes radio frequency (RF) signals as input, generates 3D human skeletons as an intermediate representation, and recognizes actions and interactions of multiple people over time. By translating the input to an intermediate skeleton-based representation, our model can learn from both vision-based and RF-based datasets, and allow the two tasks to help each other. We show that our model achieves comparable accuracy to vision-based action recognition systems in visible scenarios, yet continues to work accurately when people are not visible, hence addressing scenarios that are beyond the limit of today's vision-based action recognition.
BMAN: Bidirectional Multi-Scale Aggregation Networks for Abnormal Event Detection

Abnormal event detection is an important task in video surveillance systems. In this paper, we propose a novel bidirectional multi-scale aggregation networks (BMAN) for abnormal event detection. The proposed BMAN learns spatiotemporal patterns of normal events to detect deviations from the learned normal patterns as abnormalities. The BMAN consists of two main parts: an inter-frame predictor and an appearancemotion joint detector. The inter-frame predictor is devised to encode normal patterns, which generates an inter-frame using bidirectional multi-scale aggregation based on attention. With the feature aggregation, robustness for object scale variations and complex motions is achieved in normal pattern encoding. Based on the encoded normal patterns, abnormal events are detected by the appearance-motion joint detector in which both appearance and motion characteristics of scenes are considered. Comprehensive experiments are performed, and the results show that the proposed method outperforms the existing state-of-the-art methods. The resulting abnormal event detection is interpretable on the visual basis of where the detected events occur. Further, we validate the effectiveness of the proposed network designs by conducting ablation study and feature visualization.
DUP-Net: Denoiser and Upsampler Network for 3D Adversarial Point Clouds Defense

Neural networks are vulnerable to adversarial examples, which poses a threat to their application in security sensitive systems. We propose a Denoiser and UPsampler Network (DUP-Net) structure as defenses for 3D adversarial point cloud classification, where the two modules reconstruct surface smoothness by dropping or adding points. In this paper, statistical outlier removal (SOR) and a data-driven upsampling network are considered as denoiser and upsampler respectively. Compared with baseline defenses, DUP-Net has three advantages. First, with DUP-Net as a defense, the target model is more robust to white-box adversarial attacks. Second, the statistical outlier removal provides added robustness since it is a non-differentiable denoising operation. Third, the upsampler network can be trained on a small dataset and defends well against adversarial attacks generated from other point cloud datasets. We conduct various experiments to validate that DUP-Net is very effective as defense in practice. Our best defense eliminates 83.8% of C&W and l2 loss based attack (point shifting), 50.0% of C&W and Hausdorff distance loss based attack (point adding) and 9.0% of saliency map based attack (point dropping) under 200 dropped points on PointNet.
Exploiting Temporal Consistency for Real-Time Video Depth Estimation

Accuracy of depth estimation from static images has been significantly improved recently, by exploiting hierarchical features from deep convolutional neural networks (CNNs). Compared with static images, vast information exists among video frames and can be exploited to improve the depth estimation performance. In this work, we focus on exploring temporal information from monocular videos for depth estimation. Specifically, we take the advantage of convolutional long short-term memory (CLSTM) and propose a novel spatial-temporal CSLTM (ST-CLSTM) structure. Our ST-CLSTM structure can capture not only the spatial features but also the temporal correlations/consistency among consecutive video frames with negligible increase in computational cost. Additionally, in order to maintain the temporal consistency among the estimated depth frames, we apply the generative adversarial learning scheme and design a temporal consistency loss. The temporal consistency loss is combined with the spatial loss to update the model in an end-to-end fashion. By taking advantage of the temporal information, we build a video depth estimation framework that runs in real-time and generates visually pleasant results. Moreover, our approach is flexible and can be generalized to most existing depth estimation frameworks. Code is available at: https://tinyurl.com/STCLSTM
Blockchain-based Firmware Update Scheme Tailored for Autonomous Vehicles

Recently, Autonomous Vehicles (AVs) have gained extensive attention from both academia and industry. AVs are a complex system composed of many subsystems, making them a typical target for attackers. Therefore, the firmware of the different subsystems needs to be updated to the latest version by the manufacturer to fix bugs and introduce new features, e.g., using security patches. In this paper, we propose a distributed firmware update scheme for the AVs' subsystems, leveraging blockchain and smart contract technology. A consortium blockchain made of different AVs manufacturers is used to ensure the authenticity and integrity of firmware updates. Instead of depending on centralized third parties to distribute the new updates, we enable AVs, namely distributors, to participate in the distribution process and we take advantage of their mobility to guarantee high availability and fast delivery of the updates. To incentivize AVs to distribute the updates, a reward system is established that maintains a credit reputation for each distributor account in the blockchain. A zero-knowledge proof protocol is used to exchange the update in return for a proof of distribution in a trustless environment. Moreover, we use attribute-based encryption (ABE) scheme to ensure that only authorized AVs will be able to download and use a new update. Our analysis indicates that the additional cryptography primitives and exchanged transactions do not affect the operation of the AVs network. Also, our security analysis demonstrates that our scheme is efficient and secure against different attacks.
Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications

Recent modern displays are now able to render high dynamic range (HDR), high resolution (HR) videos of up to 8K UHD (Ultra High Definition). Consequently, UHD HDR broadcasting and streaming have emerged as high quality premium services. However, due to the lack of original UHD HDR video content, appropriate conversion technologies are urgently needed to transform the legacy low resolution (LR) standard dynamic range (SDR) videos into UHD HDR versions. In this paper, we propose a joint super-resolution (SR) and inverse tone-mapping (ITM) framework, called Deep SR-ITM, which learns the direct mapping from LR SDR video to their HR HDR version. Joint SR and ITM is an intricate task, where high frequency details must be restored for SR, jointly with the local contrast, for ITM. Our network is able to restore fine details by decomposing the input image and focusing on the separate base (low frequency) and detail (high frequency) layers. Moreover, the proposed modulation blocks apply location-variant operations to enhance local contrast. The Deep SR-ITM shows good subjective quality with increased contrast and details, outperforming the previous joint SR-ITM method.
Aligning Latent Spaces for 3D Hand Pose Estimation

Hand pose estimation from monocular RGB inputs is a highly challenging task. Many previous works for monocular settings only used RGB information for training despite the availability of corresponding data in other modalities such as depth maps. In this work, we propose to learn a joint latent representation that leverages other modalities as weak labels to boost the RGB-based hand pose estimator. By design, our architecture is highly flexible in embedding various diverse modalities such as heat maps, depth maps and point clouds. In particular, we find that encoding and decoding the point cloud of the hand surface can improve the quality of the joint latent representation. Experiments show that with the aid of other modalities during training, our proposed method boosts the accuracy of RGB-based hand pose estimation systems and significantly outperforms state-of-the-art on two public benchmarks.
Human–Robot Collaboration Acceptance Model: Development and Comparison for Germany, Japan, China and the USA

Abstract The use of robots in the national economy—especially in industrialized countries—is growing. At the same time, the interdependency between humans and robots is getting increasingly closer: they are engaging in direct contact with each other as more and more organizations let robots and humans work hand-in-hand. One factor that predicts successful human–robot interdependency is the acceptance of the robot by the human. Generally, only when an innovative assistive working system covers human needs and expectations, it is perceived to be useful and hence accepted. Furthermore, it has been found that cultural context has an impact on human–robot interaction, as people feel more comfortable interacting with a robot in a culturally normative way. Therefore this paper aims at presenting a human–robot collaboration acceptance model (HRCAM) with regard to the collaboration between humans and robots that is based on prior acceptance models, while also considering technology affinity and ethical, legal and social implications. Additionally, similarities and differences in robot acceptance are shown for four selected countries—both in comparison to the overall human–robot collaboration acceptance model and between the countries. The HRCAM additionally shows which variables influence perceived usefulness and perceived ease of use, and thus behavioral intention to use and use behavior. A further distinction is made between anchor variables, which can be influenced in the long term, and adjustment variables, which can be influenced in the short to medium term. The model therefore offers practitioners in the field of human–robot collaboration recommendations to increase the acceptance of robots.
Design of an interval Type-2 fuzzy model with justifiable uncertainty

Throughout previous design proposals of Interval Type-2 Fuzzy Logic Systems most of the research work concentrates on optimal design to best fit data behavior and rarely focus on the inner model essence of Type-2 Fuzzy Systems, which is uncertainty. In this way, failing to focus on this key aspect, which is how much uncertainty exists within the model to better represent the data. In this paper a design methodology for a Mamdani based Interval Type-2 Fuzzy Logic System (MAM-IT2FLS) with Center-Of-Sets defuzzification is presented, using descriptive statistics and granular computing theory to better define the limits of uncertainty within the Interval Type-2 Membership Functions (IT2MF) as extracted from available data. This allows us to justify the uncertainty within the entire Type-2 Fuzzy Logic model, as well as to create the fuzzy model using FCM grouping and to compute IT2MF parameters from MAM-IT2FLS rules using simple steps. This is unlike hybrid learning models with Back-Propagation that adjust IT2MF parameters with gradient based numeric optimization algorithms which are time efficient but unstable for convergence, and evolutionary computation with robust convergence and slow learning time. Experimentation is carried out with six regression benchmark datasets, measuring RMSE and R2 in order to evaluate the performance of the proposed methodology whilst maintaining justifiable uncertainty in its model.
Dual Illumination Estimation for Robust Exposure Correction

Abstract Exposure correction is one of the fundamental tasks in image processing and computational photography. While various methods have been proposed, they either fail to produce visually pleasing results, or only work well for limited types of image (e.g., underexposed images). In this paper, we present a novel automatic exposure correction method, which is able to robustly produce high‐quality results for images of various exposure conditions (e.g., underexposed, overexposed, and partially under‐ and over‐exposed). At the core of our approach is the proposed dual illumination estimation, where we separately cast the under‐and over‐exposure correction as trivial illumination estimation of the input image and the inverted input image. By performing dual illumination estimation, we obtain two intermediate exposure correction results for the input image, with one fixes the underexposed regions and the other one restores the overexposed regions. A multi‐exposure image fusion technique is then employed to adaptively blend the visually best exposed parts in the two intermediate exposure correction images and the input image into a globally well‐exposed image. Experiments on a number of challenging images demonstrate the effectiveness of the proposed approach and its superiority over the state‐of‐the‐art methods and popular automatic exposure correction tools.