Reviewerly/exHarmony huggingface dataset:
Composition and Configuration Patterns in Multiple-View Visualizations

Multiple-view visualization (MV) is a layout design technique often employed to help users see a large number of data attributes and values in a single cohesive representation. Because of its generalizability, the MV design has been widely adopted by the visualization community to help users examine and interact with large, complex, and high-dimensional data. However, although ubiquitous, there has been little work to categorize and analyze MVs in order to better understand its design space. As a result, there has been little to no guideline in how to use the MV design effectively. In this paper, we present an in-depth study of how MVs are designed in practice. We focus on two fundamental measures of multiple-view patterns: composition, which quantifies what view types and how many are there; and configuration, which characterizes spatial arrangement of view layouts in the display space. We build a new dataset containing 360 images of MVs collected from IEEE VIS, EuroVis, and PacificVis publications 2011 to 2019, and make fine-grained annotations of view types and layouts for these visualization images. From this data we conduct composition and configuration analyses using quantitative metrics of term frequency and layout topology. We identify common practices around MVs, including relationship of view types, popular view layouts, and correlation between view types and layouts. We combine the findings into a MV recommendation system, providing interactive tools to explore the design space, and support example-based design.
RIRNet

Video quality assessment (VQA), which is capable of automatically predicting the perceptual quality of source videos especially when reference information is not available, has become a major concern for video service providers due to the growing demand for video quality of experience (QoE) by end users. While significant advances have been achieved from the recent deep learning techniques, they often lead to misleading results in VQA tasks given their limitations on describing 3D spatio-temporal regularities using only fixed temporal frequency. Partially inspired by psychophysical and vision science studies revealing the speed tuning property of neurons in visual cortex when performing motion perception (i.e., sensitive to different temporal frequencies), we propose a novel no-reference (NR) VQA framework named Recurrent-In-Recurrent Network (RIRNet) to incorporate this characteristic to prompt an accurate representation of motion perception in VQA task. By fusing motion information derived from different temporal frequencies in a more efficient way, the resulting temporal modeling scheme is formulated to quantify the temporal motion effect via a hierarchical distortion description. It is found that the proposed framework is in closer agreement with quality perception of the distorted videos since it integrates concepts from motion perception in human visual system (HVS), which is manifested in the designed network structure composed of low- and high- level processing. A holistic validation of our methods on four challenging video quality databases demonstrates the superior performances over the state-of-the-art methods.
Blockchain-Based Traceability and Management for Additive Manufacturing

Additive Manufacturing (AM) is a major advancement in the digitization of manufacturing and production operations.Additive manufacturing uses three dimensional digital design, software and hardware equipment to precisely deposit layered materials for on-demand product manufacturing.The distinct advantages in enabling additive manufacturing includes cost efficiency, reduced time-to-market, flexibility and precise customization.However, several challenges such as trusted traceability, certification for quality compliance, and protecting intellectual property need to be addressed.Blockchain-based distributed ledgers provide tremendous advantages for product traceability and ensure trust among participating stakeholders.In this paper, we propose a blockchain-based solution for product traceability produced using additive manufacturing, guaranteeing secure and trusted traceability, accessibility, and immutability of transactions, and data provenance among supply chain stakeholders.Our proposed solution utilizes Ethereum smart contracts to govern and trace transactions initiated by participants involved in the manufacturing process.Decentralized storage of Inter-Planetary File Systems is used to store and share design files, IoT device records, and additional product specifications.We provide the system architecture, implementation, and detailed algorithms that demonstrate the working principles of our proposed solution for secure AM.Furthermore, we present detailed security and cost analysis of the solution highlighting its efficiency with respect to key security and performance requirements.
A novel framework for detecting social bots with deep neural networks and active learning

Microblogging is a popular online social network (OSN), which facilitates users to obtain and share news and information. Nevertheless, it is filled with a huge number of social bots that significantly disrupt the normal order of OSNs. Sina Weibo, one of the most popular Chinese OSNs in the world, is also seriously affected by social bots. With the growing development of social bots in Sina Weibo, they are increasingly indistinguishable from normal users, which presents more huge challenges in detecting social bots. Firstly, it is difficult to extract the features of social bots completely. Secondly, large-scale data collection and labeling of user data are extremely hard. Thirdly, the performance of classical classification approaches applied to social bot detection is not good enough. Therefore, this paper proposes a novel framework for detecting social bots in Sina Weibo based on deep neural networks and active learning (DABot). Specifically, 30 features from four categories, namely metadata-based, interaction-based, content-based, and timing-based are extracted to distinguish between social bots and normal users. Nine of these features are completely new features proposed in this paper. Moreover, active learning is employed to efficiently expand the labeled data. Then, a new deep neural network model called RGA is built to implement the detection of social bots, which makes use of a residual network (ResNet), a bidirectional gated recurrent unit (BiGRU), and an attention mechanism. After performance evaluation, the results show that DABot is more effective than the state-of-the-art baselines with the accuracy of 0.9887.
A Survey of Methods for Low-Power Deep Learning and Computer Vision

Deep neural networks (DNNs) are successful in many computer vision tasks. However, the most accurate DNNs require millions of parameters and operations, making them energy, computation and memory intensive. This impedes the deployment of large DNNs in low-power devices with limited compute resources. Recent research improves DNN models by reducing the memory requirement, energy consumption, and number of operations without significantly decreasing the accuracy. This paper surveys the progress of low-power deep learning and computer vision, specifically in regards to inference, and discusses the methods for compacting and accelerating DNN models. The techniques can be divided into four major categories: (1) parameter quantization and pruning, (2) compressed convolutional filters and matrix factorization, (3) network architecture search, and (4) knowledge distillation. We analyze the accuracy, advantages, disadvantages, and potential solutions to the problems with the techniques in each category. We also discuss new evaluation metrics as a guideline for future research.
GAN Slimming: All-in-One GAN Compression by a Unified Optimization Framework

Generative adversarial networks (GANs) have gained increasing popularity in various computer vision applications, and recently start to be deployed to resource-constrained mobile devices. Similar to other deep models, state-of-the-art GANs suffer from high parameter complexities. That has recently motivated the exploration of compressing GANs (usually generators). Compared to the vast literature and prevailing success in compressing deep classifiers, the study of GAN compression remains in its infancy, so far leveraging individual compression techniques instead of more sophisticated combinations. We observe that due to the notorious instability of training GANs, heuristically stacking different compression techniques will result in unsatisfactory results. To this end, we propose the first unified optimization framework combining multiple compression means for GAN compression, dubbed GAN Slimming (GS). GS seamlessly integrates three mainstream compression techniques: model distillation, channel pruning and quantization, together with the GAN minimax objective, into one unified optimization form, that can be efficiently optimized from end to end. Without bells and whistles, GS largely outperforms existing options in compressing image-to-image translation GANs. Specifically, we apply GS to compress CartoonGAN, a state-of-the-art style transfer network, by up to $$\mathbf {{47}{\times }}$$ times, with minimal visual quality degradation. Codes and pre-trained models can be found at https://github.com/TAMU-VITA/GAN-Slimming .
Fault detection based on one-class deep learning for manufacturing applications limited to an imbalanced database

Despite extensive studies for the industrial applications of deep learning, its actual usage in manufacturing sites has been extremely restrained by the difficulty in obtaining sufficient industrial data, especially for production failure cases. In this study, we introduced a fault-detection module based on one-class deep learning for imbalanced industrial time-series data, which consists of three submodules, namely, time-series prediction based on deep learning, residual calculation, and one-class classification using one-class support vector machine and isolation forest. Four different networks were used for the time-series prediction: multilayer perception (MLP), residual network (ResNet), long–short-term memory (LSTM), and ResNet–LSTM, each trained with the one-class data having only the production success cases. We adopted the residuals of the deep-learning prediction as an elaborated feature for the construction of the one-class classification. We also tested the fault-detection module with the actual mass production data of a die-casting process. By adopting the features elaborated by the deep-learning time-series prediction, we showed that the total accuracy of the one-class classification significantly improved from 90.0% to 96.0%. Especially for its capability to detect production failures, the accuracy improved from 84.0% to 96.0%. The area under the receiver operating characteristics (AUROC) also improved from 87.56% to 98.96%. ResNet showed the best performance for detecting production failures, whereas ResNet–LSTM produced better results for ensuring the production success. Our results suggest that the one-class deep learning is a promising approach for extracting important features from time-series data to realize a one-class fault-detection module.
More Is Better: Data Augmentation for Channel-Resilient RF Fingerprinting

RF fingerprinting involves identifying characteristic transmitter-imposed variations within a wireless signal. Deep neural networks (DNNs) that do not rely on handcrafting features have proven to be remarkably effective in fingerprinting tasks, as long as the channel remains invariant. However, DNNs trained at a specific location and time perform poorly on datasets collected under different channel conditions. This article proposes a data augmentation step within the training pipeline that exposes the DNN to many simulated channel and noise variations that are not present in the original dataset. We describe two approaches for data augmentation. The first approach is applied to the "transmitter data" when transmitter side data (i.e., pure signals without channel distortion) is available. The second approach is applied to the "receiver data" when only a passive dataset is available with already over-the-air transmitted signals. We show that data augmentation results in 75 percent improvement in the former case with a custom-generated dataset, and around 32-51 percent improvement in the latter case on a 5000-device WiFi dataset, compared to the case of non-augmented data fed to DNNs.
Evaluating Explanation Methods for Deep Learning in Security

Deep learning is increasingly used as a building block of security systems. Unfortunately, neural networks are hard to interpret and typically opaque to the practitioner. The machine learning community has started to address this problem by developing methods for explaining the predictions of neural networks. While several of these approaches have been successfully applied in the area of computer vision, their application in security has received little attention so far. It is an open question which explanation methods are appropriate for computer security and what requirements they need to satisfy. In this paper, we introduce criteria for comparing and evaluating explanation methods in the context of computer security. These cover general properties, such as the accuracy of explanations, as well as security-focused aspects, such as the completeness, efficiency, and robustness. Based on our criteria, we investigate six popular explanation methods and assess their utility in security systems for malware detection and vulnerability discovery. We observe significant differences between the methods and build on these to derive general recommendations for selecting and applying explanation methods in computer security.
On the Effectiveness of Image Rotation for Open Set Domain Adaptation

Open Set Domain Adaptation (OSDA) bridges the domain gap between a labeled source domain and an unlabeled target domain, while also rejecting target classes that are not present in the source. To avoid negative transfer, OSDA can be tackled by first separating the known/unknown target samples and then aligning known target samples with the source data. We propose a novel method to addresses both these problems using the self-supervised task of rotation recognition. Moreover, we assess the performance with a new open set metric that properly balances the contribution of recognizing the known classes and rejecting the unknown samples. Comparative experiments with existing OSDA methods on the standard Office-31 and Office-Home benchmarks show that: (i) our method outperforms its competitors, (ii) reproducibility for this field is a crucial issue to tackle, (iii) our metric provides a reliable tool to allow fair open set evaluation.
REVAMP<sup>2</sup>T: Real-Time Edge Video Analytics for Multicamera Privacy-Aware Pedestrian Tracking

This article presents REVAMP$^2$T, Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking, as an integrated end-to-end IoT system for privacy-built-in decentralized situational awareness. REVAMP$^2$T presents novel algorithmic and system constructs to push deep learning and video analytics next to IoT devices (i.e. video cameras). On the algorithm side, REVAMP$^2$T proposes a unified integrated computer vision pipeline for detection, re-identification, and tracking across multiple cameras without the need for storing the streaming data. At the same time, it avoids facial recognition, and tracks and re-identifies pedestrians based on their key features at runtime. On the IoT system side, REVAMP$^2$T provides infrastructure to maximize hardware utilization on the edge, orchestrates global communications, and provides system-wide re-identification, without the use of personally identifiable information, for a distributed IoT network. For the results and evaluation, this article also proposes a new metric, Accuracy$\cdot$Efficiency (\AE), for holistic evaluation of IoT systems for real-time video analytics based on accuracy, performance, and power efficiency. REVAMP$^2$T outperforms current state-of-the-art by as much as thirteen-fold \AE~improvement.
On-demand photonic entanglement synthesizer

Quantum information protocols require various types of entanglement, such as Einstein-Podolsky-Rosen, Greenberger-Horne-Zeilinger, and cluster states. In optics, on-demand preparation of these states has been realized by squeezed light sources, but such experiments require different optical circuits for different entangled states, thus lacking versatility. Here, we demonstrate an on-demand entanglement synthesizer that programmably generates all these entangled states from a single squeezed light source. This is achieved by a loop-based circuit that is dynamically controllable at nanosecond time scales and processes optical pulses in the time domain. We verify the generation of five different small-scale entangled states and a large-scale cluster state containing more than 1000 modes without changing the optical circuit. Moreover, this circuit enables storage and release of one part of the generated entangled state, thus working as a quantum memory. Our demonstration should open a way for a more general entanglement synthesizer and a scalable quantum processor.
Evaluating local explanation methods on ground truth

Evaluating local explanation methods is a difficult task due to the lack of a shared and universally accepted definition of explanation. In the literature, one of the most common ways to assess the performance of an explanation method is to measure the fidelity of the explanation with respect to the classification of a black box model adopted by an Artificial Intelligent system for making a decision. However, this kind of evaluation only measures the degree of adherence of the local explainer in reproducing the behavior of the black box classifier with respect to the final decision. Therefore, the explanation provided by the local explainer could be different in the content even though it leads to the same decision of the AI system. In this paper, we propose an approach that allows to measure to which extent the explanations returned by local explanation methods are correct with respect to a synthetic ground truth explanation. Indeed, the proposed methodology enables the generation of synthetic transparent classifiers for which the reason for the decision taken, i.e., a synthetic ground truth explanation, is available by design. Experimental results show how the proposed approach allows to easily evaluate local explanations on the ground truth and to characterize the quality of local explanation methods.
Using Transfer Learning Approach to Implement Convolutional Neural Network model to Recommend Airline Tickets by Using Online Reviews

Social Media provides an opportunity for people to share their idea about different aspects of life. Traveling is one of the essential aspects of life. In this paper, we use Bidirectional Encoder Representations from Transformers(BERT) for sentiment classification of online reviews to implement a Convolutional Neural Network model that can recommend airline tickets. Different sentiment classification models are compared based on people's online reviews from six online social platforms. The new model can classify airline tickets as an economical choice or not economical choice ticket based on various aspects of the trips, including airline customer satisfaction, travel destination, hotel information, restaurants, and tourist attractions. This work contribution is two-fold: first, it examines the importance of choosing a transfer learning model for sentiment classification of online social platforms in the recommender system's prediction accuracy. Second, the implementation of the Convolutional Neural Network model, which can classify airline tickets based on data generated by multiple online social platforms. Using the natural language processing approach to use the transfer learning model to improve the CNN model's prediction accuracy is a new approach to use online social platforms to recommend economical airline tickets to consumers.
YOLO in the Dark - Domain Adaptation Method for Merging Multiple Models

Generating models to handle new visual tasks requires additional datasets, which take considerable effort to create. We propose a method of domain adaptation for merging multiple models with less effort than creating an additional dataset. This method merges pre-trained models in different domains using glue layers and a generative model, which feeds latent features to the glue layers to train them without an additional dataset. We also propose a generative model that is created by distilling knowledge from pre-trained models. This enables the dataset to be reused to create latent features for training the glue layers. We apply this method to object detection in a low-light situation. The YOLO-in-the-Dark model comprises two models, Learning-to-See-in-the-Dark model and YOLO. We present the proposed method and report the result of domain adaptation to detect objects from RAW short-exposure low-light images. The YOLO-in-the-Dark model uses fewer computing resources than the naive approach.
Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision

One of the primary challenges limiting the applicability of deep learning is its susceptibility to learning spurious correlations rather than the underlying mechanisms of the task of interest. The resulting failure to generalise cannot be addressed by simply using more data from the same distribution. We propose an auxiliary training objective that improves the generalization capabilities of neural networks by leveraging an overlooked supervisory signal found in existing datasets. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. We show that such pairs can be identified in a number of existing datasets in computer vision (visual question answering, multi-label image classification) and natural language processing (sentiment analysis, natural language inference). The new training objective orients the gradient of a model’s decision function with pairs of counterfactual examples. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.