Reviewerly/exHarmony huggingface dataset:
FC-DCNN: A densely connected neural network for stereo estimation

We propose a novel lightweight network for stereo estimation. Our network consists of a fully-convolutional densely connected neural network (FC-DCNN) that computes matching costs between rectified image pairs. Our FC-DCNN method learns expressive features and performs some simple but effective postprocessing steps. The densely connected layer structure connects the output of each layer to the input of each subsequent layer. This network structure and the fact that we do not use any fully-connected layers or 3D convolutions leads to a very lightweight network. The output of this network is used in order to calculate matching costs and create a cost-volume. Instead of using time and memory-inefficient cost-aggregation methods such as semiglobal matching or conditional random fields in order to improve the result, we rely on filtering techniques, namely median filter and guided filter. By computing a left-right consistency check we get rid of inconsistent values. Afterwards we use a watershed foreground-background segmentation on the disparity image with removed inconsistencies. This mask is then used to refine the final prediction. We show that our method works well for both challenging indoor and outdoor scenes by evaluating it on the Middlebury, KITTI and ETH3D benchmarks respectively. Our full framework is available at https://github.com/thedodo/FC-DCNN.
Finding security threats that matter: Two industrial case studies

In the past decade, speed has become an essential trait of software development (e.g., agile, continuous integration, DevOps) and any inefficiency is considered unaffordable time waster. Such a fast pace causes challenges for architectural threat analysis. Leading techniques for threat analysis, like STRIDE, have the advantage of being systematic. However, they are not equipped to discern between important and less critical threats, while the threats are being discovered. Consequently, many threats are discarded at a later time, when their risk value is assessed. An alternative technique, called eSTRIDE, promises to remove these inefficiencies by focusing the analysis on the critical parts of the architecture. Yet, no empirical evidence exists about the actual effect of trading off systematicity, for a more focused attention on high-priority threats. This paper contributes with an empirical study comparing these two approaches in the context of two industrial case studies. We found that the two approaches yield the same number of security threats during a given time frame. However, participants using eSTRIDE found twice as many high-priority threats. The underlying analysis procedures cause similarities and differences in the execution. In addition, security expertise has an effect (albeit small) on the quality of analysis outcomes and execution.
Design and Implementation of Novel BRISI Lightweight Cipher for Resource Constrained Devices

The Internet of Things (IoT) and cyber-physical systems (CPS) has grown exponentially over the recent years, has motivated the development and deployment of the low resource devices for a wide range of applications in the IoT. Many such resource constrained devices are deployed to match the heterogeneous application requirements of IoT and CPS systems, wherein privacy and security have emerged, as the most difficult challenges, as the constrained devices are not been designed to have security features. This paper presents a lightweight cipher, based on ARX (Addition-Modulo, Rotation and XOR) operations, Fiestel structure, an amalgamation of BRIGHT and SIMON structure, hence the name BRISI. The cipher encrypts 32-bit plaintext using 64-bit key. The software implementation is performed using MATLAB tool and it fulfils the Avalanche criterion, Key-sensitivity, correlation coefficient, entropy and histogram. The proposed design is simulated using Xilinx Vivado and is implemented on Nexys-4 DDR Artix-7 and Basys-3 Artix-7 FPGA family and is evaluated for (LUT and register) power and timing
A Reconfigurable Multithreaded Accelerator for Recurrent Neural Networks

Recurrent Neural Network (RNN) is a key technology for sequential applications which require efficient and realtime implementations. Despite its popularity, efficient acceleration for RNN inference is challenging due to its recurrent nature and data dependencies. This paper proposes a multi-threaded neural processing unit (NPU) for RNN/LSTM inferences to increase processing abilities and quality of service of cloud-based NPUs by improving their hardware utilization. Besides, a custom coarse-grained multi-threaded LSTM (CGMT-LSTM) hardware architecture is introduced, which switches tasks among threads when LSTM computational kernels meet data hazard. These logical NPUs share nearly all resources of the physical NPU. When one logical NPU is stalled, another one can make progress. These optimizations improve the exploitation of parallelism to increase hardware utilization and enhance system throughput. Evaluation results show that a dual-threaded CGMT-LSTM NPU gains 27% more performance while only has 3.8% more area than a single-threaded one using a Stratix 10 FPGA. When compared with an implementation on the Tesla V100 GPU, our novel hardware architecture is 6.62 times faster and 15.88 times higher power efficiency, which demonstrates that our approach contributes to high performance energy-efficient FPGA-based multi-LSTM inference systems.
On the Usage of Generative Models for Network Anomaly Detection in Multivariate Time-Series

Despite the many attempts and approaches for anomaly de- tection explored over the years, the automatic detection of rare events in data communication networks remains a com- plex problem. In this paper we introduce Net-GAN, a novel approach to network anomaly detection in time-series, us- ing recurrent neural networks (RNNs) and generative ad- versarial networks (GAN). Different from the state of the art, which traditionally focuses on univariate measurements, Net-GAN detects anomalies in multivariate time-series, ex- ploiting temporal dependencies through RNNs. Net-GAN discovers the underlying distribution of the baseline, multi- variate data, without making any assumptions on its nature, offering a powerful approach to detect anomalies in com- plex, difficult to model network monitoring data. We further exploit the concepts behind generative models to conceive Net-VAE, a complementary approach to Net-GAN for net- work anomaly detection, based on variational auto-encoders (VAE). We evaluate Net-GAN and Net-VAE in different monitoring scenarios, including anomaly detection in IoT sensor data, and intrusion detection in network measure- ments. Generative models represent a promising approach for network anomaly detection, especially when considering the complexity and ever-growing number of time-series to monitor in operational networks.
Understanding Narrative Linearity for Telling Expressive Time-Oriented Stories

Creating expressive narrative visualization often requires choosing a well-planned narrative order that invites the audience in. The narrative can either follow the linear order of story events (chronology), or deviate from linearity (anachronies). While evidence exists that anachronies in novels and films can enhance story expressiveness, little is known about how they can be incorporated into narrative visualization. To bridge this gap, this work introduces the idea of narrative linearity to visualization and investigates how different narrative orders affect the expressiveness of time-oriented stories. First, we conducted preliminary interviews with seven experts to clarify the motivations and challenges of manipulating narrative linearity in time-oriented stories. Then, we analyzed a corpus of 80 time-oriented stories and identified six most salient patterns of narrative orders. Next, we conducted a crowdsourcing study with 221 participants. Results indicated that anachronies have the potential to make time-oriented stories more expressive without hindering comprehensibility.
t-k-means: A ROBUST AND STABLE k-means VARIANT

k-means algorithm is one of the most classical clustering methods, which has been widely and successfully used in signal processing. However, due to the thin-tailed property of the Gaussian distribution, k-means algorithm suffers from relatively poor performance on the dataset containing heavy-tailed data or outliers. Besides, standard k-means algorithm also has relatively weak stability, i.e. its results have a large variance, which reduces its credibility. In this paper, we propose a robust and stable k-means variant, dubbed the t-k-means, as well as its fast version to alleviate those problems. Theoretically, we derive the t-k-means and analyze its robustness and stability from the aspect of the loss function and the expression of the clustering center, respectively. Extensive experiments are also conducted, which verify the effectiveness and efficiency of the proposed method. The code for reproducing main results is available at https://github.com/THUYimingLi/t-k-means.
Decentralized Digital-Asset Exchanges: Issues and Evaluation

Recently, Decentralized Digital-Asset Exchanges (DDAE) have received significant attention. This paper proposes six criteria to evaluate DDAE based on the Principles of Financial Market Infrastructure (PFMI), evaluate current DDAE systems with respect to two main protocols: inter-chain and exchange protocols. This paper introduces four inter-chain protocols and six exchange protocols, summarizes these protocols into several technical models and analyzes their characteristics. These analyses provide technical reference for the DDAE design in the future.
Guidelines for Developing the VR Program in Residential Aged Care: A Preliminary Study from Staff Members’ Perspective

Despite there being evident benefits of using virtual reality (VR) in aged care, it is not yet widely used in residential aged care homes. One factor that may contribute to this is the willingness of staff to use VR as part of the social program offered in aged care homes. Therefore, we need to understand staff perceptions of VR programs, especially suggestions for improvement. In an analysis of responses from 10 staff working in residential aged care (also known as nursing homes), we found that staff have concerns about the suitability of VR for older people with cognitive impairments and mobility restrictions. Many older adults living in aged care have these conditions. Our findings suggest that providing staff with training on how to facilitate various kinds of valuable VR experiences and providing a clear picture of its benefits and drawbacks will help to make it suitable for people living in aged care. Furthermore, there should be greater investment in technological infrastructure and co-design of VR in aged care.
Mobile Edge Resource optimization for Multiplayer Interactive Virtual Reality Game

Edge computing has been regarded as an efficient approach to achieve the multiplayer interactive virtual reality (VR) game over wireless networks, where the game scenes can be rendered at the edge computing server and then the real-time video frames can be transmitted to the players. In order to ensure the fairness of the interacting players, we allocate the edge computing and wireless bandwidth resources for minimizing the average inter-player delay among different players. The proposed programming is under the constraints of the absolute delay requirements, the maximum frame per second (FPS) demands, the total bandwidth and rendering resources limits. The influence of prediction and pre-rendering field of views (FOVs) is also considered in the optimization problem to model the interaction among players. To tackle the non-convex problem efficiently, a sub-optimal algorithm which convert the original problem into several convex subproblems to optimize iteratively is designed. Finally, numerical results verify the proposed algorithm can reduce the average interplayer delay significantly with lower complexity, and it also reveals the impact of the content sizes and the channel state conditions on the edge resource allocation.
INFER: INterFerence-aware Estimation of Runtime for Concurrent CNN Execution on DPUs

Deep Learning Processor Unit (DPU) from XILINX is among the numerous accelerators that have been proposed to speed up the execution of Convolutional Neural Networks (CNNs) on embedded platforms. DPUs are available in different configurable sizes and can execute any given CNN. Neural network researchers are also rapidly bringing out newer CNN algorithms with improved performance (typically higher prediction accuracy) with a trade-off in size or energy consumption for embedded applications. To enable quick evaluation of choices among evolving CNN algorithms and accelerator configurations, we propose INFER (INterFerence-aware Estimation of Runtime). INFER is a framework to estimate the execution time of any CNN on a given size of DPU without actual implementation. Further, current FPGA platforms are capable of implementing multiple DPUs whereas many applications consist of multiple sub-tasks with each requiring separate and/or different CNNs. In such scenarios of concurrent use of multiple DPUs on an FPGA, INFER is also capable of estimating the additional time taken for execution due to the sharing of memory bandwidth. Our evaluation on various mixes of 16 standard CNNs and eight configurations of DPU shows that INFER has an average prediction error of 6.6%, which can be useful for design space exploration as well as scheduling in multi-DPU platforms.
Detection defense against adversarial attacks with saliency map

It is well established that neural networks are vulnerable to adversarial examples, which are almost imperceptible on human vision and can cause the deep models misbehave. Such phenomenon may lead to severely inestimable consequences in the safety and security critical applications. Existing defenses are trend to harden the robustness of models against adversarial attacks, for example, adversarial training technology. However, these are usually intractable to implement due to the high cost of retraining and the cumbersome operations of altering the model architecture or parameters. In this paper, we discuss the saliency map method from the view of enhancing model interpretability, it is similar to introducing the mechanism of the attention to the model, so as to comprehend the progress of object identification by the deep networks. We then propose a novel method combined with additional noises and utilize the inconsistency strategy to detect adversarial examples. Our experimental results of some representative adversarial attacks on common data sets including ImageNet and popular models show that our method can detect all the attacks with high detection success rate effectively. We compare it with the existing state-of-the-art technique, and the experiments indicate that our method is more general.
One-Shot Manipulation of Entanglement for Quantum Channels

We show that the dynamic resource theory of quantum entanglement can be formulated using the superchannel theory. In this formulation, we identify the separable channels and the class of free superchannels that preserve channel separability as free resources, and choose the swap channels as dynamic entanglement golden units. Our first result is that the one-shot dynamic entanglement cost of a bipartite quantum channel under the free superchannels is bounded by the standard log-robustness of channels. The one-shot distillable dynamic entanglement of a bipartite quantum channel under the free superchannels is found to be bounded by a resource monotone that we construct from the hypothesis-testing relative entropy of channels with minimization over separable channels. We also address the one-shot catalytic dynamic entanglement cost of a bipartite quantum channel under a larger class of free superchannels that could generate the dynamic entanglement which is asymptotically negligible; it is bounded by the generalized log-robustness of channels.
Optimizing Spatiotemporal Feature Learning in 3D Convolutional Neural Networks With Pooling Blocks

Image data contain spatial information only, thus making two-dimensional (2D) Convolutional Neural Networks (CNN) ideal for solving image classification problems. On the other hand, video data contain both spatial and temporal information that must be simultaneously analyzed to solve action recognition problems. 3D CNNs are successfully used for these tasks, but they suffer from their extensive inherent parameter set. Increasing the network's depth, as is common among 2D CNNs, and hence increasing the number of trainable parameters does not provide a good trade-off between accuracy and complexity of the 3D CNN. In this work, we propose Pooling Block (PB) as an enhanced pooling operation for optimizing action recognition by 3D CNNs. PB comprises three kernels of different sizes. The three kernels simultaneously sub-sample feature maps, and the outputs are concatenated into a single output vector. We compare our approach with three benchmark 3D CNNs (C3D, I3D, and Asymmetric 3D CNN) and three datasets (HMDB51, UCF101, and Kinetics 400). Our PB method yields significant improvement in 3D CNN performance with a comparatively small increase in the number of trainable parameters. We further investigate (1) the effect of video frame dimension and (2) the effect of the number of video frames on the performance of 3D CNNs using C3D as the benchmark.
Collaborative filtering via factorized neural networks

Plenty of deep learning models have been extensively studied over recent years to meet the needs of personalized services. In order to improve the recommendation quality, neural networks are devised more and more complex. As a result, a large number of parameters emerge in the network which significantly increases the burden of computation and storage. When these deep learning models run on the resource-limited devices, their applications are immensely restrained. In consequence, to reduce parameters in the network is of great importance in the aspect of application. In this paper, the factorized neural network model (FNN) is proposed to solve such a problem. The conventional matrix factorization method is used to decompose the weight matrix of each layer in the fully-connected network and the regularization is applied to place constraints on parameters. By our method, more than 95% of parameters in the weighted matrices can be reduced and there is a less than 1% accuracy drop.
Zoom-CAM: Generating Fine-grained Pixel Annotations from Image Labels

Current weakly supervised object localization and segmentation rely on class-discriminative visualization techniques to generate pseudo-labels for pixel-level training. Such visualization methods, including class activation mapping (CAM) and Grad-CAM, use only the deepest, lowest resolution convolutional layer, missing all information in intermediate layers. We propose Zoom-CAM: going beyond the last lowest resolution layer by integrating the importance maps over all activations in intermediate layers. Zoom-CAM captures fine-grained small-scale objects for various discriminative class instances, which are commonly missed by the baseline visualization methods. We focus on generating pixel-level pseudo-labels from class labels. The quality of our pseudo-labels evaluated on the ImageNet localization task exhibits more than 2.8% improvement on top-1 error. For weakly supervised semantic segmentation our generated pseudo-labels improve a state of the art model by 1.1%.
Eye-Blink Detection System for Virtual Keyboard

Human Computer Interaction (HCI) focuses on the interface and interaction between people and computers. The main goal of the HCI is to design a machinery that lets people interact with computers in a novel way. One of the most significant methods people use to interact is the eye movement and eye blinking, especially for people with physical disabilities. This paper presents an on-screen typing method based on eye blinking by using a virtual keyboard. The two main components are image processing to detect the eyes, and eye-blinking. A facial image is captured by the computer's camera then used to determine the eye location and size. This is done based on the famous “68 points” of the face detection approach. An eye blink is used in this system to enter a character similar to when a user presses the “Enter” button on a keyboard. This project is developed as a use case in a Saudi university. The methodology, the implementation, and results are described in details in the paper.
Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation

Inspired by the human ability to infer emotions from body language, we propose an automated framework for body language based emotion recognition starting from regular RGB videos. In collaboration with psychologists, we further extend the framework for psychiatric symptom prediction. Because a specific application domain of the proposed framework may only supply a limited amount of data, the framework is designed to work on a small training set and possess a good transferability. The proposed system in the first stage generates sequences of body language predictions based on human poses estimated from input videos. In the second stage, the predicted sequences are fed into a temporal network for emotion interpretation and psychiatric symptom prediction. We first validate the accuracy and transferability of the proposed body language recognition method on several public action recognition datasets. We then evaluate the framework on a proposed URMC dataset, which consists of conversations between a standardized patient and a behavioral health professional, along with expert annotations of body language, emotions, and potential psychiatric symptoms. The proposed framework outperforms other methods on the URMC dataset.
Refraction Compensation in Non-Destructive Testing

SAR radar imaging methods are presented, which are able to treat highly refractive fibre composite materials. The methods are based on a previous surface extraction and a subsequent refraction compensation relying on the calculation of a corresponding correction factor. Due to an efficient implementation we achieve a computational effort comparable to the standard back-projection algorithm. We demonstrate the method with examples from simulation and fibre composite materials.
Towards More Transactional Voice Assistants: Investigating the Potential for a Multimodal Voice-Activated Indoor Navigation Assistant for Blind and Sighted Travelers

Voice assistants (VAs) – like Amazon Alexa or Siri – offer hands-/eyes-free interactions that are beneficial to a range of users, including individuals who are blind, to fulfill tasks that are otherwise difficult or inaccessible. While these interfaces model conversational interactions to achieve simple tasks, there have been recent calls for VAs that model more transactional interactions for a wider range of complex activities. In this study, we explored the extension of VAs’ capabilities in the context of indoor navigation through mixed-ability focus groups with blind and sighted airport travelers. We found high overlap in the difficulties encountered by blind and sighted travelers, as well as shared interest in a voice-activated travel assistant to improve travel experiences. Leveraging user-elicited recommendations, we present interaction design examples that showcase customization of different and multiple modalities, which collectively demonstrate how VAs can more broadly achieve transactional interactions in complex task scenarios.