Reviewerly/exHarmony huggingface dataset:
A Normalized Disparity Loss for Stereo Matching Networks

Stereo vision is an essential component for lots of robot vision applications. The performance of stereo matching has been greatly improved recently thanks to deep learning technologies. However, most of the current deep networks directly use naive disparity loss to train the network, regardless of the inherent unbalance of its supervision to pixels with different depths. In this letter, we propose a normalized disparity loss to solve this problem. The new loss is constructed as the common disparity loss divided by a fitted cost function. This function is responsible for imitating the variance of the common loss along the disparity dimension. Range restriction and gradual updating strategies are further designed to ensure the robustness of the loss function during training. Experimental results on multiple public datasets show that our method can be directly applied to most state-of-the-art networks and effectively improve their disparity performance.
IFCIL: An Information Flow Configuration Language for SELinux

Security Enhanced Linux (SELinux) is a security architecture for Linux implementing mandatory access control. It has been used in numerous security-critical contexts ranging from servers to mobile devices. But this is challenging as SELinux security policies are difficult to write, understand, and maintain. Recently, the intermediate language CIL was introduced to foster the development of high-level policy languages and to write structured configurations. However, CIL lacks mechanisms for ensuring that the resulting configurations obey desired information flow policies. To remedy this, we propose IFCIL, a backward compatible extension of CIL for specifying fine-grained information flow requirements for CIL configurations. Using IFCIL, administrators can express, e.g., confidentiality, integrity, and non-interference properties. We also provide a tool to statically verify these requirements.
Deep Feature Compression with Collaborative Coding of Image Texture

In this paper, we propose a coding scheme for the deep intermediate feature and it is implemented with the collaborative compression of image texture. More specifically, we separately compress the feature and texture of the image to form two data layers. The first one is the intermediate feature layer and the second one is the texture layer. The texture layer can provide an image for users and the feature layer can be used to implement the computer vision (CV) task. With our proposed deep reconstruction network (RecNet), the texture and features cooperate to achieve a high-quality visual output as well as a high-efficiency CV task. The experimental results demonstrate the excellent performance by using our proposed method to compress the deep features.
Automated Image Captioning with Multi-layer Gated Recurrent Unit

Describing the semantic content of an image via natural language, known as image captioning, has recently attracted substantial interest in computer vision and language processing communities. Current image captioning approaches are mainly based on an encoder-decoder framework in which visual information is extracted by an image encoder and captions are generated by a text decoder, using convolution neural net-works (CNN) and recurrent neural networks (RNN), respectively. Although this framework is promising for image captioning, it has limitations in utilizing the encoded visual information for generating grammatically and semantically correct captions in the RNN decoder. More specifically, the RNN decoder is ineffective in using the contextual information from the encoded data due to its limited ability in capturing long-term complex dependencies. Inspired by the advantage of gated recurrent unit (GRU), in this paper, we propose an extension of conventional RNN by introducing a multi-layer GRU that modulates the most relevant information inside the unit to enhance the semantic coherence of captions. Experimental results on the MSCOCO dataset show the superiority of our proposed approach over the state-of-the-art approaches in several performance metrics.
Merry Go Round: Rotate a Frame and Fool a DNN

A large proportion of videos captured today are first person videos shot from wearable cameras. Similar to other computer vision tasks, Deep Neural Networks (DNNs) are the workhorse for most state-of-the-art (SOTA) egocentric vision techniques. On the other hand DNNs are known to be susceptible to Adversarial Attacks (AAs) which add imperceptible noise to the input. Both black-box, as well as white-box attacks on image as well as video analysis tasks have been shown. We observe that most AA techniques basically add intensity perturbation to an image. Even for videos, the same process is essentially repeated for each frame independently. We note that definition of imperceptibility used for images may not be applicable for videos, where a small intensity change happening randomly in two consecutive frames may still be perceptible. In this paper we make a key novel suggestion to use perturbation in optical flow to carry out AAs on a video analysis system. Such perturbation is especially useful for egocentric videos, because there is lot of shake in the egocentric videos anyways, and adding a little more, keeps it highly imperceptible. In general our idea can be seen as adding structured, parametric noise as the adversarial perturbation. Our implementation of the idea by adding 3D rotations to the frames, reveal that using our technique, one can mount a black-box AA on an egocentric activity detection system in one-third of the queries compared to the SOTA AA technique.
Progressive Multitask Learning Network for Online Chinese Signature Segmentation and Recognition

Recently, the booming of electronic devices has revolutionized the way we sign in our daily life. The sudden surge of online Chinese signatures calls for need online Chinese signature segmentation as prerequisite for downstream tasks such as building up database of Chinese characters and verifying signatures based upon individual characters. However, common approaches deriving from over-segmentation do not apply well to Chinese signatures, which have little linguistic meanings but instead have flourish, artistic styles, composite structures or even word overlaps. To cope with those difficulties, this paper exploits the benefits of signature recognition to boost segmentation performance and proposes a progressive multitask learning network (PMLNet) for online Chinese signature segmentation and recognition. PMLNet consists of a dual channel stroke feature extraction block (DSF-Block), a stacked transformer encoder block (STE-Block) and a progressive multitask learning block (PML-Block). DSF-block is used to extract stroke-wise spatial features and semantic features through dual channels; STE-block is used to model long-range dependencies among different strokes and enhance their feature representations; PML-block is used to branch interactively and progressively the signature's final segmentation and recognition. Specifically, we introduce a progressive learning strategy in PML-block to fine-tune segmentation and recognition results by fully leveraging their reciprocal relationship. The experiment results on our private database show that PMLNet achieves 8.77% higher accurate rate (AR), 9.15% higher correct rate (CR) and 9.93% higher sample-level segmentation accurate rate ( $$\mathrm {ACC_{seg}}$$ ) than SOTAs.
Research on Node Similarity in Attention Flow Network Based on GAT

The traditional mathematical model can not give the critical value of similarity in the study of node similarity in attention flow network, which makes its practical application difficult; For featureless networks, traditional deep learning methods are difficult to train. To solve the above two problems, A-NFN algorithm is proposed by using graph attention network GAT and graph embedding algorithm Node2vec. Firstly, the graph embedding algorithm Node2vec is used to perform biased random walk, and the nodes in the network are represented as high-dimensional vectors; Secondly, the attention flow data is constructed into a graph by using NetworkX toolkit; Finally, taking the vector representation of all nodes and the attention flow network structure as the input of the algorithm, the similarity between each pair of nodes is calculated, and the critical value of similarity is defined. Experiments show that the A-NFN algorithm proposed in this paper is compared with the popular graph neural network algorithm GCN in recent years, and the result accuracy is improved by 1.6 percentage points on average.
Explainable Artificial Intelligence (XAI) User Interface Design for Solving a Rubik’s Cube

Explainable Artificial Intelligence (XAI) aims to bridge the understanding between decisions made by an AI interface and the user interacting with the AI. When the goal of the AI is to teach the user how to solve a problem, user-friendly explanations of the AI’s decisions must be given to the user so they can learn how to replicate the process for themselves. This paper describes the process of defining explanations in the context of a collaborative AI platform, ALLURE, which teaches the user how to solve a Rubik’s Cube. A macro-action in our collaborative AI algorithm refers to a set of moves that takes the cube from initial state to goal state - a process that was not transparent nor accessible when we revealed back-end logic to the front-end for user engagement. By providing macro-action explanations to the user in a chatbot as well as a visual representation of the moves being performed on a virtual Rubik’s Cube, we created an XAI interface to engage and guide the user through a subset of the solutions that can later be applied to the remaining solutions of the AI. After initial usability testing, our study provides some useful and practical XAI user interface design implications.
Emulating Your eXtended World: An Emulation Environment for XR App Development

Application development for eXtended Reality (XR) platforms are similar in many ways with the development of applications for traditional computing environments (e.g., PCs and smartphones). However, XR applications differ in the fact that they need to consider the increased dimensions of user input coming from various peripherals such as head motion, gamepad controls and hand gestures. Such diversity (and complexity) in input make it difficult for XR application programs to be tested properly during the development process, and complicates the development by introducing additional levels of “physical-device” testing overhead. In this work, based on a preliminary survey conducted with 10 XR application developers, we identify the core obstacles they experience in the application testing process and present the design of XREmul. To ease an XR application's systematic and functional testing overhead, XREmul provides a set of input abstraction APIs for the application to easily access data streams coming from heterogeneous input sources, and offers a record-and-replay feature for various input modalities. This allows the application developers to easily reproduce uncommon errors generated from user inputs and can reduce the testing time for XR application functionalities. We conduct a user study to quantify the impact of testing time reduction, and also collect qualitative feedback from professional XR application developers using XREmul as a prototype system for offering an effective emulation environment for XR application development.
Proprioception-Driven Wearer Pose Estimation for Egocentric Video

Perceiving proprioception from egocentric video to estimate 3D wearer pose is an attention-grabbing visual task. Yet the invisibility of wearer body and the complex motion modality bring challenges to perceive self-motion from the human visual span. In this work, a data processing framework is designed to convert a raw egocentric video stream into a 3D wearer pose sequence. Critically, a generic and lightweight Self-Perception Excitation (SPE) module is proposed to enhance motion modeling and calibrate spatial correlation in the temporal dimension. Employing ResNet50 embedded with SPE module as a backbone, a two-stream architecture for proprioception representation pipeline is proposed to learn proprioception behaviors from RGB streams and motion streams. Then, the proprioception is incorporated as an additional key control signal in a deep reinforcement learning (DeepRL) based motion imitation policy for estimating the multi-modal wearer pose. By considering proprioception, we indicate for the first time, it is possible to understand the self from an egocentric view and further translate it into a higher understanding of wearer motion. The experimental results demonstrate that the proposed framework is able to outperform the state-of-the-art methods by a large margin on the MoCup dataset and produce highly identifiable proprioception behaviors.
OneFace: One Threshold for All

Face recognition (FR) has witnessed remarkable progress with the surge of deep learning. Current FR evaluation protocols usually adopt different thresholds to calculate the True Accept Rate (TAR) under a pre-defined False Accept Rate (FAR) for different datasets. However, in practice, when the FR model is deployed on industry systems (e.g., hardware devices), only one fixed threshold is adopted for all scenarios to distinguish whether a face image pair belongs to the same identity. Therefore, current evaluation protocols using different thresholds for different datasets are not fully compatible with the practical evaluation scenarios with one fixed threshold, and it is critical to measure the performance of FR models by using one threshold for all datasets. In this paper, we rethink the limitations of existing evaluation protocols for FR and propose to evaluate the performance of FR models from a new perspective. Specifically, in our OneFace, we first propose the One-Threshold-for-All (OTA) evaluation protocol for FR, which utilizes one fixed threshold called as Calibration Threshold to measure the performance on different datasets. Then, to improve the performance of FR models under the OTA protocol, we propose the Threshold Consistency Penalty (TCP) to improve the consistency of the thresholds among multiple domains, which includes Implicit Domain Division (IDD) as well as Calibration and Domain Thresholds Estimation (CDTE). Extensive experimental results demonstrate the effectiveness of our method for FR.
Remote Sensing Colorization Based on Bidirectional Macro-Micro Adaptive Enhancement Network

The demand for re-colorization of remote sensing images is urgent since image quality is extremely deteriorated by haze or other noises occurring in the atmospheric layer. The most challenging issue is to restore the color information with respect to preserving spatial consistency as well as to obtain object salience in context with extremely imbalanced space structure, where the former requires learning stable macroscopic semantics while the latter needs to recover microscopic pixels. In this paper, we propose a Bidirectional Macro-Micro Adaptive Enhancement (BMMAEnet) framework by adopting three modules, i.e., the Downward Micro Enhancement (DME) module, the Upward Adaptive Macro Enhancement(UAME) module, and Macro-Micro Balance (MMB) module. Firstly, the DME module is designed by adding micro details as well as suppressing macro context during the multi-branch downsampling process to supplement missing pixel details. Secondly, UAME is proposed by adaptive selecting proper level of features during multi-branch upsampling process to strengthen macro semantic constraints. In addition, MMB is designed by embedding attention-guided local details and global semantics into the decoding features to balance micro and macro information within each branch. Comprehensive comparison and ablation experiments are implemented and verify the proposed method performs overpass SOTA methods not only in pixel color value restoration performance but also in human perceptive understanding.
MT4UML: Metamorphic Testing for Unsupervised Machine Learning

One of the advantages of using unsupervised machine learning algorithms is that they don't need labeled data; thus, ultimately saving higher labeling costs for an organization. However, the computational complexity and large input space put these algorithms into the category of non-testable programs, which also suffer from the oracle problem. One popular testing approach, borrowed from the Software Engineering (SE) domain is the Metamorphic Testing (MT) technique that has been proven to be an effective approach in alleviating the oracle problem in testing such non-testable programs. We take advantage of this MT approach to make some insightful contributions that include: i) proposing a broader set of 22 Metamorphic Relations (MRs) for assessing the behavior of the K-means clustering algorithm (a prototype-based approach) and the Agglomerative clustering algorithm (a hierarchy-based approach), provided by the leading scikit-learn Python library, ii) providing a detailed analysis/reasoning to show how the proposed MRs can be used to target both the verification and validation aspects of testing the clustering algorithms under investigation, and iii) showing that verification of MRs using multiple criteria is more beneficial than relying on using just a single criterion (i.e., clusters assigned). We further applied the proposed approach to test an open source customer segmentation application and the results obtained show that, i) 10 MRs have been violated for both the K-means and Agglomerative clustering algorithms, and ii) in comparison to K-means, the Agglomerative clustering algorithm is highly susceptible to small changes in inputs and may not offer a better alternative to scenarios captured by the violated MRs.
High Speed ASIC Architectures for Aggregate Signature over BLS12-381

BLS signature is an elliptic curve cryptography with an attractive feature that signatures can be aggregated and shortened. We have designed two ASIC architectures for hashing to the elliptic curve and pairing to minimize the latency. Also, the designs are optimized for BLS12-381, a relatively new and safe curve.
Comprehensive overview on the deployment of machine learning, deep learning, reinforcement learning algorithms in Selfish mining attack in blockchain

Blockchain, a disruptive technology, has many applications in the domain of Finance, banking, real estate, insurance, supply chain, gaming industry with much more plethora of applications in near future. In spite of the decentralized, distributed, transparent, tamper-proof, data- provenance nature of the blockchain, it is subject to a lot of security attacks such as forking attacks and block withholding attacks. One such attack under the forking attack is selfish mining, which targets the reward distribution and also the difficulty adjustment algorithms(DAA). An exhaustive surve is done on the existing approaches to detect selfish mining and also on the profitability of selfish mining attacks. This survey is organized particularly around the aspects of selection or exploration of the shortest branch of the blockchain when a fork occurs. We aim to identify the implications of selecting the shorter branch of the fork in the blockchain, especially after 2016 blocks, where a difficulty adjustment occurs. Our survey focuses on the deployment of machine learning and deep learning, reinforcement methods on mitigating the selfish mining attacks in the blockchain.
Data-driven Causal Association Discovery in Manufacturing Industries

With the development of smart sensors, information storage, processing technology and computer performance, large amounts of operating data collected from production process provide opportunities as well as challenges in remaining useful life (RUL) estimation. On one hand, data-driven analysis approaches are experiencing a fast development. On the other hand, the collected variables may be redundant, noisy and high-dimensional for RUL. Thus, data dimension reduction is applied for eliminating useless information. Different from correlation-based methods, causal inference methods can obtain reliable models reflecting causal relationships among interesting variables. Thus, the latter is more suitable in data dimension reduction. In this study, we use PCMCI+, a causal discovery method based on graph model, that handles both lagged and contemporaneous relationships in multi-variable time series. We validate this method on time series data extracted directly from a medium frequency quenching machine. The obtained results confirm that PCMCI+ is able to recognize causal associations among various sensor data. For instance, variables in the same process have relatively larger causal relationships than those in different processes.
Dense-FCN: A Deep Learning Approach for Weather Radar Beam Blockage Correction

The weather radar will receive a lot of non-meteorological echo information during the body scan process, such as: object echoes, co-wave interference echoes, airplanes, flocks of birds, etc. These non-meteorological echoes will cause pollution to normal meteorological echoes, thereby affecting the accuracy of the output product. The traditional weather radar beam blocking correction method is to interpolate and fill according to the adjacent radial data, but this method cannot make full use of the deep laws of radar data and is relatively limited. In this paper, the weather radar beam blocking correction problem is regarded as an image completion problem, a fully convolutional neural network with dense connection is designed, and the multiclass cross entropy loss function is used to optimize the model. The model proposed in this paper has achieved good performance in the evaluation indicators in the field of meteorology and image completion.
DSegAN: A Deep Light-weight Segmentation-based Attention Network for Image Restoration

Feature attention is a technique used in deep neural networks to provide a discriminative processing of the various regions in an image based on their significance for enhancing the image restoration performance. In this paper, we develop a novel image restoration network, in which the feature maps extracted by the network are recalibrated using a pixel-wise feature attention and the recalibration process is guided by the structural and textural information of the image resulting from the Otsu’s method for its segmentation. It is shown that using this segmentation guidance strategy for recalibrating feature maps is indeed helpful in enhancing the quality of the restored images. The proposed image restoration network outperforms the state-of-the-art light-weight image restoration networks on benchmark datasets.