OpenRaiser/Intern-Atlas huggingface dataset:
VAC2: Visual Analysis of Combined Causality in Event Sequences
Venue: arXiv cs.CV 2022
Year: 2022

Identifying causality behind complex systems plays a significant role in different domains, such as decision making, policy implementations, and management recommendations. However, existing causality studies on temporal event sequences data mainly focus on individual causal discovery, which is incapable of exploiting combined causality. To fill the absence of combined causes discovery on temporal event sequence data,eliminating and recruiting principles are defined to balance the effectiveness and controllability on cause combinations. We also leverage the Granger causality algorithm based on the reactive point processes to describe impelling or inhibiting behavior patterns among entities. In addition, we design an informative and aesthetic visual metaphor of "electrocircuit" to encode aggregated causality for ensuring our causality visualization is non-overlapping and non-intersecting. Diverse sorting strategies and aggregation layout are also embedded into our parallel-based, directed and weighted hypergraph for illustrating combined causality. Our developed combined causality visual analysis system can help users effectively explore combined causes as well as an individual cause. This interactive system supports multi-level causality exploration with diverse ordering strategies and a focus and context technique to help users obtain different levels of information abstraction. The usefulness and effectiveness of the system are further evaluated by conducting a pilot user study and two case studies on event sequence data.
Precise Affordance Annotation for Egocentric Action Video Datasets
Venue: arXiv cs.CV 2022
Year: 2022

Object affordance is an important concept in human-object interaction, providing information on action possibilities based on human motor capacity and objects' physical property thus benefiting tasks such as action anticipation and robot imitation learning. However, existing datasets often: 1) mix up affordance with object functionality; 2) confuse affordance with goal-related action; and 3) ignore human motor capacity. This paper proposes an efficient annotation scheme to address these issues by combining goal-irrelevant motor actions and grasp types as affordance labels and introducing the concept of mechanical action to represent the action possibilities between two objects. We provide new annotations by applying this scheme to the EPIC-KITCHENS dataset and test our annotation with tasks such as affordance recognition. We qualitatively verify that models trained with our annotation can distinguish affordance and mechanical actions.
Testing Ocean Software with Metamorphic Testing
Venue: arXiv cs.SE 2022
Year: 2022

Advancing ocean science has a significant impact to the development of the world, from operating a safe navigation for vessels to maintaining a healthy and diverse ocean ecosystem. Various ocean software systems have been extensively adopted for different purposes, for instance, predicting hourly sea level elevation across shorelines, simulating large-scale ocean circulations, as well as integrating into Earth system models for weather forecasts and climate projections. Regardless of their significance, guaranteeing the trustworthiness of ocean software and modelling systems is a long-standing challenge. The testing of ocean software suffers a lot from the so-called oracle problem, which refers to the absence of test oracles mainly due to the nonlinear interactions of multiple physical variables and the high complexity in computation. In the ocean, observed tidal signals are distorted by non-deterministic physical variables, hindering us from knowing the "true" astronomical tidal constituents existing in the timeseries. In this paper, we present how to test tidal analysis and prediction (TAP) software based on metamorphic testing (MT), a simple yet effective testing approach to the oracle problem. In particular, we construct metamorphic relations from the periodic property of astronomical tide, and then use them to successfully detect a real-life defect in an open-source TAP software. We also conduct a series of experiments to further demonstrate the applicability and effectiveness of MT in the testing of TAP software. Our study not only justifies the potential of MT in testing more complex ocean software and modelling systems, but also can be expanded to assess and improve the quality of a broader range of scientific simulation software systems.
Discovery and density estimation of latent confounders in Bayesian networks with evidence lower bound
Venue: arXiv cs.LG 2022
Year: 2022

Discovering and parameterising latent confounders represent important and challenging problems in causal structure learning and density estimation respectively. In this paper, we focus on both discovering and learning the distribution of latent confounders. This task requires solutions that come from different areas of statistics and machine learning. We combine elements of variational Bayesian methods, expectation-maximisation, hill-climbing search, and structure learning under the assumption of causal insufficiency. We propose two learning strategies; one that maximises model selection accuracy, and another that improves computational efficiency in exchange for minor reductions in accuracy. The former strategy is suitable for small networks and the latter for moderate size networks. Both learning strategies perform well relative to existing solutions.
A Review of Causality for Learning Algorithms in Medical Image Analysis
Venue: arXiv cs.CV 2022
Year: 2022

Medical image analysis is a vibrant research area that offers doctors and medical practitioners invaluable insight and the ability to accurately diagnose and monitor disease. Machine learning provides an additional boost for this area. However, machine learning for medical image analysis is particularly vulnerable to natural biases like domain shifts that affect algorithmic performance and robustness. In this paper we analyze machine learning for medical image analysis within the framework of Technology Readiness Levels and review how causal analysis methods can fill a gap when creating robust and adaptable medical image analysis algorithms. We review methods using causality in medical imaging AI/ML and find that causal analysis has the potential to mitigate critical problems for clinical translation but that uptake and clinical downstream research has been limited so far.
Soft-mask: Adaptive Substructure Extractions for Graph Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

For learning graph representations, not all detailed structures within a graph are relevant to the given graph tasks. Task-relevant structures can be $localized$ or $sparse$ which are only involved in subgraphs or characterized by the interactions of subgraphs (a hierarchical perspective). A graph neural network should be able to efficiently extract task-relevant structures and be invariant to irrelevant parts, which is challenging for general message passing GNNs. In this work, we propose to learn graph representations from a sequence of subgraphs of the original graph to better capture task-relevant substructures or hierarchical structures and skip $noisy$ parts. To this end, we design soft-mask GNN layer to extract desired subgraphs through the mask mechanism. The soft-mask is defined in a continuous space to maintain the differentiability and characterize the weights of different parts. Compared with existing subgraph or hierarchical representation learning methods and graph pooling operations, the soft-mask GNN layer is not limited by the fixed sample or drop ratio, and therefore is more flexible to extract subgraphs with arbitrary sizes. Extensive experiments on public graph benchmarks show that soft-mask mechanism brings performance improvements. And it also provides interpretability where visualizing the values of masks in each layer allows us to have an insight into the structures learned by the model.
Channel Estimation for Massive MIMO systems using Tensor Cores in GPU
Venue: arXiv cs.DC 2022
Year: 2022

For efficient use of Massive MIMO systems, fast and accurate channel estimation is very important. But the Large-scale antenna array presence requires high pilot overhead for high accuracy of estimation. Also, when used with software-based processing systems like CPUs and GPUs, high processing latency becomes a major issue. To reduce Pilot overhead, a Pilot transmission scheme in combination with PN Sequence correlation based channel estimation scheme is implemented. Then, to deal with the issue of high processing latency, Tensor Cores in Nvidia GPUs are used for computing the channel estimation. Experiments are performed by using Nvidia V100 GPU in the ORBIT Testbed to show the performance of the Pilot transmission scheme. By varying factors like PN sequence length, Channel Impulse Response length, number of multiplexed transmitters, and scale of MIMO, the accuracy and processing latency of Tensor Core implementation of the Channel Estimation is evaluated.
Improving the Adversarial Robustness of NLP Models by Information Bottleneck
Venue: arXiv cs.CL 2022
Year: 2022

Existing studies have demonstrated that adversarial examples can be directly attributed to the presence of non-robust features, which are highly predictive, but can be easily manipulated by adversaries to fool NLP models. In this study, we explore the feasibility of capturing task-specific robust features, while eliminating the non-robust ones by using the information bottleneck theory. Through extensive experiments, we show that the models trained with our information bottleneck-based method are able to achieve a significant improvement in robust accuracy, exceeding performances of all the previously reported defense methods while suffering almost no performance drop in clean accuracy on SST-2, AGNEWS and IMDB datasets.
A Simplified Un-Supervised Learning Based Approach for Ink Mismatch Detection in Handwritten Hyper-Spectral Document Images
Venue: arXiv cs.CV 2022
Year: 2022

Hyper-spectral imaging has become the latest trend in the field of optical imaging systems. Among various other applications, hyper-spectral imaging has been widely used for analysis of printed and handwritten documents. This paper proposes an efficient technique for estimating the number of different but visibly similar inks present in a Hyper spectral Document Image. Our approach is based on un-supervised learning and does not require any prior knowledge of the dataset. The algorithm was tested on the iVision HHID dataset and has achieved comparable results with the state of the algorithms present in the literature. This work can prove to be effective when employed during the early stages of forgery detection in Hyper-spectral Document Images.
Surround-View Cameras based Holistic Visual Perception for Automated Driving
Venue: arXiv cs.CV 2022
Year: 2022

The formation of eyes led to the big bang of evolution. The dynamics changed from a primitive organism waiting for the food to come into contact for eating food being sought after by visual sensors. The human eye is one of the most sophisticated developments of evolution, but it still has defects. Humans have evolved a biological perception algorithm capable of driving cars, operating machinery, piloting aircraft, and navigating ships over millions of years. Automating these capabilities for computers is critical for various applications, including self-driving cars, augmented reality, and architectural surveying. Near-field visual perception in the context of self-driving cars can perceive the environment in a range of $0-10$ meters and 360° coverage around the vehicle. It is a critical decision-making component in the development of safer automated driving. Recent advances in computer vision and deep learning, in conjunction with high-quality sensors such as cameras and LiDARs, have fueled mature visual perception solutions. Until now, far-field perception has been the primary focus. Another significant issue is the limited processing power available for developing real-time applications. Because of this bottleneck, there is frequently a trade-off between performance and run-time efficiency. We concentrate on the following issues in order to address them: 1) Developing near-field perception algorithms with high performance and low computational complexity for various visual perception tasks such as geometric and semantic tasks using convolutional neural networks. 2) Using Multi-Task Learning to overcome computational bottlenecks by sharing initial convolutional layers between tasks and developing optimization strategies that balance tasks.
RadNet: Incident Prediction in Spatio-Temporal Road Graph Networks Using Traffic Forecasting
Venue: arXiv cs.LG 2022
Year: 2022

Efficient and accurate incident prediction in spatio-temporal systems is critical to minimize service downtime and optimize performance. This work aims to utilize historic data to predict and diagnose incidents using spatio-temporal forecasting. We consider the specific use case of road traffic systems where incidents take the form of anomalous events, such as accidents or broken-down vehicles. To tackle this, we develop a neural model, called RadNet, which forecasts system parameters such as average vehicle speeds for a future timestep. As such systems largely follow daily or weekly periodicity, we compare RadNet's predictions against historical averages to label incidents. Unlike prior work, RadNet infers spatial and temporal trends in both permutations, finally combining the dense representations before forecasting. This facilitates informed inference and more accurate incident detection. Experiments with two publicly available and a new road traffic dataset demonstrate that the proposed model gives up to 8% higher prediction F1 scores compared to the state-of-the-art methods.
VeriBlock: A Blockchain-Based Verifiable Trust Management Architecture with Provable Interactions
Venue: arXiv cs.DC 2022
Year: 2022

There has been considerable advancement in the use of blockchain for trust management in large-scale dynamic systems. In such systems, blockchain is mainly used to store the trust score or trust-related information of interactions among the various entities. However, present trust management architectures using blockchain lack verifiable interactions among the entities on which the trust score is calculated. In this paper, we propose a blockchain-based trust management framework that allows independent trust providers to implement different trust metrics on a common set of trust evidence and provide individual trust value. We employ geo-location as proof of interaction. Some of the existing proposals rely upon geo-location data, but they do not support trust calculation by multiple trust providers. Instead, they can only support a centralised system. Our proposed architecture does not depend upon a single centralised third-party entity to ensure trusted interactions. Our architecture is supported by provable interactions that can easily be verified using blockchain. Therefore, it allows a high degree of confidence in trust management by ensuring the actual interactions between the entities. We provide a detailed design and development of the architecture using real-world use case examples. The proof of prototype was implemented on the Ethereum blockchain platform. Experimental results demonstrate that the employment of independent trust providers adequately provides a high degree of trust scores and that the proposed architecture can be used in a real-world environment.
DRNet: Decomposition and Reconstruction Network for Remote Physiological Measurement
Venue: arXiv cs.HC 2022
Year: 2022

Remote photoplethysmography (rPPG) based physiological measurement has great application values in affective computing, non-contact health monitoring, telehealth monitoring, etc, which has become increasingly important especially during the COVID-19 pandemic. Existing methods are generally divided into two groups. The first focuses on mining the subtle blood volume pulse (BVP) signals from face videos, but seldom explicitly models the noises that dominate face video content. They are susceptible to the noises and may suffer from poor generalization ability in unseen scenarios. The second focuses on modeling noisy data directly, resulting in suboptimal performance due to the lack of regularity of these severe random noises. In this paper, we propose a Decomposition and Reconstruction Network (DRNet) focusing on the modeling of physiological features rather than noisy data. A novel cycle loss is proposed to constrain the periodicity of physiological information. Besides, a plug-and-play Spatial Attention Block (SAB) is proposed to enhance features along with the spatial location information. Furthermore, an efficient Patch Cropping (PC) augmentation strategy is proposed to synthesize augmented samples with different noise and features. Extensive experiments on different public datasets as well as the cross-database testing demonstrate the effectiveness of our approach.
DPCN++: Differentiable Phase Correlation Network for Versatile Pose Registration
Venue: arXiv cs.CV 2022
Year: 2022

Pose registration is critical in vision and robotics. This paper focuses on the challenging task of initialization-free pose registration up to 7DoF for homogeneous and heterogeneous measurements. While recent learning-based methods show promise using differentiable solvers, they either rely on heuristically defined correspondences or are prone to local minima. We present a differentiable phase correlation (DPC) solver that is globally convergent and correspondence-free. When combined with simple feature extraction networks, our general framework DPCN++ allows for versatile pose registration with arbitrary initialization. Specifically, the feature extraction networks first learn dense feature grids from a pair of homogeneous/heterogeneous measurements. These feature grids are then transformed into a translation and scale invariant spectrum representation based on Fourier transform and spherical radial aggregation, decoupling translation and scale from rotation. Next, the rotation, scale, and translation are independently and efficiently estimated in the spectrum step-by-step using the DPC solver. The entire pipeline is differentiable and trained end-to-end. We evaluate DCPN++ on a wide range of registration tasks taking different input modalities, including 2D bird's-eye view images, 3D object and scene measurements, and medical images. Experimental results demonstrate that DCPN++ outperforms both classical and learning-based baselines, especially on partially observed and heterogeneous measurements.
Federated Graph Attention Network for Rumor Detection
Venue: arXiv cs.SI 2022
Year: 2022

With the development of network technology, many social media are flourishing. Due to imperfect Internet regulation, the spread of false rumors has become a common problem on those social platforms. Social platforms can generate rumor data in their operation process, but existing rumor detection models are all constructed for a single social platform, which ignores the value of cross-platform rumor. This paper combines the federated learning paradigm with the bidirectional graph attention network rumor detection model and proposes the federated graph attention network(FedGAT) model for rumor detection. Taking the advantages of horizontal federated learning for cross-platform rumor detection, the security of each social platform's data information can be ensured. We construct both the server-side and client-side models as a bidirectional graph attention network rumor detection model in the federated learning paradigm framework. The local model on the client-side can train and verify the rumor data of the social platform in the iterative process, while the model on the server-side only plays the role of aggregating the characteristics of different social platform models and does not participate in the training of the model. Finally, we conduct simulation experiments on the rumor datasets and the experimental results show that the federated graph attention network model proposed in this paper is effective for cross-platform rumor detection.
Divergences on Monads for Relational Program Logics
Venue: arXiv cs.PL 2022
Year: 2022

Several relational program logics have been introduced for integrating reasoning about relational properties of programs and measurement of quantitative difference between computational effects. Towards a general framework for such logics, in this paper, we formalize quantitative difference between computational effects as divergence on monad, then develop a relational program logic acRL that supports generic computational effects and divergences on them. To give a categorical semantics of acRL supporting divergences, we give a method to obtain graded strong relational liftings from divergences on monads. We derive two instantiations of acRL for the verification of 1) various differential privacy of higher-order functional probabilistic programs and 2) difference of distribution of costs between higher-order functional programs with probabilistic choice and cost counting operations.
Fusing Feature Engineering and Deep Learning: A Case Study for Malware Classification
Venue: arXiv cs.CR 2022
Year: 2022

Machine learning has become an appealing signature-less approach to detect and classify malware because of its ability to generalize to never-before-seen samples and to handle large volumes of data. While traditional feature-based approaches rely on the manual design of hand-crafted features based on experts knowledge of the domain, deep learning approaches replace the manual feature engineering process by an underlying system, typically consisting of a neural network with multiple layers, that perform both feature learning and classification altogether. However, the combination of both approaches could substantially enhance detection systems. In this paper we present an hybrid approach to address the task of malware classification by fusing multiple types of features defined by experts and features learned through deep learning from raw data. In particular, our approach relies on deep learning to extract N-gram like features from the assembly language instructions and the bytes of malware, and texture patterns and shapelet-based features from malwareś grayscale image representation and structural entropy, respectively. These deep features are later passed as input to a gradient boosting model that combines the deep features and the hand-crafted features using an early-fusion mechanism. The suitability of our approach has been evaluated on the Microsoft Malware Classification Challenge benchmark and results show that the proposed solution achieves state-of-the-art performance and outperforms gradient boosting and deep learning methods in the literature.