OpenRaiser/Intern-Atlas huggingface dataset:
In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains
Venue: arXiv cs.CV 2022
Year: 2022

Medical applications have benefited greatly from the rapid advancement in computer vision. Considering patient monitoring in particular, in-bed human posture estimation offers important health-related metrics with potential value in medical condition assessments. Despite great progress in this domain, it remains challenging due to substantial ambiguity during occlusions, and the lack of large corpora of manually labeled data for model training, particularly with domains such as thermal infrared imaging which are privacy-preserving, and thus of great interest. Motivated by the effectiveness of self-supervised methods in learning features directly from data, we propose a multi-modal conditional variational autoencoder (MC-VAE) capable of reconstructing features from missing modalities seen during training. This approach is used with HRNet to enable single modality inference for in-bed pose estimation. Through extensive evaluations, we demonstrate that body positions can be effectively recognized from the available modality, achieving on par results with baseline models that are highly dependent on having access to multiple modes at inference time. The proposed framework supports future research towards self-supervised learning that generates a robust model from a single source, and expects it to generalize over many unknown distributions in clinical environments.
EdiBERT, a generative model for image editing
Venue: arXiv cs.CV 2022
Year: 2022

Advances in computer vision are pushing the limits of im-age manipulation, with generative models sampling detailed images on various tasks. However, a specialized model is often developed and trained for each specific task, even though many image edition tasks share similarities. In denoising, inpainting, or image compositing, one always aims at generating a realistic image from a low-quality one. In this paper, we aim at making a step towards a unified approach for image editing. To do so, we propose EdiBERT, a bi-directional transformer trained in the discrete latent space built by a vector-quantized auto-encoder. We argue that such a bidirectional model is suited for image manipulation since any patch can be re-sampled conditionally to the whole image. Using this unique and straightforward training objective, we show that the resulting model matches state-of-the-art performances on a wide variety of tasks: image denoising, image completion, and image composition.
Seeking Salient Facial Regions for Cross-Database Micro-Expression Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Cross-Database Micro-Expression Recognition (CDMER) aims to develop the Micro-Expression Recognition (MER) methods with strong domain adaptability, i.e., the ability to recognize the Micro-Expressions (MEs) of different subjects captured by different imaging devices in different scenes. The development of CDMER is faced with two key problems: 1) the severe feature distribution gap between the source and target databases; 2) the feature representation bottleneck of ME such local and subtle facial expressions. To solve these problems, this paper proposes a novel Transfer Group Sparse Regression method, namely TGSR, which aims to 1) optimize the measurement and better alleviate the difference between the source and target databases, and 2) highlight the valid facial regions to enhance extracted features, by the operation of selecting the group features from the raw face feature, where each region is associated with a group of raw face feature, i.e., the salient facial region selection. Compared with previous transfer group sparse methods, our proposed TGSR has the ability to select the salient facial regions, which is effective in alleviating the aforementioned problems for better performance and reducing the computational cost at the same time. We use two public ME databases, i.e., CASME II and SMIC, to evaluate our proposed TGSR method. Experimental results show that our proposed TGSR learns the discriminative and explicable regions, and outperforms most state-of-the-art subspace-learning-based domain-adaptive methods for CDMER.
Node-Level Differentially Private Graph Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Graph Neural Networks (GNNs) are a popular technique for modelling graph-structured data and computing node-level representations via aggregation of information from the neighborhood of each node. However, this aggregation implies an increased risk of revealing sensitive information, as a node can participate in the inference for multiple nodes. This implies that standard privacy-preserving machine learning techniques, such as differentially private stochastic gradient descent (DP-SGD) - which are designed for situations where each data point participates in the inference for one point only - either do not apply, or lead to inaccurate models. In this work, we formally define the problem of learning GNN parameters with node-level privacy, and provide an algorithmic solution with a strong differential privacy guarantee. We employ a careful sensitivity analysis and provide a non-trivial extension of the privacy-by-amplification technique to the GNN setting. An empirical evaluation on standard benchmark datasets demonstrates that our method is indeed able to learn accurate privacy-preserving GNNs which outperform both private and non-private methods that completely ignore graph information.
Black-box tests for algorithmic stability
Venue: arXiv cs.LG 2022
Year: 2022

Algorithmic stability is a concept from learning theory that expresses the degree to which changes to the input data (e.g., removal of a single data point) may affect the outputs of a regression algorithm. Knowing an algorithm's stability properties is often useful for many downstream applications -- for example, stability is known to lead to desirable generalization properties and predictive inference guarantees. However, many modern algorithms currently used in practice are too complex for a theoretical analysis of their stability properties, and thus we can only attempt to establish these properties through an empirical exploration of the algorithm's behavior on various data sets. In this work, we lay out a formal statistical framework for this kind of "black-box testing" without any assumptions on the algorithm or the data distribution and establish fundamental bounds on the ability of any black-box test to identify algorithmic stability.
SegDiff: Image Segmentation with Diffusion Probabilistic Models
Venue: arXiv cs.CV 2022
Year: 2022

Diffusion Probabilistic Methods are employed for state-of-the-art image generation. In this work, we present a method for extending such models for performing image segmentation. The method learns end-to-end, without relying on a pre-trained backbone. The information in the input image and in the current estimation of the segmentation map is merged by summing the output of two encoders. Additional encoding layers and a decoder are then used to iteratively refine the segmentation map, using a diffusion model. Since the diffusion model is probabilistic, it is applied multiple times, and the results are merged into a final segmentation map. The new method produces state-of-the-art results on the Cityscapes validation set, the Vaihingen building segmentation benchmark, and the MoNuSeg dataset.
Effective and efficient structure learning with pruning and model averaging strategies
Venue: arXiv cs.LG 2022
Year: 2022

Learning the structure of a Bayesian Network (BN) with score-based solutions involves exploring the search space of possible graphs and moving towards the graph that maximises a given objective function. Some algorithms offer exact solutions that guarantee to return the graph with the highest objective score, while others offer approximate solutions in exchange for reduced computational complexity. This paper describes an approximate BN structure learning algorithm, which we call Model Averaging Hill-Climbing (MAHC), that combines two novel strategies with hill-climbing search. The algorithm starts by pruning the search space of graphs, where the pruning strategy can be viewed as an aggressive version of the pruning strategies that are typically applied to combinatorial optimisation structure learning problems. It then performs model averaging in the hill-climbing search process and moves to the neighbouring graph that maximises the objective function, on average, for that neighbouring graph and over all its valid neighbouring graphs. Comparisons with other algorithms spanning different classes of learning suggest that the combination of aggressive pruning with model averaging is both effective and efficient, particularly in the presence of data noise.
Null Model-Based Data Augmentation for Graph Classification
Venue: arXiv cs.SI 2022
Year: 2022

In network science, the null model is typically used to generate a series of graphs based on randomization as a term of comparison to verify whether a network in question displays some non-trivial features such as community structure. Since such non-trivial features play a significant role in graph classification, the null model could be useful for network data augmentation to enhance classification performance. In this paper, we propose a novel technique that combines the null model with data augmentation for graph classification. Moreover, we propose four standard null model-based augmentation methods and four approximate null model-based augmentation methods to verify and improve the performance of our graph classification technique. Our experiments demonstrate that the proposed augmentation technique has significantly achieved general improvement on the tested datasets. In addition, we find that the standard null model-based augmentation methods always outperform the approximate ones, depending on the design mechanisms of the null models. Our results indicate that the choice of non-trivial features is significant for increasing the performance of augmentation models for different network structures, which also provides a new perspective of data augmentation for studying various graph classification methods.
A Few-Shot Meta-Learning based Siamese Neural Network using Entropy Features for Ransomware Classification
Venue: arXiv cs.CR 2022
Year: 2022

Ransomware defense solutions that can quickly detect and classify different ransomware classes to formulate rapid response plans have been in high demand in recent years. Though the applicability of adopting deep learning techniques to provide automation and self-learning provision has been proven in many application domains, the lack of data available for ransomware (and other malware)samples has been raised as a barrier to developing effective deep learning-based solutions. To address this concern, we propose a few-shot meta-learning based Siamese Neural Network that not only detects ransomware attacks but is able to classify them into different classes. Our proposed model utilizes the entropy feature directly extracted from ransomware binary files to retain more fine-grained features associated with different ransomware signatures. These entropy features are used further to train and optimize our model using a pre-trained network (e.g. VGG-16) in a meta-learning fashion. This approach generates more accurate weight factors, compared to feature images are used, to avoid the bias typically associated with a model trained with a limited number of training samples. Our experimental results show that our proposed model is highly effective in providing a weighted F1-score exceeding the rate>86% compared
Collaborative Artificial Intelligence Needs Stronger Assurances Driven by Risks
Venue: arXiv cs.HC 2022
Year: 2022

Collaborative AI systems (CAISs) aim at working together with humans in a shared space to achieve a common goal. This critical setting yields hazardous circumstances that could harm human beings. Thus, building such systems with strong assurances of compliance with requirements, domain-specific standards and regulations is of greatest importance. Only few scale impact has been reported so far for such systems since much work remains to manage possible risks. We identify emerging problems in this context and then we report our vision, as well as the progress of our multidisciplinary research team composed of software/systems, and mechatronics engineers to develop a risk-driven assurance process for CAISs.
PreViTS: Contrastive Pretraining with Video Tracking Supervision
Venue: arXiv cs.CV 2022
Year: 2022

Videos are a rich source for self-supervised learning (SSL) of visual representations due to the presence of natural temporal transformations of objects. However, current methods typically randomly sample video clips for learning, which results in an imperfect supervisory signal. In this work, we propose PreViTS, an SSL framework that utilizes an unsupervised tracking signal for selecting clips containing the same object, which helps better utilize temporal transformations of objects. PreViTS further uses the tracking signal to spatially constrain the frame regions to learn from and trains the model to locate meaningful objects by providing supervision on Grad-CAM attention maps. To evaluate our approach, we train a momentum contrastive (MoCo) encoder on VGG-Sound and Kinetics-400 datasets with PreViTS. Training with PreViTS outperforms representations learnt by contrastive strategy alone on video downstream tasks, obtaining state-of-the-art performance on action classification. PreViTS helps learn feature representations that are more robust to changes in background and context, as seen by experiments on datasets with background changes. Learning from large-scale videos with PreViTS could lead to more accurate and robust visual feature representations.
Temporally Resolution Decrement: Utilizing the Shape Consistency for Higher Computational Efficiency
Venue: arXiv cs.CV 2022
Year: 2022

Image resolution that has close relations with accuracy and computational cost plays a pivotal role in network training. In this paper, we observe that the reduced image retains relatively complete shape semantics but loses extensive texture information. Inspired by the consistency of the shape semantics as well as the fragility of the texture information, we propose a novel training strategy named Temporally Resolution Decrement. Wherein, we randomly reduce the training images to a smaller resolution in the time domain. During the alternate training with the reduced images and the original images, the unstable texture information in the images results in a weaker correlation between the texture-related patterns and the correct label, naturally enforcing the model to rely more on shape properties that are robust and conform to the human decision rule. Surprisingly, our approach greatly improves both the training and inference efficiency of convolutional neural networks. On ImageNet classification, using only 33\% calculation quantity (randomly reducing the training image to 112$\times$112 within 90\% epochs) can still improve ResNet-50 from 76.32\% to 77.71\%. Superimposed with the strong training procedure of ResNet-50 on ImageNet, our method achieves 80.42\% top-1 accuracy with saving 37.5\% calculation overhead. To the best of our knowledge this is the highest ImageNet single-crop accuracy on ResNet-50 under 224$\times$224 without extra data or distillation.
Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification
Venue: arXiv cs.LG 2022
Year: 2022

Multi-label classification (MLC) is a prediction task where each sample can have more than one label. We propose a novel contrastive learning boosted multi-label prediction model based on a Gaussian mixture variational autoencoder (C-GMVAE), which learns a multimodal prior space and employs a contrastive loss. Many existing methods introduce extra complex neural modules like graph neural networks to capture the label correlations, in addition to the prediction modules. We find that by using contrastive learning in the supervised setting, we can exploit label information effectively in a data-driven manner, and learn meaningful feature and label embeddings which capture the label correlations and enhance the predictive power. Our method also adopts the idea of learning and aligning latent spaces for both features and labels. In contrast to previous works based on a unimodal prior, C-GMVAE imposes a Gaussian mixture structure on the latent space, to alleviate the posterior collapse and over-regularization issues. C-GMVAE outperforms existing methods on multiple public datasets and can often match other models' full performance with only 50% of the training data. Furthermore, we show that the learnt embeddings provide insights into the interpretation of label-label interactions.
Automatic deforestation detectors based on frequentist statistics and their extensions for other spatial objects
Venue: arXiv cs.CV 2022
Year: 2022

This paper is devoted to the problem of detection of forest and non-forest areas on Earth images. We propose two statistical methods to tackle this problem: one based on multiple hypothesis testing with parametric distribution families, another one -- on non-parametric tests. The parametric approach is novel in the literature and relevant to a larger class of problems -- detection of natural objects, as well as anomaly detection. We develop mathematical background for each of the two methods, build self-sufficient detection algorithms using them and discuss practical aspects of their implementation. We also compare our algorithms with those from standard machine learning using satellite data.
Memory-efficient array redistribution through portable collective communication
Venue: arXiv cs.DC 2022
Year: 2022

Modern large-scale deep learning workloads highlight the need for parallel execution across many devices in order to fit model data into hardware accelerator memories. In these settings, array redistribution may be required during a computation, but can also become a bottleneck if not done efficiently. In this paper we address the problem of redistributing multi-dimensional array data in SPMD computations, the most prevalent form of parallelism in deep learning. We present a type-directed approach to synthesizing array redistributions as sequences of MPI-style collective operations. We prove formally that our synthesized redistributions are memory-efficient and perform no excessive data transfers. Array redistribution for SPMD computations using collective operations has also been implemented in the context of the XLA SPMD partitioner, a production-grade tool for partitioning programs across accelerator systems. We evaluate our approach against the XLA implementation and find that our approach delivers a geometric mean speedup of $1.22\times$, with maximum speedups as a high as $5.7\times$, while offering provable memory guarantees, making our system particularly appealing for large-scale models.
ViF-SD2E: A Robust Weakly-Supervised Method for Neural Decoding
Venue: arXiv cs.NE 2022
Year: 2022

Neural decoding plays a vital role in the interaction between the brain and the outside world. In this paper, we directly decode the movement track of a finger based on the neural signals of a macaque. Supervised regression methods may overfit to actual labels containing noise, and require a high labeling cost, while unsupervised approaches often have unsatisfactory accuracy. Besides, the spatial and temporal information is often ignored or not well exploited by those methods. This motivates us to propose a robust weakly-supervised method, called ViF-SD2E, for neural decoding. In particular, it consists of a space-division (SD) module and a exploration--exploitation (2E) strategy, to effectively exploit both the spatial information of the outside world and the temporal information of neural activity, where the SD2E output is analogized with the weak 0/1 vision-feedback (ViF) label for training. It is worth noting that the designed ViF-SD2E is based on a symmetric phenomenon between the unsupervised decoding trajectory and the real trajectory in previous observations, then a cognitive pattern of fuzzy (robust) interaction in the nervous system may be discovered by us. Extensive experiments demonstrate the effectiveness of our method, which can be sometimes comparable to supervised counterparts.
Simultaneously Transmitting and Reflecting Reconfigurable Intelligent Surface Assisted NOMA Networks
Venue: arXiv cs.IT 2022
Year: 2022

Simultaneously transmitting/refracting and reflecting reconfigurable intelligent surface (STAR-RIS) has been introduced to achieve full coverage area. This paper investigate the performance of STAR-RIS assisted non-orthogonal multiple access (NOMA) networks over Rician fading channels, where the incidence signals sent by base station are reflected and transmitted to the nearby user and distant user, respectively. To evaluate the performance of STAR-RIS-NOMA networks, we derive new approximate expressions of outage probability and ergodic rate for a pair of users, in which the imperfect successive interference cancellation (ipSIC) and perfect SIC (pSIC) schemes are taken into consideration. Based on the asymptotic expressions, the diversity orders of the nearby user with ipSIC/pSIC and distant user are achieved carefully. The high signal-to-noise ratio slopes of ergodic rates for nearby user with pSIC and distant user are equal to $one$ and $zero$, respectively. In addition, the system throughput of STAR-RIS-NOMA is discussed in delay-limited and delay-tolerant modes. Simulation results are provided to verify the accuracy of the theoretical analyses and demonstrate that: 1) The outage probability of STAR-RIS-NOMA outperforms that of STAR-RIS assisted orthogonal multiple access (OMA) and conventional cooperative communication systems; 2) With the increasing of reflecting elements $K$ and Rician factor $κ$, the STAR-RIS-NOMA networks are capable of attaining the enhanced performance; and 3) The ergodic rates of STAR-RIS-NOMA are superior to that of STAR-RIS-OMA.
Efficient Calling Conventions for Irregular Architectures
Venue: arXiv cs.PL 2022
Year: 2022

We empirically evaluated thousands of different C calling conventions for irregular microcontroller architectures, and found potential for improvement over the calling conventions previously used in the Small Device C Compiler (SDCC). The improvements in code size and speed are substantial enough that SDCC made changes to its default calling convention, breaking ABI compatibility.
KPDrop: Improving Absent Keyphrase Generation
Venue: arXiv cs.CL 2022
Year: 2022

Keyphrase generation is the task of generating phrases (keyphrases) that summarize the main topics of a given document. Keyphrases can be either present or absent from the given document. While the extraction of present keyphrases has received much attention in the past, only recently a stronger focus has been placed on the generation of absent keyphrases. However, generating absent keyphrases is challenging; even the best methods show only a modest degree of success. In this paper, we propose a model-agnostic approach called keyphrase dropout (or KPDrop) to improve absent keyphrase generation. In this approach, we randomly drop present keyphrases from the document and turn them into artificial absent keyphrases during training. We test our approach extensively and show that it consistently improves the absent performance of strong baselines in both supervised and resource-constrained semi-supervised settings.