OpenRaiser/Intern-Atlas huggingface dataset:
LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech
Venue: arXiv cs.SD 2021
Year: 2021

An effective approach to automatically predict the subjective rating for synthetic speech is to train on a listening test dataset with human-annotated scores. Although each speech sample in the dataset is rated by several listeners, most previous works only used the mean score as the training target. In this work, we present LDNet, a unified framework for mean opinion score (MOS) prediction that predicts the listener-wise perceived quality given the input speech and the listener identity. We reflect recent advances in LD modeling, including design choices of the model architecture, and propose two inference methods that provide more stable results and efficient computation. We conduct systematic experiments on the voice conversion challenge (VCC) 2018 benchmark and a newly collected large-scale MOS dataset, providing an in-depth analysis of the proposed framework. Results show that the mean listener inference method is a better way to utilize the mean scores, whose effectiveness is more obvious when having more ratings per sample.
Speeding-Up Back-Propagation in DNN: Approximate Outer Product with Memory
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, an algorithm for approximate evaluation of back-propagation in DNN training is considered, which we term Approximate Outer Product Gradient Descent with Memory (Mem-AOP-GD). The Mem-AOP-GD algorithm implements an approximation of the stochastic gradient descent by considering only a subset of the outer products involved in the matrix multiplications that encompass backpropagation. In order to correct for the inherent bias in this approximation, the algorithm retains in memory an accumulation of the outer products that are not used in the approximation. We investigate the performance of the proposed algorithm in terms of DNN training loss under two design parameters: (i) the number of outer products used for the approximation, and (ii) the policy used to select such outer products. We experimentally show that significant improvements in computational complexity as well as accuracy can indeed be obtained through Mem-AOPGD.
Branch Predicting with Sparse Distributed Memories
Venue: arXiv cs.AR 2021
Year: 2021

Modern processors rely heavily on speculation to keep the pipeline filled and consequently execute and commit instructions as close to maximum capacity as possible. To improve instruction-level parallelism, the processor core needs to fetch and decode multiple instructions per cycle and has come to rely on incredibly accurate branch prediction. However, this comes at cost of the increased area and complexity which is needed for modern high accuracy branch predictors. The key idea described in this work is to use hyperdimensional computing and sparse distributed memory principles to create a novel branch predictor that can deliver complex predictions for a fraction of the current area. Sparse distributed memories can store vast amounts of data in a compressed manner, theoretically enabling branch histories larger and more precise than the branch predictors used today to be stored with equal or smaller area footprint. Furthermore, as all the data is in a hashed format and due to the nature of the hashing scheme used, it is inherently harder to manipulate with known side-channel attacks. We describe our proof-of-concept and evaluate it against a state-of-the-art academic TAGE predictor. Our experiments are conducted on realistic synthetic branch predictor patterns and the Championship Branch Prediction traces and show competitive accuracy. Finally, we describe techniques that can be used to solve some of the challenges of processing with hyperdimensional vectors in order to deliver timely predictions.
Domain Generalisation for Apparent Emotional Facial Expression Recognition across Age-Groups
Venue: arXiv cs.CV 2021
Year: 2021

Apparent emotional facial expression recognition has attracted a lot of research attention recently. However, the majority of approaches ignore age differences and train a generic model for all ages. In this work, we study the effect of using different age-groups for training apparent emotional facial expression recognition models. To this end, we study Domain Generalisation in the context of apparent emotional facial expression recognition from facial imagery across different age groups. We first compare several domain generalisation algorithms on the basis of out-of-domain-generalisation, and observe that the Class-Conditional Domain-Adversarial Neural Networks (CDANN) algorithm has the best performance. We then study the effect of variety and number of age-groups used during training on generalisation to unseen age-groups and observe that an increase in the number of training age-groups tends to increase the apparent emotional facial expression recognition performance on unseen age-groups. We also show that exclusion of an age-group during training tends to affect more the performance of the neighbouring age groups.
Graph Partner Neural Networks for Semi-Supervised Learning on Graphs
Venue: arXiv cs.LG 2021
Year: 2021

Graph Convolutional Networks (GCNs) are powerful for processing graph-structured data and have achieved state-of-the-art performance in several tasks such as node classification, link prediction, and graph classification. However, it is inevitable for deep GCNs to suffer from an over-smoothing issue that the representations of nodes will tend to be indistinguishable after repeated graph convolution operations. To address this problem, we propose the Graph Partner Neural Network (GPNN) which incorporates a de-parameterized GCN and a parameter-sharing MLP. We provide empirical and theoretical evidence to demonstrate the effectiveness of the proposed MLP partner on tackling over-smoothing while benefiting from appropriate smoothness. To further tackle over-smoothing and regulate the learning process, we introduce a well-designed consistency contrastive loss and KL divergence loss. Besides, we present a graph enhancement technique to improve the overall quality of edges in graphs. While most GCNs can work with shallow architecture only, GPNN can obtain better results through increasing model depth. Experiments on various node classification tasks have demonstrated the state-of-the-art performance of GPNN. Meanwhile, extensive ablation studies are conducted to investigate the contributions of each component in tackling over-smoothing and improving performance.
Electric Vehicle Automatic Charging System Based on Vision-force Fusion
Venue: arXiv cs.RO 2021
Year: 2021

Electric vehicles are an emerging means of transportation with environmental friendliness. The automatic charging is a hot topic in this field that is full of challenges. We introduce a complete automatic charging system based on vision-force fusion, which includes perception, planning and control for robot manipulations of the system. We design the whole system in simulation and transfer it to the real world. The experimental results prove the effectiveness of our system.
Color Image Segmentation Using Multi-Objective Swarm Optimizer and Multi-level Histogram Thresholding
Venue: arXiv cs.CV 2021
Year: 2021

Rapid developments in swarm intelligence optimizers and computer processing abilities make opportunities to design more accurate, stable, and comprehensive methods for color image segmentation. This paper presents a new way for unsupervised image segmentation by combining histogram thresholding methods (Kapur's entropy and Otsu's method) and different multi-objective swarm intelligence algorithms (MOPSO, MOGWO, MSSA, and MOALO) to thresholding 3D histogram of a color image. More precisely, this method first combines the objective function of traditional thresholding algorithms to design comprehensive objective functions then uses multi-objective optimizers to find the best thresholds during the optimization of designed objective functions. Also, our method uses a vector objective function in 3D space that could simultaneously handle the segmentation of entire image color channels with the same thresholds. To optimize this vector objective function, we employ multiobjective swarm optimizers that can optimize multiple objective functions at the same time. Therefore, our method considers dependencies between channels to find the thresholds that satisfy objective functions of color channels (which we name as vector objective function) simultaneously. Segmenting entire color channels with the same thresholds also benefits from the fact that our proposed method needs fewer thresholds to segment the image than other thresholding algorithms; thus, it requires less memory space to save thresholds. It helps a lot when we want to segment many images to many regions. The subjective and objective results show the superiority of this method to traditional thresholding methods that separately threshold histograms of a color image.
Leveraging MoCap Data for Human Mesh Recovery
Venue: arXiv cs.CV 2021
Year: 2021

Training state-of-the-art models for human body pose and shape recovery from images or videos requires datasets with corresponding annotations that are really hard and expensive to obtain. Our goal in this paper is to study whether poses from 3D Motion Capture (MoCap) data can be used to improve image-based and video-based human mesh recovery methods. We find that fine-tune image-based models with synthetic renderings from MoCap data can increase their performance, by providing them with a wider variety of poses, textures and backgrounds. In fact, we show that simply fine-tuning the batch normalization layers of the model is enough to achieve large gains. We further study the use of MoCap data for video, and introduce PoseBERT, a transformer module that directly regresses the pose parameters and is trained via masked modeling. It is simple, generic and can be plugged on top of any state-of-the-art image-based model in order to transform it in a video-based model leveraging temporal information. Our experimental results show that the proposed approaches reach state-of-the-art performance on various datasets including 3DPW, MPI-INF-3DHP, MuPoTS-3D, MCB and AIST. Test code and models will be available soon.
Natural Attribute-based Shift Detection
Venue: arXiv cs.LG 2021
Year: 2021

Despite the impressive performance of deep networks in vision, language, and healthcare, unpredictable behaviors on samples from the distribution different than the training distribution cause severe problems in deployment. For better reliability of neural-network-based classifiers, we define a new task, natural attribute-based shift (NAS) detection, to detect the samples shifted from the training distribution by some natural attribute such as age of subjects or brightness of images. Using the natural attributes present in existing datasets, we introduce benchmark datasets in vision, language, and medical for NAS detection. Further, we conduct an extensive evaluation of prior representative out-of-distribution (OOD) detection methods on NAS datasets and observe an inconsistency in their performance. To understand this, we provide an analysis on the relationship between the location of NAS samples in the feature space and the performance of distance- and confidence-based OOD detection methods. Based on the analysis, we split NAS samples into three categories and further suggest a simple modification to the training objective to obtain an improved OOD detection method that is capable of detecting samples from all NAS categories.
Reconfigurable Intelligent Surface-Enhanced OFDM Communications via Delay Adjustable Metasurface
Venue: arXiv cs.IT 2021
Year: 2021

Reconfigurable intelligent surface (RIS) is a promising technology for establishing spectral- and energy-efficient wireless networks. In this paper, we study RIS-enhanced orthogonal frequency division multiplexing (OFDM) communications, which generalize the existing RIS-driven context focusing only on frequency-flat channels. Firstly, we introduce the delay adjustable metasurface (DAM) relying on varactor diodes. In contrast to existing reflecting elements, each one in DAM is capable of storing and retrieving the impinging electromagnetic waves upon dynamically controlling its electromagnetically induced transparency (EIT) properties, thus additionally imposing an extra delay onto the reflected incident signals. Secondly, we formulate the rate-maximization problem by jointly optimizing the transmit power allocation and the RIS reflection coefficients as well as the RIS delays. Furthermore, to address the coupling among optimization variables, we propose an efficient algorithm to achieve a high-quality solution for the formulated non-convex design problem by alternately optimizing the transmit power allocation and the RIS reflection pattern, including the reflection coefficients and the delays. Thirdly, to circumvent the high complexity for optimizing the RIS reflection coefficients, we conceive a low-complexity scheme upon aligning the strongest taps of all reflected channels, while ensuring that the maximum delay spread after introducing extra RIS delays does not exceed the length of the cyclic prefix (CP). Finally, simulation results demonstrate that the proposed design significantly improves the OFDM rate performance as well as the RIS's adaptability to wideband signals compared to baseline schemes without employing DAM.
Fair Tree Classifier using Strong Demographic Parity
Venue: arXiv cs.LG 2021
Year: 2021

When dealing with sensitive data in automated data-driven decision-making, an important concern is to learn predictors with high performance towards a class label, whilst minimising for the discrimination towards any sensitive attribute, like gender or race, induced from biased data. A few hybrid tree optimisation criteria exist that combine classification performance and fairness. Although the threshold-free ROC-AUC is the standard for measuring traditional classification model performance, current fair tree classification methods mainly optimise for a fixed threshold on both the classification task as well as the fairness metric. In this paper, we propose a compound splitting criterion which combines threshold-free (i.e., strong) demographic parity with ROC-AUC termed SCAFF -- Splitting Criterion AUC for Fairness -- and easily extends to bagged and boosted tree frameworks. Our method simultaneously leverages multiple sensitive attributes of which the values may be multicategorical or intersectional, and is tunable with respect to the unavoidable performance-fairness trade-off. In our experiments, we demonstrate how SCAFF generates models with performance and fairness with respect to binary, multicategorical, and multiple sensitive attributes.
Impact of review valence and perceived uncertainty on purchase of time-constrained and discounted search goods
Venue: arXiv cs.CY 2021
Year: 2021

Increasing online shoppers have generated enormous amount of data in form of reviews (text) and sales data. Aggregate reviews in form of rating (stars) have become noticeable indicators of product quality and vendor performance to prospective consumers at first sight. Consumers subjected to product discount deadlines search for ways in which they could evaluate product and vendor service using a comprehensible benchmark. Considering the effect of time pressure on consumers, aggregate reviews, known as review valence, become a viable indicator of product quality. This study investigates how purchase decisions for new products are affected by past customer aggregate ratings when a soon-to-expire discount is being offered. We examine the role that a consumer's attitude towards review valence (RV) plays as an antecedent to that consumer's reliance on RV in a purchase decision for time-discounted search goods. Considering review credibility, diagnosticity, and effectiveness as determinants of consumer attitude in a time-constrained search and purchase environment, we follow the approach-avoidance conflict theory to examine the role of review valence and perceived uncertainty in a time-constrained environment. The data was collected through an online survey and analyzed using structural equation modelling. This study provides significant implications for practitioners as they can better understand how review valence can influence a purchase decision. Empirical analysis includes two contributions: 1. It helps to understand how consumer attitude toward review valence, when positively influenced by the determinants, can lead to reliance on review valence, further influencing purchase decision; 2. Time constrained purchase-related perceived uncertainty negatively moderates the relationship between consumer attitude and reliance on review valence.
Mixed Reality using Illumination-aware Gradient Mixing in Surgical Telepresence: Enhanced Multi-layer Visualization
Venue: arXiv cs.MM 2021
Year: 2021

Background and aim: Surgical telepresence using augmented perception has been applied, but mixed reality is still being researched and is only theoretical. The aim of this work is to propose a solution to improve the visualization in the final merged video by producing globally consistent videos when the intensity of illumination in the input source and target video varies. Methodology: The proposed system uses an enhanced multi-layer visualization with illumination-aware gradient mixing using Illumination Aware Video Composition algorithm. Particle Swarm Optimization Algorithm is used to find the best sample pair from foreground and background region and image pixel correlation to estimate the alpha matte. Particle Swarm Optimization algorithm helps to get the original colour and depth of the unknown pixel in the unknown region. Result: Our results showed improved accuracy caused by reducing the Mean squared Error for selecting the best sample pair for unknown region in 10 each sample for bowel, jaw and breast. The amount of this reduction is 16.48% from the state of art system. As a result, the visibility accuracy is improved from 89.4 to 97.7% which helped to clear the hand vision even in the difference of light. Conclusion: Illumination effect and alpha pixel correlation improves the visualization accuracy and produces a globally consistent composition results and maintains the temporal coherency when compositing two videos with high and inverse illumination effect. In addition, this paper provides a solution for selecting the best sampling pair for the unknown region to obtain the original colour and depth.
Verifying Compliance in Process Choreographies: Foundations, Algorithms, and Implementation
Venue: arXiv cs.LO 2021
Year: 2021

The current push towards interoperability drives companies to collaborate through process choreographies. At the same time, they face a jungle of continuously changing regulations, e.g., due to the pandemic and developments such as the BREXIT, which strongly affect cross-organizational collaborations. Think of, for example, supply chains spanning several countries with different and maybe even conflicting COVID19 traveling restrictions. Hence, providing automatic compliance verification in process choreographies is crucial for any cross-organizational business process. A particular challenge concerns the restricted visibility of the partner processes at the presence of global compliance rules (GCR), i.e., rules that span across the process of several partners. This work deals with the question how to verify global compliance if affected tasks are not fully visible. Our idea is to decompose GCRs into so called assertions that can be checked by each affected partner whereby the decomposition is both correct and lossless. The algorithm exploits transitivity properties of the underlying rule specification, and its correctness and complexity are proven, considering advanced aspects such as loops. The algorithm is implemented in a proof-of-concept prototype, including a model checker for verifying compliance. The applicability of the approach is further demonstrated on a real-world manufacturing use case.
HRFormer: High-Resolution Transformer for Dense Prediction
Venue: arXiv cs.CV 2021
Year: 2021

We present a High-Resolution Transformer (HRFormer) that learns high-resolution representations for dense prediction tasks, in contrast to the original Vision Transformer that produces low-resolution representations and has high memory and computational cost. We take advantage of the multi-resolution parallel design introduced in high-resolution convolutional networks (HRNet), along with local-window self-attention that performs self-attention over small non-overlapping image windows, for improving the memory and computation efficiency. In addition, we introduce a convolution into the FFN to exchange information across the disconnected image windows. We demonstrate the effectiveness of the High-Resolution Transformer on both human pose estimation and semantic segmentation tasks, e.g., HRFormer outperforms Swin transformer by $1.3$ AP on COCO pose estimation with $50\%$ fewer parameters and $30\%$ fewer FLOPs. Code is available at: https://github.com/HRNet/HRFormer.
FMFCC-A: A Challenging Mandarin Dataset for Synthetic Speech Detection
Venue: arXiv cs.SD 2021
Year: 2021

As increasing development of text-to-speech (TTS) and voice conversion (VC) technologies, the detection of synthetic speech has been suffered dramatically. In order to promote the development of synthetic speech detection model against Mandarin TTS and VC technologies, we have constructed a challenging Mandarin dataset and organized the accompanying audio track of the first fake media forensic challenge of China Society of Image and Graphics (FMFCC-A). The FMFCC-A dataset is by far the largest publicly-available Mandarin dataset for synthetic speech detection, which contains 40,000 synthesized Mandarin utterances that generated by 11 Mandarin TTS systems and two Mandarin VC systems, and 10,000 genuine Mandarin utterances collected from 58 speakers. The FMFCC-A dataset is divided into the training, development and evaluation sets, which are used for the research of detection of synthesized Mandarin speech under various previously unknown speech synthesis systems or audio post-processing operations. In addition to describing the construction of the FMFCC-A dataset, we provide a detailed analysis of two baseline methods and the top-performing submissions from the FMFCC-A, which illustrates the usefulness and challenge of FMFCC-A dataset. We hope that the FMFCC-A dataset can fill the gap of lack of Mandarin datasets for synthetic speech detection.
Goal Agnostic Planning using Maximum Likelihood Paths in Hypergraph World Models
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we present a hypergraph--based machine learning algorithm, a datastructure--driven maintenance method, and a planning algorithm based on a probabilistic application of Dijkstra's algorithm. Together, these form a goal agnostic automated planning engine for an autonomous learning agent which incorporates beneficial properties of both classical Machine Learning and traditional Artificial Intelligence. We prove that the algorithm determines optimal solutions within the problem space, mathematically bound learning performance, and supply a mathematical model analyzing system state progression through time yielding explicit predictions for learning curves, goal achievement rates, and response to abstractions and uncertainty. To validate performance, we exhibit results from applying the agent to three archetypal planning problems, including composite hierarchical domains, and highlight empirical findings which illustrate properties elucidated in the analysis.