OpenRaiser/Intern-Atlas huggingface dataset:
Controllable Accented Text-to-Speech Synthesis
Venue: arXiv cs.SD 2022
Year: 2022

Accented text-to-speech (TTS) synthesis seeks to generate speech with an accent (L2) as a variant of the standard version (L1). Accented TTS synthesis is challenging as L2 is different from L1 in both in terms of phonetic rendering and prosody pattern. Furthermore, there is no easy solution to the control of the accent intensity in an utterance. In this work, we propose a neural TTS architecture, that allows us to control the accent and its intensity during inference. This is achieved through three novel mechanisms, 1) an accent variance adaptor to model the complex accent variance with three prosody controlling factors, namely pitch, energy and duration; 2) an accent intensity modeling strategy to quantify the accent intensity; 3) a consistency constraint module to encourage the TTS system to render the expected accent intensity at a fine level. Experiments show that the proposed system attains superior performance to the baseline models in terms of accent rendering and intensity control. To our best knowledge, this is the first study of accented TTS synthesis with explicit intensity control.
SR-GCL: Session-Based Recommendation with Global Context Enhanced Augmentation in Contrastive Learning
Venue: arXiv cs.IR 2022
Year: 2022

Session-based recommendations aim to predict the next behavior of users based on ongoing sessions. The previous works have been modeling the session as a variable-length of a sequence of items and learning the representation of both individual items and the aggregated session. Recent research has applied graph neural networks with an attention mechanism to capture complicated item transitions and dependencies by modeling the sessions into graph-structured data. However, they still face fundamental challenges in terms of data and learning methodology such as sparse supervision signals and noisy interactions in sessions, leading to sub-optimal performance. In this paper, we propose SR-GCL, a novel contrastive learning framework for a session-based recommendation. As a crucial component of contrastive learning, we propose two global context enhanced data augmentation methods while maintaining the semantics of the original session. The extensive experiment results on two real-world E-commerce datasets demonstrate the superiority of SR-GCL as compared to other state-of-the-art methods.
Color Recommendation for Vector Graphic Documents based on Multi-Palette Representation
Venue: arXiv cs.CV 2022
Year: 2022

Vector graphic documents present multiple visual elements, such as images, shapes, and texts. Choosing appropriate colors for multiple visual elements is a difficult but crucial task for both amateurs and professional designers. Instead of creating a single color palette for all elements, we extract multiple color palettes from each visual element in a graphic document, and then combine them into a color sequence. We propose a masked color model for color sequence completion and recommend the specified colors based on color context in multi-palette with high probability. We train the model and build a color recommendation system on a large-scale dataset of vector graphic documents. The proposed color recommendation method outperformed other state-of-the-art methods by both quantitative and qualitative evaluations on color prediction and our color recommendation system received positive feedback from professional designers in an interview study.
DIG: Draping Implicit Garment over the Human Body
Venue: arXiv cs.CV 2022
Year: 2022

Existing data-driven methods for draping garments over human bodies, despite being effective, cannot handle garments of arbitrary topology and are typically not end-to-end differentiable. To address these limitations, we propose an end-to-end differentiable pipeline that represents garments using implicit surfaces and learns a skinning field conditioned on shape and pose parameters of an articulated body model. To limit body-garment interpenetrations and artifacts, we propose an interpenetration-aware pre-processing strategy of training data and a novel training loss that penalizes self-intersections while draping garments. We demonstrate that our method yields more accurate results for garment reconstruction and deformation with respect to state of the art methods. Furthermore, we show that our method, thanks to its end-to-end differentiability, allows to recover body and garments parameters jointly from image observations, something that previous work could not do.
MnTTS: An Open-Source Mongolian Text-to-Speech Synthesis Dataset and Accompanied Baseline
Venue: arXiv cs.SD 2022
Year: 2022

This paper introduces a high-quality open-source text-to-speech (TTS) synthesis dataset for Mongolian, a low-resource language spoken by over 10 million people worldwide. The dataset, named MnTTS, consists of about 8 hours of transcribed audio recordings spoken by a 22-year-old professional female Mongolian announcer. It is the first publicly available dataset developed to promote Mongolian TTS applications in both academia and industry. In this paper, we share our experience by describing the dataset development procedures and faced challenges. To demonstrate the reliability of our dataset, we built a powerful non-autoregressive baseline system based on FastSpeech2 model and HiFi-GAN vocoder, and evaluated it using the subjective mean opinion score (MOS) and real time factor (RTF) metrics. Evaluation results show that the powerful baseline system trained on our dataset achieves MOS above 4 and RTF about $3.30\times10^{-1}$, which makes it applicable for practical use. The dataset, training recipe, and pretrained TTS models are freely available \footnote{\label{github}\url{https://github.com/walker-hyf/MnTTS}}.
An Attention Matrix for Every Decision: Faithfulness-based Arbitration Among Multiple Attention-Based Interpretations of Transformers in Text Classification
Venue: arXiv cs.CL 2022
Year: 2022

Transformers are widely used in natural language processing, where they consistently achieve state-of-the-art performance. This is mainly due to their attention-based architecture, which allows them to model rich linguistic relations between (sub)words. However, transformers are difficult to interpret. Being able to provide reasoning for its decisions is an important property for a model in domains where human lives are affected. With transformers finding wide use in such fields, the need for interpretability techniques tailored to them arises. We propose a new technique that selects the most faithful attention-based interpretation among the several ones that can be obtained by combining different head, layer and matrix operations. In addition, two variations are introduced towards (i) reducing the computational complexity, thus being faster and friendlier to the environment, and (ii) enhancing the performance in multi-label data. We further propose a new faithfulness metric that is more suitable for transformer models and exhibits high correlation with the area under the precision-recall curve based on ground truth rationales. We validate the utility of our contributions with a series of quantitative and qualitative experiments on seven datasets.
Non-Negative Matrix Factorization with Scale Data Structure Preservation
Venue: arXiv cs.LG 2022
Year: 2022

The model described in this paper belongs to the family of non-negative matrix factorization methods designed for data representation and dimension reduction. In addition to preserving the data positivity property, it aims also to preserve the structure of data during matrix factorization. The idea is to add, to the NMF cost function, a penalty term to impose a scale relationship between the pairwise similarity matrices of the original and transformed data points. The solution of the new model involves deriving a new parametrized update scheme for the coefficient matrix, which makes it possible to improve the quality of reduced data when used for clustering and classification. The proposed clustering algorithm is compared to some existing NMF-based algorithms and to some manifold learning-based algorithms when applied to some real-life datasets. The obtained results show the effectiveness of the proposed algorithm.
Improving Utility for Privacy-Preserving Analysis of Correlated Columns using Pufferfish Privacy
Venue: arXiv cs.CR 2022
Year: 2022

Surveys are an important tool for many areas of social science research, but privacy concerns can complicate the collection and analysis of survey data. Differentially private analyses of survey data can address these concerns, but at the cost of accuracy - especially for high-dimensional statistics. We present a novel privacy mechanism, the Tabular DDP Mechanism, designed for high-dimensional statistics with incomplete correlation. The Tabular DDP Mechanism satisfies dependent differential privacy, a variant of Pufferfish privacy; it works by building a causal model of the sensitive data, then calibrating noise to the level of correlation between statistics. An empirical evaluation on survey data shows that the Tabular DDP Mechanism can significantly improve accuracy over the Laplace mechanism.
Delivering Ultra-Reliable Low-Latency Communications via Statistical Radio Maps
Venue: arXiv cs.IT 2022
Year: 2022

High reliability guarantees for Ultra-Reliable Low-Latency Communications (URLLC) require accurate knowledge of channel statistics, used as an input for rate selection. Exploiting the spatial consistency of channel statistics arises as a promising solution, allowing a base station to predict the propagation conditions and select the communication parameters for a new user from samples collected from previous users of the network. Based on this idea, this article provides a timely framework to exploit long-range channel spatial correlation through so-called statistical radio maps, enabling URLLC communications with given statistical guarantees. The framework is exemplified by predicting the channel capacity distribution both in a location-based radio map and in a latent space rendered by a channel chart, the latter being a localization-free approach based on channel state information (CSI). It is also shown how to use the maps to select the transmission rate in a new location that achieves a target level of reliability. Finally, several future directions and research challenges are also discussed.
Distributed architecture for resource description and discovery in the IoT
Venue: arXiv cs.NI 2022
Year: 2022

Nowadays, the Internet of Things (IoT) creates a vast ecosystem of intelligent objects interconnected via the Internet, allowing them to exchange information and to interact. This paradigm has been extended to a new concept, called the Web of Things (WoT), considering that every physical object can be accessed and controlled using Web-based languages and protocols, such as: the CoAP protocol which is becoming the most accepted and suitable protocol in this context. Moreover, the architectures currently proposed for the creation of IoT environments lack efficient and standard support for the discovery, selection and composition of IoT services and their integration in a scalable and interoperable way. To overcome this, in this work, we propose a hybrid and distributed CoAP-based architecture, considering all these aspects by combining the Fog Computing paradigm and structured P2P overlay networks. Furthermore, we describe the different components of our architecture and explain the interaction between them.
Learning Agile Flight Maneuvers: Deep SE(3) Motion Planning and Control for Quadrotors
Venue: arXiv cs.RO 2022
Year: 2022

Agile flights of autonomous quadrotors in cluttered environments require constrained motion planning and control subject to translational and rotational dynamics. Traditional model-based methods typically demand complicated design and heavy computation. In this paper, we develop a novel deep reinforcement learning-based method that tackles the challenging task of flying through a dynamic narrow gate. We design a model predictive controller with its adaptive tracking references parameterized by a deep neural network (DNN). These references include the traversal time and the quadrotor SE(3) traversal pose that encourage the robot to fly through the gate with maximum safety margins from various initial conditions. To cope with the difficulty of training in highly dynamic environments, we develop a reinforce-imitate learning framework to train the DNN efficiently that generalizes well to diverse settings. Furthermore, we propose a binary search algorithm that allows online adaption of the SE(3) references to dynamic gates in real-time. Finally, through extensive high-fidelity simulations, we show that our approach is robust to the gate's velocity uncertainties and adaptive to different gate trajectories and orientations.
Canadian Traveller Problem with Predictions
Venue: arXiv cs.DS 2022
Year: 2022

In this work, we consider the $k$-Canadian Traveller Problem ($k$-CTP) under the learning-augmented framework proposed by Lykouris & Vassilvitskii. $k$-CTP is a generalization of the shortest path problem, and involves a traveller who knows the entire graph in advance and wishes to find the shortest route from a source vertex $s$ to a destination vertex $t$, but discovers online that some edges (up to $k$) are blocked once reaching them. A potentially imperfect predictor gives us the number and the locations of the blocked edges. We present a deterministic and a randomized online algorithm for the learning-augmented $k$-CTP that achieve a tradeoff between consistency (quality of the solution when the prediction is correct) and robustness (quality of the solution when there are errors in the prediction). Moreover, we prove a matching lower bound for the deterministic case establishing that the tradeoff between consistency and robustness is optimal, and show a lower bound for the randomized algorithm. Finally, we prove several deterministic and randomized lower bounds on the competitive ratio of $k$-CTP depending on the prediction error, and complement them, in most cases, with matching upper bounds.
Multi-Tenant Cloud FPGA: A Survey on Security
Venue: arXiv cs.CR 2022
Year: 2022

With the exponentially increasing demand for performance and scalability in cloud applications and systems, data center architectures evolved to integrate heterogeneous computing fabrics that leverage CPUs, GPUs, and FPGAs. FPGAs differ from traditional processing platforms such as CPUs and GPUs in that they are reconfigurable at run-time, providing increased and customized performance, flexibility, and acceleration. FPGAs can perform large-scale search optimization, acceleration, and signal processing tasks compared with power, latency, and processing speed. Many public cloud provider giants, including Amazon, Huawei, Microsoft, Alibaba, etc., have already started integrating FPGA-based cloud acceleration services. While FPGAs in cloud applications enable customized acceleration with low power consumption, it also incurs new security challenges that still need to be reviewed. Allowing cloud users to reconfigure the hardware design after deployment could open the backdoors for malicious attackers, potentially putting the cloud platform at risk. Considering security risks, public cloud providers still don't offer multi-tenant FPGA services. This paper analyzes the security concerns of multi-tenant cloud FPGAs, gives a thorough description of the security problems associated with them, and discusses upcoming future challenges in this field of study.
Model-Assisted Labeling via Explainability for Visual Inspection of Civil Infrastructures
Venue: arXiv cs.CV 2022
Year: 2022

Labeling images for visual segmentation is a time-consuming task which can be costly, particularly in application domains where labels have to be provided by specialized expert annotators, such as civil engineering. In this paper, we propose to use attribution methods to harness the valuable interactions between expert annotators and the data to be annotated in the case of defect segmentation for visual inspection of civil infrastructures. Concretely, a classifier is trained to detect defects and coupled with an attribution-based method and adversarial climbing to generate and refine segmentation masks corresponding to the classification outputs. These are used within an assisted labeling framework where the annotators can interact with them as proposal segmentation masks by deciding to accept, reject or modify them, and interactions are logged as weak labels to further refine the classifier. Applied on a real-world dataset resulting from the automated visual inspection of bridges, our proposed method is able to save more than 50\% of annotators' time when compared to manual annotation of defects.
Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism
Venue: arXiv cs.CV 2022
Year: 2022

In this paper two new learning-based eXplainable AI (XAI) methods for deep convolutional neural network (DCNN) image classifiers, called L-CAM-Fm and L-CAM-Img, are proposed. Both methods use an attention mechanism that is inserted in the original (frozen) DCNN and is trained to derive class activation maps (CAMs) from the last convolutional layer's feature maps. During training, CAMs are applied to the feature maps (L-CAM-Fm) or the input image (L-CAM-Img) forcing the attention mechanism to learn the image regions explaining the DCNN's outcome. Experimental evaluation on ImageNet shows that the proposed methods achieve competitive results while requiring a single forward pass at the inference stage. Moreover, based on the derived explanations a comprehensive qualitative analysis is performed providing valuable insight for understanding the reasons behind classification errors, including possible dataset biases affecting the trained classifier.
Timely Multi-Process Estimation with Erasures
Venue: arXiv cs.IT 2022
Year: 2022

We consider a multi-process remote estimation system observing $K$ independent Ornstein-Uhlenbeck processes. In this system, a shared sensor samples the $K$ processes in such a way that the long-term average sum mean square error (MSE) is minimized. The sensor operates under a total sampling frequency constraint $f_{\max}$ and samples the processes according to a Maximum-Age-First (MAF) schedule. The samples from all processes consume random processing delays, and then are transmitted over an erasure channel with probability $ε$. Aided by optimal structural results, we show that the optimal sampling policy, under some conditions, is a \emph{threshold policy}. We characterize the optimal threshold and the corresponding optimal long-term average sum MSE as a function of $K$, $f_{\max}$, $ε$, and the statistical properties of the observed processes.
Siamese Network-based Lightweight Framework for Tomato Leaf Disease Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Automatic tomato disease recognition from leaf images is vital to avoid crop losses by applying control measures on time. Even though recent deep learning-based tomato disease recognition methods with classical training procedures showed promising recognition results, they demand large labelled data and involve expensive training. The traditional deep learning models proposed for tomato disease recognition also consume high memory and storage because of a high number of parameters. While lightweight networks overcome some of these issues to a certain extent, they continue to show low performance and struggle to handle imbalanced data. In this paper, a novel Siamese network-based lightweight framework is proposed for automatic tomato leaf disease recognition. This framework achieves the highest accuracy of 96.97% on the tomato subset obtained from the PlantVillage dataset and 95.48% on the Taiwan tomato leaf disease dataset. Experimental results further confirm that the proposed framework is effective with imbalanced and small data. The backbone deep network integrated with this framework is lightweight with approximately 2.9629 million trainable parameters, which is way lower than existing lightweight deep networks.
The Microsoft System for VoxCeleb Speaker Recognition Challenge 2022
Venue: arXiv cs.SD 2022
Year: 2022

In this report, we describe our submitted system for track 2 of the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC-22). We fuse a variety of good-performing models ranging from supervised models to self-supervised learning(SSL) pre-trained models. The supervised models, trained using VoxCeleb-2 dev data, consist of ECAPA-TDNN and Res2Net in a very deep structure. The SSL pre-trained models, wav2vec and wavLM, are trained using large scale unlabeled speech data up to million hours. These models are cascaded with ECAPA-TDNN and further fine-tuned in a supervised fashion to extract the speaker representations. All 13 models are applied with score normalization and calibration and then fused into the the submitted system. We also explore the audio quality measures in the calibration stage such as duration, SNR, T60, and MOS. The best submitted system achieves 0.073 in minDCF and 1.436% in EER on the VoxSRC-22 evaluation set.
Recurrence-free Survival Prediction under the Guidance of Automatic Gross Tumor Volume Segmentation for Head and Neck Cancers
Venue: arXiv cs.CV 2022
Year: 2022

For Head and Neck Cancers (HNC) patient management, automatic gross tumor volume (GTV) segmentation and accurate pre-treatment cancer recurrence prediction are of great importance to assist physicians in designing personalized management plans, which have the potential to improve the treatment outcome and quality of life for HNC patients. In this paper, we developed an automated primary tumor (GTVp) and lymph nodes (GTVn) segmentation method based on combined pre-treatment positron emission tomography/computed tomography (PET/CT) scans of HNC patients. We extracted radiomics features from the segmented tumor volume and constructed a multi-modality tumor recurrence-free survival (RFS) prediction model, which fused the prediction results from separate CT radiomics, PET radiomics, and clinical models. We performed 5-fold cross-validation to train and evaluate our methods on the MICCAI 2022 HEad and neCK TumOR segmentation and outcome prediction challenge (HECKTOR) dataset. The ensemble prediction results on the testing cohort achieved Dice scores of 0.77 and 0.73 for GTVp and GTVn segmentation, respectively, and a C-index value of 0.67 for RFS prediction. The code is publicly available (https://github.com/wangkaiwan/HECKTOR-2022-AIRT). Our team's name is AIRT.
OpenPneu: Compact platform for pneumatic actuation with multi-channels
Venue: arXiv cs.RO 2022
Year: 2022

This paper presents a compact system, OpenPneu, to support the pneumatic actuation for multi-chambers on soft robots. Micro-pumps are employed in the system to generate airflow and therefore no extra input as compressed air is needed. Our system conducts modular design to provide good scalability, which has been demonstrated on a prototype with ten air channels. Each air channel of OpenPneu is equipped with both the inflation and the deflation functions to provide a full range pressure supply from positive to negative with a maximal flow rate at 1.7 L/min. High precision closed-loop control of pressures has been built into our system to achieve stable and efficient dynamic performance in actuation. An open-source control interface and API in Python are provided. We also demonstrate the functionality of OpenPneu on three soft robotic systems with up to 10 chambers.