OpenRaiser/Intern-Atlas huggingface dataset:
A Speaker Verification Backend with Robust Performance across Conditions
Venue: arXiv cs.SD 2021
Year: 2021

In this paper, we address the problem of speaker verification in conditions unseen or unknown during development. A standard method for speaker verification consists of extracting speaker embeddings with a deep neural network and processing them through a backend composed of probabilistic linear discriminant analysis (PLDA) and global logistic regression score calibration. This method is known to result in systems that work poorly on conditions different from those used to train the calibration model. We propose to modify the standard backend, introducing an adaptive calibrator that uses duration and other automatically extracted side-information to adapt to the conditions of the inputs. The backend is trained discriminatively to optimize binary cross-entropy. When trained on a number of diverse datasets that are labeled only with respect to speaker, the proposed backend consistently and, in some cases, dramatically improves calibration, compared to the standard PLDA approach, on a number of held-out datasets, some of which are markedly different from the training data. Discrimination performance is also consistently improved. We show that joint training of the PLDA and the adaptive calibrator is essential -- the same benefits cannot be achieved when freezing PLDA and fine-tuning the calibrator. To our knowledge, the results in this paper are the first evidence in the literature that it is possible to develop a speaker verification system with robust out-of-the-box performance on a large variety of conditions.
Secret Key Agreement and Secure Omniscience of Tree-PIN Source with Linear Wiretapper
Venue: arXiv cs.IT 2021
Year: 2021

While the wiretap secret key capacity remains unknown for general source models even in the two-user case, we obtained a single-letter characterization for a large class of multi-user source models with a linear wiretapper who can observe any linear combinations of the source. We introduced the idea of irreducible sources to show existence of an optimal communication scheme that achieves perfect omniscience with minimum leakage of information to the wiretapper. This implies a duality between the problems of wiretap secret key agreement and secure omniscience, and such duality potentially holds for more general sources.
A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Recent advancements in machine learning and signal processing domains have resulted in an extensive surge of interest in Deep Neural Networks (DNNs) due to their unprecedented performance and high accuracy for different and challenging problems of significant engineering importance. However, when such deep learning architectures are utilized for making critical decisions such as the ones that involve human lives (e.g., in control systems and medical applications), it is of paramount importance to understand, trust, and in one word "explain" the argument behind deep models' decisions. In many applications, artificial neural networks (including DNNs) are considered as black-box systems, which do not provide sufficient clue on their internal processing actions. Although some recent efforts have been initiated to explain the behaviors and decisions of deep networks, explainable artificial intelligence (XAI) domain, which aims at reasoning about the behavior and decisions of DNNs, is still in its infancy. The aim of this paper is to provide a comprehensive overview on Understanding, Visualization, and Explanation of the internal and overall behavior of DNNs.
Evaluation of Point Pattern Features for Anomaly Detection of Defect within Random Finite Set Framework
Venue: arXiv cs.CV 2021
Year: 2021

Defect detection in the manufacturing industry is of utmost importance for product quality inspection. Recently, optical defect detection has been investigated as an anomaly detection using different deep learning methods. However, the recent works do not explore the use of point pattern features, such as SIFT for anomaly detection using the recently developed set-based methods. In this paper, we present an evaluation of different point pattern feature detectors and descriptors for defect detection application. The evaluation is performed within the random finite set framework. Handcrafted point pattern features, such as SIFT as well as deep features are used in this evaluation. Random finite set-based defect detection is compared with state-of-the-arts anomaly detection methods. The results show that using point pattern features, such as SIFT as data points for random finite set-based anomaly detection achieves the most consistent defect detection accuracy on the MVTec-AD dataset.
Llama: A Heterogeneous & Serverless Framework for Auto-Tuning Video Analytics Pipelines
Venue: arXiv cs.DC 2021
Year: 2021

The proliferation of camera-enabled devices and large video repositories has led to a diverse set of video analytics applications. These applications rely on video pipelines, represented as DAGs of operations, to transform videos, process extracted metadata, and answer questions like, "Is this intersection congested?" The latency and resource efficiency of pipelines can be optimized using configurable knobs for each operation (e.g., sampling rate, batch size, or type of hardware used). However, determining efficient configurations is challenging because (a) the configuration search space is exponentially large, and (b) the optimal configuration depends on users' desired latency and cost targets, (c) input video contents may exercise different paths in the DAG and produce a variable amount intermediate results. Existing video analytics and processing systems leave it to the users to manually configure operations and select hardware resources. We present Llama: a heterogeneous and serverless framework for auto-tuning video pipelines. Given an end-to-end latency target, Llama optimizes for cost efficiency by (a) calculating a latency target for each operation invocation, and (b) dynamically running a cost-based optimizer to assign configurations across heterogeneous hardware that best meet the calculated per-invocation latency target. This makes the problem of auto-tuning large video pipelines tractable and allows us to handle input-dependent behavior, conditional branches in the DAG, and execution variability. We describe the algorithms in Llama and evaluate it on a cloud platform using serverless CPU and GPU resources. We show that compared to state-of-the-art cluster and serverless video analytics and processing systems, Llama achieves 7.8x lower latency and 16x cost reduction on average.
Optimizing QoS for Erasure-Coded Wireless Data Centers
Venue: arXiv cs.IT 2021
Year: 2021

Cloud computing facilitates the access of applications and data from any location by a distributed storage system. Erasure codes offer better data replication technique with reduced storage costs for more reliability. This paper considers the erasure-coded data center with multiple servers in a wireless network where each is equipped with a base-station. The cause of latency in the file retrieval process is mainly due to queuing delays at each server. This work puts forth a stochastic optimization framework for obtaining the optimal scheduling policy that maximizes users' quality of service (QoS) while adhering to the latency requirements. We further show that the problem has non-linear functions of expectations in objective and constraints and is impossible to solve with traditional SGD like algorithms. We propose a new algorithm that addresses compositional structure in the problem. Further, we show that the proposed algorithm achieves a faster convergence rate than the best-known results. Finally, we test the efficacy of the proposed method in a simulated environment.
Online Cycle Detection for Models with Mode-Dependent Input and Output Dependencies
Venue: arXiv cs.SE 2021
Year: 2021

In the fields of co-simulation and component-based modelling, designers import models as building blocks to create a composite model that provides more complex functionalities. Modelling tools perform instantaneous cycle detection (ICD) on the composite models having feedback loops to reject the models if the loops are mathematically unsound and to improve simulation performance. In this case, the analysis relies heavily on the availability of dependency information from the imported models. However, the cycle detection problem becomes harder when the model's input to output dependencies are mode-dependent, i.e. changes for certain events generated internally or externally as inputs. The number of possible modes created by composing such models increases significantly and unknown factors such as environmental inputs make the offline (statical) ICD a difficult task. In this paper, an online ICD method is introduced to address this issue for the models used in cyber-physical systems. The method utilises an oracle as a central source of information that can answer whether the individual models can make mode transition without creating instantaneous cycles. The oracle utilises three types of data-structures created offline that are adaptively chosen during online (runtime) depending on the frequency as well as the number of models that make mode transitions. During the analysis, the models used online are stalled from running, resulting in the discrepancy with the physical system. The objective is to detect an absence of the instantaneous cycle while minimising the stall time of the model simulation that is induced from the analysis. The benchmark results show that our method is an adequate alternative to the offline analysis methods and significantly reduces the analysis time.
General-Purpose Speech Representation Learning through a Self-Supervised Multi-Granularity Framework
Venue: arXiv cs.SD 2021
Year: 2021

This paper presents a self-supervised learning framework, named MGF, for general-purpose speech representation learning. In the design of MGF, speech hierarchy is taken into consideration. Specifically, we propose to use generative learning approaches to capture fine-grained information at small time scales and use discriminative learning approaches to distill coarse-grained or semantic information at large time scales. For phoneme-scale learning, we borrow idea from the masked language model but tailor it for the continuous speech signal by replacing classification loss with a contrastive loss. We corroborate our design by evaluating MGF representation on various downstream tasks, including phoneme classification, speaker classification, speech recognition, and emotion classification. Experiments verify that training at different time scales needs different training targets and loss functions, which in general complement each other and lead to a better performance.
Key Technology Considerations in Developing and Deploying Machine Learning Models in Clinical Radiology Practice
Venue: arXiv cs.LG 2021
Year: 2021

The use of machine learning to develop intelligent software tools for interpretation of radiology images has gained widespread attention in recent years. The development, deployment, and eventual adoption of these models in clinical practice, however, remains fraught with challenges. In this paper, we propose a list of key considerations that machine learning researchers must recognize and address to make their models accurate, robust, and usable in practice. Namely, we discuss: insufficient training data, decentralized datasets, high cost of annotations, ambiguous ground truth, imbalance in class representation, asymmetric misclassification costs, relevant performance metrics, generalization of models to unseen datasets, model decay, adversarial attacks, explainability, fairness and bias, and clinical validation. We describe each consideration and identify techniques to address it. Although these techniques have been discussed in prior research literature, by freshly examining them in the context of medical imaging and compiling them in the form of a laundry list, we hope to make them more accessible to researchers, software developers, radiologists, and other stakeholders.
Convergence Voting: From Pairwise Comparisons to Consensus
Venue: arXiv cs.MA 2021
Year: 2021

An important aspect of AI design and ethics is to create systems that reflect aggregate preferences of the society. To this end, the techniques of social choice theory are often utilized. We propose a new social choice function motivated by the PageRank algorithm. The function ranks voting options based on the Condorcet graph of pairwise comparisons. To this end, we transform the Condorcet graph into a Markov chain whose stationary distribution provides the scores of the options. We show how the values in the stationary distribution can be interpreted as quantified aggregate support for the voting options, to which the community of voters converges through an imaginary sequence of negotiating steps. Because of that, we suggest the name "convergence voting" for the new voting scheme, and "negotiated community support" for the resulting stationary allocation of scores. Our social choice function can be viewed as a consensus voting method, sitting somewhere between Copeland and Borda. On the one hand, it does not necessarily choose the Condorcet winner, as strong support from a part of the society can outweigh mediocre uniform support. On the other hand, the influence of unpopular candidates on the outcome is smaller than in the primary technique of consensus voting, i.e., the Borda count. We achieve that without having to introduce an ad hoc weighting that some other methods do.
A generalised feature for low level vision
Venue: arXiv cs.CV 2021
Year: 2021

This papers presents a novel quantised transform (the Sinclair-Town or ST transform for short) that subsumes the rolls of both edge-detector, MSER style region detector and corner detector. The transform is similar to the $unsharp$ transform but the difference from the local mean is quantised to 3 values (dark-neutral-light). The transform naturally leads to the definition of an appropriate local scale. A range of methods for extracting shape features form the transformed image are presented. The generalized feature provides a robust basis for establishing correspondence between images. The transform readily admits more complicated kernel behaviour including multi-scale and asymmetric elements to prefer shorter scale or oriented local features.
Information-Theoretic Bounds on the Moments of the Generalization Error of Learning Algorithms
Venue: arXiv cs.IT 2021
Year: 2021

Generalization error bounds are critical to understanding the performance of machine learning models. In this work, building upon a new bound of the expected value of an arbitrary function of the population and empirical risk of a learning algorithm, we offer a more refined analysis of the generalization behaviour of a machine learning models based on a characterization of (bounds) to their generalization error moments. We discuss how the proposed bounds -- which also encompass new bounds to the expected generalization error -- relate to existing bounds in the literature. We also discuss how the proposed generalization error moment bounds can be used to construct new generalization error high-probability bounds.
Music source separation conditioned on 3D point clouds
Venue: arXiv cs.SD 2021
Year: 2021

Recently, significant progress has been made in audio source separation by the application of deep learning techniques. Current methods that combine both audio and visual information use 2D representations such as images to guide the separation process. However, in order to (re)-create acoustically correct scenes for 3D virtual/augmented reality applications from recordings of real music ensembles, detailed information about each sound source in the 3D environment is required. This demand, together with the proliferation of 3D visual acquisition systems like LiDAR or rgb-depth cameras, stimulates the creation of models that can guide the audio separation using 3D visual information. This paper proposes a multi-modal deep learning model to perform music source separation conditioned on 3D point clouds of music performance recordings. This model extracts visual features using 3D sparse convolutions, while audio features are extracted using dense convolutions. A fusion module combines the extracted features to finally perform the audio source separation. It is shown, that the presented model can distinguish the musical instruments from a single 3D point cloud frame, and perform source separation qualitatively similar to a reference case, where manually assigned instrument labels are provided.
InfoColorizer: Interactive Recommendation of Color Palettes for Infographics
Venue: arXiv cs.HC 2021
Year: 2021

When designing infographics, general users usually struggle with getting desired color palettes using existing infographic authoring tools, which sometimes sacrifice customizability, require design expertise, or neglect the influence of elements' spatial arrangement. We propose a data-driven method that provides flexibility by considering users' preferences, lowers the expertise barrier via automation, and tailors suggested palettes to the spatial layout of elements. We build a recommendation engine by utilizing deep learning techniques to characterize good color design practices from data, and further develop InfoColorizer, a tool that allows users to obtain color palettes for their infographics in an interactive and dynamic manner. To validate our method, we conducted a comprehensive four-part evaluation, including case studies, a controlled user study, a survey study, and an interview study. The results indicate that InfoColorizer can provide compelling palette recommendations with adequate flexibility, allowing users to effectively obtain high-quality color design for input infographics with low effort.
Optical Stochastic Computing Architectures Using Photonic Crystal Nanocavities
Venue: arXiv cs.ET 2021
Year: 2021

Stochastic computing allows a drastic reduction in hardware complexity using serial processing of bit streams. While the induced high computing latency can be overcome using integrated optics technology, the design of realistic optical stochastic computing architectures calls for energy efficient switching devices. Photonics Crystal (PhC) nanocavities are $μm^2$ scale devices offering 100fJ switching operation under picoseconds-scale switching speed. Fabrication process allows controlling the Quality factor of each nanocavity resonance, leading to opportunities to implement architectures involving cascaded gates and multi-wavelength signaling. In this report, we investigate the design of cascaded gates architecture using nanocavities in the context of stochastic computing. We propose a transmission model considering key nanocavity device parameters, such as Quality factors, resonance wavelength and switching efficiency. The model is calibrated with experimental measurements. We propose the design of XOR gate and multiplexer. We illustrate the use of the gates to design an edge detection filter. System-level exploration of laser power, bit-stream length and bit-error rate is carried out for the processing of gray-scale images. The results show that the proposed architecture leads to 8.5nJ/pixel energy consumption and 512ns/pixel processing time.
Data Generation Using Pass-phrase-dependent Deep Auto-encoders for Text-Dependent Speaker Verification
Venue: arXiv cs.SD 2021
Year: 2021

In this paper, we propose a novel method that trains pass-phrase specific deep neural network (PP-DNN) based auto-encoders for creating augmented data for text-dependent speaker verification (TD-SV). Each PP-DNN auto-encoder is trained using the utterances of a particular pass-phrase available in the target enrollment set with two methods: (i) transfer learning and (ii) training from scratch. Next, feature vectors of a given utterance are fed to the PP-DNNs and the output from each PP-DNN at frame-level is considered one new set of generated data. The generated data from each PP-DNN is then used for building a TD-SV system in contrast to the conventional method that considers only the evaluation data available. The proposed approach can be considered as the transformation of data to the pass-phrase specific space using a non-linear transformation learned by each PP-DNN. The method develops several TD-SV systems with the number equal to the number of PP-DNNs separately trained for each pass-phrases for the evaluation. Finally, the scores of the different TD-SV systems are fused for decision making. Experiments are conducted on the RedDots challenge 2016 database for TD-SV using short utterances. Results show that the proposed method improves the performance for both conventional cepstral feature and deep bottleneck feature using both Gaussian mixture model - universal background model (GMM-UBM) and i-vector framework.
Top-down Discourse Parsing via Sequence Labelling
Venue: arXiv cs.CL 2021
Year: 2021

We introduce a top-down approach to discourse parsing that is conceptually simpler than its predecessors (Kobayashi et al., 2020; Zhang et al., 2020). By framing the task as a sequence labelling problem where the goal is to iteratively segment a document into individual discourse units, we are able to eliminate the decoder and reduce the search space for splitting points. We explore both traditional recurrent models and modern pre-trained transformer models for the task, and additionally introduce a novel dynamic oracle for top-down parsing. Based on the Full metric, our proposed LSTM model sets a new state-of-the-art for RST parsing.
Learning to Match Mathematical Statements with Proofs
Venue: arXiv cs.CL 2021
Year: 2021

We introduce a novel task consisting in assigning a proof to a given mathematical statement. The task is designed to improve the processing of research-level mathematical texts. Applying Natural Language Processing (NLP) tools to research level mathematical articles is both challenging, since it is a highly specialized domain which mixes natural language and mathematical formulae. It is also an important requirement for developing tools for mathematical information retrieval and computer-assisted theorem proving. We release a dataset for the task, consisting of over 180k statement-proof pairs extracted from mathematical research articles. We carry out preliminary experiments to assess the difficulty of the task. We first experiment with two bag-of-words baselines. We show that considering the assignment problem globally and using weighted bipartite matching algorithms helps a lot in tackling the task. Finally, we introduce a self-attention-based model that can be trained either locally or globally and outperforms baselines by a wide margin.
Outlier-Robust Learning of Ising Models Under Dobrushin's Condition
Venue: arXiv cs.LG 2021
Year: 2021

We study the problem of learning Ising models satisfying Dobrushin's condition in the outlier-robust setting where a constant fraction of the samples are adversarially corrupted. Our main result is to provide the first computationally efficient robust learning algorithm for this problem with near-optimal error guarantees. Our algorithm can be seen as a special case of an algorithm for robustly learning a distribution from a general exponential family. To prove its correctness for Ising models, we establish new anti-concentration results for degree-$2$ polynomials of Ising models that may be of independent interest.