Miracle-dz/UniFAR-FTUs huggingface dataset:
Hierarchical Correlation Clustering and Tree Preserving Embedding

We propose a hierarchical correlation clustering method that extends the well-known correlation clustering to produce hierarchical clusters. We then investigate embedding the respective hierarchy to be used for (tree preserving) embedding and feature extraction. We study the connection of such an embedding to single linkage embedding and minimax distances, and in particular study minimax distances for correlation clustering. Finally, we demonstrate the performance of our methods on several UCI and 20 newsgroup datasets.
Text Classification with Lexicon from PreAttention Mechanism

A comprehensive and high-quality lexicon plays a crucial role in traditional text classification approaches. And it improves the utilization of the linguistic knowledge. Although it is helpful for the task, the lexicon has got little attention in recent neural network models. Firstly, getting a high-quality lexicon is not easy. We lack an effective automated lexicon extraction method, and most lexicons are hand crafted, which is very inefficient for big data. What's more, there is no an effective way to use a lexicon in a neural network. To address those limitations, we propose a Pre-Attention mechanism for text classification in this paper, which can learn attention of different words according to their effects in the classification tasks. The words with different attention can form a domain lexicon. Experiments on three benchmark text classification tasks show that our models get competitive result comparing with the state-of-the-art methods. We get 90.5% accuracy on Stanford Large Movie Review dataset, 82.3% on Subjectivity dataset, 93.7% on Movie Reviews. And compared with the text classification model without Pre-Attention mechanism, those with Pre-Attention mechanism improve by 0.9%-2.4% accuracy, which proves the validity of the Pre-Attention mechanism. In addition, the Pre-Attention mechanism performs well followed by different types of neural networks (e.g., convolutional neural networks and Long Short-Term Memory networks). For the same dataset, when we use Pre-Attention mechanism to get attention value followed by different neural networks, those words with high attention values have a high degree of coincidence, which proves the versatility and portability of the Pre-Attention mechanism. we can get stable lexicons by attention values, which is an inspiring method of information extraction.
Federated Extra-Trees with Privacy Preserving

It is commonly observed that the data are scattered everywhere and difficult to be centralized. The data privacy and security also become a sensitive topic. The laws and regulations such as the European Union's General Data Protection Regulation (GDPR) are designed to protect the public's data privacy. However, machine learning requires a large amount of data for better performance, and the current circumstances put deploying real-life AI applications in an extremely difficult situation. To tackle these challenges, in this paper we propose a novel privacy-preserving federated machine learning model, named Federated Extra-Trees, which applies local differential privacy in the federated trees model. A secure multi-institutional machine learning system was developed to provide superior performance by processing the modeling jointly on different clients without exchanging any raw data. We have validated the accuracy of our work by conducting extensive experiments on public datasets and the efficiency and robustness were also verified by simulating the real-world scenarios. Overall, we presented an extensible, scalable and practical solution to handle the data island problem.
Uncertainty in Structured Prediction

Uncertainty estimation is important for ensuring safety and robustness of AI systems. While most research in the area has focused on un-structured prediction tasks, limited work has investigated general uncertainty estimation approaches for structured prediction. Thus, this work aims to investigate uncertainty estimation for structured prediction tasks within a single unified and interpretable probabilistic ensemble-based framework. We consider: uncertainty estimation for sequence data at the token-level and complete sequence-level; interpretations for, and applications of, various measures of uncertainty; and discuss both the theoretical and practical challenges associated with obtaining them. This work also provides baselines for token-level and sequence-level error detection, and sequence-level out-of-domain input detection on the WMT14 English-French and WMT17 English-German translation and LibriSpeech speech recognition datasets.
Shape Optimization of Actuators over Banach Spaces for Nonlinear Systems

In this paper, optimal actuator shape for nonlinear parabolic systems is discussed. The system under study is an abstract differential equation with a locally Lipschitz nonlinear part. A quadratic cost on the state and input of the system is considered. The existence of an optimal actuator shape has been established in the literature. This paper focuses on driving the optimality conditions for actuator shapes belonging to a Banach space. The application of the theory to the optimal actuator shape design for railway track model is considered.
Nonparametric Bayesian Deconvolution of a Symmetric Unimodal Density

We consider nonparametric measurement error density deconvolution subject to heteroscedastic measurement errors as well as symmetry about zero and shape constraints, in particular unimodality. The problem is motivated by applications where the observed data are estimated effect sizes from regressions on multiple factors, where the target is the distribution of the true effect sizes. We exploit the fact that any symmetric and unimodal density can be expressed as a mixture of symmetric uniform densities, and model the mixing density in a new way using a Dirichlet process location-mixture of Gamma distributions. We do the computations within a Bayesian context, describe a simple scalable implementation that is linear in the sample size, and show that the estimate of the unknown target density is consistent. Within our application context of regression effect sizes, the target density is likely to have a large probability near zero (the near null effects) coupled with a heavy-tailed distribution (the actual effects). Simulations show that unlike standard deconvolution methods, our Constrained Bayesian Deconvolution method does a much better job of reconstruction of the target density. Applications to a genome-wise association study (GWAS) and microarray data reveal similar results.
NoiseBreaker: Gradual Image Denoising Guided by Noise Analysis

Fully supervised deep-learning based denoisers are currently the most performing image denoising solutions. However, they require clean reference images. When the target noise is complex, e.g. composed of an unknown mixture of primary noises with unknown intensity, fully supervised solutions are hindered by the difficulty to build a suited training set for the problem.This paper proposes a gradual denoising strategy called NoiseBreaker that iteratively detects the dominating noise in an image, and removes it using a tailored denoiser. The method is shown to strongly outperform state of the art blind denoisers on mixture noises. Moreover, noise analysis is demonstrated to guide denoisers efficiently not only on noise type, but also on noise intensity. NoiseBreaker provides an insight on the nature of the encountered noise, and it makes it possible to update an existing denoiser with novel noise profiles. This feature makes the method adaptive to varied denoising cases.
Distributional Sliced-Wasserstein and Applications to Generative Modeling

Sliced-Wasserstein distance (SWD) and its variation, Max Sliced-Wasserstein distance (Max-SWD), have been widely used in the recent years due to their fast computation and scalability when the probability measures lie in very high dimension. However, these distances still have their weakness, SWD requires a lot of projection samples because it uses the uniform distribution to sample projecting directions, Max-SWD uses only one projection, causing it to lose a large amount of information. In this paper, we propose a novel distance that finds optimal penalized probability measure over the slices, which is named Distributional Sliced-Wasserstein distance (DSWD). We show that the DSWD is a generalization of both SWD and Max-SWD, and the proposed distance could be found by searching for the push-forward measure over a set of measures satisfying some certain constraints. Moreover, similar to SWD, we can extend Generalized Sliced-Wasserstein distance (GSWD) to Distributional Generalized Sliced-Wasserstein distance (DGSWD). Finally, we carry out extensive experiments to demonstrate the favorable generative modeling performances of our distances over the previous sliced-based distances in large-scale real datasets.
On Succinctness and Recognisability of Alternating Good-for-Games Automata

We study alternating good-for-games (GFG) automata, i.e., alternating automata where both conjunctive and disjunctive choices can be resolved in an online manner, without knowledge of the suffix of the input word still to be read. We show that they can be exponentially more succinct than both their nondeterministic and universal counterparts. Furthermore, we lift many results from nondeterministic parity GFG automata to alternating ones: a single exponential determinisation procedure, an Exptime upper bound to the GFGness problem, a PTime algorithm for the GFGness problem of weak automata, and a reduction from a positive solution to the $G_2$ conjecture to a PTime algorithm for the GFGness problem of parity automata with a fixed index. The $G_2$ conjecture states that a nondeterministic parity automaton A is GFG if and only if a token game, known as the $G_2$ game, played on A is won by the first player. So far, it had only been proved for Buchi automata; we provide further evidence for it by proving it for coBuchi automata. We also study the complexity of deciding "half-GFGness", a property specific to alternating automata that only requires nondeterministic choices to be resolved in an online manner. We show that this problem is strictly more difficult than GFGness check, already for alternating automata on finite words.
Latent Poisson models for networks with heterogeneous density

Empirical networks are often globally sparse, with a small average number of connections per node, when compared to the total size of the network. However, this sparsity tends not to be homogeneous, and networks can also be locally dense, for example, with a few nodes connecting to a large fraction of the rest of the network, or with small groups of nodes with a large probability of connections between them. Here we show how latent Poisson models that generate hidden multigraphs can be effective at capturing this density heterogeneity, while being more tractable mathematically than some of the alternatives that model simple graphs directly. We show how these latent multigraphs can be reconstructed from data on simple graphs, and how this allows us to disentangle disassortative degree-degree correlations from the constraints of imposed degree sequences, and to improve the identification of community structure in empirically relevant scenarios.
A forward-backward algorithm with different inertial terms for the minimization of the sum of two non-convex functions

We investigate an inertial forward-backward algorithm in connection with the minimization of the sum of a non-smooth and possible non-convex and a non-convex differentiable function. The algorithm is formulated in the spirit of the famous FISTA method, however the setting is non-convex and we allow different inertial terms. We also treat the case when the non-smooth function is convex and we show that in this case a better step-size can be allowed. We prove some abstract convergence results which applied to our numerical scheme allow us to show that the generated sequences converge to a critical point of the objective function, provided a regularization of the objective function satisfies the Kurdyka-{\L}ojasiewicz property.
K-means++: Few More Steps Yield Constant Approximation

The k-means++ algorithm of Arthur and Vassilvitskii (SODA 2007) is a state-of-the-art algorithm for solving the k-means clustering problem and is known to give an O(log k)-approximation in expectation. Recently, Lattanzi and Sohler (ICML 2019) proposed augmenting k-means++ with O(k log log k) local search steps to yield a constant approximation (in expectation) to the k-means clustering problem. In this paper, we improve their analysis to show that, for any arbitrarily small constant $\eps > 0$, with only $\eps k$ additional local search steps, one can achieve a constant approximation guarantee (with high probability in k), resolving an open problem in their paper.
MARVEL: A Decoupled Model-driven Approach for Efficiently Mapping Convolutions on Spatial DNN Accelerators

The efficiency of a spatial DNN accelerator depends heavily on the compiler's ability to generate optimized mappings for a given DNN's operators (layers) on to the accelerator's compute and memory resources. Searching for the optimal mapping is challenging because of a massive space of possible data-layouts and loop transformations for the DNN layers. For example, there are over 10^19 valid mappings for a single convolution layer on average for mapping ResNet50 and MobileNetV2 on a representative DNN edge accelerator. This challenge gets exacerbated with new layer types (e.g., depth-wise and point-wise convolutions) and diverse hardware accelerator configurations. To address this challenge, we propose a decoupled off-chip/on-chip approach that decomposes the mapping space into off-chip and on-chip subspaces, and first optimizes the off-chip subspace followed by the on-chip subspace. The motivation for this decomposition is to dramatically reduce the size of the search space, and to also prioritize the optimization of off-chip data movement, which is 2-3 orders of magnitude more compared to the on-chip data movement. We introduce {\em Marvel}, which implements the above approach by leveraging two cost models to explore the two subspaces -- a classical distinct-block (DB) locality cost model for the off-chip subspace, and a state-of-the-art DNN accelerator behavioral cost model, MAESTRO, for the on-chip subspace. Our approach also considers dimension permutation, a form of data-layouts, in the mapping space formulation along with the loop transformations.
Langevin DQN

Algorithms that tackle deep exploration -- an important challenge in reinforcement learning -- have relied on epistemic uncertainty representation through ensembles or other hypermodels, exploration bonuses, or visitation count distributions. An open question is whether deep exploration can be achieved by an incremental reinforcement learning algorithm that tracks a single point estimate, without additional complexity required to account for epistemic uncertainty. We answer this question in the affirmative. In particular, we develop Langevin DQN, a variation of DQN that differs only in perturbing parameter updates with Gaussian noise, and demonstrate through a computational study that the algorithm achieves deep exploration. We also provide an intuition for why Langevin DQN performs deep exploration.
Synaptic clock as a neural substrate of consciousness.

In this study the temporal aspect of consciousness is analyzed. We start from the notion that while conscious experience seems to change constantly, yet for any contents of experience to be consciously perceived they must last for some non-zero duration of time, which appears to constitute a certain conflict. We posit that, in terms of phenomenological analysis of consciousness, the temporal aspect, and this apparent conflict in particular, is the most basic property, likely inherent to any conceivable form of consciousness. We then put forward arguments for the synaptic clock to be a content-specific neural substrate of consciousness, showing how it would correspond to this temporal aspect. This proposal is considered in light of the information integration theory of consciousness, and other theories that relate consciousness to processes of learning and memory. It is outlined how this can offer a concrete way of relating the properties of consciousness directly to the neural plasticity mechanisms of learning and memory. In this regard, we propose a viewpoint, in which an association between different contents of conscious experience can be created in a form of relational memory only if they occur at the very same moment of subjective time, with moments of subjective time having different durations depending on the type of information processed, proportional to the time units of corresponding synaptic clocks, and being in principle different for different brain regions and nervous systems in different animal species. Finally, using this viewpoint, we consider the two alternative views on the structure of consciousness, namely a static and a dynamic one, and argue in favor of the latter, proposing that consciousness can be best understood if change is considered its only dimension.
Performance Guarantees of Distributed Algorithms for QoS in Wireless Ad Hoc Networks

Consider a wireless network where each communication link has a minimum bandwidth quality-of-service requirement. Certain pairs of wireless links interfere with each other due to being in the same vicinity, and this interference is modeled by a conflict graph. Given the conflict graph and link bandwidth requirements, the objective is to determine, using only localized information, whether the demands of all the links can be satisfied. At one extreme, each node knows the demands of only its neighbors; at the other extreme, there exists an optimal, centralized scheduler that has global information. The present work interpolates between these two extremes by quantifying the tradeoff between the degree of decentralization and the performance of the distributed algorithm. This open problem is resolved for the primary interference model, and the following general result is obtained: if each node knows the demands of all links in a ball of radius $d$ centered at the node, then there is a distributed algorithm whose performance is away from that of an optimal, centralized algorithm by a factor of at most ${{ (2d+3)}}/{{ (2d+2)}}$ . The tradeoff between performance and complexity of the distributed algorithm is also analyzed. It is shown that for line networks under the protocol interference model, the row constraints are a factor of at most 3 away from optimal. Both bounds are best possible.
The Rényi Gaussian Process

In this article we introduce an alternative closed form lower bound on the Gaussian process ($\mathcal{GP}$) likelihood based on the Renyi $\alpha$-divergence. This new lower bound can be viewed as a convex combination of the Nystrom approximation and the exact $\mathcal{GP}$. The key advantage of this bound, is its capability to control and tune the enforced regularization on the model and thus is a generalization of the traditional sparse variational $\mathcal{GP}$ regression. From the theoretical perspective, we show that with probability at least $1-\delta$, the Renyi $\alpha$-divergence between the variational distribution and the true posterior becomes arbitrarily small as the number of data points increase.
Comparative Proteomics of Extended-Spectrum Cephalosporin-Resistant Neisseria gonorrhoeae Isolates Demonstrates Altered Protein Synthesis, Metabolism, Substance Transport, and Membrane Permeability

Neisseria gonorrhoeae isolates exhibit resistance to extended-spectrum cephalosporins (ESCs), the last remaining option for first-line empirical monotherapy. Here, we investigated the proteomic profiles of N. gonorrhoeae clinical isolates with ESC-resistance to support exploration of the antimicrobial resistance mechanisms for N. gonorrhoeae. We used comparative iTRAQ quantitative proteomics to investigate differential protein expression of three ESC-resistant N. gonorrhoeae clinical isolates using N. gonorrhoeae ATCC49226 as a reference strain. The expression of 40 proteins was downregulated and expression of 56 proteins was upregulated in all three ESC-resistant N. gonorrhoeae isolates. Proteins with predicted function of translation, ribosomal structure and biogenesis, as well as components of the Type IV secretory systems, were significantly upregulated. Two differentially expressed proteins of ABC transporters were also reported by other teams in proteomics studies of N. gonorrhoeae isolates under antimicrobial stress conditions. Differentially expressed proteins are involved in energy production and metabolism of carbohydrates and amino acids. Our results indicated that amino acid and carbohydrate metabolism, cell membrane structure, interbacterial DNA transfer, and ribosome components might be involved in mediating ESC-resistance in N. gonorrhoeae. These findings facilitate a better understanding of the mechanisms of ESC-resistance in N. gonorrhoeae and provide useful information for identifying novel targets in the development of antimicrobials against N. gonorrhoeae.
Preferential HLA-B27 Allorecognition Displayed by Multiple Cross-Reactive Antiviral CD8+ T Cell Receptors

T cells provide essential immunosurveillance to combat and eliminate infection from pathogens, yet these cells can also induce unwanted immune responses via T cell receptor (TCR) cross-reactivity, also known as heterologous immunity. Indeed, pathogen-induced TCR cross-reactivity has shown to be a common, robust, and functionally potent mechanism that can trigger a spectrum of human immunopathologies associated with either transplant rejection, drug allergy, and autoimmunity. Here, we report that several virus-specific CD8+ T cells directed against peptides derived from chronic viruses (EBV, CMV, and HIV-1) presented by high frequency HLA-A and -B allomorphs differentially cross-react toward HLA-B27 allotypes in a highly focused and hierarchical manner. Given the commonality of cross-reactive T cells and their potential contribution to adverse outcomes in allogeneic transplants, our study demonstrates that multiple antiviral T cells recognizing the same HLA allomorph could pose an extra layer of complexity for organ matching.