Reviewerly/exHarmony huggingface dataset:
Perturbative estimation of stochastic gradients

In this paper we introduce a family of stochastic gradient estimation techniques based of the perturbative expansion around the mean of the sampling distribution. We characterize the bias and variance of the resulting Taylor-corrected estimators using the Lagrange error formula. Furthermore, we introduce a family of variance reduction techniques that can be applied to other gradient estimators. Finally, we show that these new perturbative methods can be extended to discrete functions using analytic continuation. Using this technique, we derive a new gradient descent method for training stochastic networks with binary weights. In our experiments, we show that the perturbative correction improves the convergence of stochastic variational inference both in the continuous and in the discrete case.
NEWSTRADCOIN: A Blockchain Based Privacy Preserving Secure NEWS Trading Network

In order to stay up to date with world issues and cutting-edge technol-ogies, the newspaper plays a crucial role. However, collecting news is not a very easy task. Currently, news publishers are collecting news from their correspond-ents through social networks, email, phone call, fax etc. and sometimes they buy news from the agencies. However, the existing news sharing networks may not provide security for data integrity and any third party may obstruct the regular flow of news sharing. Moreover, the existing news schemes are very vulnerable in case of disclosing the identity. Therefore, a universal platform is needed in the era of globalization where anyone can share and trade news from anywhere in the world securely, without the interference of third-party, and without disclosing the identity of an individual. Recently, blockchain has gained popularity because of its security mechanism over data, identity, etc. Blockchain enables a distrib-uted way of managing transactions where each participant of the network holds the same copy of the transactions. Therefore, with the help of pseudonymity, fault-tolerance, immutability and the distributed structure of blockchain, a scheme (termed as NEWSTRADCOIN) is presented in this paper in which not only news can be shared securely but also anyone can earn money by selling news. The proposed NEWSTRADCOIN can provide a universal platform where publishers can directly obtain news from news-gatherers in a secure way by main-taining data integrity, without experiencing the interference of a third-party, and without disclosing the identity of the news gatherer and publishers.
Re-Ranking Words to Improve Interpretability of Automatically Generated Topics

Topics models, such as LDA, are widely used in Natural Language Processing. Making their output interpretable is an important area of research with applications to areas such as the enhancement of exploratory search interfaces and the development of interpretable machine learning models. Conventionally, topics are represented by their n most probable words, however, these representations are often difficult for humans to interpret. This paper explores the re-ranking of topic words to generate more interpretable topic representations. A range of approaches are compared and evaluated in two experiments. The first uses crowdworkers to associate topics represented by different word rankings with related documents. The second experiment is an automatic approach based on a document retrieval task applied on multiple domains. Results in both experiments demonstrate that re-ranking words improves topic interpretability and that the most effective re-ranking schemes were those which combine information about the importance of words both within topics and their relative frequency in the entire corpus. In addition, close correlation between the results of the two evaluation approaches suggests that the automatic method proposed here could be used to evaluate re-ranking methods without the need for human judgements.
Synthesizing a 4D Spatio-Angular Consistent Light Field from a Single Image

Synthesizing a densely sampled light field from a single image is highly beneficial for many applications. The conventional method reconstructs a depth map and relies on physical-based rendering and a secondary network to improve the synthesized novel views. Simple pixel-based loss also limits the network by making it rely on pixel intensity cue rather than geometric reasoning. In this study, we show that a different geometric representation, namely, appearance flow, can be used to synthesize a light field from a single image robustly and directly. A single end-to-end deep neural network that does not require a physical-based approach nor a post-processing subnetwork is proposed. Two novel loss functions based on known light field domain knowledge are presented to enable the network to preserve the spatio-angular consistency between sub-aperture images effectively. Experimental results show that the proposed model successfully synthesizes dense light fields and qualitatively and quantitatively outperforms the previous model . The method can be generalized to arbitrary scenes, rather than focusing on a particular class of object. The synthesized light field can be used for various applications, such as depth estimation and refocusing.
Making Neural Machine Reading Comprehension Faster

This study aims at solving the Machine Reading Comprehension problem where questions have to be answered given a context passage. The challenge is to develop a computationally faster model which will have improved inference time. State of the art in many natural language understanding tasks, BERT model, has been used and knowledge distillation method has been applied to train two smaller models. The developed models are compared with other models which have been developed with the same intention.
How to Estimate the Ability of a Metaheuristic Algorithm to Guide Heuristics During Optimization

Metaheuristics are general methods that guide application of concrete heuristic(s) to problems that are too hard to solve using exact algorithms. However, even though a growing body of literature has been devoted to their statistical evaluation, the approaches proposed so far are able to assess only coupled effects of metaheuristics and heuristics. They do not reveal us anything about how efficient the examined metaheuristic is at guiding its subordinate heuristic(s), nor do they provide us information about how much the heuristic component of the combined algorithm contributes to the overall performance. In this paper, we propose a simple yet effective methodology of doing so by deriving a naive, placebo metaheuristic from the one being studied and comparing the distributions of chosen performance metrics for the two methods. We propose three measures of difference between the two distributions. Those measures, which we call BER values (benefit, equivalence, risk) are based on a preselected threshold of practical significance which represents the minimal difference between two performance scores required for them to be considered practically different. We illustrate usefulness of our methodology on the example of Simulated Annealing, Boolean Satisfiability Problem, and the Flip heuristic.
Training neural networks to encode symbols enables combinatorial generalization

Combinatorial generalization - the ability to understand and produce novel combinations of already familiar elements - is considered to be a core capacity of the human mind and a major challenge to neural network models. A significant body of research suggests that conventional neural networks can't solve this problem unless they are endowed with mechanisms specifically engineered for the purpose of representing symbols. In this paper we introduce a novel way of representing symbolic structures in connectionist terms - the vectors approach to representing symbols (VARS), which allows training standard neural architectures to encode symbolic knowledge explicitly at their output layers. In two simulations, we show that neural networks not only can learn to produce VARS representations, but in doing so they achieve combinatorial generalization in their symbolic and non-symbolic output. This adds to other recent work that has shown improved combinatorial generalization under specific training conditions, and raises the question of whether specific mechanisms or training routines are needed to support symbolic processing.
Addressing Model Vulnerability to Distributional Shifts over Image Transformation Sets

We are concerned with the vulnerability of computer vision models to distributional shifts. We formulate a combinatorial optimization problem that allows evaluating the regions in the image space where a given model is more vulnerable, in terms of image transformations applied to the input, and face it with standard search algorithms. We further embed this idea in a training procedure, where we define new data augmentation rules according to the image transformations that the current model is most vulnerable to, over iterations. An empirical evaluation on classification and semantic segmentation problems suggests that the devised algorithm allows to train models that are more robust against content-preserving image manipulations and, in general, against distributional shifts.
Many Task Learning with Task Routing

Typical multi-task learning (MTL) methods rely on architectural adjustments and a large trainable parameter set to jointly optimize over several tasks. However, when the number of tasks increases so do the complexity of the architectural adjustments and resource requirements. In this paper, we introduce a method which applies a conditional feature-wise transformation over the convolutional activations that enables a model to successfully perform a large number of tasks. To distinguish from regular MTL, we introduce Many Task Learning (MaTL) as a special case of MTL where more than 20 tasks are performed by a single model. Our method dubbed Task Routing (TR) is encapsulated in a layer we call the Task Routing Layer (TRL), which applied in an MaTL scenario successfully fits hundreds of classification tasks in one model. We evaluate our method on 5 datasets against strong baselines and state-of-the-art approaches.
Optimal Random Sampling from Distributed Streams Revisited

We give an improved algorithm for drawing a random sample from a large data stream when the input elements are distributed across multiple sites which communicate via a central coordinator. At any point in time the set of elements held by the coordinator represent a uniform random sample from the set of all the elements observed so far. When compared with prior work, our algorithms asymptotically improve the total number of messages sent in the system as well as the computation required of the coordinator. We also present a matching lower bound, showing that our protocol sends the optimal number of messages up to a constant factor with large probability. As a byproduct, we obtain an improved algorithm for finding the heavy hitters across multiple distributed sites.
A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet

Neural speech synthesis algorithms are a promising new approach for coding speech at very low bitrate. They have so far demonstrated quality that far exceeds traditional vocoders, at the cost of very high complexity. In this work, we present a low-bitrate neural vocoder based on the LPCNet model. The use of linear prediction and sparse recurrent networks makes it possible to achieve real-time operation on general-purpose hardware. We demonstrate that LPCNet operating at 1.6 kb/s achieves significantly higher quality than MELP and that uncompressed LPCNet can exceed the quality of a waveform codec operating at low bitrate. This opens the way for new codec designs based on neural synthesis models.
Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling

In this paper, we address the problem of effectively self-training neural networks in a low-resource setting. Self-training is frequently used to automatically increase the amount of training data. However, in a low-resource scenario, it is less effective due to unreliable annotations created using self-labeling of unlabeled data. We propose to combine self-training with noise handling on the self-labeled data. Directly estimating noise on the combined clean training set and self-labeled data can lead to corruption of the clean data and hence, performs worse. Thus, we propose the Clean and Noisy Label Neural Network which trains on clean and noisy self-labeled data simultaneously by explicitly modelling clean and noisy labels separately. In our experiments on Chunking and NER, this approach performs more robustly than the baselines. Complementary to this explicit approach, noise can also be handled implicitly with the help of an auxiliary learning task. To such a complementary approach, our method is more beneficial than other baseline methods and together provides the best performance overall.
W-Net: Reinforced U-Net for Density Map Estimation

Crowd management is of paramount importance when it comes to preventing stampedes and saving lives, especially in a countries like China and India where the combined population is a third of the global population. Millions of people convene annually all around the nation to celebrate a myriad of events and crowd count estimation is the linchpin of the crowd management system that could prevent stampedes and save lives. We present a network for crowd counting which reports state of the art results on crowd counting benchmarks. Our contributions are, first, a U-Net inspired model which affords us to report state of the art results. Second, we propose an independent decoding Reinforcement branch which helps the network converge much earlier and also enables the network to estimate density maps with high Structural Similarity Index (SSIM). Third, we discuss the drawbacks of the contemporary architectures and empirically show that even though our architecture achieves state of the art results, the merit may be due to the encoder-decoder pipeline instead. Finally, we report the error analysis which shows that the contemporary line of work is at saturation and leaves certain prominent problems unsolved.
The Multiset Partition Algebra

We introduce the multiset partition algebra $\mathcal{MP}_k(\xi)$ over $F[\xi]$, where $F$ is a field of characteristic $0$ and $k$ is a positive integer. When $\xi$ is specialized to a positive integer $n$, we establish the Schur-Weyl duality between the actions of resulting algebra $\mathcal{MP}_k(n)$ and the symmetric group $S_n$ on $\text{Sym}^k(F^n)$. The construction of $\mathcal{MP}_k(\xi)$ generalizes to any vector $\lambda$ of non-negative integers yielding the algebra $\mathcal{MP}_{\lambda}(\xi)$ over $F[\xi]$ so that there is Schur-Weyl duality between the actions of $\mathcal{MP}_{\lambda}(n)$ and $S_n$ on $\text{Sym}^{\lambda}(F^n)$. We find the generating function for the multiplicity of each irreducible representation of $S_n$ in $\text{Sym}^\lambda(F^n)$, as $\lambda$ varies, in terms of a plethysm of Schur functions. As consequences we obtain an indexing set for the irreducible representations of $\mathcal{MP}_k(n)$, and the generating function for the multiplicity of an irreducible polynomial representation of $GL_n(F)$ when restricted to $S_n$. We show that $\mathcal{MP}_\lambda(\xi)$ embeds inside the partition algebra $\mathcal{P}_{|\lambda|}(\xi)$. Using this embedding, over $F$, we prove that $\mathcal{MP}_{\lambda}(\xi)$ is a cellular algebra, and $\mathcal{MP}_{\lambda}(\xi)$ is semisimple when $\xi$ is not an integer or $\xi$ is an integer such that $\xi\geq 2|\lambda|-1$. We give an insertion algorithm based on Robinson-Schensted-Knuth correspondence realizing the decomposition of $\mathcal{MP}_{\lambda}(n)$ as $\mathcal{MP}_{\lambda}(n)\times \mathcal{MP}_{\lambda}(n)$-module.
On the Benefits of Populations on the Exploitation Speed of Standard Steady-State Genetic Algorithms

It is generally accepted that populations are useful for the global exploration of multi-modal optimisation problems. Indeed, several theoretical results are available showing such advantages over single-trajectory search heuristics. In this paper we provide evidence that evolving populations via crossover and mutation may also benefit the optimisation time for hillclimbing unimodal functions. In particular, we prove bounds on the expected runtime of the standard ($\mu$+1)~GA for OneMax that are lower than its unary black box complexity and decrease in the leading constant with the population size up to $\mu=O(\sqrt{\log n})$. Our analysis suggests that the optimal mutation strategy is to flip two bits most of the time. To achieve the results we provide two interesting contributions to the theory of randomised search heuristics: 1) A novel application of drift analysis which compares absorption times of different Markov chains without defining an explicit potential function. 2) The inversion of fundamental matrices to calculate the absorption times of the Markov chains. The latter strategy was previously proposed in the literature but to the best of our knowledge this is the first time is has been used to show non-trivial bounds on expected runtimes.
Testing isomorphism of circular-arc graphs in polynomial time

A graph is said to be circular-arc if the vertices can be associated with arcs of a circle so that two vertices are adjacent if and only if the corresponding arcs overlap. It is proved that the isomorphism of circular-arc graphs can be tested by the Weisfeiler-Leman algorithm after individualization of two vertices.
BSEA-1 - A Stream Cipher Backdooring Technique

Recent years have shown that more than ever governments and intelligence agencies try to control and bypass the cryptographic means used for the protection of data. Backdooring encryption algorithms is considered as the best way to enforce cryptographic control. Implementation backdoors (at the protocol/implementation/ma\-nagement level) are generally considered. In this paper we propose to address the most critical issue of backdoors: mathematical backdoors or by-design backdoors, which are put directly at the mathematical design of the encryption algorithm. Considering a particular family (among all the possible ones) of backdoors, we present BSEA-1, a stream cipher algorithm which contains a design backdoor enabling an effective cryptanalysis. The BSEA-1 algorithm uses a 120-bit key. The exploitation of the backdoor enables to break the cipher with around 2 Kbits of knowplaintext in a few seconds.
Improved Dynamic Time Warping (DTW) Approach for Online Signature Verification

Online signature verification is the process of verifying time series signature data which is generally obtained from the tablet-based device. Unlike offline signature images, the online signature image data consists of points that are arranged in a sequence of time. The aim of this research is to develop an improved approach to map the strokes in both test and reference signatures. Current methods make use of the Dynamic Time Warping (DTW) algorithm and its variant to segment them before comparing each of its data dimension. This paper presents a modified DTW algorithm with the proposed Lost Box Recovery Algorithm aims to improve the mapping performance for online signature verification
Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Learning

Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) is a momentum version of stochastic gradient descent with properly injected Gaussian noise to find a global minimum. In this paper, non-asymptotic convergence analysis of SGHMC is given in the context of non-convex optimization, where subsampling techniques are used over an i.i.d dataset for gradient updates. Our results complement those of [RRT17] and improve on those of [GGZ18].
Down-Scaling with Learned Kernels in Multi-Scale Deep Neural Networks for Non-Uniform Single Image Deblurring

Multi-scale approach has been used for blind image / video deblurring problems to yield excellent performance for both conventional and recent deep-learning-based state-of-the-art methods. Bicubic down-sampling is a typical choice for multi-scale approach to reduce spatial dimension after filtering with a fixed kernel. However, this fixed kernel may be sub-optimal since it may destroy important information for reliable deblurring such as strong edges. We propose convolutional neural network (CNN)-based down-scale methods for multi-scale deep-learning-based non-uniform single image deblurring. We argue that our CNN-based down-scaling effectively reduces the spatial dimension of the original image, while learned kernels with multiple channels may well-preserve necessary details for deblurring tasks. For each scale, we adopt to use RCAN (Residual Channel Attention Networks) as a backbone network to further improve performance. Our proposed method yielded state-of-the-art performance on GoPro dataset by large margin. Our proposed method was able to achieve 2.59dB higher PSNR than the current state-of-the-art method by Tao. Our proposed CNN-based down-scaling was the key factor for this excellent performance since the performance of our network without it was decreased by 1.98dB. The same networks trained with GoPro set were also evaluated on large-scale Su dataset and our proposed method yielded 1.15dB better PSNR than the Tao's method. Qualitative comparisons on Lai dataset also confirmed the superior performance of our proposed method over other state-of-the-art methods.
Cyclically covering subspaces in $\mathbb{F}_2^n$

A subspace of $\mathbb{F}_2^n$ is called cyclically covering if every vector in $\mathbb{F}_2^n$ has a cyclic shift which is inside the subspace. Let $h_2(n)$ denote the largest possible codimension of a cyclically covering subspace of $\mathbb{F}_2^n$. We show that $h_2(p)= 2$ for every prime $p$ such that 2 is a primitive root modulo $p$, which, assuming Artin's conjecture, answers a question of Peter Cameron from 1991. We also prove various bounds on $h_2(ab)$ depending on $h_2(a)$ and $h_2(b)$ and extend some of our results to a more general set-up proposed by Cameron, Ellis and Raynaud.
Fast Underwater Image Enhancement for Improved Visual Perception

In this paper, we present a conditional generative adversarial network-based model for real-time underwater image enhancement. To supervise the adversarial training, we formulate an objective function that evaluates the perceptual image quality based on its global content, color, local texture, and style information. We also present EUVP, a large-scale dataset of a paired and unpaired collection of underwater images (of `poor' and `good' quality) that are captured using seven different cameras over various visibility conditions during oceanic explorations and human-robot collaborative experiments. In addition, we perform several qualitative and quantitative evaluations which suggest that the proposed model can learn to enhance underwater image quality from both paired and unpaired training. More importantly, the enhanced images provide improved performances of standard models for underwater object detection, human pose estimation, and saliency prediction. These results validate that it is suitable for real-time preprocessing in the autonomy pipeline by visually-guided underwater robots. The model and associated training pipelines are available at https://github.com/xahidbuffon/funie-gan.