Reviewerly/exHarmony huggingface dataset:
Model-based Clustering using Automatic Differentiation: Confronting Misspecification and High-Dimensional Data

We study two practically important cases of model based clustering using Gaussian Mixture Models: (1) when there is misspecification and (2) on high dimensional data, in the light of recent advances in Gradient Descent (GD) based optimization using Automatic Differentiation (AD). Our simulation studies show that EM has better clustering performance, measured by Adjusted Rand Index, compared to GD in cases of misspecification, whereas on high dimensional data GD outperforms EM. We observe that both with EM and GD there are many solutions with high likelihood but poor cluster interpretation. To address this problem we design a new penalty term for the likelihood based on the Kullback Leibler divergence between pairs of fitted components. Closed form expressions for the gradients of this penalized likelihood are difficult to derive but AD can be done effortlessly, illustrating the advantage of AD-based optimization. Extensions of this penalty for high dimensional data and for model selection are discussed. Numerical experiments on synthetic and real datasets demonstrate the efficacy of clustering using the proposed penalized likelihood approach.
StructureBoost: Efficient Gradient Boosting for Structured Categorical Variables

Gradient boosting methods based on Structured Categorical Decision Trees (SCDT) have been demonstrated to outperform numerical and one-hot-encodings on problems where the categorical variable has a known underlying structure. However, the enumeration procedure in the SCDT is infeasible except for categorical variables with low or moderate cardinality. We propose and implement two methods to overcome the computational obstacles and efficiently perform Gradient Boosting on complex structured categorical variables. The resulting package, called StructureBoost, is shown to outperform established packages such as CatBoost and LightGBM on problems with categorical predictors that contain sophisticated structure. Moreover, we demonstrate that StructureBoost can make accurate predictions on unseen categorical values due to its knowledge of the underlying structure.
Quaternion Capsule Networks

Capsules are grouping of neurons that allow to represent sophisticated information of a visual entity such as pose and features. In the view of this property, Capsule Networks outperform CNNs in challenging tasks like object recognition in unseen viewpoints, and this is achieved by learning the transformations between the object and its parts with the help of high dimensional representation of pose information. In this paper, we present Quaternion Capsules (QCN) where pose information of capsules and their transformations are represented by quaternions. Quaternions are immune to the gimbal lock, have straightforward regularization of the rotation representation for capsules, and require less number of parameters than matrices. The experimental results show that QCNs generalize better to novel viewpoints with fewer parameters, and also achieve on-par or better performances with the state-of-the-art Capsule architectures on well-known benchmarking datasets.
Sharp Analysis of Smoothed Bellman Error Embedding

The \textit{Smoothed Bellman Error Embedding} algorithm~\citep{dai2018sbeed}, known as SBEED, was proposed as a provably convergent reinforcement learning algorithm with general nonlinear function approximation. It has been successfully implemented with neural networks and achieved strong empirical results. In this work, we study the theoretical behavior of SBEED in batch-mode reinforcement learning. We prove a near-optimal performance guarantee that depends on the representation power of the used function classes and a tight notion of the distribution shift. Our results improve upon prior guarantees for SBEED in ~\citet{dai2018sbeed} in terms of the dependence on the planning horizon and on the sample size. Our analysis builds on the recent work of ~\citet{Xie2020} which studies a related algorithm MSBO, that could be interpreted as a \textit{non-smooth} counterpart of SBEED.
PathGAN: Local Path Planning with Attentive Generative Adversarial Networks

To achieve autonomous driving without high-definition maps, we present a model capable of generating multiple plausible paths from egocentric images for autonomous vehicles. Our generative model comprises two neural networks: the feature extraction network (FEN) and path generation network (PGN). The FEN extracts meaningful features from an egocentric image, whereas the PGN generates multiple paths from the features, given a driving intention and speed. To ensure that the paths generated are plausible and consistent with the intention, we introduce an attentive discriminator and train it with the PGN under generative adversarial networks framework. We also devise an interaction model between the positions in the paths and the intentions hidden in the positions and design a novel PGN architecture that reflects the interaction model, resulting in the improvement of the accuracy and diversity of the generated paths. Finally, we introduce ETRIDriving, a dataset for autonomous driving in which the recorded sensor data are labeled with discrete high-level driving actions, and demonstrate the state-of-the-art performance of the proposed model on ETRIDriving in terms of accuracy and diversity.
Guided Exploration with Proximal Policy Optimization using a Single Demonstration

Solving sparse reward tasks through exploration is one of the major challenges in deep reinforcement learning, especially in three-dimensional, partially-observable environments. Critically, the algorithm proposed in this article uses a single human demonstration to solve hard-exploration problems. We train an agent on a combination of demonstrations and own experience to solve problems with variable initial conditions. We adapt this idea and integrate it with the proximal policy optimization (PPO). The agent is able to increase its performance and to tackle harder problems by replaying its own past trajectories prioritizing them based on the obtained reward and the maximum value of the trajectory. We compare different variations of this algorithm to behavioral cloning on a set of hard-exploration tasks in the Animal-AI Olympics environment. To the best of our knowledge, learning a task in a three-dimensional environment with comparable difficulty has never been considered before using only one human demonstration.
C2G-Net: Exploiting Morphological Properties for Image Classification

In this paper we propose C2G-Net, a pipeline for image classification that exploits the morphological properties of images containing a large number of similar objects like biological cells. C2G-Net consists of two components: (1) Cell2Grid, an image compression algorithm that identifies objects using segmentation and arranges them on a grid, and (2) DeepLNiNo, a CNN architecture with less than 10,000 trainable parameters aimed at facilitating model interpretability. To test the performance of C2G-Net we used multiplex immunohistochemistry images for predicting relapse risk in colon cancer. Compared to conventional CNN architectures trained on raw images, C2G-Net achieved similar prediction accuracy while training time was reduced by 85% and its model was is easier to interpret.
Topological 4-manifolds with 4-dimensional fundamental group

Let $π$ be a group satisfying the Farrell-Jones conjecture and assume that $Bπ$ is a 4-dimensional Poincaré duality space. We consider topological, closed, connected manifolds with fundamental group $π$ whose canonical map to $Bπ$ has degree 1 and show that two such manifolds are s-cobordant if and only if their equivariant intersection forms are isometric and they have the same Kirby-Siebenmann invariant. If $π$ is good in the sense of Freedman, it follows that two such manifolds are homeomorphic if and only if they are homotopy equivalent and have the same Kirby--Siebenmann invariant. This shows rigidity in many cases that lie between aspherical 4-manifolds, where rigidity is expected by Borel's conjecture, and simply connected manifolds where rigidity is a consequence of Freedman's classification results.
Fast Training of Deep Neural Networks Robust to Adversarial Perturbations

Deep neural networks are capable of training fast and generalizing well within many domains. Despite their promising performance, deep networks have shown sensitivities to perturbations of their inputs (e.g., adversarial examples) and their learned feature representations are often difficult to interpret, raising concerns about their true capability and trustworthiness. Recent work in adversarial training, a form of robust optimization in which the model is optimized against adversarial examples, demonstrates the ability to improve performance sensitivities to perturbations and yield feature representations that are more interpretable. Adversarial training, however, comes with an increased computational cost over that of standard (i.e., nonrobust) training, rendering it impractical for use in large-scale problems. Recent work suggests that a fast approximation to adversarial training shows promise for reducing training time and maintaining robustness in the presence of perturbations bounded by the infinity norm. In this work, we demonstrate that this approach extends to the Euclidean norm and preserves the human-aligned feature representations that are common for robust models. Additionally, we show that using a distributed training scheme can further reduce the time to train robust deep networks. Fast adversarial training is a promising approach that will provide increased security and explainability in machine learning applications for which robust optimization was previously thought to be impractical.
DRIVE: A Digital Network Oracle for Cooperative Intelligent Transportation Systems

In a world where Artificial Intelligence revolutionizes inference, prediction and decision-making tasks, Digital Twins emerge as game-changing tools. A case in point is the development and optimization of Cooperative Intelligent Transportation Systems (C-ITSs): a confluence of cyber-physical digital infrastructure and (semi)automated mobility. Herein we introduce Digital Twin for self-dRiving Intelligent VEhicles (DRIVE). The developed framework tackles shortcomings of traditional vehicular and network simulators. It provides a flexible, modular, and scalable implementation to ensure large-scale, city-wide experimentation with a moderate computational cost. The defining feature of our Digital Twin is a unique architecture allowing for submission of sequential queries, to which the Digital Twin provides instantaneous responses with the "state of the world", and hence is an Oracle. With such bidirectional interaction with external intelligent agents and realistic mobility traces, DRIVE provides the environment for development, training and optimization of Machine Learning based C-ITS solutions.
Optimal Witnessing of Healthcare IoT Data Using Blockchain Logging Contract

Verification of data generated by wearable sensors is increasingly becoming of concern to health service providers and insurance companies. There is a need for a verification framework that various authorities can request a verification service for the local network data of a target IoT device. In this paper, we leverage blockchain as a distributed platform to realize an on-demand verification scheme. This allows authorities to automatically transact with connected devices for witnessing services. A public request is made for witness statements on the data of a target IoT that is transmitted on its local network, and subsequently, devices (in close vicinity of the target IoT) offer witnessing service. Our contributions are threefold: (1) We develop a system architecture based on blockchain and smart contract that enables authorities to dynamically avail a verification service for data of a subject device from a distributed set of witnesses which are willing to provide (in a privacy-preserving manner) their local wireless measurement in exchange of monetary return; (2) We then develop a method to optimally select witnesses in such a way that the verification error is minimized subject to monetary cost constraints; (3) Lastly, we evaluate the efficacy of our scheme using real Wi-Fi session traces collected from a five-storeyed building with more than thirty access points, representative of a hospital. According to the current pricing schedule of the Ethereum public blockchain, our scheme enables healthcare authorities to verify data transmitted from a typical wearable device with the verification error of the order 0.01% at cost of less than two dollars for one-hour witnessing service.
Provably Safe PAC-MDP Exploration Using Analogies

A key challenge in applying reinforcement learning to safety-critical domains is understanding how to balance exploration (needed to attain good performance on the task) with safety (needed to avoid catastrophic failure). Although a growing line of work in reinforcement learning has investigated this area of "safe exploration," most existing techniques either 1) do not guarantee safety during the actual exploration process; and/or 2) limit the problem to a priori known and/or deterministic transition dynamics with strong smoothness assumptions. Addressing this gap, we propose Analogous Safe-state Exploration (ASE), an algorithm for provably safe exploration in MDPs with unknown, stochastic dynamics. Our method exploits analogies between state-action pairs to safely learn a near-optimal policy in a PAC-MDP sense. Additionally, ASE also guides exploration towards the most task-relevant states, which empirically results in significant improvements in terms of sample efficiency, when compared to existing methods.
Detile: Fine-Grained Information Leak Detection in Script Engines

Memory disclosure attacks play an important role in the exploitation of memory corruption vulnerabilities. By analyzing recent research, we observe that bypasses of defensive solutions that enforce control-flow integrity or attempt to detect return-oriented programming require memory disclosure attacks as a fundamental first step. However, research lags behind in detecting such information leaks. In this paper, we tackle this problem and present a system for fine-grained, automated detection of memory disclosure attacks against scripting engines. The basic insight is as follows: scripting languages, such as JavaScript in web browsers, are strictly sandboxed. They must not provide any insights about the memory layout in their contexts. In fact, any such information potentially represents an ongoing memory disclosure attack. Hence, to detect information leaks, our system creates a clone of the scripting engine process with a re-randomized memory layout. The clone is instrumented to be synchronized with the original process. Any inconsistency in the script contexts of both processes appears when a memory disclosure was conducted to leak information about the memory layout. Based on this detection approach, we have designed and implemented Detile (\underline{det}ection of \underline{i}nformation \underline{le}aks), a prototype for the JavaScript engine in Microsoft's Internet Explorer 10/11 on Windows 8.0/8.1. An empirical evaluation shows that our tool can successfully detect memory disclosure attacks even against this proprietary software.
Confounding Ghost Channels and Causality: A New Approach to Causal Information Flows

Information theory provides a fundamental framework for the quantification of information flows through channels, formally Markov kernels. However, quantities such as mutual information and conditional mutual information do not necessarily reflect the causal nature of such flows. We argue that this is often the result of conditioning based on sigma algebras that are not associated with the given channels. We propose a version of the (conditional) mutual information based on families of sigma algebras that are coupled with the underlying channel. This leads to filtrations which allow us to prove a corresponding causal chain rule as a basic requirement within the presented approach.
A Term-Rewriting Semantics for Imperative Style Programming

Term rewriting systems have a simple syntax and semantics and facilitate proofs of correctness. However, they are not as popular in industry or academia as imperative languages. We define a term rewriting based abstract programming language with an imperative style and a precise semantics allowing programs to be translatable into efficient imperative languages, to obtain proofs of correctness together with efficient execution. This language is designed to facilitate translations into correct programs in imperative languages with assignment statements, iteration, recursion, arrays, pointers, and side effects. It can also be used in place of a pseudo-programming language to specify algorithms.
Covariate Distribution Aware Meta-learning

Meta-learning has proven to be successful for few-shot learning across the regression, classification, and reinforcement learning paradigms. Recent approaches have adopted Bayesian interpretations to improve gradient-based meta-learners by quantifying the uncertainty of the post-adaptation estimates. Most of these works almost completely ignore the latent relationship between the covariate distribution $(p(x))$ of a task and the corresponding conditional distribution $p(y|x)$. In this paper, we identify the need to explicitly model the meta-distribution over the task covariates in a hierarchical Bayesian framework. We begin by introducing a graphical model that leverages the samples from the marginal $p(x)$ to better infer the posterior over the optimal parameters of the conditional distribution $(p(y|x))$ for each task. Based on this model we propose a computationally feasible meta-learning algorithm by introducing meaningful relaxations in our final objective. We demonstrate the gains of our algorithm over initialization based meta-learning baselines on popular classification benchmarks. Finally, to understand the potential benefit of modeling task covariates we further evaluate our method on a synthetic regression dataset.
Sub-picosecond all-optical switching in a hybrid VO2:silicon waveguide at 1550 nm

Achieving ultrafast all-optical switching in a silicon waveguide geometry is a key milestone on the way to an integrated platform capable of handling the increasing demands for higher speed and higher capacity for information transfer. Given the weak electro-optic and thermo-optic effects in silicon, there has been intense interest in hybrid structures in which that switching could be accomplished by integrating another material into the waveguide, including the phase-changing material, vanadium dioxide (VO2). It has long been known that the phase transition in VO2 can be triggered by ultrafast laser pulses, and that pump-laser fluence is a critical parameter governing the recovery time of thin films irradiated by femtosecond laser pulses near 800 nm. However, thin-film experiments are not a priori reliable guides to using VO2 for all-optical switching in on-chip silicon photonics because of the large changes in VO2 optical constants in the telecommunications band, the requirement of low insertion loss, and the limits on switching energy permissible in integrated photonic systems. Here we report the first measurements to show that the reversible, ultrafast photo-induced phase transition in VO2 can be harnessed to achieve sub-picosecond switching when small VO2 volumes are integrated in a silicon waveguide as a modulating element. Switching energies above threshold are of order 600 fJ/switch. These results suggest that VO2 can now be pursued as a strong candidate for all-optical switching with sub-picosecond on-off times.
Gradient Origin Networks

This paper proposes a new type of generative model that is able to quickly learn a latent representation without an encoder. This is achieved using empirical Bayes to calculate the expectation of the posterior, which is implemented by initialising a latent vector with zeros, then using the gradient of the log-likelihood of the data with respect to this zero vector as new latent points. The approach has similar characteristics to autoencoders, but with a simpler architecture, and is demonstrated in a variational autoencoder equivalent that permits sampling. This also allows implicit representation networks to learn a space of implicit functions without requiring a hypernetwork, retaining their representation advantages across datasets. The experiments show that the proposed method converges faster, with significantly lower reconstruction error than autoencoders, while requiring half the parameters.
Benefiting from Bicubically Down-Sampled Images for Learning Real-World Image Super-Resolution

Super-resolution (SR) has traditionally been based on pairs of high-resolution images (HR) and their low-resolution (LR) counterparts obtained artificially with bicubic downsampling. However, in real-world SR, there is a large variety of realistic image degradations and analytically modeling these realistic degradations can prove quite difficult. In this work, we propose to handle real-world SR by splitting this ill-posed problem into two comparatively more well-posed steps. First, we train a network to transform real LR images to the space of bicubically downsampled images in a supervised manner, by using both real LR/HR pairs and synthetic pairs. Second, we take a generic SR network trained on bicubically downsampled images to super-resolve the transformed LR image. The first step of the pipeline addresses the problem by registering the large variety of degraded images to a common, well understood space of images. The second step then leverages the already impressive performance of SR on bicubically downsampled images, sidestepping the issues of end-to-end training on datasets with many different image degradations. We demonstrate the effectiveness of our proposed method by comparing it to recent methods in real-world SR and show that our proposed approach outperforms the state-of-the-art works in terms of both qualitative and quantitative results, as well as results of an extensive user study conducted on several real image datasets.
Announcing CzEng 2.0 Parallel Corpus with over 2 Gigawords

We present a new release of the Czech-English parallel corpus CzEng 2.0 consisting of over 2 billion words (2 "gigawords") in each language. The corpus contains document-level information and is filtered with several techniques to lower the amount of noise. In addition to the data in the previous version of CzEng, it contains new authentic and also high-quality synthetic parallel data. CzEng is freely available for research and educational purposes.
News Sentiment Analysis

Modern technological era has reshaped traditional lifestyle in several domains. The medium of publishing news and events has become faster with the advancement of Information Technology. IT has also been flooded with immense amounts of data, which is being published every minute of every day, by millions of users, in the shape of comments, blogs, news sharing through blogs, social media micro-blogging websites and many more. Manual traversal of such huge data is a challenging job, thus, sophisticated methods are acquired to perform this task automatically and efficiently. News reports events that comprise of emotions - good, bad, neutral. Sentiment analysis is utilized to investigate human emotions present in textual information. This paper presents a lexicon-based approach for sentiment analysis of news articles. The experiments have been performed on BBC news data set, which expresses the applicability and validation of the adopted approach.