OpenRaiser/Intern-Atlas huggingface dataset:
On the Convergence Proof of AMSGrad and a New Version
Venue: arXiv cs.LG 2019
Year: 2019

The adaptive moment estimation algorithm Adam (Kingma and Ba) is a popular optimizer in the training of deep neural networks. However, Reddi et al. have recently shown that the convergence proof of Adam is problematic and proposed a variant of Adam called AMSGrad as a fix. In this paper, we show that the convergence proof of AMSGrad is also problematic. Concretely, the problem in the convergence proof of AMSGrad is in handling the hyper-parameters, treating them as equal while they are not. This is also the neglected issue in the convergence proof of Adam. We provide an explicit counter-example of a simple convex optimization setting to show this neglected issue. Depending on manipulating the hyper-parameters, we present various fixes for this issue. We provide a new convergence proof for AMSGrad as the first fix. We also propose a new version of AMSGrad called AdamX as another fix. Our experiments on the benchmark dataset also support our theoretical results.
Hybrid Approaches to Detect Comments Violating Macro Norms on Reddit
Venue: arXiv cs.SI 2019
Year: 2019

In this dataset paper, we present a three-stage process to collect Reddit comments that are removed comments by moderators of several subreddits, for violating subreddit rules and guidelines. Other than the fact that these comments were flagged by moderators for violating community norms, we do not have any other information regarding the nature of the violations. Through this procedure, we collect over 2M comments removed by moderators of 100 different Reddit communities, and publicly release the data. Working with this dataset of removed comments, we identify 8 macro norms---norms that are widely enforced on most parts of Reddit. We extract these macro norms by employing a hybrid approach---classification, topic modeling, and open-coding---on comments identified to be norm violations within at least 85 out of the 100 study subreddits. Finally, we label over 40K Reddit comments removed by moderators according to the specific type of macro norm being violated, and make this dataset publicly available. By breaking down a collection of removed comments into more granular types of macro norm violation, our dataset can be used to train more nuanced machine learning classifiers for online moderation.
The Discussion on Hulls of Cyclic Codes over the Ring $\Re =\Z_4+v\Z_4$, $v^2=v$
Venue: arXiv cs.IT 2019
Year: 2019

For odd length $n$, the cyclic codes construction over $\Re= \Z_4[v]/ \langle v^2-v \rangle$ is provided. The hulls of cyclic codes over $\Re$ are studied. The average $2$-dimension $E(n)$ of the hulls of cyclic codes over $\Re$ is also conferred. Among these, the various examples of generators of hulls of cyclic codes over $\Re$ are provided, whose $\Z_4$-images are good $\Z_4$-linear codes with good parameters.
Extending planning knowledge using ontologies for goal opportunities
Venue: arXiv cs.AI 2019
Year: 2019

Approaches to goal-directed behaviour including online planning and opportunistic planning tackle a change in the environment by generating alternative goals to avoid failures or seize opportunities. However, current approaches only address unanticipated changes related to objects or object types already defined in the planning task that is being solved. This article describes a domain-independent approach that advances the state of the art by extending the knowledge of a planning task with relevant objects of new types. The approach draws upon the use of ontologies, semantic measures, and ontology alignment to accommodate newly acquired data that trigger the formulation of goal opportunities inducing a better-valued plan.
Learning to Control Highly Accelerated Ballistic Movements on Muscular Robots
Venue: arXiv cs.RO 2019
Year: 2019

High-speed and high-acceleration movements are inherently hard to control. Applying learning to the control of such motions on anthropomorphic robot arms can improve the accuracy of the control but might damage the system. The inherent exploration of learning approaches can lead to instabilities and the robot reaching joint limits at high speeds. Having hardware that enables safe exploration of high-speed and high-acceleration movements is therefore desirable. To address this issue, we propose to use robots actuated by Pneumatic Artificial Muscles (PAMs). In this paper, we present a four degrees of freedom (DoFs) robot arm that reaches high joint angle accelerations of up to 28000 deg/s^2 while avoiding dangerous joint limits thanks to the antagonistic actuation and limits on the air pressure ranges. With this robot arm, we are able to tune control parameters using Bayesian optimization directly on the hardware without additional safety considerations. The achieved tracking performance on a fast trajectory exceeds previous results on comparable PAM-driven robots. We also show that our system can be controlled well on slow trajectories with PID controllers due to careful construction considerations such as minimal bending of cables, lightweight kinematics and minimal contact between PAMs and PAMs with the links. Finally, we propose a novel technique to control the the co-contraction of antagonistic muscle pairs. Experimental results illustrate that choosing the optimal co-contraction level is vital to reach better tracking performance. Through the use of PAM-driven robots and learning, we do a small step towards the future development of robots capable of more human-like motions.
Robust Building-based Registration of Airborne LiDAR Data and Optical Imagery on Urban Scenes
Venue: arXiv cs.CV 2019
Year: 2019

The motivation of this paper is to address the problem of registering airborne LiDAR data and optical aerial or satellite imagery acquired from different platforms, at different times, with different points of view and levels of detail. In this paper, we present a robust registration method based on building regions, which are extracted from optical images using mean shift segmentation, and from LiDAR data using a 3D point cloud filtering process. The matching of the extracted building segments is then carried out using Graph Transformation Matching (GTM) which allows to determine a common pattern of relative positions of segment centers. Thanks to this registration, the relative shifts between the data sets are significantly reduced, which enables a subsequent fine registration and a resulting high-quality data fusion.
Parametrization of stochastic inputs using generative adversarial networks with application in geology
Venue: arXiv cs.LG 2019
Year: 2019

We investigate artificial neural networks as a parametrization tool for stochastic inputs in numerical simulations. We address parametrization from the point of view of emulating the data generating process, instead of explicitly constructing a parametric form to preserve predefined statistics of the data. This is done by training a neural network to generate samples from the data distribution using a recent deep learning technique called generative adversarial networks. By emulating the data generating process, the relevant statistics of the data are replicated. The method is assessed in subsurface flow problems, where effective parametrization of underground properties such as permeability is important due to the high dimensionality and presence of high spatial correlations. We experiment with realizations of binary channelized subsurface permeability and perform uncertainty quantification and parameter estimation. Results show that the parametrization using generative adversarial networks is very effective in preserving visual realism as well as high order statistics of the flow responses, while achieving a dimensionality reduction of two orders of magnitude.
AI Meets Austen: Towards Human-Robot Discussions of Literary Metaphor
Venue: arXiv cs.HC 2019
Year: 2019

Artificial intelligence is revolutionizing formal education, fueled by innovations in learning assessment, content generation, and instructional delivery. Informal, lifelong learning settings have been the subject of less attention. We provide a proof-of-concept for an embodied book discussion companion, designed to stimulate conversations with readers about particularly creative metaphors in fiction literature. We collect ratings from 26 participants, each of whom discuss Jane Austen's "Pride and Prejudice" with the robot across one or more sessions, and find that participants rate their interactions highly. This suggests that companion robots could be an interesting entryway for the promotion of lifelong learning and cognitive exercise in future applications.
Graph pattern detection: Hardness for all induced patterns and faster non-induced cycles
Venue: arXiv cs.CC 2019
Year: 2019

We consider the pattern detection problem in graphs: given a constant size pattern graph $H$ and a host graph $G$, determine whether $G$ contains a subgraph isomorphic to $H$. Our main results are: * We prove that if a pattern $H$ contains a $k$-clique subgraph, then detecting whether an $n$ node host graph contains a not necessarily induced copy of $H$ requires at least the time for detecting whether an $n$ node graph contains a $k$-clique. The previous result of this nature required that $H$ contains a $k$-clique which is disjoint from all other $k$-cliques of $H$. * We show that if the famous Hadwiger conjecture from graph theory is true, then detecting whether an $n$ node host graph contains a not necessarily induced copy of a pattern with chromatic number $t$ requires at least the time for detecting whether an $n$ node graph contains a $t$-clique. This implies that: (1) under Hadwiger's conjecture for every $k$-node pattern $H$, finding an induced copy of $H$ requires at least the time of $\sqrt k$-clique detection, and at least size $ω(n^{\sqrt{k}/4})$ for any constant depth circuit, and (2) unconditionally, detecting an induced copy of a random $G(k,p)$ pattern w.h.p. requires at least the time of $Θ(k/\log k)$-clique detection, and hence also at least size $n^{Ω(k/\log k)}$ for circuits of constant depth. * Finally, we consider the case when the pattern is a directed cycle on $k$ nodes, and we would like to detect whether a directed $m$-edge graph $G$ contains a $k$-Cycle as a not necessarily induced subgraph. We resolve a 14 year old conjecture of [Yuster-Zwick SODA'04] on the complexity of $k$-Cycle detection by giving a tight analysis of their $k$-Cycle algorithm. Our analysis improves the best bounds for $k$-Cycle detection in directed graphs, for all $k>5$.
Information Bottleneck and its Applications in Deep Learning
Venue: arXiv cs.LG 2019
Year: 2019

Information Theory (IT) has been used in Machine Learning (ML) from early days of this field. In the last decade, advances in Deep Neural Networks (DNNs) have led to surprising improvements in many applications of ML. The result has been a paradigm shift in the community toward revisiting previous ideas and applications in this new framework. Ideas from IT are no exception. One of the ideas which is being revisited by many researchers in this new era, is Information Bottleneck (IB); a formulation of information extraction based on IT. The IB is promising in both analyzing and improving DNNs. The goal of this survey is to review the IB concept and demonstrate its applications in deep learning. The information theoretic nature of IB, makes it also a good candidate in showing the more general concept of how IT can be used in ML. Two important concepts are highlighted in this narrative on the subject, i) the concise and universal view that IT provides on seemingly unrelated methods of ML, demonstrated by explaining how IB relates to minimal sufficient statistics, stochastic gradient descent, and variational auto-encoders, and ii) the common technical mistakes and problems caused by applying ideas from IT, which is discussed by a careful study of some recent methods suffering from them.
Towards Real-Time Automatic Portrait Matting on Mobile Devices
Venue: arXiv cs.CV 2019
Year: 2019

We tackle the problem of automatic portrait matting on mobile devices. The proposed model is aimed at attaining real-time inference on mobile devices with minimal degradation of model performance. Our model MMNet, based on multi-branch dilated convolution with linear bottleneck blocks, outperforms the state-of-the-art model and is orders of magnitude faster. The model can be accelerated four times to attain 30 FPS on Xiaomi Mi 5 device with moderate increase in the gradient error. Under the same conditions, our model has an order of magnitude less number of parameters and is faster than Mobile DeepLabv3 while maintaining comparable performance. The accompanied implementation can be found at \url{https://github.com/hyperconnect/MMNet}.
Evolved embodied phase coordination enables robust quadruped robot locomotion
Venue: arXiv cs.RO 2019
Year: 2019

Overcoming robotics challenges in the real world requires resilient control systems capable of handling a multitude of environments and unforeseen events. Evolutionary optimization using simulations is a promising way to automatically design such control systems, however, if the disparity between simulation and the real world becomes too large, the optimization process may result in dysfunctional real-world behaviors. In this paper, we address this challenge by considering embodied phase coordination in the evolutionary optimization of a quadruped robot controller based on central pattern generators. With this method, leg phases, and indirectly also inter-leg coordination, are influenced by sensor feedback.By comparing two very similar control systems we gain insight into how the sensory feedback approach affects the evolved parameters of the control system, and how the performances differs in simulation, in transferal to the real world, and to different real-world environments. We show that evolution enables the design of a control system with embodied phase coordination which is more complex than previously seen approaches, and that this system is capable of controlling a real-world multi-jointed quadruped robot.The approach reduces the performance discrepancy between simulation and the real world, and displays robustness towards new environments.
From independent sets and vertex colorings to isotropic spaces and isotropic decompositions
Venue: arXiv cs.DS 2019
Year: 2019

In the 1970's, Lovász built a bridge between graphs and alternating matrix spaces, in the context of perfect matchings (FCT 1979). A similar connection between bipartite graphs and matrix spaces plays a key role in the recent resolutions of the non-commutative rank problem (Garg-Gurvits-Oliveira-Wigderson, FOCS 2016; Ivanyos-Qiao-Subrahmanyam, ITCS 2017). In this paper, we lay the foundation for another bridge between graphs and alternating matrix spaces, in the context of independent sets and vertex colorings. The corresponding structures in alternating matrix spaces are isotropic spaces and isotropic decompositions, both useful structures in group theory and manifold theory. We first show that the maximum independent set problem and the vertex c-coloring problem reduce to the maximum isotropic space problem and the isotropic c-decomposition problem, respectively. Next, we show that several topics and results about independent sets and vertex colorings have natural correspondences for isotropic spaces and decompositions. These include algorithmic problems, such as the maximum independent set problem for bipartite graphs, and exact exponential-time algorithms for the chromatic number, as well as mathematical questions, such as the number of maximal independent sets, and the relation between the maximum degree and the chromatic number. These connections lead to new interactions between graph theory and algebra. Some results have concrete applications to group theory and manifold theory, and we initiate a variant of these structures in the context of quantum information theory. Finally, we propose several open questions for further exploration. This paper is dedicated to the memory of Ker-I Ko.
Towards Motion Invariant Authentication for On-Body IoT Devices
Venue: arXiv cs.CR 2019
Year: 2019

As the rapid proliferation of on-body Internet of Things (IoT) devices, their security vulnerabilities have raised serious privacy and safety issues. Traditional efforts to secure these devices against impersonation attacks mainly rely on either dedicated sensors or specified user motions, impeding their wide-scale adoption. This paper transcends these limitations with a general security solution by leveraging ubiquitous wireless chips available in IoT devices. Particularly, representative time and frequency features are first extracted from received signal strengths (RSSs) to characterize radio propagation profiles. Then, an adversarial multi-player network is developed to recognize underlying radio propagation patterns and facilitate on-body device authentication. We prove that at equilibrium, our adversarial model can extract all information about propagation patterns and eliminate any irrelevant information caused by motion variances. We build a prototype of our system using universal software radio peripheral (USRP) devices and conduct extensive experiments with both static and dynamic body motions in typical indoor and outdoor environments. The experimental results show that our system achieves an average authentication accuracy of 90.4%, with a high area under the receiver operating characteristic curve (AUROC) of 0.958 and better generalization performance in comparison with the conventional non-adversarial-based approach.
OpenMX Viewer: A web-based crystalline and molecular graphical user interface program
Venue: arXiv cs.HC 2019
Year: 2019

The OpenMX Viewer (Open source package for Material eXplorer Viewer) is a web-based graphical user interface (GUI) program for visualization and analysis of crystalline and molecular structures and 3D grid data in the Gaussian cube format such as electron density and molecular orbitals. The web-based GUI program enables us to quickly visualize crystalline and molecular structures by dragging and dropping XYZ, CIF, or OpenMX input/output files, and analyze static/dynamic structural properties conveniently in a web browser. Several basic functionalities such as analysis of Mulliken charges, molecular dynamics, geometry optimization and band structure are included. In addition, based on marching cubes, marching tetrahedra and surface nets algorithms with Affine transformation, 3D isosurface techniques are supported to visualize electron density and molecular/crystalline orbitals in the cube format with superposition of a crystalline or molecular structure. Furthermore, the Band Structure Viewer is implemented for showing a band structure in a web browser. By accessing the website of the OpenMX Viewer, the latest OpenMX Viewer is always available for users to visualize various structures and analyze their properties without installations, upgrades, updates, registration, sign-in and terminal commands.
Task Oriented Channel State Information Quantization
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we propose a new perspective for quantizing a signal and more specifically the channel state information (CSI). The proposed point of view is fully relevant for a receiver which has to send a quantized version of the channel state to the transmitter. Roughly, the key idea is that the receiver sends the right amount of information to the transmitter so that the latter be able to take its (resource allocation) decision. More formally, the decision task of the transmitter is to maximize an utility function u(x;g) with respect to x (e.g., a power allocation vector) given the knowledge of a quantized version of the function parameters g. We exhibit a special case of an energy-efficient power control (PC) problem for which the optimal task oriented CSI quantizer (TOCQ) can be found analytically. For more general utility functions, we propose to use neural networks (NN) based learning. Simulations show that the compression rate obtained by adapting the feedback information rate to the function to be optimized may be significantly increased.
A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relations
Venue: arXiv cs.LG 2019
Year: 2019

We investigate the use of discrete and continuous versions of physics-informed neural network methods for learning unknown dynamics or constitutive relations of a dynamical system. For the case of unknown dynamics, we represent all the dynamics with a deep neural network (DNN). When the dynamics of the system are known up to the specification of constitutive relations (that can depend on the state of the system), we represent these constitutive relations with a DNN. The discrete versions combine classical multistep discretization methods for dynamical systems with neural network based machine learning methods. On the other hand, the continuous versions utilize deep neural networks to minimize the residual function for the continuous governing equations. We use the case of a fedbatch bioreactor system to study the effectiveness of these approaches and discuss conditions for their applicability. Our results indicate that the accuracy of the trained neural network models is much higher for the cases where we only have to learn a constitutive relation instead of the whole dynamics. This finding corroborates the well-known fact from scientific computing that building as much structural information is available into an algorithm can enhance its efficiency and/or accuracy.
Weighted Point Cloud Augmentation for Neural Network Training Data Class-Imbalance
Venue: arXiv cs.CV 2019
Year: 2019

Recent developments in the field of deep learning for 3D data have demonstrated promising potential for end-to-end learning directly from point clouds. However, many real-world point clouds contain a large class im-balance due to the natural class im-balance observed in nature. For example, a 3D scan of an urban environment will consist mostly of road and facade, whereas other objects such as poles will be under-represented. In this paper we address this issue by employing a weighted augmentation to increase classes that contain fewer points. By mitigating the class im-balance present in the data we demonstrate that a standard PointNet++ deep neural network can achieve higher performance at inference on validation data. This was observed as an increase of F1 score of 19% and 25% on two test benchmark datasets; ScanNet and Semantic3D respectively where no class im-balance pre-processing had been performed. Our networks performed better on both highly-represented and under-represented classes, which indicates that the network is learning more robust and meaningful features when the loss function is not overly exposed to only a few classes.