OpenRaiser/Intern-Atlas huggingface dataset:
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima
Venue: arXiv cs.LG 2018
Year: 2018

We consider the problem of learning a one-hidden-layer neural network with non-overlapping convolutional layer and ReLU activation, i.e., $f(\mathbf{Z}, \mathbf{w}, \mathbf{a}) = \sum_j a_jσ(\mathbf{w}^T\mathbf{Z}_j)$, in which both the convolutional weights $\mathbf{w}$ and the output weights $\mathbf{a}$ are parameters to be learned. When the labels are the outputs from a teacher network of the same architecture with fixed weights $(\mathbf{w}^*, \mathbf{a}^*)$, we prove that with Gaussian input $\mathbf{Z}$, there is a spurious local minimizer. Surprisingly, in the presence of the spurious local minimizer, gradient descent with weight normalization from randomly initialized weights can still be proven to recover the true parameters with constant probability, which can be boosted to probability $1$ with multiple restarts. We also show that with constant probability, the same procedure could also converge to the spurious local minimum, showing that the local minimum plays a non-trivial role in the dynamics of gradient descent. Furthermore, a quantitative analysis shows that the gradient descent dynamics has two phases: it starts off slow, but converges much faster after several iterations.
Algebraic Soft Decoding of Reed-Solomon Codes Using Module Minimization
Venue: arXiv cs.IT 2018
Year: 2018

The interpolation based algebraic decoding for Reed-Solomon (RS) codes can correct errors beyond half of the code's minimum Hamming distance. Using soft information, the algebraic soft decoding (ASD) further improves the decoding performance. This paper presents a unified study of two classical ASD algorithms in which the computationally expensive interpolation is solved by the module minimization (MM) technique. An explicit module basis construction for the two ASD algorithms will be introduced. Compared with Koetter's interpolation, the MM interpolation enables the algebraic Chase decoding and the Koetter-Vardy decoding perform less finite field arithmetic operations. Re-encoding transform is applied to further reduce the decoding complexity. Computational cost of the two ASD algorithms as well as their re-encoding transformed variants are analyzed. This research shows re-encoding transform attributes to a lower decoding complexity by reducing the degree of module generators. Furthermore, Monte-Carlo simulation of the two ASD algorithms has been performed to show their decoding and complexity competency.
Generalizing inference systems by coaxioms
Venue: arXiv cs.LO 2018
Year: 2018

After surveying classical results, we introduce a generalized notion of inference system to support structural recursion on non-well-founded data types. Besides axioms and inference rules with the usual meaning, a generalized inference system allows coaxioms, which are, intuitively, axioms which can only be applied "at infinite depth" in a proof tree. This notion nicely subsumes standard inference systems and their inductive and coinductive interpretation, while providing more flexibility. Indeed, the classical results can be extended to our generalized framework, interpreting recursive definitions as fixed points which are not necessarily the least, nor the greatest one. This allows formal reasoning in cases where the inductive and coinductive interpretation do not provide the intended meaning, or are mixed together.
On data recovery with restraints on the spectrum range and the process range
Venue: arXiv cs.IT 2018
Year: 2018

The paper considers recovery of signals from incomplete observations and a problem of determination of the allowed quantity of missed observations, i.e. the problem of determination of the size of the uniqueness sets for a given data recovery procedures. The paper suggests a way to bypass solution of this uniqueness problem via imposing restrictions investigates possibility of data recovery for classes of finite sequences under a special discretization of the process range. It is shown that these sequences can be dense in the space of all sequences and that the uniqueness sets for them can be singletons. Some robustness with respect to rounding of input data can be achieved via including additional observations.
An All-Memristor Deep Spiking Neural Computing System: A Step Towards Realizing the Low Power,Stochastic Brain
Venue: arXiv cs.ET 2018
Year: 2018

Deep 'Analog Artificial Neural Networks' (ANNs) perform complex classification problems with remarkably high accuracy. However, they rely on humongous amount of power to perform the calculations, veiling the accuracy benefits. The biological brain on the other hand is significantly more powerful than such networks and consumes orders of magnitude less power, indicating us about some conceptual mismatch. Given that the biological neurons communicate using energy efficient trains of spikes, and the behavior is non-deterministic, incorporating these effects in Deep Artificial Neural Networks may drive us few steps towards a more realistic neuron. In this work, we propose how the inherent stochasticity of nano-scale resistive devices can be harnessed to emulate the functionality of a spiking neuron that can be incorporated in deep stochastic Spiking Neural Networks (SNN). At the algorithmic level, we propose how the training can be modified to convert an ANN to an SNN while supporting the stochastic activation function offered by these devices. We devise circuit architectures to incorporate stochastic memristive neurons along with memristive crossbars which perform the functionality of the synaptic weights. We tested the proposed All Memristor deep stochastic SNN for image classification and observed only about 1% degradation in accuracy with the ANN baseline after incorporating the circuit and device related non-idealities. We witnessed that the network is robust to certain variations and consumes ~ 6.4x less energy than its CMOS counterpart.
Optimizing colormaps with consideration for color vision deficiency to enable accurate interpretation of scientific data
Venue: arXiv cs.CV 2018
Year: 2018

Color vision deficiency (CVD) affects more than 4% of the population and leads to a different visual perception of colors. Though this has been known for decades, colormaps with many colors across the visual spectra are often used to represent data, leading to the potential for misinterpretation or difficulty with interpretation by someone with this deficiency. Until the creation of the module presented here, there were no colormaps mathematically optimized for CVD using modern color appearance models. While there have been some attempts to make aesthetically pleasing or subjectively tolerable colormaps for those with CVD, our goal was to make optimized colormaps for the most accurate perception of scientific data by as many viewers as possible. We developed a Python module, cmaputil, to create CVD-optimized colormaps, which imports colormaps and modifies them to be perceptually uniform in CVD-safe colorspace while linearizing and maximizing the brightness range. The module is made available to the science community to enable others to easily create their own CVDoptimized colormaps. Here, we present an example CVD-optimized colormap created with this module that is optimized for viewing by those without a CVD as well as those with redgreen colorblindness. This colormap, cividis, enables nearly-identical visual-data interpretation to both groups, is perceptually uniform in hue and brightness, and increases in brightness linearly.
CNN training with graph-based sample preselection: application to handwritten character recognition
Venue: arXiv cs.LG 2018
Year: 2018

In this paper, we present a study on sample preselection in large training data set for CNN-based classification. To do so, we structure the input data set in a network representation, namely the Relative Neighbourhood Graph, and then extract some vectors of interest. The proposed preselection method is evaluated in the context of handwritten character recognition, by using two data sets, up to several hundred thousands of images. It is shown that the graph-based preselection can reduce the training data set without degrading the recognition accuracy of a non pretrained CNN shallow model.
LocNet: Global localization in 3D point clouds for mobile vehicles
Venue: arXiv cs.RO 2018
Year: 2018

Global localization in 3D point clouds is a challenging problem of estimating the pose of vehicles without any prior knowledge. In this paper, a solution to this problem is presented by achieving place recognition and metric pose estimation in the global prior map. Specifically, we present a semi-handcrafted representation learning method for LiDAR point clouds using siamese LocNets, which states the place recognition problem to a similarity modeling problem. With the final learned representations by LocNet, a global localization framework with range-only observations is proposed. To demonstrate the performance and effectiveness of our global localization system, KITTI dataset is employed for comparison with other algorithms, and also on our long-time multi-session datasets for evaluation. The result shows that our system can achieve high accuracy.
Viral content propagation in Online Social Networks
Venue: arXiv cs.SI 2018
Year: 2018

Information flows are the result of a constant exchange in Online Social Networks (OSNs). OSN users create and share varying types of information in real-time throughout a day. Virality is introduced as a term to describe information that reaches a wide audience within a small time-frame. As a case, we measure propagation of information submitted in Reddit, identify different patterns and present a multi OSN diffusion analysis on Twitter, Facebook, and 2 hosting domains for images and multimedia, ImgUr and YouTube. Our results indicate that positive content is the most shared and presents the highest virality probability, and the overall virality probability of user created information is low. Finally, we underline the problems of limited access in OSN data. Keywords: Online Social Networks, Virality, Diffusion, Viral Content, Reddit, Twitter, Facebook, ImgUr, YouTube
Generative Adversarial Perturbations
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we propose novel generative models for creating adversarial examples, slightly perturbed images resembling natural images but maliciously crafted to fool pre-trained models. We present trainable deep neural networks for transforming images to adversarial perturbations. Our proposed models can produce image-agnostic and image-dependent perturbations for both targeted and non-targeted attacks. We also demonstrate that similar architectures can achieve impressive results in fooling classification and semantic segmentation models, obviating the need for hand-crafting attack methods for each task. Using extensive experiments on challenging high-resolution datasets such as ImageNet and Cityscapes, we show that our perturbations achieve high fooling rates with small perturbation norms. Moreover, our attacks are considerably faster than current iterative methods at inference time.
Concurrent interactive visualization and handling of molecular structures over the Internet in web browsers
Venue: arXiv cs.CY 2018
Year: 2018

This preprint presents a web app (essentially a web page-based program) with which two or more users (peers) can view and handle 3D molecular structures in a concurrent, interactive way through their web browsers. This means they can share orientation and zoom level, commands and other operations in almost real time over the Internet through standard web pages. This web app, open source and built with the open source components JSmol for molecular visualization and Peer.js for WebRTC connection, provides a practical tool for online collaboration and teaching at a distance. More broadly, it illustrates the strong integrability of technologies for client-side web programming, and paves the way for similar apps for concurrent work in other disciplines. Web app is available at: http://lucianoabriata.altervista.org/jsinscience/concurrent-jsmol/concurrent-jsmol-visualization.html
Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
Venue: arXiv cs.CV 2018
Year: 2018

Learning-based pattern classifiers, including deep networks, have shown impressive performance in several application domains, ranging from computer vision to cybersecurity. However, it has also been shown that adversarial input perturbations carefully crafted either at training or at test time can easily subvert their predictions. The vulnerability of machine learning to such wild patterns (also referred to as adversarial examples), along with the design of suitable countermeasures, have been investigated in the research field of adversarial machine learning. In this work, we provide a thorough overview of the evolution of this research area over the last ten years and beyond, starting from pioneering, earlier work on the security of non-deep learning algorithms up to more recent work aimed to understand the security properties of deep learning algorithms, in the context of computer vision and cybersecurity tasks. We report interesting connections between these apparently-different lines of work, highlighting common misconceptions related to the security evaluation of machine-learning algorithms. We review the main threat models and attacks defined to this end, and discuss the main limitations of current work, along with the corresponding future challenges towards the design of more secure learning algorithms.
Efficient Data Collection Over Multiple Access Wireless Sensors Network
Venue: arXiv cs.IT 2018
Year: 2018

Data collection in Wireless Sensor Networks (WSN) draws significant attention, due to emerging interest in technologies raging from Internet of Things (IoT) networks to simple "Presence" applications, which identify the status of the devices (active or inactive). Numerous Medium Access Control (MAC) protocols for WSN, which can address the challenge of data collection in dense networks, were suggested over the years. Most of these protocols utilize the traditional layering approach, in which the MAC layer is unaware of the encapsulated packet payload, and therefore there is no connection between the data collected, the physical layer and the signaling mechanisms. Nonetheless, in many of the applications that intend to utilize such protocols, nodes may need to exchange very little information, and do so only sporadically, that is, while the number of devices in the network can be very large, only a subset wishes to transmit at any given time. Thus, a tailored protocol, which matches the signaling, physical layer and access control to traffic patterns is required. In this work, we design and analyze a data collection protocol based on information theoretic principles. In the suggested protocol, the sink collects messages from up to K sensors simultaneously, out of a large population of sensors, without knowing in advance which sensors will transmit, and without requiring any synchronization, coordination or management overhead. In other words, neither the sink nor the other sensors need to know who are the actively transmitting sensors, and this data is decoded directly from the channel output. We provide a simple codebook construction with very simple encoding and decoding procedures. We further design a secure version of the protocol.
CycleGAN Face-off
Venue: arXiv cs.CV 2018
Year: 2018

Face-off is an interesting case of style transfer where the facial expressions and attributes of one person could be fully transformed to another face. We are interested in the unsupervised training process which only requires two sequences of unaligned video frames from each person and learns what shared attributes to extract automatically. In this project, we explored various improvements for adversarial training (i.e. CycleGAN[Zhu et al., 2017]) to capture details in facial expressions and head poses and thus generate transformation videos of higher consistency and stability.
A novel algorithm for online inexact string matching and its FPGA implementation
Venue: arXiv cs.DC 2018
Year: 2018

Accelerating inexact string matching procedures is of utmost importance when dealing with practical applications where huge amount of data must be processed in real time, as usual in bioinformatics or cybersecurity. Inexact matching procedures can yield multiple shadow hits, which must be filtered, according to some criterion, to obtain a concise and meaningful list of occurrences. The filtering procedures are often computationally demanding and are performed offline in a post-processing phase. This paper introduces a novel algorithm for Online Approximate String Matching (OASM) able to filter shadow hits on the fly, according to general purpose priority rules that greedily assign priorities to overlapping hits. An FPGA hardware implementation of OASM is proposed and compared with a serial software version. Even when implemented on entry level FPGAs, the proposed procedure can reach a high degree of parallelism and superior performance in time compared to the software implementation, while keeping low the usage of logic elements. This makes the developed architecture very competitive in terms of both performance and cost of the overall computing system.
Static/Dynamic Filtering for Mesh Geometry
Venue: arXiv cs.GR 2018
Year: 2018

The joint bilateral filter, which enables feature-preserving signal smoothing according to the structural information from a guidance, has been applied for various tasks in geometry processing. Existing methods either rely on a static guidance that may be inconsistent with the input and lead to unsatisfactory results, or a dynamic guidance that is automatically updated but sensitive to noises and outliers. Inspired by recent advances in image filtering, we propose a new geometry filtering technique called static/dynamic filter, which utilizes both static and dynamic guidances to achieve state-of-the-art results. The proposed filter is based on a nonlinear optimization that enforces smoothness of the signal while preserving variations that correspond to features of certain scales. We develop an efficient iterative solver for the problem, which unifies existing filters that are based on static or dynamic guidances. The filter can be applied to mesh face normals followed by vertex position update, to achieve scale-aware and feature-preserving filtering of mesh geometry. It also works well for other types of signals defined on mesh surfaces, such as texture colors. Extensive experimental results demonstrate the effectiveness of the proposed filter for various geometry processing applications such as mesh denoising, geometry feature enhancement, and texture color filtering.
Usability of Humanly Computable Passwords
Venue: arXiv cs.HC 2018
Year: 2018

Reusing passwords across multiple websites is a common practice that compromises security. Recently, Blum and Vempala have proposed password strategies to help people calculate, in their heads, passwords for different sites without dependence on third-party tools or external devices. Thus far, the security and efficiency of these "mental algorithms" has been analyzed only theoretically. But are such methods usable? We present the first usability study of humanly computable password strategies, involving a learning phase (to learn a password strategy), then a rehearsal phase (to login to a few websites), and multiple follow-up tests. In our user study, with training, participants were able to calculate a deterministic eight-character password for an arbitrary new website in under 20 seconds.
Inhabitation for Non-idempotent Intersection Types
Venue: arXiv cs.LO 2018
Year: 2018

The inhabitation problem for intersection types in the lambda-calculus is known to be undecidable. We study the problem in the case of non-idempotent intersection, considering several type assignment systems, which characterize the solvable or the strongly normalizing lambda-terms. We prove the decidability of the inhabitation problem for all the systems considered, by providing sound and complete inhabitation algorithms for them.
A Random Sample Partition Data Model for Big Data Analysis
Venue: arXiv cs.DC 2018
Year: 2018

Big data sets must be carefully partitioned into statistically similar data subsets that can be used as representative samples for big data analysis tasks. In this paper, we propose the random sample partition (RSP) data model to represent a big data set as a set of non-overlapping data subsets, called RSP data blocks, where each RSP data block has a probability distribution similar to the whole big data set. Under this data model, efficient block level sampling is used to randomly select RSP data blocks, replacing expensive record level sampling to select sample data from a big distributed data set on a computing cluster. We show how RSP data blocks can be employed to estimate statistics of a big data set and build models which are equivalent to those built from the whole big data set. In this approach, analysis of a big data set becomes analysis of few RSP data blocks which have been generated in advance on the computing cluster. Therefore, the new method for data analysis based on RSP data blocks is scalable to big data.
Interpretable Policies for Reinforcement Learning by Genetic Programming
Venue: arXiv cs.AI 2018
Year: 2018

The search for interpretable reinforcement learning policies is of high academic and industrial interest. Especially for industrial systems, domain experts are more likely to deploy autonomously learned controllers if they are understandable and convenient to evaluate. Basic algebraic equations are supposed to meet these requirements, as long as they are restricted to an adequate complexity. Here we introduce the genetic programming for reinforcement learning (GPRL) approach based on model-based batch reinforcement learning and genetic programming, which autonomously learns policy equations from pre-existing default state-action trajectory samples. GPRL is compared to a straight-forward method which utilizes genetic programming for symbolic regression, yielding policies imitating an existing well-performing, but non-interpretable policy. Experiments on three reinforcement learning benchmarks, i.e., mountain car, cart-pole balancing, and industrial benchmark, demonstrate the superiority of our GPRL approach compared to the symbolic regression method. GPRL is capable of producing well-performing interpretable reinforcement learning policies from pre-existing default trajectory data.
On the Gaussian Multiple Access Wiretap Channel and the Gaussian Wiretap Channel with a Helper: Achievable Schemes and Upper Bounds
Venue: arXiv cs.IT 2018
Year: 2018

We study deterministic approximations of the Gaussian two-user multiple access wiretap channel (G-MAC-WT) and the Gaussian wiretap channel with a helper (G-WT-H). These approximations enable results beyond the recently shown 2/3 and 1/2 secure degrees of freedom (s.d.o.f.) for the G-MAC-WT and the G-WT-H, respectively. While the s.d.o.f. were obtained by real interference alignment, our approach uses signal-scale alignment. We show achievable schemes which are independent of the rationality of the channel gains. Moreover, our results can differentiate between channel strengths, in particular between both users, and establishes secrecy rates dependent on this difference. We can show that the resulting achievable secrecy rates tend to the s.d.o.f. for vanishing channel gain differences. Moreover, we extend previous and develop new techniques to prove general s.d.o.f. bounds for varying channel strengths and show that our achievable schemes reach the bounds for certain channel gain parameters. We believe that our analysis is the next step towards a constant-gap analysis of the G-MAC-WT and the G-WT-H.