OpenRaiser/Intern-Atlas huggingface dataset:
Image Reconstruction Using Deep Learning
Venue: arXiv cs.CV 2018
Year: 2018

This paper proposes a deep learning architecture that attains statistically significant improvements over traditional algorithms in Poisson image denoising espically when the noise is strong. Poisson noise commonly occurs in low-light and photon- limited settings, where the noise can be most accurately modeled by the Poission distribution. Poisson noise traditionally prevails only in specific fields such as astronomical imaging. However, with the booming market of surveillance cameras, which commonly operate in low-light environments, or mobile phones, which produce noisy night scene pictures due to lower-grade sensors, the necessity for an advanced Poisson image denoising algorithm has increased. Deep learning has achieved amazing breakthroughs in other imaging problems, such image segmentation and recognition, and this paper proposes a deep learning denoising network that outperforms traditional algorithms in Poisson denoising especially when the noise is strong. The architecture incorporates a hybrid of convolutional and deconvolutional layers along with symmetric connections. The denoising network achieved statistically significant 0.38dB, 0.68dB, and 1.04dB average PSNR gains over benchmark traditional algorithms in experiments with image peak values 4, 2, and 1. The denoising network can also operate with shorter computational time while still outperforming the benchmark algorithm by tuning the reconstruction stride sizes.
A State-Space Model for Assimilating Passenger and Vehicle Flow Data with User Feedback in a Transit Network
Venue: arXiv cs.SY 2018
Year: 2018

This note explores the idea of utilising a state-space model, congruent with the underlying equations of the Kalman filter with control input, for reconstructing the state of crowdedness in a transit network. The envisaged role of the proposed scheme is twofold: first, to provide an estimate of the state of crowdedness given input data on vehicle movement, on passenger inflow/outflow at stations and on measured crowdedness; second, to trigger localised requests for feedback based on the estimated system state as well as on the data assimilation performance indices. The latter is applicable to a scenario where the crowdedness is measured through passenger feedback. The feedback loop is conceptualised to be realised with a participatory crowd-sensing smartphone-based system in which reported perceived levels of crowdedness are assimilated in near-real-time with the aim of improving the estimation of the model state. Presented model is also applicable for assimilating other relevant measurements, for instance, vehicle weighing, automatic passenger counting, aggregated smartcard data or passive wireless device monitoring data.
Wafer Quality Inspection using Memristive LSTM, ANN, DNN and HTM
Venue: arXiv cs.ET 2018
Year: 2018

The automated wafer inspection and quality control is a complex and time-consuming task, which can speed up using neuromorphic memristive architectures, as a separate inspection device or integrating directly into sensors. This paper presents the performance analysis and comparison of different neuromorphic architectures for patterned wafer quality inspection and classification. The application of non-volatile memristive devices in these architectures ensures low power consumption, small on-chip area scalability. We demonstrate that Long-Short Term Memory (LSTM) outperforms other architectures for the same number of training iterations, and has relatively low on-chip area and power consumption.
Object Detection and Motion Planning for Automated Welding of Tubular Joints
Venue: arXiv cs.RO 2018
Year: 2018

Automatic welding of tubular TKY joints is an important and challenging task for the marine and offshore industry. In this paper, a framework for tubular joint detection and motion planning is proposed. The pose of the real tubular joint is detected using RGB-D sensors, which is used to obtain a real-to-virtual mapping for positioning the workpiece in a virtual environment. For motion planning, a Bi-directional Transition based Rapidly exploring Random Tree (BiTRRT) algorithm is used to generate trajectories for reaching the desired goals. The complete framework is verified with experiments, and the results show that the robot welding torch is able to transit without collision to desired goals which are close to the tubular joint.
Obladi: Oblivious Serializable Transactions in the Cloud
Venue: arXiv cs.DC 2018
Year: 2018

This paper presents the design and implementation of Obladi, the first system to provide ACID transactions while also hiding access patterns. Obladi uses as its building block oblivious RAM, but turns the demands of supporting transactions into a performance opportunity. By executing transactions within epochs and delaying commit decisions until an epoch ends, Obladi reduces the amortized bandwidth costs of oblivious storage and increases overall system throughput. These performance gains, combined with new oblivious mechanisms for concurrency control and recovery, allow Obladi to execute OLTP workloads with reasonable throughput: it comes within 5x to 12x of a non-oblivious baseline on the TPC-C, SmallBank, and FreeHealth applications. Latency overheads, however, are higher (70x on TPC-C).
A novel active learning framework for classification: using weighted rank aggregation to achieve multiple query criteria
Venue: arXiv cs.LG 2018
Year: 2018

Multiple query criteria active learning (MQCAL) methods have a higher potential performance than conventional active learning methods in which only one criterion is deployed for sample selection. A central issue related to MQCAL methods concerns the development of an integration criteria strategy (ICS) that makes full use of all criteria. The conventional ICS adopted in relevant research all facilitate the desired effects, but several limitations still must be addressed. For instance, some of the strategies are not sufficiently scalable during the design process, and the number and type of criteria involved are dictated. Thus, it is challenging for the user to integrate other criteria into the original process unless modifications are made to the algorithm. Other strategies are too dependent on empirical parameters, which can only be acquired by experience or cross-validation and thus lack generality; additionally, these strategies are counter to the intention of active learning, as samples need to be labeled in the validation set before the active learning process can begin. To address these limitations, we propose a novel MQCAL method for classification tasks that employs a third strategy via weighted rank aggregation. The proposed method serves as a heuristic means to select high-value samples of high scalability and generality and is implemented through a three-step process: (1) the transformation of the sample selection to sample ranking and scoring, (2) the computation of the self-adaptive weights of each criterion, and (3) the weighted aggregation of each sample rank list. Ultimately, the sample at the top of the aggregated ranking list is the most comprehensively valuable and must be labeled. Several experiments generating 257 wins, 194 ties and 49 losses against other state-of-the-art MQCALs are conducted to verify that the proposed method can achieve superior results.
Predicting the confirmation time of Bitcoin transactions
Venue: arXiv cs.PF 2018
Year: 2018

We study the probabilistic distribution of the confirmation time of Bitcoin transactions, conditional on the current memory pool (i.e., the queue of transactions awaiting confirmation). The results of this paper are particularly interesting for users that want to make a Bitcoin transaction during `heavy-traffic situations', when the transaction demand exceeds the block capacity. In such situations, Bitcoin users tend to bid up the transaction fees, in order to gain priority over other users that pay a lower fee. We argue that the time until a Bitcoin transaction is confirmed can be modelled as a particular stochastic fluid queueing process (to be precise: a Cramér-Lundberg process). We approximate the queueing process in two different ways. The first approach leads to a lower bound on the confirmation probability, which becomes increasingly tight as traffic decreases. The second approach relies on a diffusion approximation with a continuity correction, which becomes increasingly accurate as traffic intensifies. The accuracy of the approximations under different traffic loads are evaluated in a simulation study.
Operations on Partial Orders
Venue: arXiv cs.LO 2018
Year: 2018

We define analogues of Boolean operations on not necessarily complete partial orders, they often have as results sets of elements rather than single elements. It proves useful to add to such sets X if they are intended to be sup(X) or inf(X), even if sup and inf do not always exist. We then define the height of an element as the maximal length of chains going from BOTTOM to that element, and use height to define probability measures.
Conditional WaveGAN
Venue: arXiv cs.CV 2018
Year: 2018

Generative models are successfully used for image synthesis in the recent years. But when it comes to other modalities like audio, text etc little progress has been made. Recent works focus on generating audio from a generative model in an unsupervised setting. We explore the possibility of using generative models conditioned on class labels. Concatenation based conditioning and conditional scaling were explored in this work with various hyper-parameter tuning methods. In this paper we introduce Conditional WaveGANs (cWaveGAN). Find our implementation at https://github.com/acheketa/cwavegan
Predictive Embeddings for Hate Speech Detection on Twitter
Venue: arXiv cs.CL 2018
Year: 2018

We present a neural-network based approach to classifying online hate speech in general, as well as racist and sexist speech in particular. Using pre-trained word embeddings and max/mean pooling from simple, fully-connected transformations of these embeddings, we are able to predict the occurrence of hate speech on three commonly used publicly available datasets. Our models match or outperform state of the art F1 performance on all three datasets using significantly fewer parameters and minimal feature preprocessing compared to previous methods.
Bohemian Upper Hessenberg Matrices
Venue: arXiv cs.SC 2018
Year: 2018

We look at Bohemian matrices, specifically those with entries from $\{-1, 0, {+1}\}$. More, we specialize the matrices to be upper Hessenberg, with subdiagonal entries $\pm1$. Many properties remain after these specializations, some of which surprised us. We find two recursive formulae for the characteristic polynomials of upper Hessenberg matrices. Focusing on only those matrices whose characteristic polynomials have maximal height allows us to explicitly identify these polynomials and give a lower bound on their height. This bound is exponential in the order of the matrix. We count stable matrices, normal matrices, and neutral matrices, and tabulate the results of our experiments. We prove a theorem about the only possible kinds of normal matrices amongst a specific family of Bohemian upper Hessenberg matrices.
Multi-Scale Verification of Distributed Synchronisation
Venue: arXiv cs.LO 2018
Year: 2018

Algorithms for the synchronisation of clocks across networks are both common and important within distributed systems. We here address not only the formal modelling of these algorithms, but also the formal verification of their behaviour. Of particular importance is the strong link between the very different levels of abstraction at which the algorithms may be verified. Our contribution is primarily the formalisation of this connection between individual models and population-based models, and the subsequent verification that is then possible. While the technique is applicable across a range of synchronisation algorithms, we particularly focus on the synchronisation of (biologically-inspired) pulse-coupled oscillators, a widely used approach in practical distributed systems. For this application domain, different levels of abstraction are crucial: models based on the behaviour of an individual process are able to capture the details of distinguished nodes in possibly heterogenous networks, where each node may exhibit different behaviour. On the other hand, collective models assume homogeneous sets of processes, and allow the behaviour of the network to be analysed at the global level. System-wide parameters may be easily adjusted, for example environmental factors inhibiting the reliability of the shared communication medium. This work provides a formal bridge across the abstraction gap separating the individual models and the population-based models for this important class of synchronisation algorithms.
Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation
Venue: arXiv cs.LG 2018
Year: 2018

We propose a method to efficiently learn diverse strategies in reinforcement learning for query reformulation in the tasks of document retrieval and question answering. In the proposed framework an agent consists of multiple specialized sub-agents and a meta-agent that learns to aggregate the answers from sub-agents to produce a final answer. Sub-agents are trained on disjoint partitions of the training data, while the meta-agent is trained on the full training set. Our method makes learning faster, because it is highly parallelizable, and has better generalization performance than strong baselines, such as an ensemble of agents trained on the full data. We show that the improved performance is due to the increased diversity of reformulation strategies.
Introducing Noise in Decentralized Training of Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

It has been shown that injecting noise into the neural network weights during the training process leads to a better generalization of the resulting model. Noise injection in the distributed setup is a straightforward technique and it represents a promising approach to improve the locally trained models. We investigate the effects of noise injection into the neural networks during a decentralized training process. We show both theoretically and empirically that noise injection has no positive effect in expectation on linear models, though. However for non-linear neural networks we empirically show that noise injection substantially improves model quality helping to reach a generalization ability of a local model close to the serial baseline.
Supervised Nonnegative Matrix Factorization to Predict ICU Mortality Risk
Venue: arXiv cs.LG 2018
Year: 2018

ICU mortality risk prediction is a tough yet important task. On one hand, due to the complex temporal data collected, it is difficult to identify the effective features and interpret them easily; on the other hand, good prediction can help clinicians take timely actions to prevent the mortality. These correspond to the interpretability and accuracy problems. Most existing methods lack of the interpretability, but recently Subgraph Augmented Nonnegative Matrix Factorization (SANMF) has been successfully applied to time series data to provide a path to interpret the features well. Therefore, we adopted this approach as the backbone to analyze the patient data. One limitation of the raw SANMF method is its poor prediction ability due to its unsupervised nature. To deal with this problem, we proposed a supervised SANMF algorithm by integrating the logistic regression loss function into the NMF framework and solved it with an alternating optimization procedure. We used the simulation data to verify the effectiveness of this method, and then we applied it to ICU mortality risk prediction and demonstrated its superiority over other conventional supervised NMF methods.
A Short Survey of Topological Data Analysis in Time Series and Systems Analysis
Venue: arXiv cs.IR 2018
Year: 2018

Topological Data Analysis (TDA) is the collection of mathematical tools that capture the structure of shapes in data. Despite computational topology and computational geometry, the utilization of TDA in time series and signal processing is relatively new. In some recent contributions, TDA has been utilized as an alternative to the conventional signal processing methods. Specifically, TDA is been considered to deal with noisy signals and time series. In these applications, TDA is used to find the shapes in data as the main properties, while the other properties are assumed much less informative. In this paper, we will review recent developments and contributions where topological data analysis especially persistent homology has been applied to time series analysis, dynamical systems and signal processing. We will cover problem statements such as stability determination, risk analysis, systems behaviour, and predicting critical transitions in financial markets.
On the loss landscape of a class of deep neural networks with no bad local valleys
Venue: arXiv cs.LG 2018
Year: 2018

We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily close to zero. This implies that these networks have no sub-optimal strict local minima.
Building a Lemmatizer and a Spell-checker for Sorani Kurdish
Venue: arXiv cs.CL 2018
Year: 2018

The present paper aims at presenting a lemmatization and a word-level error correction system for Sorani Kurdish. We propose a hybrid approach based on the morphological rules and a n-gram language model. We have called our lemmatization and error correction systems Peyv and Rênûs respectively, which are the first tools presented for Sorani Kurdish to the best of our knowledge. The Peyv lemmatizer has shown 86.7% accuracy. As for Rênûs, using a lexicon, we have obtained 96.4% accuracy while without a lexicon, the correction system has 87% accuracy. As two fundamental text processing tools, these tools can pave the way for further researches on more natural language processing applications for Sorani Kurdish.
Cursive Scene Text Analysis by Deep Convolutional Linear Pyramids
Venue: arXiv cs.CV 2018
Year: 2018

The camera captured images have various aspects to investigate. Generally, the emphasis of research depends on the interesting regions. Sometimes the focus could be on color segmentation, object detection or scene text analysis. The image analysis, visibility and layout analysis are the tasks easier for humans as suggested by behavioral trait of humans, but in contrast when these same tasks are supposed to perform by machines then it seems to be challenging. The learning machines always learn from the properties associated to provided samples. The numerous approaches are designed in recent years for scene text extraction and recognition and the efforts are underway to improve the accuracy. The convolutional approach provided reasonable results on non-cursive text analysis appeared in natural images. The work presented in this manuscript exploited the strength of linear pyramids by considering each pyramid as a feature of the provided sample. Each pyramid image process through various empirically selected kernels. The performance was investigated by considering Arabic text on each image pyramid of EASTR-42k dataset. The error rate of 0.17% was reported on Arabic scene text recognition.
FanStore: Enabling Efficient and Scalable I/O for Distributed Deep Learning
Venue: arXiv cs.DC 2018
Year: 2018

Emerging Deep Learning (DL) applications introduce heavy I/O workloads on computer clusters. The inherent long lasting, repeated, and random file access pattern can easily saturate the metadata and data service and negatively impact other users. In this paper, we present FanStore, a transient runtime file system that optimizes DL I/O on existing hardware/software stacks. FanStore distributes datasets to the local storage of compute nodes, and maintains a global namespace. With the techniques of system call interception, distributed metadata management, and generic data compression, FanStore provides a POSIX-compliant interface with native hardware throughput in an efficient and scalable manner. Users do not have to make intrusive code changes to use FanStore and take advantage of the optimized I/O. Our experiments with benchmarks and real applications show that FanStore can scale DL training to 512 compute nodes with over 90\% scaling efficiency.
Effective Cloud Detection and Segmentation using a Gradient-Based Algorithm for Satellite Imagery; Application to improve PERSIANN-CCS
Venue: arXiv cs.CV 2018
Year: 2018

Being able to effectively identify clouds and monitor their evolution is one important step toward more accurate quantitative precipitation estimation and forecast. In this study, a new gradient-based cloud-image segmentation technique is developed using tools from image processing techniques. This method integrates morphological image gradient magnitudes to separable cloud systems and patches boundaries. A varying scale-kernel is implemented to reduce the sensitivity of image segmentation to noise and capture objects with various finenesses of the edges in remote-sensing images. The proposed method is flexible and extendable from single- to multi-spectral imagery. Case studies were carried out to validate the algorithm by applying the proposed segmentation algorithm to synthetic radiances for channels of the Geostationary Operational Environmental Satellites (GOES-R) simulated by a high-resolution weather prediction model. The proposed method compares favorably with the existing cloud-patch-based segmentation technique implemented in the PERSIANN-CCS (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network - Cloud Classification System) rainfall retrieval algorithm. Evaluation of event-based images indicates that the proposed algorithm has potential to improve rain detection and estimation skills with an average of more than 45% gain comparing to the segmentation technique used in PERSIANN-CCS and identifying cloud regions as objects with accuracy rates up to 98%.
Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Biped
Venue: arXiv cs.RO 2018
Year: 2018

Learning controllers for bipedal robots is a challenging problem, often requiring expert knowledge and extensive tuning of parameters that vary in different situations. Recently, deep reinforcement learning has shown promise at automatically learning controllers for complex systems in simulation. This has been followed by a push towards learning controllers that can be transferred between simulation and hardware, primarily with the use of domain randomization. However, domain randomization can make the problem of finding stable controllers even more challenging, especially for underactuated bipedal robots. In this work, we explore whether policies learned in simulation can be transferred to hardware with the use of high-fidelity simulators and structured controllers. We learn a neural network policy which is a part of a more structured controller. While the neural network is learned in simulation, the rest of the controller stays fixed, and can be tuned by the expert as needed. We show that using this approach can greatly speed up the rate of learning in simulation, as well as enable transfer of policies between simulation and hardware. We present our results on an ATRIAS robot and explore the effect of action spaces and cost functions on the rate of transfer between simulation and hardware. Our results show that structured policies can indeed be learned in simulation and implemented on hardware successfully. This has several advantages, as the structure preserves the intuitive nature of the policy, and the neural network improves the performance of the hand-designed policy. In this way, we propose a way of using neural networks to improve expert designed controllers, while maintaining ease of understanding.