OpenRaiser/Intern-Atlas huggingface dataset:
Revisiting Small Batch Training for Deep Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

Modern deep neural network training is typically based on mini-batch stochastic gradient optimization. While the use of large mini-batches increases the available computational parallelism, small batch training has been shown to provide improved generalization performance and allows a significantly smaller memory footprint, which might also be exploited to improve machine throughput. In this paper, we review common assumptions on learning rate scaling and training duration, as a basis for an experimental comparison of test performance for different mini-batch sizes. We adopt a learning rate that corresponds to a constant average weight update per gradient calculation (i.e., per unit cost of computation), and point out that this results in a variance of the weight updates that increases linearly with the mini-batch size $m$. The collected experimental results for the CIFAR-10, CIFAR-100 and ImageNet datasets show that increasing the mini-batch size progressively reduces the range of learning rates that provide stable convergence and acceptable test performance. On the other hand, small mini-batch sizes provide more up-to-date gradient calculations, which yields more stable and reliable training. The best performance has been consistently obtained for mini-batch sizes between $m = 2$ and $m = 32$, which contrasts with recent work advocating the use of mini-batch sizes in the thousands.
AMMCOA - Nomadic 5G Private Networks
Venue: arXiv cs.NI 2018
Year: 2018

This paper presents ideas and concepts for interconnected off-road vehicles, like harvesters or tandem rollers, which span a mobile network by themselves to be, to some extent, independent from the network infrastructure. Multiple RATs are used to cover a variety of use cases, including mmW technology for short range high bandwidth communication and ranging. The AMMCOA Project aims at providing connectivity in areas where the infrastructure network is either at long range distances or out off range entirely. In this case, for certain network services, like authentication of UEs, to be available, some of the core network functionalities have to be instantiated locally. This effectively turns the local network into a light version of a core network. In this paper we give an overview of the project's use cases, their requirements, the architectural idea of a local autonomous network (Trust Zone, 5G Island, 5G Private Networks) and the AMMCOA network architecture envisioned so far.
Verifying Text Summaries of Relational Data Sets
Venue: arXiv cs.DB 2018
Year: 2018

We present a novel natural language query interface, the AggChecker, aimed at text summaries of relational data sets. The tool focuses on natural language claims that translate into an SQL query and a claimed query result. Similar in spirit to a spell checker, the AggChecker marks up text passages that seem to be inconsistent with the actual data. At the heart of the system is a probabilistic model that reasons about the input document in a holistic fashion. Based on claim keywords and the document structure, it maps each text claim to a probability distribution over associated query translations. By efficiently executing tens to hundreds of thousands of candidate translations for a typical input document, the system maps text claims to correctness probabilities. This process becomes practical via a specialized processing backend, avoiding redundant work via query merging and result caching. Verification is an interactive process in which users are shown tentative results, enabling them to take corrective actions if necessary. Our system was tested on a set of 53 public articles containing 392 claims. Our test cases include articles from major newspapers, summaries of survey results, and Wikipedia articles. Our tool revealed erroneous claims in roughly a third of test cases. A detailed user study shows that users using our tool are in average six times faster at checking text summaries, compared to generic SQL interfaces. In fully automated verification, our tool achieves significantly higher recall and precision than baselines from the areas of natural language query interfaces and fact-checking.
A Reduction from Unbounded Linear Mixed Arithmetic Problems into Bounded Problems
Venue: arXiv cs.LO 2018
Year: 2018

We present a combination of the Mixed-Echelon-Hermite transformation and the Double-Bounded Reduction for systems of linear mixed arithmetic that preserve satisfiability and can be computed in polynomial time. Together, the two transformations turn any system of linear mixed constraints into a bounded system, i.e., a system for which termination can be achieved easily. Existing approaches for linear mixed arithmetic, e.g., branch-and-bound and cuts from proofs, only explore a finite search space after application of our two transformations. Instead of generating a priori bounds for the variables, e.g., as suggested by Papadimitriou, unbounded variables are eliminated through the two transformations. The transformations orient themselves on the structure of an input system instead of computing a priori (over-)approximations out of the available constants. Experiments provide further evidence to the efficiency of the transformations in practice. We also present a polynomial method for converting certificates of (un)satisfiability from the transformed to the original system.
A Survey of User Expectations and Tool Limitations in Collaborative Scientific Authoring and Reviewing
Venue: arXiv cs.DL 2018
Year: 2018

Collaborative scientific authoring is increasingly being supported by software tools. Traditionally, desktop-based authoring tools had the most advanced editing features, allowed for more formatting options, and included more import/export filters. Web-based tools have excelled in their collaboration support. Recently, developers on both sides have been trying to close this gap by extending desktop-based tools to better support collaboration and by making web-based tools richer in functionality. To verify to what extent these developments actually meet the needs of researchers, we gathered precise requirements towards better tool support for scientific authoring and reviewing workflows by interviewing 213 users and studying a corpus of 27 documents. We present the design of the survey and interpret its results. The conclusion is that WYSIWYG and offline desktop authoring tools continue to be more popular among academics than text-based and online editors.
Learning More Robust Features with Adversarial Training
Venue: arXiv cs.LG 2018
Year: 2018

In recent years, it has been found that neural networks can be easily fooled by adversarial examples, which is a potential safety hazard in some safety-critical applications. Many researchers have proposed various method to make neural networks more robust to white-box adversarial attacks, but an effective method have not been found so far. In this short paper, we focus on the robustness of the features learned by neural networks. We show that the features learned by neural networks are not robust, and find that the robustness of the learned features is closely related to the resistance against adversarial examples of neural networks. We also find that adversarial training against fast gradients sign method (FGSM) does not make the leaned features very robust, even if it can make the trained networks very resistant to FGSM attack. Then we propose a method, which can be seen as an extension of adversarial training, to train neural networks to learn more robust features. We perform experiments on MNIST and CIFAR-10 to evaluate our method, and the experiment results show that this method greatly improves the robustness of the learned features and the resistance to adversarial attacks.
A Self-paced Regularization Framework for Partial-Label Learning
Venue: arXiv cs.LG 2018
Year: 2018

Partial label learning (PLL) aims to solve the problem where each training instance is associated with a set of candidate labels, one of which is the correct label. Most PLL algorithms try to disambiguate the candidate label set, by either simply treating each candidate label equally or iteratively identifying the true label. Nonetheless, existing algorithms usually treat all labels and instances equally, and the complexities of both labels and instances are not taken into consideration during the learning stage. Inspired by the successful application of self-paced learning strategy in machine learning field, we integrate the self-paced regime into the partial label learning framework and propose a novel Self-Paced Partial-Label Learning (SP-PLL) algorithm, which could control the learning process to alleviate the problem by ranking the priorities of the training examples together with their candidate labels during each learning iteration. Extensive experiments and comparisons with other baseline methods demonstrate the effectiveness and robustness of the proposed method.
Pathologies of Neural Models Make Interpretations Difficult
Venue: arXiv cs.CL 2018
Year: 2018

One way to interpret neural model predictions is to highlight the most important input features---for example, a heatmap visualization over the words in an input sentence. In existing interpretation methods for NLP, a word's importance is determined by either input perturbation---measuring the decrease in model confidence when that word is removed---or by the gradient with respect to that word. To understand the limitations of these methods, we use input reduction, which iteratively removes the least important word from the input. This exposes pathological behaviors of neural models: the remaining words appear nonsensical to humans and are not the ones determined as important by interpretation methods. As we confirm with human experiments, the reduced examples lack information to support the prediction of any label, but models still make the same predictions with high confidence. To explain these counterintuitive results, we draw connections to adversarial examples and confidence calibration: pathological behaviors reveal difficulties in interpreting neural models trained with maximum likelihood. To mitigate their deficiencies, we fine-tune the models by encouraging high entropy outputs on reduced examples. Fine-tuned models become more interpretable under input reduction without accuracy loss on regular examples.
Large Scale Automated Reading of Frontal and Lateral Chest X-Rays using Dual Convolutional Neural Networks
Venue: arXiv cs.CV 2018
Year: 2018

The MIMIC-CXR dataset is (to date) the largest released chest x-ray dataset consisting of 473,064 chest x-rays and 206,574 radiology reports collected from 63,478 patients. We present the results of training and evaluating a collection of deep convolutional neural networks on this dataset to recognize multiple common thorax diseases. To the best of our knowledge, this is the first work that trains CNNs for this task on such a large collection of chest x-ray images, which is over four times the size of the largest previously released chest x-ray corpus (ChestX-Ray14). We describe and evaluate individual CNN models trained on frontal and lateral CXR view types. In addition, we present a novel DualNet architecture that emulates routine clinical practice by simultaneously processing both frontal and lateral CXR images obtained from a radiological exam. Our DualNet architecture shows improved performance in recognizing findings in CXR images when compared to applying separate baseline frontal and lateral classifiers.
DeepPET: A deep encoder-decoder network for directly solving the PET reconstruction inverse problem
Venue: arXiv cs.CV 2018
Year: 2018

Positron emission tomography (PET) is a cornerstone of modern radiology. The ability to detect cancer and metastases in whole body scans fundamentally changed cancer diagnosis and treatment. One of the main bottlenecks in the clinical application is the time it takes to reconstruct the anatomical image from the deluge of data in PET imaging. State-of-the art methods based on expectation maximization can take hours for a single patient and depend on manual fine-tuning. This results not only in financial burden for hospitals but more importantly leads to less efficient patient handling, evaluation, and ultimately diagnosis and treatment for patients. To overcome this problem we present a novel PET image reconstruction technique based on a deep convolutional encoder-decoder network, that takes PET sinogram data as input and directly outputs full PET images. Using realistic simulated data, we demonstrate that our network is able to reconstruct images >100 times faster, and with comparable image quality (in terms of root mean squared error) relative to conventional iterative reconstruction techniques.
Designing Practical PTASes for Minimum Feedback Vertex Set in Planar Graphs
Venue: arXiv cs.DS 2018
Year: 2018

We present two algorithms for the minimum feedback vertex set problem in planar graphs: an $O(n \log n)$ PTAS using a linear kernel and balanced separator, and a heuristic algorithm using kernelization and local search. We implemented these algorithms and compared their performance with Becker and Geiger's 2-approximation algorithm. We observe that while our PTAS is competitive with the 2-approximation algorithm on large planar graphs, its running time is much longer. And our heuristic algorithm can outperform the 2-approximation algorithm on most large planar graphs and provide a trade-off between running time and solution quality, i.e. a "PTAS behavior".
A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers
Venue: arXiv cs.RO 2018
Year: 2018

Lane change is a crucial vehicle maneuver which needs coordination with surrounding vehicles. Automated lane changing functions built on rule-based models may perform well under pre-defined operating conditions, but they may be prone to failure when unexpected situations are encountered. In our study, we proposed a Reinforcement Learning based approach to train the vehicle agent to learn an automated lane change behavior such that it can intelligently make a lane change under diverse and even unforeseen scenarios. Particularly, we treated both state space and action space as continuous, and designed a Q-function approximator that has a closed- form greedy policy, which contributes to the computation efficiency of our deep Q-learning algorithm. Extensive simulations are conducted for training the algorithm, and the results illustrate that the Reinforcement Learning based vehicle agent is capable of learning a smooth and efficient driving policy for lane change maneuvers.
Spatial Image Steganography Based on Generative Adversarial Network
Venue: arXiv cs.MM 2018
Year: 2018

With the recent development of deep learning on steganalysis, embedding secret information into digital images faces great challenges. In this paper, a secure steganography algorithm by using adversarial training is proposed. The architecture contain three component modules: a generator, an embedding simulator and a discriminator. A generator based on U-NET to translate a cover image into an embedding change probability is proposed. To fit the optimal embedding simulator and propagate the gradient, a function called Tanh-simulator is proposed. As for the discriminator, the selection-channel awareness (SCA) is incorporated to resist the SCA based steganalytic methods. Experimental results have shown that the proposed framework can increase the security performance dramatically over the recently reported method ASDL-GAN, while the training time is only 30% of that used by ASDL-GAN. Furthermore, it also performs better than the hand-crafted steganographic algorithm S-UNIWARD.
AI Meets Physical World -- Exploring Robot Cooking
Venue: arXiv cs.RO 2018
Year: 2018

This paper describes our recent research effort to bring the computer intelligence into the physical world so that robots could perform physically interactive manipulation tasks. Our proposed approach first gives robots the ability to learn manipulation skills by "watching" online instructional videos. After "watching" over 200 instructional videos, a functional object-oriented network (FOON) is constructed to represent the observed manipulation skills. Using the network, robots can take a high-level task command such as "I want BBQ Ribs for dinner," decipher the task goal, seek the correct objects to operate on, and then generate and execute a sequence of manipulation motions that can deal with physical interactions in a new condition. To best facilitate manipulation motions in the physical world, we also developed new grasping strategies for robots to hold objects with a firm grasp to withstand the disturbance during physical interactions.
Swarm Intelligence: Past, Present and Future
Venue: arXiv cs.NE 2018
Year: 2018

Many optimization problems in science and engineering are challenging to solve, and the current trend is to use swarm intelligence (SI) and SI-based algorithms to tackle such challenging problems. Some significant developments have been made in recent years, though there are still many open problems in this area. This paper provides a short but timely analysis about SI-based algorithms and their links with self-organization. Different characteristics and properties are analyzed here from both mathematical and qualitative perspectives. Future research directions are outlined and open questions are also highlighted.
A Fluid-Flow Interpretation of SCED Scheduling
Venue: arXiv cs.NI 2018
Year: 2018

We show that a fluid-flow interpretation of Service Curve Earliest Deadline First (SCED) scheduling simplifies deadline derivations for this scheduler. By exploiting the recently reported isomorphism between min-plus and max-plus network calculus, and expressing deadlines in a max-plus algebra, deadline computations no longer require pseudo-inverse computations. SCED deadlines are provided for general convex or concave piecewise linear service curves.
Subatomic systems need not be subatomic
Venue: arXiv cs.LO 2018
Year: 2018

Subatomic systems were recently introduced to identify the structural principles underpinning the normalization of proofs. "Subatomic" means that we can reformulate logical systems in accordance with two principles. Their atomic formulas become instances of sub-atoms, non-commutative self-dual relations among logical constants, and their rules are derivable by means of a unique deductive scheme, the medial shape. One of the results is that the cut-elimination of subatomic systems implies the cut-elimination of every standard system we can represent sub-atomically. We here introduce Subatomic systems-1.1. They relax and widen the properties that the sub-atoms of Subatomic systems can satisfy while maintaining the use of the medial shape as their only inference principle. Since sub-atoms can operate directly on variables we introduce P. The cut-elimination of P is a corollary of the cut-elimination that we prove for Subatomic systems-1.1. Moreover, P is sound and complete with respect to the clone at the top of Post's Lattice. I.e. P proves all and only the tautologies that contain conjunctions, disjunctions and projections. So, P extends Propositional logic without any encoding of its atoms as sub-atoms of P. This shows that the logical principles underpinning Subatomic systems also apply outside the sub-atomic level which they are conceived to work at. We reinforce this point of view by introducing the set R of medial shapes. The formulas that the rules in R deal with belong to the union of two disjoint clones of Post's Lattice. The SAT-problem of the first clone is in P-Time. The SAT-problem of the other is NP-Time complete. So, R and the proof technology of Subatomic systems could help to identify proof-theoretical properties that highlight the phase transition from P-Time to NP-Time complete satisfiability.
SolidWorx: A Resilient and Trustworthy Transactive Platform for Smart and Connected Communities
Venue: arXiv cs.DC 2018
Year: 2018

Internet of Things and data sciences are fueling the development of innovative solutions for various applications in Smart and Connected Communities (SCC). These applications provide participants with the capability to exchange not only data but also resources, which raises the concerns of integrity, trust, and above all the need for fair and optimal solutions to the problem of resource allocation. This exchange of information and resources leads to a problem where the stakeholders of the system may have limited trust in each other. Thus, collaboratively reaching consensus on when, how, and who should access certain resources becomes problematic. This paper presents SolidWorx, a blockchain-based platform that provides key mechanisms required for arbitrating resource consumption across different SCC applications in a domain-agnostic manner. For example, it introduces and implements a hybrid-solver pattern, where complex optimization computation is handled off-blockchain while solution validation is performed by a smart contract. To ensure correctness, the smart contract of SolidWorx is generated and verified.
Word Embedding Perturbation for Sentence Classification
Venue: arXiv cs.CL 2018
Year: 2018

In this technique report, we aim to mitigate the overfitting problem of natural language by applying data augmentation methods. Specifically, we attempt several types of noise to perturb the input word embedding, such as Gaussian noise, Bernoulli noise, and adversarial noise, etc. We also apply several constraints on different types of noise. By implementing these proposed data augmentation methods, the baseline models can gain improvements on several sentence classification tasks.
A Deep Convolutional Neural Network for Lung Cancer Diagnostic
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we examine the strength of deep learning technique for diagnosing lung cancer on medical image analysis problem. Convolutional neural networks (CNNs) models become popular among the pattern recognition and computer vision research area because of their promising outcome on generating high-level image representations. We propose a new deep learning architecture for learning high-level image representation to achieve high classification accuracy with low variance in medical image binary classification tasks. We aim to learn discriminant compact features at beginning of our deep convolutional neural network. We evaluate our model on Kaggle Data Science Bowl 2017 (KDSB17) data set, and compare it with some related works proposed in the Kaggle competition.
Automatic Language Identification in Texts: A Survey
Venue: arXiv cs.CL 2018
Year: 2018

Language identification (LI) is the problem of determining the natural language that a document or part thereof is written in. Automatic LI has been extensively researched for over fifty years. Today, LI is a key part of many text processing pipelines, as text processing techniques generally assume that the language of the input text is known. Research in this area has recently been especially active. This article provides a brief history of LI research, and an extensive survey of the features and methods used so far in the LI literature. For describing the features and methods we introduce a unified notation. We discuss evaluation methods, applications of LI, as well as off-the-shelf LI systems that do not require training by the end user. Finally, we identify open issues, survey the work to date on each issue, and propose future directions for research in LI.