OpenRaiser/Intern-Atlas huggingface dataset:
Towards Understanding Generalization in Gradient-Based Meta-Learning
Venue: arXiv cs.LG 2019
Year: 2019

In this work we study generalization of neural networks in gradient-based meta-learning by analyzing various properties of the objective landscapes. We experimentally demonstrate that as meta-training progresses, the meta-test solutions, obtained after adapting the meta-train solution of the model, to new tasks via few steps of gradient-based fine-tuning, become flatter, lower in loss, and further away from the meta-train solution. We also show that those meta-test solutions become flatter even as generalization starts to degrade, thus providing an experimental evidence against the correlation between generalization and flat minima in the paradigm of gradient-based meta-leaning. Furthermore, we provide empirical evidence that generalization to new tasks is correlated with the coherence between their adaptation trajectories in parameter space, measured by the average cosine similarity between task-specific trajectory directions, starting from a same meta-train solution. We also show that coherence of meta-test gradients, measured by the average inner product between the task-specific gradient vectors evaluated at meta-train solution, is also correlated with generalization. Based on these observations, we propose a novel regularizer for MAML and provide experimental evidence for its effectiveness.
Adversarial Security Attacks and Perturbations on Machine Learning and Deep Learning Methods
Venue: arXiv cs.LG 2019
Year: 2019

The ever-growing big data and emerging artificial intelligence (AI) demand the use of machine learning (ML) and deep learning (DL) methods. Cybersecurity also benefits from ML and DL methods for various types of applications. These methods however are susceptible to security attacks. The adversaries can exploit the training and testing data of the learning models or can explore the workings of those models for launching advanced future attacks. The topic of adversarial security attacks and perturbations within the ML and DL domains is a recent exploration and a great interest is expressed by the security researchers and practitioners. The literature covers different adversarial security attacks and perturbations on ML and DL methods and those have their own presentation styles and merits. A need to review and consolidate knowledge that is comprehending of this increasingly focused and growing topic of research; however, is the current demand of the research communities. In this review paper, we specifically aim to target new researchers in the cybersecurity domain who may seek to acquire some basic knowledge on the machine learning and deep learning models and algorithms, as well as some of the relevant adversarial security attacks and perturbations.
A General Framework of Learning Multi-Vehicle Interaction Patterns from Videos
Venue: arXiv cs.RO 2019
Year: 2019

Semantic learning and understanding of multi-vehicle interaction patterns in a cluttered driving environment are essential but challenging for autonomous vehicles to make proper decisions. This paper presents a general framework to gain insights into intricate multi-vehicle interaction patterns from bird's-eye view traffic videos. We adopt a Gaussian velocity field to describe the time-varying multi-vehicle interaction behaviors and then use deep autoencoders to learn associated latent representations for each temporal frame. Then, we utilize a hidden semi-Markov model with a hierarchical Dirichlet process as a prior to segment these sequential representations into granular components, also called traffic primitives, corresponding to interaction patterns. Experimental results demonstrate that our proposed framework can extract traffic primitives from videos, thus providing a semantic way to analyze multi-vehicle interaction patterns, even for cluttered driving scenarios that are far messier than human beings can cope with.
Learnability for the Information Bottleneck
Venue: arXiv cs.LG 2019
Year: 2019

The Information Bottleneck (IB) method (\cite{tishby2000information}) provides an insightful and principled approach for balancing compression and prediction for representation learning. The IB objective $I(X;Z)-βI(Y;Z)$ employs a Lagrange multiplier $β$ to tune this trade-off. However, in practice, not only is $β$ chosen empirically without theoretical guidance, there is also a lack of theoretical understanding between $β$, learnability, the intrinsic nature of the dataset and model capacity. In this paper, we show that if $β$ is improperly chosen, learning cannot happen -- the trivial representation $P(Z|X)=P(Z)$ becomes the global minimum of the IB objective. We show how this can be avoided, by identifying a sharp phase transition between the unlearnable and the learnable which arises as $β$ is varied. This phase transition defines the concept of IB-Learnability. We prove several sufficient conditions for IB-Learnability, which provides theoretical guidance for choosing a good $β$. We further show that IB-learnability is determined by the largest confident, typical, and imbalanced subset of the examples (the conspicuous subset), and discuss its relation with model capacity. We give practical algorithms to estimate the minimum $β$ for a given dataset. We also empirically demonstrate our theoretical conditions with analyses of synthetic datasets, MNIST, and CIFAR10.
Towards Markerless Grasp Capture
Venue: arXiv cs.CV 2019
Year: 2019

Humans excel at grasping objects and manipulating them. Capturing human grasps is important for understanding grasping behavior and reconstructing it realistically in Virtual Reality (VR). However, grasp capture - capturing the pose of a hand grasping an object, and orienting it w.r.t. the object - is difficult because of the complexity and diversity of the human hand, and occlusion. Reflective markers and magnetic trackers traditionally used to mitigate this difficulty introduce undesirable artifacts in images and can interfere with natural grasping behavior. We present preliminary work on a completely marker-less algorithm for grasp capture from a video depicting a grasp. We show how recent advances in 2D hand pose estimation can be used with well-established optimization techniques. Uniquely, our algorithm can also capture hand-object contact in detail and integrate it in the grasp capture process. This is work in progress, find more details at https://contactdb. cc.gatech.edu/grasp_capture.html.
Block based Singular Value Decomposition approach to matrix factorization for recommender systems
Venue: arXiv cs.LG 2019
Year: 2019

With the abundance of data in recent years, interesting challenges are posed in the area of recommender systems. Producing high quality recommendations with scalability and performance is the need of the hour. Singular Value Decomposition(SVD) based recommendation algorithms have been leveraged to produce better results. In this paper, we extend the SVD technique further for scalability and performance in the context of 1) multi-threading 2) multiple computational units (with the use of Graphical Processing Units) and 3) distributed computation. We propose block based matrix factorization (BMF) paired with SVD. This enabled us to take advantage of SVD over basic matrix factorization(MF) while taking advantage of parallelism and scalability through BMF. We used Compute Unified Device Architecture (CUDA) platform and related hardware for leveraging Graphical Processing Unit (GPU) along with block based SVD to demonstrate the advantages in terms of performance and memory.
Containment Graphs, Posets, and Related Classes of Graphs
Venue: arXiv cs.DM 2019
Year: 2019

In this paper, we introduce the notion of the containment graph of a family of sets and containment classes of graphs and posets. Let $Z$ be a family of nonempty sets. We call a (simple, finite) graph G = (V, E) a $Z$-containment graph provided one can assign to each vertex $v_i \in V $ a set $S_i \in Z$ such that $v_i v_j \in E$ if and only if $S_i \subset S_j$ or $S_j \subset S_i$ . Similarly, we call a (strict) partially ordered set $P = (V, <)$ a $Z$-containment poset if to each $v_i \in V $ we can assign a set $S_i \in Z$ such that $v_i < v_j$ if and only if $S_i \subset S_j$. Obviously, $G$ is the comparability graph of $P$. We give some basic results on containment graphs and investigate the containment graphs of iso-oriented boxes in $d$-space. We present a characterization of those classes of posets and graphs that have containment representations by sets of a specific type, and we extend our results to ``injective'' containment classes. After that we discuss similar characterizations for intersection, overlap, and disjointedness classes of graphs. Finally, in the last section we discuss the nonexistence of a characterization theorem for ``strong'' containment classes of graphs.
OGNet: Salient Object Detection with Output-guided Attention Module
Venue: arXiv cs.CV 2019
Year: 2019

Attention mechanisms are widely used in salient object detection models based on deep learning, which can effectively promote the extraction and utilization of useful information by neural networks. However, most of the existing attention modules used in salient object detection are input with the processed feature map itself, which easily leads to the problem of `blind overconfidence'. In this paper, instead of applying the widely used self-attention module, we present an output-guided attention module built with multi-scale outputs to overcome the problem of `blind overconfidence'. We also construct a new loss function, the intractable area F-measure loss function, which is based on the F-measure of the hard-to-handle area to improve the detection effect of the model in the edge areas and confusing areas of an image. Extensive experiments and abundant ablation studies are conducted to evaluate the effect of our methods and to explore the most suitable structure for the model. Tests on several data sets show that our model performs very well, even though it is very lightweight.
AVDNet: A Small-Sized Vehicle Detection Network for Aerial Visual Data
Venue: arXiv cs.CV 2019
Year: 2019

Detection of small-sized targets in aerial views is a challenging task due to the smallness of vehicle size, complex background, and monotonic object appearances. In this letter, we propose a one-stage vehicle detection network (AVDNet) to robustly detect small-sized vehicles in aerial scenes. In AVDNet, we introduced ConvRes residual blocks at multiple scales to alleviate the problem of vanishing features for smaller objects caused because of the inclusion of deeper convolutional layers. These residual blocks, along with enlarged output feature map, ensure the robust representation of the salient features for small sized objects. Furthermore, we proposed a recurrent-feature aware visualization (RFAV) technique to analyze the network behavior. We also created a new airborne image data set (ABD) by annotating 1396 new objects in 79 aerial images for our experiments. The effectiveness of AVDNet is validated on VEDAI, DLR- 3K, DOTA, and the combined (VEDAI, DLR-3K, DOTA, and ABD) data set. Experimental results demonstrate the significant performance improvement of the proposed method over state-of-the-art detection techniques in terms of mAP, computation, and space complexity.
Multi-Antenna Covert Communications with Random Access Protocol
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we consider the issue of covert communications with random access protocol. We consider that the legitimate user Bob has no priori knowledge about packet arrival time and thus employs data-aided frame detection based on reference sequence. The warden user Willie tries to detect this covert communication by using a radiometer. The detection performance analysis is provided for both Bob and Willie under Rayleigh fading channel. It is demonstrated that the covert performance can benefit from the increase of the frame length or the number of receive antennas from frame detection perspective. Numerical results are provided to verify the proposed studies.
Canada Protocol: an ethical checklist for the use of Artificial Intelligence in Suicide Prevention and Mental Health
Venue: arXiv cs.CY 2019
Year: 2019

Introduction: To improve current public health strategies in suicide prevention and mental health, governments, researchers and private companies increasingly use information and communication technologies, and more specifically Artificial Intelligence and Big Data. These technologies are promising but raise ethical challenges rarely covered by current legal systems. It is essential to better identify, and prevent potential ethical risks. Objectives: The Canada Protocol - MHSP is a tool to guide and support professionals, users, and researchers using AI in mental health and suicide prevention. Methods: A checklist was constructed based upon ten international reports on AI and ethics and two guides on mental health and new technologies. 329 recommendations were identified, of which 43 were considered as applicable to Mental Health and AI. The checklist was validated, using a two round Delphi Consultation. Results: 16 experts participated in the first round of the Delphi Consultation and 8 participated in the second round. Of the original 43 items, 38 were retained. They concern five categories: "Description of the Autonomous Intelligent System" (n=8), "Privacy and Transparency" (n=8), "Security" (n=6), "Health-Related Risks" (n=8), "Biases" (n=8). The checklist was considered relevant by most users, and could need versions tailored to each category of target users.
A Note on Linear Complementary Pairs of Group Codes
Venue: arXiv cs.IT 2019
Year: 2019

We give a short and elementary proof of the fact that for a linear complementary pair $(C,D)$, where $C$ and $D$ are $2$-sided ideals in a group algebra, $D$ is uniquely determined by $C$ and the dual code $D^\perp$ is permutation equivalent to $C$. This includes earlier results of Carlet et al. and Güneri et al. on nD cyclic codes which have been proved by subtle and lengthy calculations in the space of polynomials.
Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches
Venue: arXiv cs.LG 2019
Year: 2019

Despite the recent success of deep transfer learning approaches in NLP, there is a lack of quantitative studies demonstrating the gains these models offer in low-shot text classification tasks over existing paradigms. Deep transfer learning approaches such as BERT and ULMFiT demonstrate that they can beat state-of-the-art results on larger datasets, however when one has only 100-1000 labelled examples per class, the choice of approach is less clear, with classical machine learning and deep transfer learning representing valid options. This paper compares the current best transfer learning approach with top classical machine learning approaches on a trinary sentiment classification task to assess the best paradigm. We find that BERT, representing the best of deep transfer learning, is the best performing approach, outperforming top classical machine learning algorithms by 9.7% on average when trained with 100 examples per class, narrowing to 1.8% at 1000 labels per class. We also show the robustness of deep transfer learning in moving across domains, where the maximum loss in accuracy is only 0.7% in similar domain tasks and 3.2% cross domain, compared to classical machine learning which loses up to 20.6%.
Machine Learning based Simulation Optimisation for Trailer Management
Venue: arXiv cs.NE 2019
Year: 2019

In many situations, simulation models are developed to handle complex real-world business optimisation problems. For example, a discrete-event simulation model is used to simulate the trailer management process in a big Fast-Moving Consumer Goods company. To address the problem of finding suitable inputs to this simulator for optimising fleet configuration, we propose a simulation optimisation approach in this paper. The simulation optimisation model combines a metaheuristic search (genetic algorithm), with an approximation model filter (feed-forward neural network) to optimise the parameter configuration of the simulation model. We introduce an ensure probability that overrules the rejection of potential solutions by the approximation model and we demonstrate its effectiveness. In addition, we evaluate the impact of the parameters of the optimisation model on its effectiveness and show the parameters such as population size, filter threshold, and mutation probability can have a significant impact on the overall optimisation performance. Moreover, we compare the proposed method with a single global approximation model approach and a random-based approach. The results show the effectiveness of our method in terms of computation time and solution quality.
Competing Against Equilibria in Zero-Sum Games with Evolving Payoffs
Venue: arXiv cs.LG 2019
Year: 2019

We study the problem of repeated play in a zero-sum game in which the payoff matrix may change, in a possibly adversarial fashion, on each round; we call these Online Matrix Games. Finding the Nash Equilibrium (NE) of a two player zero-sum game is core to many problems in statistics, optimization, and economics, and for a fixed game matrix this can be easily reduced to solving a linear program. But when the payoff matrix evolves over time our goal is to find a sequential algorithm that can compete with, in a certain sense, the NE of the long-term-averaged payoff matrix. We design an algorithm with small NE regret--that is, we ensure that the long-term payoff of both players is close to minimax optimum in hindsight. Our algorithm achieves near-optimal dependence with respect to the number of rounds and depends poly-logarithmically on the number of available actions of the players. Additionally, we show that the naive reduction, where each player simply minimizes its own regret, fails to achieve the stated objective regardless of which algorithm is used. We also consider the so-called bandit setting, where the feedback is significantly limited, and we provide an algorithm with small NE regret using one-point estimates of each payoff matrix.
Cross-Media Usage of Social Big Data for Emergency Services and Volunteer Communities: Approaches, Development and Challenges of Multi-Platform Social Media Services
Venue: arXiv cs.SI 2019
Year: 2019

The use of social media is ubiquitous and nowadays well-established in our everyday life, but increasingly also before, during or after emergencies. The produced data is spread across several types of social media and can be used by different actors, such as emergency services or volunteer communities. There are already systems available that support the process of gathering, analysing and distributing information through social media. However, dependent on the goal of analysis, the analysis methods and available systems are limited based on technical or business-oriented restrictions. This paper presents the design of a cross-platform Social Media API, which was integrated and evaluated within multiple emergency scenarios. Based on the lessons learned, we outline the core challenges from the practical development and theoretical findings, focusing (1) cross-platform gathering and data management, (2) trustability and information quality, (3) tailorability and adjustable data operations, and (4) queries, performance, and technical development.
Delta -- new logic programming language and Delta-methodology for p-computable programs on Turing Complete Languages
Venue: arXiv cs.LO 2019
Year: 2019

In paper describes the new logic programming language Delta, which have a many good properties. Delta-programs is p-computable, verifiable and can translation on other languages. Also we describe the Delta-methodology for constructing p-computable programs in high-level languages such as PHP, Java, JavaScript, C++, Pascal, Delphi, Python, Solidity and other. We would like to especially note the use of the Delta methodology for creating Smart Contracts and for Internet of things. We change the concept of the formula and define D-formulas(or Delta programs) are special list-formulas. Then we define the execution of a program how is the process of checking truth D-formula on a dynamic model. Main idea our paper consider program how list-formula from another formulas on dynamic models. And we created by iterations new Delta-programs use simple base formulas for this. Also we entered a dynamic models how models where we save final values of variables when check formula on this model.
Modern CNNs for IoT Based Farms
Venue: arXiv cs.CY 2019
Year: 2019

Recent introduction of ICT in agriculture has brought a number of changes in the way farming is done. This means use of Internet of Things(IoT), Cloud Computing(CC), Big Data (BD) and automation to gain better control over the process of farming. As the use of these technologies in farms has grown exponentially with massive data production, there is need to develop and use state-of-the-art tools in order to gain more insight from the data within reasonable time. In this paper, we present an initial understanding of Convolutional Neural Network (CNN), the recent architectures of state-of-the-art CNN and their underlying complexities. Then we propose a classification taxonomy tailored for agricultural application of CNN. Finally, we present a comprehensive review of research dedicated to applications of state-of-the-art CNNs in agricultural production systems. Our contribution is in two-fold. First, for end users of agricultural deep learning tools, our benchmarking finding can serve as a guide to selecting appropriate architecture to use. Second, for agricultural software developers of deep learning tools, our in-depth analysis explains the state-of-the-art CNN complexities and points out possible future directions to further optimize the running performance.
A Simpler Approach to Linear Programming
Venue: arXiv cs.DM 2019
Year: 2019

Dantzig and Eaves claimed that fundamental duality theorems of linear programming were a trivial consequence of Fourier elimination. Another property of Fourier elimination is considered here, regarding the existence of implicit equalities rather than solvability. This leads to a different interpretation of duality theory which allows us to use Gaussian elimination to decide solvability of systems of linear inequalities, for bounded systems.
Comparison of Classical Machine Learning Approaches on Bangla Textual Emotion Analysis
Venue: arXiv cs.CL 2019
Year: 2019

Detecting emotions from text is an extension of simple sentiment polarity detection. Instead of considering only positive or negative sentiments, emotions are conveyed using more tangible manner; thus, they can be expressed as many shades of gray. This paper manifests the results of our experimentation for fine-grained emotion analysis on Bangla text. We gathered and annotated a text corpus consisting of user comments from several Facebook groups regarding socio-economic and political issues, and we made efforts to extract the basic emotions (sadness, happiness, disgust, surprise, fear, anger) conveyed through these comments. Finally, we compared the results of the five most popular classical machine learning techniques namely Naive Bayes, Decision Tree, k-Nearest Neighbor (k-NN), Support Vector Machine (SVM) and K-Means Clustering with several combinations of features. Our best model (SVM with a non-linear radial-basis function (RBF) kernel) achieved an overall average accuracy score of 52.98% and an F1 score (macro) of 0.3324