OpenRaiser/Intern-Atlas huggingface dataset:
Customizing Sequence Generation with Multi-Task Dynamical Systems
Venue: arXiv cs.LG 2019
Year: 2019

Dynamical system models (including RNNs) often lack the ability to adapt the sequence generation or prediction to a given context, limiting their real-world application. In this paper we show that hierarchical multi-task dynamical systems (MTDSs) provide direct user control over sequence generation, via use of a latent code $\mathbf{z}$ that specifies the customization to the individual data sequence. This enables style transfer, interpolation and morphing within generated sequences. We show the MTDS can improve predictions via latent code interpolation, and avoid the long-term performance degradation of standard RNN approaches.
Finding Interpretable Concept Spaces in Node Embeddings using Knowledge Bases
Venue: arXiv cs.LG 2019
Year: 2019

In this paper we propose and study the novel problem of explaining node embeddings by finding embedded human interpretable subspaces in already trained unsupervised node representation embeddings. We use an external knowledge base that is organized as a taxonomy of human-understandable concepts over entities as a guide to identify subspaces in node embeddings learned from an entity graph derived from Wikipedia. We propose a method that given a concept finds a linear transformation to a subspace where the structure of the concept is retained. Our initial experiments show that we obtain low error in finding fine-grained concepts.
Keyphrase Generation: A Multi-Aspect Survey
Venue: arXiv cs.CL 2019
Year: 2019

Extractive keyphrase generation research has been around since the nineties, but the more advanced abstractive approach based on the encoder-decoder framework and sequence-to-sequence learning has been explored only recently. In fact, more than a dozen of abstractive methods have been proposed in the last three years, producing meaningful keyphrases and achieving state-of-the-art scores. In this survey, we examine various aspects of the extractive keyphrase generation methods and focus mostly on the more recent abstractive methods that are based on neural networks. We pay particular attention to the mechanisms that have driven the perfection of the later. A huge collection of scientific article metadata and the corresponding keyphrases is created and released for the research community. We also present various keyphrase generation and text summarization research patterns and trends of the last two decades.
Incorporating Fine-grained Events in Stock Movement Prediction
Venue: arXiv cs.CE 2019
Year: 2019

Considering event structure information has proven helpful in text-based stock movement prediction. However, existing works mainly adopt the coarse-grained events, which loses the specific semantic information of diverse event types. In this work, we propose to incorporate the fine-grained events in stock movement prediction. Firstly, we propose a professional finance event dictionary built by domain experts and use it to extract fine-grained events automatically from finance news. Then we design a neural model to combine finance news with fine-grained event structure and stock trade data to predict the stock movement. Besides, in order to improve the generalizability of the proposed method, we design an advanced model that uses the extracted fine-grained events as the distant supervised label to train a multi-task framework of event extraction and stock prediction. The experimental results show that our method outperforms all the baselines and has good generalizability.
Modeling Cyber-Physical Human Systems via an Interplay Between Reinforcement Learning and Game Theory
Venue: arXiv cs.MA 2019
Year: 2019

Predicting the outcomes of cyber-physical systems with multiple human interactions is a challenging problem. This article reviews a game theoretical approach to address this issue, where reinforcement learning is employed to predict the time-extended interaction dynamics. We explain that the most attractive feature of the method is proposing a computationally feasible approach to simultaneously model multiple humans as decision makers, instead of determining the decision dynamics of the intelligent agent of interest and forcing the others to obey certain kinematic and dynamic constraints imposed by the environment. We present two recent exploitations of the method to model 1) unmanned aircraft integration into the National Airspace System and 2) highway traffic. We conclude the article by providing ongoing and future work about employing, improving and validating the method. We also provide related open problems and research opportunities.
Non-Uniform Time-Step Deep Q-Network for Carrier-Sense Multiple Access in Heterogeneous Wireless Networks
Venue: arXiv cs.NI 2019
Year: 2019

This paper investigates a new class of carrier-sense multiple access (CSMA) protocols that employ deep reinforcement learning (DRL) techniques, referred to as carrier-sense deep-reinforcement learning multiple access (CS-DLMA). The goal of CS-DLMA is to enable efficient and equitable spectrum sharing among a group of co-located heterogeneous wireless networks. Existing CSMA protocols, such as the medium access control (MAC) of WiFi, are designed for a homogeneous network in which all nodes adopt the same protocol. Such protocols suffer from severe performance degradation in a heterogeneous environment where there are nodes adopting other MAC protocols. CS-DLMA aims to circumvent this problem by making use of DRL. In particular, this paper adopts alpha-fairness as the general objective of CS-DLMA. With alpha-fairness, CS-DLMA can achieve a range of different objectives when coexisting with other MACs by changing the value of alpha. A salient feature of CS-DLMA is that it can achieve these objectives without knowing the coexisting MACs through a learning process based on DRL. The underpinning DRL technique in CS-DLMA is deep Q-network (DQN). However, the conventional DQN algorithms are not suitable for CS-DLMA due to their uniform time-step assumption. In CSMA protocols, time steps are non-uniform in that the time duration required for carrier sensing is smaller than the duration of data transmission. This paper introduces a non-uniform time-step formulation of DQN to address this issue. Our simulation results show that CS-DLMA can achieve the general alpha-fairness objective when coexisting with TDMA, ALOHA, and WiFi protocols by adjusting its own transmission strategy. Interestingly, we also find that CS-DLMA is more Pareto efficient than other CSMA protocols when coexisting with WiFi.
Shape Constrained Network for Eye Segmentation in the Wild
Venue: arXiv cs.CV 2019
Year: 2019

Semantic segmentation of eyes has long been a vital pre-processing step in many biometric applications. Majority of the works focus only on high resolution eye images, while little has been done to segment the eyes from low quality images in the wild. However, this is a particularly interesting and meaningful topic, as eyes play a crucial role in conveying the emotional state and mental well-being of a person. In this work, we take two steps toward solving this problem: (1) We collect and annotate a challenging eye segmentation dataset containing 8882 eye patches from 4461 facial images of different resolutions, illumination conditions and head poses; (2) We develop a novel eye segmentation method, Shape Constrained Network (SCN), that incorporates shape prior into the segmentation network training procedure. Specifically, we learn the shape prior from our dataset using VAE-GAN, and leverage the pre-trained encoder and discriminator to regularise the training of SegNet. To improve the accuracy and quality of predicted masks, we replace the loss of SegNet with three new losses: Intersection-over-Union (IoU) loss, shape discriminator loss and shape embedding loss. Extensive experiments shows that our method outperforms state-of-the-art segmentation and landmark detection methods in terms of mean IoU (mIoU) accuracy and the quality of segmentation masks. The eye segmentation database is available at https://www.dropbox.com/s/yvveouvxsvti08x/Eye_Segmentation_Database.zip?dl=0.
Learning Cluster Structured Sparsity by Reweighting
Venue: arXiv cs.LG 2019
Year: 2019

Recently, the paradigm of unfolding iterative algorithms into finite-length feed-forward neural networks has achieved a great success in the area of sparse recovery. Benefit from available training data, the learned networks have achieved state-of-the-art performance in respect of both speed and accuracy. However, the structure behind sparsity, imposing constraint on the support of sparse signals, is often an essential prior knowledge but seldom considered in the existing networks. In this paper, we aim at bridging this gap. Specifically, exploiting the iterative reweighted $\ell_1$ minimization (IRL1) algorithm, we propose to learn the cluster structured sparsity (CSS) by rewegihting adaptively. In particular, we first unfold the Reweighted Iterative Shrinkage Algorithm (RwISTA) into an end-to-end trainable deep architecture termed as RW-LISTA. Then instead of the element-wise reweighting, the global and local reweighting manner are proposed for the cluster structured sparse learning. Numerical experiments further show the superiority of our algorithm against both classical algorithms and learning-based networks on different tasks.
Communications and Networking Technologies for Intelligent Drone Cruisers
Venue: arXiv cs.NI 2019
Year: 2019

Future mobile communication networks require an Aerial Base Station (ABS) with fast mobility and long-term hovering capabilities. At present, unmanned aerial vehicles (UAV) or drones do not have long flight times and are mainly used for monitoring, surveillance, and image post-processing. On the other hand, the traditional airship is too large and not easy to take off and land. Therefore, we propose to develop an "Artificial Intelligence (AI) Drone-Cruiser" base station that can help 5G mobile communication systems and beyond quickly recover the network after a disaster and handle the instant communications by the flash crowd. The drone-cruiser base station can overcome the communications problem for three types of flash crowds, such as in stadiums, parades, and large plaza so that an appropriate number of aerial base stations can be accurately deployed to meet large and dynamic traffic demands. Artificial intelligence can solve these problems by analyzing the collected data, and then adjust the system parameters in the framework of Self-Organizing Network (SON) to achieve the goals of self-configuration, self-optimization, and self-healing. With the help of AI technologies, 5G networks can become more intelligent. This paper aims to provide a new type of service, On-Demand Aerial Base Station as a Service. This work needs to overcome the following five technical challenges: innovative design of drone-cruisers for the long-time hovering, crowd estimation and prediction, rapid 3D wireless channel learning and modeling, 3D placement of aerial base stations and the integration of WiFi front-haul and millimeter wave/WiGig back-haul networks.
A Heuristic for Maximizing the Lifetime of Data Aggregation in Wireless Sensor Networks
Venue: arXiv cs.NI 2019
Year: 2019

Recently, many researchers have studied efficiently gathering data in wireless sensor networks to minimize the total energy consumption when a fixed number of data are allowed to be aggregated into one packet. However, minimizing the total energy consumption does not imply the network lifetime is maximized. In this paper, we study the problem of scheduling data aggregation trees working for different time periods to maximize the network lifetime when a fixed number of data are allowed to be aggregated into one packet. In addition, we propose a heuristic to balance the lifetime of nodes in data aggregation trees such that the network lifetime is maximized. Simulation results show that the proposed heuristic provides a good performance.
Situation-aware Re-configuration of Production Processes
Venue: arXiv cs.DC 2019
Year: 2019

Manufacturing of products, nowadays, due to the advances in Information Technologies (IT) that turn products into smart devices, and the evolution in device connectivity, becomes increasingly complex. Connected product networks (CPN) and cyber-physical systems (CPS) are bringing new challenges to manufacturing companies. These challenges, among others things, lead to increased demands for optimisation and personalisation of production processes. This move, which as a result drives the market of machines and products, poses new challenges to the manufacturing domain that needs flexible solutions to replace the rigid traditional methodologies and tools. The current work suggests a cloud-based platform for optimisation of machines, products and processes, allowing for situational awareness on the users' site, involving technologies for event prediction and optimisation.
Anticipating Illegal Maritime Activities from Anomalous Multiscale Fleet Behaviors
Venue: arXiv cs.MA 2019
Year: 2019

Illegal fishing is prevalent throughout the world and heavily impacts the health of our oceans, the sustainability and profitability of fisheries, and even acts to destabilize geopolitical relations. To achieve the United Nations' Sustainable Development Goal of "Life Below Water", our ability to detect and predict illegal fishing must improve. Recent advances have been made through the use of vessel location data, however, most analyses to date focus on anomalous spatial behaviors of vessels one at a time. To improve predictions, we develop a method inspired by complex systems theory to monitor the anomalous multi-scale behavior of whole fleets as they respond to nearby illegal activities. Specifically, we analyze changes in the multiscale geospatial organization of fishing fleets operating on the Patagonia Shelf, an important fishing region with chronic exposure to illegal fishing. We show that legally operating (and visible) vessels respond anomalously to nearby illegal activities (by vessels that are difficult to detect). Indeed, precursor behaviors are identified, suggesting a path towards pre-empting illegal activities. This approach offers a promising step towards a global system for detecting, predicting and deterring illegal activities at sea in near real-time. Doing so will be a big step forward to achieving sustainable life underwater.
Channel Estimation for Extremely Large-Scale Massive MIMO Systems
Venue: arXiv cs.IT 2019
Year: 2019

Extremely large-scale massive multiple-input multiple-output (MIMO) has shown considerable potential in future mobile communications. However, the use of extremely large aperture arrays has led to near-field and spatial non-stationary channel conditions, which result in changes to transceiver design and channel state information that should be acquired. This letter focuses on the channel estimation problem and describes the non-stationary channel through mapping between subarrays and scatterers. We propose subarray-wise and scatterer-wise channel estimation methods to estimate the near-field non-stationary channel from the view of subarray and scatterer, respectively. Numerical results demonstrate that subarray-wise method can derive accurate channel estimation results with low complexity, whereas the scatterer-wise method can accurately position the scatterers and identify almost all the mappings between subarrays and scatterers.
Neural Memory Plasticity for Anomaly Detection
Venue: arXiv cs.NE 2019
Year: 2019

In the domain of machine learning, Neural Memory Networks (NMNs) have recently achieved impressive results in a variety of application areas including visual question answering, trajectory prediction, object tracking, and language modelling. However, we observe that the attention based knowledge retrieval mechanisms used in current NMNs restricts them from achieving their full potential as the attention process retrieves information based on a set of static connection weights. This is suboptimal in a setting where there are vast differences among samples in the data domain; such as anomaly detection where there is no consistent criteria for what constitutes an anomaly. In this paper, we propose a plastic neural memory access mechanism which exploits both static and dynamic connection weights in the memory read, write and output generation procedures. We demonstrate the effectiveness and flexibility of the proposed memory model in three challenging anomaly detection tasks in the medical domain: abnormal EEG identification, MRI tumour type classification and schizophrenia risk detection in children. In all settings, the proposed approach outperforms the current state-of-the-art. Furthermore, we perform an in-depth analysis demonstrating the utility of neural plasticity for the knowledge retrieval process and provide evidence on how the proposed memory model generates sparse yet informative memory outputs.
A preference learning framework for multiple criteria sorting with diverse additive value models and valued assignment examples
Venue: arXiv cs.LG 2019
Year: 2019

We present a preference learning framework for multiple criteria sorting. We consider sorting procedures applying an additive value model with diverse types of marginal value functions (including linear, piecewise-linear, splined, and general monotone ones) under a unified analytical framework. Differently from the existing sorting methods that infer a preference model from crisp decision examples, where each reference alternative is assigned to a unique class, our framework allows to consider valued assignment examples in which a reference alternative can be classified into multiple classes with respective credibility degrees. We propose an optimization model for constructing a preference model from such valued examples by maximizing the credible consistency among reference alternatives. To improve the predictive ability of the constructed model on new instances, we employ the regularization techniques. Moreover, to enhance the capability of addressing large-scale datasets, we introduce a state-of-the-art algorithm that is widely used in the machine learning community to solve the proposed optimization model in a computationally efficient way. Using the constructed additive value model, we determine both crisp and valued assignments for non-reference alternatives. Moreover, we allow the Decision Maker to prioritize importance of classes and give the method a flexibility to adjust classification performance across classes according to the specified priorities. The practical usefulness of the analytical framework is demonstrated on a real-world dataset by comparing it to several existing sorting methods.
Combinational Class Activation Maps for Weakly Supervised Object Localization
Venue: arXiv cs.CV 2019
Year: 2019

Weakly supervised object localization has recently attracted attention since it aims to identify both class labels and locations of objects by using image-level labels. Most previous methods utilize the activation map corresponding to the highest activation source. Exploiting only one activation map of the highest probability class is often biased into limited regions or sometimes even highlights background regions. To resolve these limitations, we propose to use activation maps, named combinational class activation maps (CCAM), which are linear combinations of activation maps from the highest to the lowest probability class. By using CCAM for localization, we suppress background regions to help highlighting foreground objects more accurately. In addition, we design the network architecture to consider spatial relationships for localizing relevant object regions. Specifically, we integrate non-local modules into an existing base network at both low- and high-level layers. Our final model, named non-local combinational class activation maps (NL-CCAM), obtains superior performance compared to previous methods on representative object localization benchmarks including ILSVRC 2016 and CUB-200-2011. Furthermore, we show that the proposed method has a great capability of generalization by visualizing other datasets.
Stripe-based and Attribute-aware Network: A Two-Branch Deep Model for Vehicle Re-identification
Venue: arXiv cs.CV 2019
Year: 2019

Vehicle re-identification (Re-ID) has been attracting increasing interest in the field of computer vision due to the growing utilization of surveillance cameras in public security. However, vehicle Re-ID still suffers a similarity challenge despite the efforts made to solve this problem. This challenge involves distinguishing different instances with nearly identical appearances. In this paper, we propose a novel two-branch stripe-based and attribute-aware deep convolutional neural network (SAN) to learn the efficient feature embedding for vehicle Re-ID task. The two-branch neural network, consisting of stripe-based branch and attribute-aware branches, can adaptively extract the discriminative features from the visual appearance of vehicles. A horizontal average pooling and dimension-reduced convolutional layers are inserted into the stripe-based branch to achieve part-level features. Meanwhile, the attribute-aware branch extracts the global feature under the supervision of vehicle attribute labels to separate the similar vehicle identities with different attribute annotations. Finally, the part-level and global features are concatenated together to form the final descriptor of the input image for vehicle Re-ID. The final descriptor not only can separate vehicles with different attributes but also distinguish vehicle identities with the same attributes. The extensive experiments on both VehicleID and VeRi databases show that the proposed SAN method outperforms other state-of-the-art vehicle Re-ID approaches.
Efficiency in managing peer-review of scientific manuscripts -- editors' perspective
Venue: arXiv cs.DL 2019
Year: 2019

The purpose of this paper is to introduce a model for measuring the efficiency in managing peer-review of scientific manuscripts by editors. The approach employed is based on the assumption that the editorial aim is to manage publication with high efficiency, employing the least amount of editorial resources. Efficiency is defined in this research as a measure based on 7 variables. An on-line survey was constructed and editors of journals originating from Serbia regularly publishing articles in the field of chemistry were invited to participate. An evaluation of the model is given based on responses from 24 journals and 50 editors. With this investigation we aimed to contribute to our understanding of the peer-review process and, possibly, offer a tool to improve the "efficiency" in journal editing. The proposed protocol may be adapted by other journals in order to assess the managing potential of editors.
VAIS Hate Speech Detection System: A Deep Learning based Approach for System Combination
Venue: arXiv cs.CL 2019
Year: 2019

Nowadays, Social network sites (SNSs) such as Facebook, Twitter are common places where people show their opinions, sentiments and share information with others. However, some people use SNSs to post abuse and harassment threats in order to prevent other SNSs users from expressing themselves as well as seeking different opinions. To deal with this problem, SNSs have to use a lot of resources including people to clean the aforementioned content. In this paper, we propose a supervised learning model based on the ensemble method to solve the problem of detecting hate content on SNSs in order to make conversations on SNSs more effective. Our proposed model got the first place for public dashboard with 0.730 F1 macro-score and the third place with 0.584 F1 macro-score for private dashboard at the sixth international workshop on Vietnamese Language and Speech Processing 2019.