OpenRaiser/Intern-Atlas huggingface dataset:
Active Learning for Video Description With Cluster-Regularized Ensemble Ranking
Venue: arXiv cs.CV 2020
Year: 2020

Automatic video captioning aims to train models to generate text descriptions for all segments in a video, however, the most effective approaches require large amounts of manual annotation which is slow and expensive. Active learning is a promising way to efficiently build a training set for video captioning tasks while reducing the need to manually label uninformative examples. In this work we both explore various active learning approaches for automatic video captioning and show that a cluster-regularized ensemble strategy provides the best active learning approach to efficiently gather training sets for video captioning. We evaluate our approaches on the MSR-VTT and LSMDC datasets using both transformer and LSTM based captioning models and show that our novel strategy can achieve high performance while using up to 60% fewer training data than the strong state of the art baselines.
Farsighted Risk Mitigation of Lateral Movement Using Dynamic Cognitive Honeypots
Venue: arXiv cs.NI 2020
Year: 2020

Lateral movement of advanced persistent threats has posed a severe security challenge. Due to the stealthy and persistent nature of the lateral movement, defenders need to consider time and spatial locations holistically to discover latent attack paths across a large time-scale and achieve long-term security for the target assets. In this work, we propose a time-expanded random network to model the stochastic service links in the user-host enterprise network and the adversarial lateral movement. We design cognitive honeypots at idle production nodes and disguise honey links as service links to detect and deter the adversarial lateral movement. The location of the honeypot changes randomly at different times and increases the honeypots' stealthiness. Since the defender does not know whether, when, and where the initial intrusion and the lateral movement occur, the honeypot policy aims to reduce the target assets' Long-Term Vulnerability (LTV) for proactive and persistent protection. We further characterize three tradeoffs, i.e., the probability of interference, the stealthiness level, and the roaming cost. To counter the curse of multiple attack paths, we propose an iterative algorithm and approximate the LTV with the union bound for computationally efficient deployment of cognitive honeypots. The results of the vulnerability analysis illustrate the bounds, trends, and a residue of LTV when the adversarial lateral movement has infinite duration. Besides honeypot policies, we obtain a critical threshold of compromisability to guide the design and modification of the current system parameters for a higher level of long-term security. We show that the target node can achieve zero vulnerability under infinite stages of lateral movement if the probability of movement deterrence is not less than the threshold.
Superpixel Based Graph Laplacian Regularization for Sparse Hyperspectral Unmixing
Venue: arXiv cs.CV 2020
Year: 2020

An efficient spatial regularization method using superpixel segmentation and graph Laplacian regularization is proposed for sparse hyperspectral unmixing method. Since it is likely to find spectrally similar pixels in a homogeneous region, we use a superpixel segmentation algorithm to extract the homogeneous regions by considering the image boundaries. We first extract the homogeneous regions, which are called superpixels, then a weighted graph in each superpixel is constructed by selecting $K$-nearest pixels in each superpixel. Each node in the graph represents the spectrum of a pixel and edges connect the similar pixels inside the superpixel. The spatial similarity is investigated using graph Laplacian regularization. Sparsity regularization for abundance matrix is provided using a weighted sparsity promoting norm. Experimental results on simulated and real data sets show the superiority of the proposed algorithm over the well-known algorithms in the literature.
Emotion Correlation Mining Through Deep Learning Models on Natural Language Text
Venue: arXiv cs.CL 2020
Year: 2020

Emotion analysis has been attracting researchers' attention. Most previous works in the artificial intelligence field focus on recognizing emotion rather than mining the reason why emotions are not or wrongly recognized. Correlation among emotions contributes to the failure of emotion recognition. In this paper, we try to fill the gap between emotion recognition and emotion correlation mining through natural language text from web news. Correlation among emotions, expressed as the confusion and evolution of emotion, is primarily caused by human emotion cognitive bias. To mine emotion correlation from emotion recognition through text, three kinds of features and two deep neural network models are presented. The emotion confusion law is extracted through orthogonal basis. The emotion evolution law is evaluated from three perspectives, one-step shift, limited-step shifts, and shortest path transfer. The method is validated using three datasets-the titles, the bodies, and the comments of news articles, covering both objective and subjective texts in varying lengths (long and short). The experimental results show that, in subjective comments, emotions are easily mistaken as anger. Comments tend to arouse emotion circulations of love-anger and sadness-anger. In objective news, it is easy to recognize text emotion as love and cause fear-joy circulation. That means, journalists may try to attract attention using fear and joy words but arouse the emotion love instead; After news release, netizens generate emotional comments to express their intense emotions, i.e., anger, sadness, and love. These findings could provide insights for applications regarding affective interaction such as network public sentiment, social media communication, and human-computer interaction.
Preparation of Sentiment tagged Parallel Corpus and Testing its effect on Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

In the current work, we explore the enrichment in the machine translation output when the training parallel corpus is augmented with the introduction of sentiment analysis. The paper discusses the preparation of the same sentiment tagged English-Bengali parallel corpus. The preparation of raw parallel corpus, sentiment analysis of the sentences and the training of a Character Based Neural Machine Translation model using the same has been discussed extensively in this paper. The output of the translation model has been compared with a base-line translation model using automated metrics such as BLEU and TER as well as manually.
Universal Fake News Collection System using Debunking Tweets
Venue: arXiv cs.CY 2020
Year: 2020

Large numbers of people use Social Networking Services (SNS) for easy access to various news, but they have more opportunities to obtain and share ``fake news'' carrying false information. Partially to combat fake news, several fact-checking sites such as Snopes and PolitiFact have been founded. Nevertheless, these sites rely on time-consuming and labor-intensive tasks. Moreover, their available languages are not extensive. To address these difficulties, we propose a new fake news collection system based on rule-based (unsupervised) frameworks that can be extended easily for various languages. The system collects news with high probability of being fake by debunking tweets by users and presents event clusters gathering higher attention. Our system currently functions in two languages: English and Japanese. It shows event clusters, 65\% of which are actually fake. In future studies, it will be applied to other languages and will be published with a large fake news dataset.
A Deep Learning-based Detector for Brown Spot Disease in Passion Fruit Plant Leaves
Venue: arXiv cs.CV 2020
Year: 2020

Pests and diseases pose a key challenge to passion fruit farmers across Uganda and East Africa in general. They lead to loss of investment as yields reduce and losses increases. As the majority of the farmers, including passion fruit farmers, in the country are smallholder farmers from low-income households, they do not have the sufficient information and means to combat these challenges. While, passion fruits have the potential to improve the well-being of these farmers as they have a short maturity period and high market value , without the required knowledge about the health of their crops, farmers cannot intervene promptly to turn the situation around. For this work, we have partnered with the Uganda National Crop Research Institute (NaCRRI) to develop a dataset of expertly labelled passion fruit plant leaves and fruits, both diseased and healthy. We have made use of their extension service to collect images from 5 districts in Uganda, With the dataset in place, we are employing state-of-the-art techniques in machine learning, and specifically deep learning, techniques at scale for object detection and classification to correctly determine the health status of passion fruit plants and provide an accurate diagnosis for positive detections.This work focuses on two major diseases woodiness (viral) and brown spot (fungal) diseases.
BUT-FIT at SemEval-2020 Task 5: Automatic detection of counterfactual statements with deep pre-trained language representation models
Venue: arXiv cs.CL 2020
Year: 2020

This paper describes BUT-FIT's submission at SemEval-2020 Task 5: Modelling Causal Reasoning in Language: Detecting Counterfactuals. The challenge focused on detecting whether a given statement contains a counterfactual (Subtask 1) and extracting both antecedent and consequent parts of the counterfactual from the text (Subtask 2). We experimented with various state-of-the-art language representation models (LRMs). We found RoBERTa LRM to perform the best in both subtasks. We achieved the first place in both exact match and F1 for Subtask 2 and ranked second for Subtask 1.
Dual Half-integrality for Uncrossable Cut Cover and its Application to Maximum Half-Integral Flow
Venue: arXiv cs.DS 2020
Year: 2020

Given an edge weighted graph and a forest $F$, the $\textit{2-edge connectivity augmentation problem}$ is to pick a minimum weighted set of edges, $E'$, such that every connected component of $E'\cup F$ is 2-edge connected. Williamson et al. gave a 2-approximation algorithm (WGMV) for this problem using the primal-dual schema. We show that when edge weights are integral, the WGMV procedure can be modified to obtain a half-integral dual. The 2-edge connectivity augmentation problem has an interesting connection to routing flow in graphs where the union of supply and demand is planar. The half-integrality of the dual leads to a tight 2-approximate max-half-integral-flow min-multicut theorem.
The Role of UAV-IoT Networks in Future Wildfire Detection
Venue: arXiv cs.NI 2020
Year: 2020

The challenge of wildfire management and detection is recently gaining increased attention due to the increased severity and frequency of wildfires worldwide. Popular fire detection techniques such as satellite imaging and remote camera-based sensing suffer from late detection and low reliability while early wildfire detection is a key to prevent massive fires. In this paper, we propose a novel wildfire detection solution based on unmanned aerial vehicles assisted Internet of things (UAV-IoT) networks. The main objective is to (1) study the performance and reliability of the UAV-IoT networks for wildfire detection and (2) present a guideline to optimize the UAV-IoT network to improve fire detection probability under limited budgets. We focus on optimizing the IoT devices' density and number of UAVs covering the forest area such that a lower bound of the wildfires detection probability is maximized within a limited time and budget. At any time after the fire ignition, the IoT devices within a limited distance from the fire can detect it. These IoT devices can then report their measurements only when the UAV is nearby. Discrete-time Markov chain (DTMC) analysis is utilized to compute the fire detection probability at discrete time. Before declaring fire detection, a validation state is designed to account for IoT devices' practical limitations such as miss-detection and false alarm probabilities. Numerical results suggest that given enough system budget, the UAV-IoT based fire detection can offer a faster and more reliable wildfire detection solution than the state of the art satellite imaging techniques.
Optimization of XNOR Convolution for Binary Convolutional Neural Networks on GPU
Venue: arXiv cs.CV 2020
Year: 2020

Binary convolutional networks have lower computational load and lower memory foot-print compared to their full-precision counterparts. So, they are a feasible alternative for the deployment of computer vision applications on limited capacity embedded devices. Once trained on less resource-constrained computational environments, they can be deployed for real-time inference on such devices. In this study, we propose an implementation of binary convolutional network inference on GPU by focusing on optimization of XNOR convolution. Experimental results show that using GPU can provide a speed-up of up to $42.61\times$ with a kernel size of $3\times3$. The implementation is publicly available at https://github.com/metcan/Binary-Convolutional-Neural-Network-Inference-on-GPU
Word embedding and neural network on grammatical gender -- A case study of Swedish
Venue: arXiv cs.CL 2020
Year: 2020

We analyze the information provided by the word embeddings about the grammatical gender in Swedish. We wish that this paper may serve as one of the bridges to connect the methods of computational linguistics and general linguistics. Taking nominal classification in Swedish as a case study, we first show how the information about grammatical gender in language can be captured by word embedding models and artificial neural networks. Then, we match our results with previous linguistic hypotheses on assignment and usage of grammatical gender in Swedish and analyze the errors made by the computational model from a linguistic perspective.
Symmetric Positive Semi-definite Riemannian Geometry with Application to Domain Adaptation
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we present new results on the Riemannian geometry of symmetric positive semi-definite (SPSD) matrices. First, based on an existing approximation of the geodesic path, we introduce approximations of the logarithmic and exponential maps. Second, we present a closed-form expression for Parallel Transport (PT). Third, we derive a canonical representation for a set of SPSD matrices. Based on these results, we propose an algorithm for Domain Adaptation (DA) and demonstrate its performance in two applications: fusion of hyper-spectral images and motion identification.
Learning Stable Manoeuvres in Quadruped Robots from Expert Demonstrations
Venue: arXiv cs.RO 2020
Year: 2020

With the research into development of quadruped robots picking up pace, learning based techniques are being explored for developing locomotion controllers for such robots. A key problem is to generate leg trajectories for continuously varying target linear and angular velocities, in a stable manner. In this paper, we propose a two pronged approach to address this problem. First, multiple simpler policies are trained to generate trajectories for a discrete set of target velocities and turning radius. These policies are then augmented using a higher level neural network for handling the transition between the learned trajectories. Specifically, we develop a neural network-based filter that takes in target velocity, radius and transforms them into new commands that enable smooth transitions to the new trajectory. This transformation is achieved by learning from expert demonstrations. An application of this is the transformation of a novice user's input into an expert user's input, thereby ensuring stable manoeuvres regardless of the user's experience. Training our proposed architecture requires much less expert demonstrations compared to standard neural network architectures. Finally, we demonstrate experimentally these results in the in-house quadruped Stoch 2.
Signed Distance Fields Dynamic Diffuse Global Illumination
Venue: arXiv cs.GR 2020
Year: 2020

Global Illumination (GI) is of utmost importance in the field of photo-realistic rendering. However, its computation has always been very complex, especially diffuse GI. State of the art real-time GI methods have limitations of different nature, such as light leaking, performance issues, special hardware requirements, noise corruption, bounce number limitations, among others. To overcome these limitations, we propose a novel approach of computing dynamic diffuse GI with a signed distance fields approximation of the scene and discretizing the space domain of the irradiance function. With this approach, we are able to estimate real-time diffuse GI for dynamic lighting and geometry, without any precomputations and supporting multi-bounce GI, providing good quality lighting and high performance at the same time. Our algorithm is also able to achieve better scalability, and manage both large open scenes and indoor high-detailed scenes without being corrupted by noise.
Towards Ecologically Valid Research on Language User Interfaces
Venue: arXiv cs.CL 2020
Year: 2020

Language User Interfaces (LUIs) could improve human-machine interaction for a wide variety of tasks, such as playing music, getting insights from databases, or instructing domestic robots. In contrast to traditional hand-crafted approaches, recent work attempts to build LUIs in a data-driven way using modern deep learning methods. To satisfy the data needs of such learning algorithms, researchers have constructed benchmarks that emphasize the quantity of collected data at the cost of its naturalness and relevance to real-world LUI use cases. As a consequence, research findings on such benchmarks might not be relevant for developing practical LUIs. The goal of this paper is to bootstrap the discussion around this issue, which we refer to as the benchmarks' low ecological validity. To this end, we describe what we deem an ideal methodology for machine learning research on LUIs and categorize five common ways in which recent benchmarks deviate from it. We give concrete examples of the five kinds of deviations and their consequences. Lastly, we offer a number of recommendations as to how to increase the ecological validity of machine learning research on LUIs.
Paid Prioritization with Content Competition
Venue: arXiv cs.GT 2020
Year: 2020

We study the effects of allowing paid prioritization arrangements in a market with content provider (CP) competition. We consider competing CPs who pay prioritization fees to a monopolistic ISP so as to offset the ISP's cost for investing in infrastructure to support fast lanes. Unlike prior works, our proposed model of users' content consumption accounts for multi-purchasing (i.e., users simultaneously subscribing to more than one CP). This model allows us to account for the "attention" received by each CP, and consequently to draw a contrast between how subscription-revenues and ad-revenues are impacted by paid prioritization. We show that there exist incentives for the ISP to build additional fast lanes subsidized by CPs with sufficiently high revenue (from either subscription fees or advertisements). We show that non-prioritized content providers need not lose users, yet may lose revenue from advertisements due to decreased attention from users. We further show that users will consume a wider variety of content in a prioritized regime, and that they can attain higher welfare provided that non-prioritized traffic is not throttled. We discuss some policy and practical implications of these findings and numerically validate them.
Camera-Based Piano Sheet Music Identification
Venue: arXiv cs.CV 2020
Year: 2020

This paper presents a method for large-scale retrieval of piano sheet music images. Our work differs from previous studies on sheet music retrieval in two ways. First, we investigate the problem at a much larger scale than previous studies, using all solo piano sheet music images in the entire IMSLP dataset as a searchable database. Second, we use cell phone images of sheet music as our input queries, which lends itself to a practical, user-facing application. We show that a previously proposed fingerprinting method for sheet music retrieval is far too slow for a real-time application, and we diagnose its shortcomings. We propose a novel hashing scheme called dynamic n-gram fingerprinting that significantly reduces runtime while simultaneously boosting retrieval accuracy. In experiments on IMSLP data, our proposed method achieves a mean reciprocal rank of 0.85 and an average runtime of 0.98 seconds per query.
Translate the Facial Regions You Like Using Region-Wise Normalization
Venue: arXiv cs.CV 2020
Year: 2020

Though GAN (Generative Adversarial Networks) based technique has greatly advanced the performance of image synthesis and face translation, only few works available in literature provide region based style encoding and translation. We propose in this paper a region-wise normalization framework, for region level face translation. While per-region style is encoded using available approach, we build a so called RIN (region-wise normalization) block to individually inject the styles into per-region feature maps and then fuse them for following convolution and upsampling. Both shape and texture of different regions can thus be translated to various target styles. A region matching loss has also been proposed to significantly reduce the inference between regions during the translation process. Extensive experiments on three publicly available datasets, i.e. Morph, RaFD and CelebAMask-HQ, suggest that our approach demonstrate a large improvement over state-of-the-art methods like StarGAN, SEAN and FUNIT. Our approach has further advantages in precise control of the regions to be translated. As a result, region level expression changes and step by step make up can be achieved. The video demo is available at https://youtu.be/ceRqsbzXAfk.
CommuNety: A Deep Learning System for the Prediction of Cohesive Social Communities
Venue: arXiv cs.SI 2020
Year: 2020

Effective mining of social media, which consists of a large number of users is a challenging task. Traditional approaches rely on the analysis of text data related to users to accomplish this task. However, text data lacks significant information about the social users and their associated groups. In this paper, we propose CommuNety, a deep learning system for the prediction of cohesive social networks using images. The proposed deep learning model consists of hierarchical CNN architecture to learn descriptive features related to each cohesive network. The paper also proposes a novel Face Co-occurrence Frequency algorithm to quantify existence of people in images, and a novel photo ranking method to analyze the strength of relationship between different individuals in a predicted social network. We extensively evaluate the proposed technique on PIPA dataset and compare with state-of-the-art methods. Our experimental results demonstrate the superior performance of the proposed technique for the prediction of relationship between different individuals and the cohesiveness of communities.