Reviewerly/exHarmony huggingface dataset:
Sentiment Recognition for Short Annotated GIFs Using Visual-Textual Fusion

With the rapid development of social media, visual sentiment analysis from image or video has become a hot spot in visual understanding researches. In this work, we propose an effective approach using visual and textual fusion for sentiment analysis of short GIF videos with textual descriptions. We extract both sequence-level and frame-level visual features for each given GIF video. Next, we build a visual sentiment classifier by using the extracted features. We also define a mapping function, which converts the sentiment probability from the classifier to a sentiment score used in our fusion function. At the same time, for the accompanied textual annotations, we employ the Synset forest to extract the sets of the meaningful sentiment words and utilize the SentiWordNet3.0 model to obtain the textual sentiment score. Then, we design a joint visual-textual sentiment score function weighted with visual sentiment component and textual sentiment one. To make the function more robust, we introduce a noticeable difference threshold to further process the fused sentiment score. Finally, we adopt a grid search technique to obtain relevant model hyper-parameters by optimizing a sentiment aware score function. Experimental results and analysis extensively demonstrate the effectiveness of the proposed sentiment recognition scheme on three benchmark datasets including T-GIF dataset, GSO-2016 dataset and Adjusted-GIFGIF dataset.
Uniform Error Bounds for Gaussian Process Regression with Application to Safe Control

Data-driven models are subject to model errors due to limited and noisy training data. Key to the application of such models in safety-critical domains is the quantification of their model error. Gaussian processes provide such a measure and uniform error bounds have been derived, which allow safe control based on these models. However, existing error bounds require restrictive assumptions. In this paper, we employ the Gaussian process distribution and continuity arguments to derive a novel uniform error bound under weaker assumptions. Furthermore, we demonstrate how this distribution can be used to derive probabilistic Lipschitz constants and analyze the asymptotic behavior of our bound. Finally, we derive safety conditions for the control of unknown dynamical systems based on Gaussian process models and evaluate them in simulations of a robotic manipulator.
Neural Attentive Bag-of-Entities Model for Text Classification

This study proposes a Neural Attentive Bagof-Entities model, which is a neural network model that performs text classification using entities in a knowledge base.Entities provide unambiguous and relevant semantic signals that are beneficial for capturing semantics in texts.We combine simple high-recall entity detection based on a dictionary, to detect entities in a document, with a novel neural attention mechanism that enables the model to focus on a small number of unambiguous and relevant entities.We tested the effectiveness of our model using two standard text classification datasets (i.e., the 20 Newsgroups and R8 datasets) and a popular factoid question answering dataset based on a trivia quiz game.As a result, our model achieved state-of-the-art results on all datasets.The source code of the proposed model is available online at https://github.com/wikipedia2vec/wikipedia2vec.
Cross-predicting the dynamics of an optically injected single-mode semiconductor laser using reservoir computing

In real-world dynamical systems, technical limitations may prevent complete access to their dynamical variables. Such a lack of information may cause significant problems, especially when monitoring or controlling the dynamics of the system is required or when decisions need to be taken based on the dynamical state of the system. Cross-predicting the missing data is, therefore, of considerable interest. Here, we use a machine learning algorithm based on reservoir computing to perform cross-prediction of unknown variables of a chaotic dynamical laser system. In particular, we chose a realistic model of an optically injected single-mode semiconductor laser. While the intensity of the laser can often be acquired easily, measuring the phase of the electric field and the carriers in real time, although possible, requires a more demanding experimental scheme. We demonstrate that the dynamics of two of the three dynamical variables describing the state of the laser can be reconstructed accurately from the knowledge of only one variable, if our algorithm has been trained beforehand with all three variables for a limited period of time. We analyze the accuracy of the method depending on the parameters of the laser system and the reservoir. Finally, we test the robustness of the cross-prediction method when adding noise to the time series. The suggested reservoir computing state observer might be used in many applications, including reconstructing time series, recovering lost time series data and testing data encryption security in cryptography based on chaotic synchronization of lasers.
Safe, Aggressive Quadrotor Flight via Reachability-Based Trajectory Design

Abstract Quadrotors can provide services such as infrastructure inspection and search-and-rescue, which require operating autonomously in cluttered environments. Autonomy is typically achieved with receding-horizon planning, where a short plan is executed while a new one is computed, because sensors receive limited information at any time. To ensure safety and prevent robot loss, plans must be verified as collision free despite uncertainty (e.g, tracking error). Existing spline-based planners dilate obstacles uniformly to compensate for uncertainty, which can be conservative. On the other hand, reachability-based planners can include trajectory-dependent uncertainty as a function of the planned trajectory. This work applies Reachability-based Trajectory Design (RTD) to plan quadrotor trajectories that are safe despite trajectory-dependent tracking error. This is achieved by using zonotopes in a novel way for online planning. Simulations show aggressive flight up to 5 m/s with zero crashes in 500 cluttered, randomized environments.
A Good Sample is Hard to Find: Noise Injection Sampling and Self-Training for Neural Language Generation Models

Deep neural networks (DNN) are quickly becoming the de facto standard modeling method for many natural language generation (NLG) tasks.In order for such models to truly be useful, they must be capable of correctly generating utterances for novel meaning representations (MRs) at test time.In practice, even sophisticated DNNs with various forms of semantic control frequently fail to generate utterances faithful to the input MR.In this paper, we propose an architecture agnostic selftraining method to sample novel MR/text utterance pairs to augment the original training data.Remarkably, after training on the augmented data, even simple encoder-decoder models with greedy decoding are capable of generating semantically correct utterances that are as good as state-of-the-art outputs in both automatic and human evaluations of quality.
Metal: A Metadata-Hiding File-Sharing System

File-sharing systems like Dropbox offer insufficient privacy because a compromised server can see the file contents in the clear.Although encryption can hide such contents from the servers, metadata leakage remains significant.The goal of our work is to develop a file-sharing system that hides metadataincluding user identities and file access patterns.Metal is the first file-sharing system that hides such metadata from malicious users and that has a latency of only a few seconds.The core of Metal consists of a new two-server multi-user oblivious RAM (ORAM) scheme, which is secure against malicious users, a metadata-hiding access control protocol, and a capability sharing protocol.Compared with the state-of-the-art malicious-user filesharing scheme PIR-MCORAM (Maffei et al.'17), which does not hide user identities, Metal hides the user identities and is 500× faster (in terms of amortized latency) or 10 5 × faster (in terms of worst-case latency).
Visualizing Distributed System Executions

Distributed systems pose unique challenges for software developers. Understanding the system’s communication topology and reasoning about concurrent activities of system hosts can be difficult. The standard approach, analyzing system logs, can be a tedious and complex process that involves reconstructing a system log from multiple hosts’ logs, reconciling timestamps among hosts with non-synchronized clocks, and understanding what took place during the execution encoded by the log. This article presents a novel approach for tackling three tasks frequently performed during analysis of distributed system executions: (1) understanding the relative ordering of events, (2) searching for specific patterns of interaction between hosts, and (3) identifying structural similarities and differences between pairs of executions. Our approach consists of XVector , which instruments distributed systems to capture partial ordering information that encodes the happens-before relation between events, and ShiViz , which processes the resulting logs and presents distributed system executions as interactive time-space diagrams. Two user studies with a total of 109 students and a case study with 2 developers showed that our method was effective, helping participants answer statistically significantly more system-comprehension questions correctly, with a very large effect size.
Person tracking algorithm based on convolutional neural network for indoor video surveillance

In this paper, a person tracking algorithm for indoor video surveillance is presented. The algorithm contains the following steps: person detection, person features formation, features similarity calculation for the detected objects, postprocessing, person indexing, and person visibility determination in the current frame. Convolutional Neural Network (CNN) YOLO v3 is used for person detection. Person features are formed based on H channel in HSV color space histograms and a modified CNN ResNet. The proposed architecture includes 29 convolutional and one fully connected layer. As the output, it forms a 128-feature vector for every input image. CNN model was trained to perform feature extraction. Experiments were conducted using MOT methodology on stable camera videos in indoor environment. Main characteristics of the presented algorithm are calculated and discussed, confirming its effectiveness in comparison with the current approaches for person tracking in an indoor environment. Our algorithm performs real time processing for object detection and tracking using CUDA technology and a graphics card NVIDIA GTX 1060.
Augmented EMTCNN: A Fast and Accurate Facial Landmark Detection Network

Facial landmarks represent prominent feature points on the face that can be used as anchor points in many face-related tasks. So far, a lot of research has been done with the aim of achieving efficient extraction of landmarks from facial images. Employing a large number of feature points for landmark detection and tracking usually requires excessive processing time. On the contrary, relying on too few feature points cannot accurately represent diverse landmark properties, such as shape. To extract the 68 most popular facial landmark points efficiently, in our previous study, we proposed a model called EMTCNN that extended the multi-task cascaded convolutional neural network for real-time face landmark detection. To improve the detection accuracy, in this study, we augment the EMTCNN model by using two convolution techniques—dilated convolution and CoordConv. The former makes it possible to increase the filter size without a significant increase in computation time. The latter enables the spatial coordinate information of landmarks to be reflected in the model. We demonstrate that our model can improve the detection accuracy while maintaining the processing speed.
Unsupervised FAQ Retrieval with Question Generation and BERT

We focus on the task of Frequently Asked Questions (FAQ) retrieval.A given user query can be matched against the questions and/or the answers in the FAQ.We present a fully unsupervised method that exploits the FAQ pairs to train two BERT models.The two models match user queries to FAQ answers and questions, respectively.We alleviate the missing labeled data of the latter by automatically generating high-quality question paraphrases.We show that our model is on par and even outperforms supervised models on existing datasets.
Enhanced Bagging (eBagging): A Novel Approach for Ensemble Learning

Bagging is one of the well-known ensemble learning methods, which combines several classifiers trained on different subsamples of the dataset. However, a drawback of bagging is its random selection, where the classification performance depends on chance to choose a suitable subset of training objects. This paper proposes a novel modified version of bagging, named enhanced Bagging (eBagging), which uses a new mechanism (error-based bootstrapping) when constructing training sets in order to cope with this problem. In the experimental setting, the proposed eBagging technique was tested on 33 well-known benchmark datasets and compared with both bagging, random forest and boosting techniques using well-known classification algorithms: Support Vector Machines (SVM), decision trees (C4.5), k-Nearest Neighbour (kNN) and Naive Bayes (NB). The results show that eBagging outperforms its counterparts by classifying the data points more accurately while reducing the training error
Dual Access Control for Cloud-Based Data Storage and Sharing

Cloud-based data storage service has drawn increasing interests from both academic and industry in the recent years due to its efficient and low cost management. Since it provides services in an open network, it is urgent for service providers to make use of secure data storage and sharing mechanism to ensure data confidentiality and service user privacy. To protect sensitive data from being compromised, the most widely used method is encryption. However, simply encrypting data (e.g., via AES) cannot fully address the practical need of data management. Besides, an effective access control over download request also needs to be considered so that Economic Denial of Sustainability (EDoS) attacks cannot be launched to hinder users from enjoying service. In this article, we consider the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dual access control</i> , in the context of cloud-based storage, in the sense that we design a control mechanism over both data access and download request without loss of security and efficiency. Two dual access control systems are designed in this article, where each of them is for a distinct designed setting. The security and experimental analysis for the systems are also presented.
On Tractable Representations of Binary Neural Networks

We consider the compilation of a binary neural network’s decision function into tractable representations such as Ordered Binary Decision Diagrams (OBDDs) and Sentential Decision Diagrams (SDDs). Obtaining this function as an OBDD/SDD facilitates the explanation and formal veriﬁcation of a neural network’s behavior. First, we consider the task of verifying the robustness of a neural network, and show how we can compute the expected robustness of a neural network, given an OBDD/SDD representation of it. Next, we consider a more efﬁcient approach for compiling neural networks, based on a pseudo-polynomial time algorithm for compiling a neuron. We then provide a case study in a handwritten digits dataset, highlighting how two neural networks trained from the same dataset can have very high accuracies, yet have very different levels of robustness. Finally, in experiments, we show that it is feasible to obtain compact representations of neural networks as SDDs.
Online Topology Inference from Streaming Stationary Graph Signals with Partial Connectivity Information

We develop online graph learning algorithms from streaming network data. Our goal is to track the (possibly) time-varying network topology, and affect memory and computational savings by processing the data on-the-fly as they are acquired. The setup entails observations modeled as stationary graph signals generated by local diffusion dynamics on the unknown network. Moreover, we may have a priori information on the presence or absence of a few edges as in the link prediction problem. The stationarity assumption implies that the observations’ covariance matrix and the so-called graph shift operator (GSO—a matrix encoding the graph topology) commute under mild requirements. This motivates formulating the topology inference task as an inverse problem, whereby one searches for a sparse GSO that is structurally admissible and approximately commutes with the observations’ empirical covariance matrix. For streaming data, said covariance can be updated recursively, and we show online proximal gradient iterations can be brought to bear to efficiently track the time-varying solution of the inverse problem with quantifiable guarantees. Specifically, we derive conditions under which the GSO recovery cost is strongly convex and use this property to prove that the online algorithm converges to within a neighborhood of the optimal time-varying batch solution. Numerical tests illustrate the effectiveness of the proposed graph learning approach in adapting to streaming information and tracking changes in the sought dynamic network.
Advanced Dropout: A Model-free Methodology for Bayesian Dropout Optimization

Due to lack of data, overfitting ubiquitously exists in real-world applications of deep neural networks (DNNs). We propose advanced dropout, a model-free methodology, to mitigate overfitting and improve the performance of DNNs. The advanced dropout technique applies a model-free and easily implemented distribution with parametric prior, and adaptively adjusts dropout rate. Specifically, the distribution parameters are optimized by stochastic gradient variational Bayes in order to carry out an end-to-end training. We evaluate the effectiveness of the advanced dropout against nine dropout techniques on seven computer vision datasets (five small-scale datasets and two large-scale datasets) with various base models. The advanced dropout outperforms all the referred techniques on all the datasets.We further compare the effectiveness ratios and find that advanced dropout achieves the highest one on most cases. Next, we conduct a set of analysis of dropout rate characteristics, including convergence of the adaptive dropout rate, the learned distributions of dropout masks, and a comparison with dropout rate generation without an explicit distribution. In addition, the ability of overfitting prevention is evaluated and confirmed. Finally, we extend the application of the advanced dropout to uncertainty inference, network pruning, text classification, and regression. The proposed advanced dropout is also superior to the corresponding referred methods. Codes are available at https://github.com/PRIS-CV/AdvancedDropout.
JAICOB: A Data Science Chatbot

The application of natural language to improve students' interaction with information systems is demonstrated to be beneficial.In particular, advances in cognitive computing enable a new way of interaction that accelerates insight from existing information sources, thereby contributing to the process of learning.This work aims at researching the application of cognitive computing in blended learning environments.We propose a modular cognitive agent architecture for pedagogical question answering, featuring social dialogue (small talk), improved for a specific knowledge domain.This system has been implemented as a personal agent to assist students in learning Data Science and Machine Learning techniques.Its implementation includes the training of machine learning models and natural language understanding algorithms in a humanlike interface.The effectiveness of the system has been validated through an experiment.
Short-Term and Long-Term Context Aggregation Network for Video Inpainting

Video inpainting aims to restore missing regions of a video and has many applications such as video editing and object removal. However, existing methods either suffer from inaccurate short-term context aggregation or rarely explore long-term frame information. In this work, we present a novel context aggregation network to effectively exploit both short-term and long-term frame information for video inpainting. In the encoding stage, we propose boundary-aware short-term context aggregation, which aligns and aggregates, from neighbor frames, local regions that are closely related to the boundary context of missing regions into the target frame (The target frame refers to the current input frame under inpainting.). Furthermore, we propose dynamic long-term context aggregation to globally refine the feature map generated in the encoding stage using long-term frame features, which are dynamically updated throughout the inpainting process. Experiments show that it outperforms state-of-the-art methods with better inpainting results and fast inpainting speed.
Comparing Natural Language Processing Techniques for Alzheimer’s Dementia Prediction in Spontaneous Speech

Alzheimer's Dementia (AD) is an incurable, debilitating, and progressive neurodegenerative condition that affects cognitive function. Early diagnosis is important as therapeutics can delay progression and give those diagnosed vital time. Developing models that analyse spontaneous speech could eventually provide an efficient diagnostic modality for earlier diagnosis of AD. The Alzheimer's Dementia Recognition through Spontaneous Speech task offers acoustically pre-processed and balanced datasets for the classification and prediction of AD and associated phenotypes through the modelling of spontaneous speech. We exclusively analyse the supplied textual transcripts of the spontaneous speech dataset, building and comparing performance across numerous models for the classification of AD vs controls and the prediction of Mental Mini State Exam scores. We rigorously train and evaluate Support Vector Machines (SVMs), Gradient Boosting Decision Trees (GBDT), and Conditional Random Fields (CRFs) alongside deep learning Transformer based models. We find our top performing models to be a simple Term Frequency-Inverse Document Frequency (TF-IDF) vectoriser as input into a SVM model and a pre-trained Transformer based model 'DistilBERT' when used as an embedding layer into simple linear models. We demonstrate test set scores of 0.81-0.82 across classification metrics and a RMSE of 4.58.
Word Rotator’s Distance

A key principle in assessing textual similarity is measuring the degree of semantic overlap between two texts by considering the word alignment.Such alignment-based approaches are intuitive and interpretable; however, they are empirically inferior to the simple cosine similarity between general-purpose sentence vectors.To address this issue, we focus on and demonstrate the fact that the norm of word vectors is a good proxy for word importance, and their angle is a good proxy for word similarity.Alignment-based approaches do not distinguish them, whereas sentence-vector approaches automatically use the norm as the word importance.Accordingly, we propose a method that first decouples word vectors into their norm and direction, and then computes alignment-based similarity using earth mover's distance (i.e., optimal transport cost), which we refer to as word rotator's distance.Besides, we find how to "grow" the norm and direction of word vectors (vector converter), which is a new systematic approach derived from sentence-vector estimation methods.On several textual similarity datasets, the combination of these simple proposed methods outperformed not only alignment-based approaches but also strong baselines. 1