OpenRaiser/Intern-Atlas huggingface dataset:
An analysis of noise folding for low-rank matrix recovery
Venue: arXiv cs.IT 2020
Year: 2020

Previous work regarding low-rank matrix recovery has concentrated on the scenarios in which the matrix is noise-free and the measurements are corrupted by noise. However, in practical application, the matrix itself is usually perturbed by random noise preceding to measurement. This paper concisely investigates this scenario and evidences that, for most measurement schemes utilized in compressed sensing, the two models are equivalent with the central distinctness that the noise associated with (\ref{eq.3}) is larger by a factor to $mn/M$, where $m,~n$ are the dimension of the matrix and $M$ is the number of measurements. Additionally, this paper discusses the reconstruction of low-rank matrices in the setting, presents sufficient conditions based on the associating null space property to guarantee the robust recovery and obtains the number of measurements. Furthermore, for the non-Gaussian noise scenario, we further explore it and give the corresponding result. The simulation experiments conducted, on the one hand show effect of noise variance on recovery performance, on the other hand demonstrate the verifiability of the proposed model.
Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learning
Venue: arXiv cs.CV 2020
Year: 2020

One of the key factors of enabling machine learning models to comprehend and solve real-world tasks is to leverage multimodal data. Unfortunately, annotation of multimodal data is challenging and expensive. Recently, self-supervised multimodal methods that combine vision and language were proposed to learn multimodal representations without annotation. However, these methods often choose to ignore the presence of high levels of noise and thus yield sub-optimal results. In this work, we show that the problem of noise estimation for multimodal data can be reduced to a multimodal density estimation task. Using multimodal density estimation, we propose a noise estimation building block for multimodal representation learning that is based strictly on the inherent correlation between different modalities. We demonstrate how our noise estimation can be broadly integrated and achieves comparable results to state-of-the-art performance on five different benchmark datasets for two challenging multimodal tasks: Video Question Answering and Text-To-Video Retrieval. Furthermore, we provide a theoretical probabilistic error bound substantiating our empirical results and analyze failure cases. Code: https://github.com/elad-amrani/ssml.
Generalizable semi-supervised learning method to estimate mass from sparsely annotated images
Venue: arXiv cs.CV 2020
Year: 2020

Mass flow estimation is of great importance to several industries, and it can be quite challenging to obtain accurate estimates due to limitation in expense or general infeasibility. In the context of agricultural applications, yield monitoring is a key component to precision agriculture and mass flow is the critical factor to measure. Measuring mass flow allows for field productivity analysis, cost minimization, and adjustments to machine efficiency. Methods such as volume or force-impact have been used to measure mass flow; however, these methods are limited in application and accuracy. In this work, we use deep learning to develop and test a vision system that can accurately estimate the mass of sugarcane while running in real-time on a sugarcane harvester during operation. The deep learning algorithm that is used to estimate mass flow is trained using very sparsely annotated images (semi-supervised) using only final load weights (aggregated weights over a certain period of time). The deep neural network (DNN) succeeds in capturing the mass of sugarcane accurately and surpasses older volumetric-based methods, despite highly varying lighting and material colors in the images. The deep neural network is initially trained to predict mass on laboratory data (bamboo) and then transfer learning is utilized to apply the same methods to estimate mass of sugarcane. Using a vision system with a relatively lightweight deep neural network we are able to estimate mass of bamboo with an average error of 4.5% and 5.9% for a select season of sugarcane.
Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robot
Venue: arXiv cs.RO 2020
Year: 2020

Model Predictive Control (MPC) is a powerful control technique that handles constraints, takes the system's dynamics into account, and optimizes for a given cost function. In practice, however, it often requires an expert to craft and tune this cost function and find trade-offs between different state penalties to satisfy simple high level objectives. In this paper, we use Reinforcement Learning and in particular value learning to approximate the value function given only high level objectives, which can be sparse and binary. Building upon previous works, we present improvements that allowed us to successfully deploy the method on a real world unmanned ground vehicle. Our experiments show that our method can learn the cost function from scratch and without human intervention, while reaching a performance level similar to that of an expert-tuned MPC. We perform a quantitative comparison of these methods with standard MPC approaches both in simulation and on the real robot.
Introduction to deep learning
Venue: arXiv cs.LG 2020
Year: 2020

Deep Learning (DL) has made a major impact on data science in the last decade. This chapter introduces the basic concepts of this field. It includes both the basic structures used to design deep neural networks and a brief survey of some of its popular use cases.
Stretching the capacity of Hardware Transactional Memory in IBM POWER architectures
Venue: arXiv cs.DC 2020
Year: 2020

The hardware transactional memory (HTM) implementations in commercially available processors are significantly hindered by their tight capacity constraints. In practice, this renders current HTMs unsuitable to many real-world workloads of in-memory databases. This paper proposes SI-HTM, which stretches the capacity bounds of the underlying HTM, thus opening HTM to a much broader class of applications. SI-HTM leverages the HTM implementation of the IBM POWER architecture with a software layer to offer a single-version implementation of Snapshot Isolation. When compared to HTM- and software-based concurrency control alternatives, SI-HTM exhibits improved scalability, achieving speedups of up to 300% relatively to HTM on in-memory database benchmarks.
Recognizing Affiliation: Using Behavioural Traces to Predict the Quality of Social Interactions in Online Games
Venue: arXiv cs.HC 2020
Year: 2020

Online social interactions in multiplayer games can be supportive and positive or toxic and harmful; however, few methods can easily assess interpersonal interaction quality in games. We use behavioural traces to predict affiliation between dyadic strangers, facilitated through their social interactions in an online gaming setting. We collected audio, video, in-game, and self-report data from 23 dyads, extracted 75 features, trained Random Forest and Support Vector Machine models, and evaluated their performance predicting binary (high/low) as well as continuous affiliation toward a partner. The models can predict both binary and continuous affiliation with up to 79.1% accuracy (F1) and 20.1% explained variance (R2) on unseen data, with features based on verbal communication demonstrating the highest potential. Our findings can inform the design of multiplayer games and game communities, and guide the development of systems for matchmaking and mitigating toxic behaviour in online games.
Synthetic Error Dataset Generation Mimicking Bengali Writing Pattern
Venue: arXiv cs.CL 2020
Year: 2020

While writing Bengali using English keyboard, users often make spelling mistakes. The accuracy of any Bengali spell checker or paragraph correction module largely depends on the kind of error dataset it is based on. Manual generation of such error dataset is a cumbersome process. In this research, We present an algorithm for automatic misspelled Bengali word generation from correct word through analyzing Bengali writing pattern using QWERTY layout English keyboard. As part of our analysis, we have formed a list of most commonly used Bengali words, phonetically similar replaceable clusters, frequently mispressed replaceable clusters, frequently mispressed insertion prone clusters and some rules for Juktakkhar (constant letter clusters) handling while generating errors.
Neural Operator: Graph Kernel Network for Partial Differential Equations
Venue: arXiv cs.LG 2020
Year: 2020

The classical development of neural networks has been primarily for mappings between a finite-dimensional Euclidean space and a set of classes, or between two finite-dimensional Euclidean spaces. The purpose of this work is to generalize neural networks so that they can learn mappings between infinite-dimensional spaces (operators). The key innovation in our work is that a single set of network parameters, within a carefully designed network architecture, may be used to describe mappings between infinite-dimensional spaces and between different finite-dimensional approximations of those spaces. We formulate approximation of the infinite-dimensional mapping by composing nonlinear activation functions and a class of integral operators. The kernel integration is computed by message passing on graph networks. This approach has substantial practical consequences which we will illustrate in the context of mappings between input data to partial differential equations (PDEs) and their solutions. In this context, such learned networks can generalize among different approximation methods for the PDE (such as finite difference or finite element methods) and among approximations corresponding to different underlying levels of resolution and discretization. Experiments confirm that the proposed graph kernel network does have the desired properties and show competitive performance compared to the state of the art solvers.
Study of Linear Precoding and Stream Combining for Rate Splitting in MU-MIMO Systems
Venue: arXiv cs.IT 2020
Year: 2020

This paper develops stream combining techniques for rate-splitting (RS) multiple-antenna systems with multiple users to enhance the common rate. We propose linear combining techniques based on the Min-Max, the maximum ratio and the minimum mean-square error criteria along with Regularized Block Diagonalization (RBD) precoders for RS-based multiuser multiple-antenna systems. An analysis of the sum rate performance is carried out, leading to closed-form expressions. Simulations show that the proposed combining schemes offer a significant sum rate performance gain over conventional linear precoding schemes.
Super Resolution Using Segmentation-Prior Self-Attention Generative Adversarial Network
Venue: arXiv cs.CV 2020
Year: 2020

Convolutional Neural Network (CNN) is intensively implemented to solve super resolution (SR) tasks because of its superior performance. However, the problem of super resolution is still challenging due to the lack of prior knowledge and small receptive field of CNN. We propose the Segmentation-Piror Self-Attention Generative Adversarial Network (SPSAGAN) to combine segmentation-priors and feature attentions into a unified framework. This combination is led by a carefully designed weighted addition to balance the influence of feature and segmentation attentions, so that the network can emphasize textures in the same segmentation category and meanwhile focus on the long-distance feature relationship. We also propose a lightweight skip connection architecture called Residual-in-Residual Sparse Block (RRSB) to further improve the super-resolution performance and save computation. Extensive experiments show that SPSAGAN can generate more realistic and visually pleasing textures compared to state-of-the-art SFTGAN and ESRGAN on many SR datasets.
MIMO Amplify-and-Forward Precoding for Networked Control Systems
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we consider a MIMO networked control system (NCS) in which a sensor amplifies and forwards the observed MIMO plant state to a remote controller via a MIMO fading channel. We focus on the MIMO amplify-and-forward (AF) precoding design at the sensor to minimize a weighted average state estimation error at the remote controller subject to an average communication power gain constraint of the sensor. The MIMO AF precoding design is formulated as an infinite horizon average cost Markov decision process (MDP). To deal with the curse of dimensionality associated with the MDP, we propose a novel continuous-time perturbation approach and derive an asymptotically optimal closed-form priority function for the MDP. Based on this, we derive a closed-form first-order optimal dynamic MIMO AF precoding solution, and the solution has an event-driven control structure. Specifically, the sensor activates the strongest eigenchannel to deliver a dynamically weighted combination of the plant states to the controller when the accumulated state estimation error exceeds a dynamic threshold. We further establish technical conditions for ensuring the stability of the MIMO NCS, and show that the mean square error of the plant state estimation is $\mathcal{O}\left(\frac{1}{\bar{F}}\right)$, where $\bar{F}$ is the maximum AF gain of the MIMO AF precoding.
TTPP: Temporal Transformer with Progressive Prediction for Efficient Action Anticipation
Venue: arXiv cs.CV 2020
Year: 2020

Video action anticipation aims to predict future action categories from observed frames. Current state-of-the-art approaches mainly resort to recurrent neural networks to encode history information into hidden states, and predict future actions from the hidden representations. It is well known that the recurrent pipeline is inefficient in capturing long-term information which may limit its performance in predication task. To address this problem, this paper proposes a simple yet efficient Temporal Transformer with Progressive Prediction (TTPP) framework, which repurposes a Transformer-style architecture to aggregate observed features, and then leverages a light-weight network to progressively predict future features and actions. Specifically, predicted features along with predicted probabilities are accumulated into the inputs of subsequent prediction. We evaluate our approach on three action datasets, namely TVSeries, THUMOS-14, and TV-Human-Interaction. Additionally we also conduct a comprehensive study for several popular aggregation and prediction strategies. Extensive results show that TTPP not only outperforms the state-of-the-art methods but also more efficient.
An Online Learning Based Path Selection for Multipath Video Telephony Service in Overlay
Venue: arXiv cs.NI 2020
Year: 2020

Even real time video telephony services have been pervasively applied, providing satisfactory quality of experience to users is still a challenge task especially in wireless networks. Multipath transmission is a promising solution to improve video quality by aggregating bandwidth. In existing multipath transmission solutions, sender concurrently splits traffic on default routing paths and has no flexibility to select paths. When default paths fall into severe congestion and the available bandwidth decreases, sender has to decrease video quality by reducing resolution or encoding bitrate. Deploying relay servers in current infrastructure to form overlay network provides path diversity. An online learning approach based on multi-armed bandits is applied for path selection to harvest maximum profit. Further, a congestion control algorithm adapted from BBR is implemented to probe bandwidth and to avoid link congestion. To maintain throughput stability and fairness, a smaller probe up gain value is used and the cycle length in bandwidth probe phase is randomized. To reduce delay, the inflight packets will be reduced to match with the estimated bandwidth delay product in the probe down phase. Experiments are conducted to verify the effectiveness the proposed solution to improve throughput and quality in video communication service.
Generating Emotionally Aligned Responses in Dialogues using Affect Control Theory
Venue: arXiv cs.CL 2020
Year: 2020

State-of-the-art neural dialogue systems excel at syntactic and semantic modelling of language, but often have a hard time establishing emotional alignment with the human interactant during a conversation. In this work, we bring Affect Control Theory (ACT), a socio-mathematical model of emotions for human-human interactions, to the neural dialogue generation setting. ACT makes predictions about how humans respond to emotional stimuli in social situations. Due to this property, ACT and its derivative probabilistic models have been successfully deployed in several applications of Human-Computer Interaction, including empathetic tutoring systems, assistive healthcare devices and two-person social dilemma games. We investigate how ACT can be used to develop affect-aware neural conversational agents, which produce emotionally aligned responses to prompts and take into consideration the affective identities of the interactants.
Multi-task Learning Based Neural Bridging Reference Resolution
Venue: arXiv cs.CL 2020
Year: 2020

We propose a multi task learning-based neural model for resolving bridging references tackling two key challenges. The first challenge is the lack of large corpora annotated with bridging references. To address this, we use multi-task learning to help bridging reference resolution with coreference resolution. We show that substantial improvements of up to 8 p.p. can be achieved on full bridging resolution with this architecture. The second challenge is the different definitions of bridging used in different corpora, meaning that hand-coded systems or systems using special features designed for one corpus do not work well with other corpora. Our neural model only uses a small number of corpus independent features, thus can be applied to different corpora. Evaluations with very different bridging corpora (ARRAU, ISNOTES, BASHI and SCICORP) suggest that our architecture works equally well on all corpora, and achieves the SoTA results on full bridging resolution for all corpora, outperforming the best reported results by up to 36.3 p.p..
Towards Solving Large-scale Expensive Optimization Problems Efficiently Using Coordinate Descent Algorithm
Venue: arXiv cs.NE 2020
Year: 2020

Many real-world problems are categorized as large-scale problems, and metaheuristic algorithms as an alternative method to solve large-scale problem; they need the evaluation of many candidate solutions to tackle them prior to their convergence, which is not affordable for practical applications since the most of them are computationally expensive. In other words, these problems are not only large-scale but also computationally expensive, that makes them very difficult to solve. There is no efficient surrogate model to support large-scale expensive global optimization (LSEGO) problems. As a result, the algorithms should address LSEGO problems using a limited computational budget to be applicable in real-world applications. Coordinate Descent (CD) algorithm is an optimization strategy based on the decomposition of a n-dimensional problem into n one-dimensional problem. To the best our knowledge, there is no significant study to assess benchmark functions with various dimensions and landscape properties to investigate CD algorithm. In this paper, we propose a modified Coordinate Descent algorithm (MCD) to tackle LSEGO problems with a limited computational budget. Our proposed algorithm benefits from two leading steps, namely, finding the region of interest and then shrinkage of the search space by folding it into the half with exponential speed. One of the main advantages of the proposed algorithm is being free of any control parameters, which makes it far from the intricacies of the tuning process. The proposed algorithm is compared with cooperative co-evolution with delta grouping on 20 benchmark functions with dimension 1000. Also, we conducted some experiments on CEC-2017, D=10, 30, 50, and 100, to investigate the behavior of MCD algorithm in lower dimensions. The results show that MCD is beneficial not only in large-scale problems, but also in low-scale optimization problems.
Progressive Growing of Neural ODEs
Venue: arXiv cs.LG 2020
Year: 2020

Neural Ordinary Differential Equations (NODEs) have proven to be a powerful modeling tool for approximating (interpolation) and forecasting (extrapolation) irregularly sampled time series data. However, their performance degrades substantially when applied to real-world data, especially long-term data with complex behaviors (e.g., long-term trend across years, mid-term seasonality across months, and short-term local variation across days). To address the modeling of such complex data with different behaviors at different frequencies (time spans), we propose a novel progressive learning paradigm of NODEs for long-term time series forecasting. Specifically, following the principle of curriculum learning, we gradually increase the complexity of data and network capacity as training progresses. Our experiments with both synthetic data and real traffic data (PeMS Bay Area traffic data) show that our training methodology consistently improves the performance of vanilla NODEs by over 64%.
OVC-Net: Object-Oriented Video Captioning with Temporal Graph and Detail Enhancement
Venue: arXiv cs.CV 2020
Year: 2020

Traditional video captioning requests a holistic description of the video, yet the detailed descriptions of the specific objects may not be available. Without associating the moving trajectories, these image-based data-driven methods cannot understand the activities from the spatio-temporal transitions in the inter-object visual features. Besides, adopting ambiguous clip-sentence pairs in training, it goes against learning the multi-modal functional mappings owing to the one-to-many nature. In this paper, we propose a novel task to understand the videos in object-level, named object-oriented video captioning. We introduce the video-based object-oriented video captioning network (OVC)-Net via temporal graph and detail enhancement to effectively analyze the activities along time and stably capture the vision-language connections under small-sample condition. The temporal graph provides useful supplement over previous image-based approaches, allowing to reason the activities from the temporal evolution of visual features and the dynamic movement of spatial locations. The detail enhancement helps to capture the discriminative features among different objects, with which the subsequent captioning module can yield more informative and precise descriptions. Thereafter, we construct a new dataset, providing consistent object-sentence pairs, to facilitate effective cross-modal learning. To demonstrate the effectiveness, we conduct experiments on the new dataset and compare it with the state-of-the-art video captioning methods. From the experimental results, the OVC-Net exhibits the ability of precisely describing the concurrent objects, and achieves the state-of-the-art performance.