OpenRaiser/Intern-Atlas huggingface dataset:
Recursive Experts: An Efficient Optimal Mixture of Learning Systems in Dynamic Environments
Venue: arXiv cs.LG 2020
Year: 2020

Sequential learning systems are used in a wide variety of problems from decision making to optimization, where they provide a 'belief' (opinion) to nature, and then update this belief based on the feedback (result) to minimize (or maximize) some cost or loss (conversely, utility or gain). The goal is to reach an objective by exploiting the temporal relation inherent to the nature's feedback (state). By exploiting this relation, specific learning systems can be designed that perform asymptotically optimal for various applications. However, if the framework of the problem is not stationary, i.e., the nature's state sometimes changes arbitrarily, the past cumulative belief revision done by the system may become useless and the system may fail if it lacks adaptivity. While this adaptivity can be directly implemented in specific cases (e.g., convex optimization), it is mostly not straightforward for general learning tasks. To this end, we propose an efficient optimal mixture framework for general sequential learning systems, which we call the recursive experts for dynamic environments. For this purpose, we design hyper-experts that incorporate the learning systems at our disposal and recursively merge in a specific way to achieve minimax optimal regret bounds up to constant factors. The multiplicative increases in computational complexity from the initial system to our adaptive system are only logarithmic-in-time factors.
Road to the White House: Analyzing the Relations Between Mainstream and Social Media During the U.S. Presidential Primaries
Venue: arXiv cs.SI 2020
Year: 2020

Information is crucial to the function of a democratic society where well-informed citizens can make rational political decisions. While in the past political entities were primarily utilizing newspaper and later television to inform the public, with the rise of the Internet and online social media, the political arena has transformed into a more complex structure. Now, more than ever, people express themselves online while mainstream news agencies attempt to seize the power of the Internet to spread their agenda. To grasp the political coexistence of mainstream media and online social media, in this paper, we perform an analysis between these two sources of information in the context of the U.S. 2020 presidential election. In particular, we collect data during the 2020 Democratic Party presidential primaries pertaining to the candidates and by analyzing this data, we highlight similarities and differences between these two main types of sources, detect the potential impact they have on each other, and understand how this impact relationship can change over time. To supplement these two main sources and to establish a baseline, we also include Google Trends search results and Polling results for each of the candidates that are being analyzed.
High-Resolution Augmentation for Automatic Template-Based Matching of Human Models
Venue: arXiv cs.CV 2020
Year: 2020

We propose a new approach for 3D shape matching of deformable human shapes. Our approach is based on the joint adoption of three different tools: an intrinsic spectral matching pipeline, a morphable model, and an extrinsic details refinement. By operating in conjunction, these tools allow us to greatly improve the quality of the matching while at the same time resolving the key issues exhibited by each tool individually. In this paper we present an innovative High-Resolution Augmentation (HRA) strategy that enables highly accurate correspondence even in the presence of significant mesh resolution mismatch between the input shapes. This augmentation provides an effective workaround for the resolution limitations imposed by the adopted morphable model. The HRA in its global and localized versions represents a novel refinement strategy for surface subdivision methods. We demonstrate the accuracy of the proposed pipeline on multiple challenging benchmarks, and showcase its effectiveness in surface registration and texture transfer.
Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

Despite being the seventh most widely spoken language in the world, Bengali has received much less attention in machine translation literature due to being low in resources. Most publicly available parallel corpora for Bengali are not large enough; and have rather poor quality, mostly because of incorrect sentence alignments resulting from erroneous sentence segmentation, and also because of a high volume of noise present in them. In this work, we build a customized sentence segmenter for Bengali and propose two novel methods for parallel corpus creation on low-resource setups: aligner ensembling and batch filtering. With the segmenter and the two methods combined, we compile a high-quality Bengali-English parallel corpus comprising of 2.75 million sentence pairs, more than 2 million of which were not available before. Training on neural models, we achieve an improvement of more than 9 BLEU score over previous approaches to Bengali-English machine translation. We also evaluate on a new test set of 1000 pairs made with extensive quality control. We release the segmenter, parallel corpus, and the evaluation set, thus elevating Bengali from its low-resource status. To the best of our knowledge, this is the first ever large scale study on Bengali-English machine translation. We believe our study will pave the way for future research on Bengali-English machine translation as well as other low-resource languages. Our data and code are available at https://github.com/csebuetnlp/banglanmt.
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference
Venue: arXiv cs.LG 2020
Year: 2020

The neural attention mechanism plays an important role in many natural language processing applications. In particular, the use of multi-head attention extends single-head attention by allowing a model to jointly attend information from different perspectives. Without explicit constraining, however, multi-head attention may suffer from attention collapse, an issue that makes different heads extract similar attentive features, thus limiting the model's representation power. In this paper, for the first time, we provide a novel understanding of multi-head attention from a Bayesian perspective. Based on the recently developed particle-optimization sampling techniques, we propose a non-parametric approach that explicitly improves the repulsiveness in multi-head attention and consequently strengthens model's expressiveness. Remarkably, our Bayesian interpretation provides theoretical inspirations on the not-well-understood questions: why and how one uses multi-head attention. Extensive experiments on various attention models and applications demonstrate that the proposed repulsive attention can improve the learned feature diversity, leading to more informative representations with consistent performance improvement on various tasks.
Deriving Visual Semantics from Spatial Context: An Adaptation of LSA and Word2Vec to generate Object and Scene Embeddings from Images
Venue: arXiv cs.CV 2020
Year: 2020

Embeddings are an important tool for the representation of word meaning. Their effectiveness rests on the distributional hypothesis: words that occur in the same context carry similar semantic information. Here, we adapt this approach to index visual semantics in images of scenes. To this end, we formulate a distributional hypothesis for objects and scenes: Scenes that contain the same objects (object context) are semantically related. Similarly, objects that appear in the same spatial context (within a scene or subregions of a scene) are semantically related. We develop two approaches for learning object and scene embeddings from annotated images. In the first approach, we adapt LSA and Word2vec's Skipgram and CBOW models to generate two sets of embeddings from object co-occurrences in whole images, one for objects and one for scenes. The representational space spanned by these embeddings suggests that the distributional hypothesis holds for images. In an initial application of this approach, we show that our image-based embeddings improve scene classification models such as ResNet18 and VGG-11 (3.72\% improvement on Top5 accuracy, 4.56\% improvement on Top1 accuracy). In the second approach, rather than analyzing whole images of scenes, we focus on co-occurrences of objects within subregions of an image. We illustrate that this method yields a sensible hierarchical decomposition of a scene into collections of semantically related objects. Overall, these results suggest that object and scene embeddings from object co-occurrences and spatial context yield semantically meaningful representations as well as computational improvements for downstream applications such as scene classification.
Longformer for MS MARCO Document Re-ranking Task
Venue: arXiv cs.IR 2020
Year: 2020

Two step document ranking, where the initial retrieval is done by a classical information retrieval method, followed by neural re-ranking model, is the new standard. The best performance is achieved by using transformer-based models as re-rankers, e.g., BERT. We employ Longformer, a BERT-like model for long documents, on the MS MARCO document re-ranking task. The complete code used for training the model can be found on: https://github.com/isekulic/longformer-marco
Robust 2D Assembly Sequencing via Geometric Planning with Learned Scores
Venue: arXiv cs.RO 2020
Year: 2020

To compute robust 2D assembly plans, we present an approach that combines geometric planning with a deep neural network. We train the network using the Box2D physics simulator with added stochastic noise to yield robustness scores--the success probabilities of planned assembly motions. As running a simulation for every assembly motion is impractical, we train a convolutional neural network to map assembly operations, given as an image pair of the subassemblies before and after they are mated, to a robustness score. The neural network prediction is used within a planner to quickly prune out motions that are not robust. We demonstrate this approach on two-handed planar assemblies, where the motions are one-step translations. Results suggest that the neural network can learn robustness to plan robust sequences an order of magnitude faster than physics simulation.
An enhanced performance for H.265/SHVC based on combined AEGBM3D filter and back-propagation neural network
Venue: arXiv cs.MM 2020
Year: 2020

This paper deals with the latest video coding standard H265 SHVC, a scalable extension to High Efficiency Video Coding (HEVC). HEVC introduces new coding tools compared to its predecessor and is backward compatible with all types of electronic gadgets. The gadgets with different display capabilities cannot be offered the same quality video due to the constraints in transmission bandwidth is a major problem. One solution to this problem will be the compression of the video sequence which is focused in this paper to preserve or increase PSNR while reducing bit-rate besides a novel method implemented in SHVC encoder. The novel method undergoes a combined AEGBM3D (adaptive edge guided block-matching and 3D) filtering and back-propagation technique. The technique includes an AEGBM3D filter which avoids spatial redundancy and de-noise frames; hence enhancement in PSNR is achieved. The obtained PSNR of the video is compared with the set threshold PSNR to maintain PSNR above the threshold by repeated AEGBM3D filtering. The BP technique based on the neural network machine learning approach continually restrains the output if the input block does not contain a feature they were trained to recognize. This frequent control over the output produces few bits; hence reduction in bit-rate is achieved. The simulation results show that the proposed technique delivers an average increment of 0.16 and 0.25dB in PSNR and an average decrement of 28 and 37% in bit-rate for 1.5 and 2 times spatial ratios respectively, compared with the existing methods.
TADOC: Text Analytics Directly on Compression
Venue: arXiv cs.DS 2020
Year: 2020

This article provides a comprehensive description of Text Analytics Directly on Compression (TADOC), which enables direct document analytics on compressed textual data. The article explains the concept of TADOC and the challenges to its effective realizations. Additionally, a series of guidelines and technical solutions that effectively address those challenges, including the adoption of a hierarchical compression method and a set of novel algorithms and data structure designs, are presented. Experiments on six data analytics tasks of various complexities show that TADOC can save 90.8% storage space and 87.9% memory usage, while halving data processing times.
Unsupervised Anomaly Detection on Temporal Multiway Data
Venue: arXiv cs.LG 2020
Year: 2020

Temporal anomaly detection looks for irregularities over space-time. Unsupervised temporal models employed thus far typically work on sequences of feature vectors, and much less on temporal multiway data. We focus our investigation on two-way data, in which a data matrix is observed at each time step. Leveraging recent advances in matrix-native recurrent neural networks, we investigated strategies for data arrangement and unsupervised training for temporal multiway anomaly detection. These include compressing-decompressing, encoding-predicting, and temporal data differencing. We conducted a comprehensive suite of experiments to evaluate model behaviors under various settings on synthetic data, moving digits, and ECG recordings. We found interesting phenomena not previously reported. These include the capacity of the compact matrix LSTM to compress noisy data near perfectly, making the strategy of compressing-decompressing data ill-suited for anomaly detection under the noise. Also, long sequence of vectors can be addressed directly by matrix models that allow very long context and multiple step prediction. Overall, the encoding-predicting strategy works very well for the matrix LSTMs in the conducted experiments, thanks to its compactness and better fit to the data dynamics.
Two and Three-Party Digital Goods Auctions: Scalable Privacy Analysis
Venue: arXiv cs.IT 2020
Year: 2020

A digital goods auction is a type of auction where potential buyers bid the maximal price that they are willing to pay for a certain item, which a seller can produce at a negligible cost and in unlimited quantity. To maximise her benefits, the aim for the seller is to find the optimal sales price, which every buyer whose bid is not lower will pay. For fairness and privacy purposes, buyers may be concerned about protecting the confidentiality of their bids. Secure Multi-Party Computation is a domain of Cryptography that would allow the seller to compute the optimal sales price while guaranteeing that the bids remain secret. Paradoxically, as a function of the buyers' bids, the sales price inevitably reveals some private information. Generic frameworks and entropy-based techniques based on Quantitative Information Flow have been developed in order to quantify and restrict those leakages. Due to their combinatorial nature, these techniques do not scale to large input spaces. In this work, we aim at scaling those privacy analyses to large input spaces in the particular case of digital goods auctions. We derive closed-form formulas for the posterior min-entropy of private inputs in two and three-party auctions, which enables us to effectively quantify the information leaks for arbitrarily large input spaces. We also provide supportive experimental evidence that enables us to formulate a conjecture that would allow us to extend our results to any number of parties.
PPTP: Price-based Path-specified Transport Protocol for Named Data Network using Blockchain
Venue: arXiv cs.NI 2020
Year: 2020

Serving as a potential future Internet architecture, Named Data Network (NDN) offers superior information-centric architectural support for mobile ad-hoc networking. Using NDN as an underlying protocol, end-user devices (e.g. IoT device and smart phone) formulate a multi-hop (mesh) network, in which certain devices play a role of forwarding packets for others and/or act as gateways to access the Internet. Nevertheless, an autonomous (selfish) node in an ad-hoc network has two disincentives for forwarding packets for others: energy expenditure and possible delays for its own data. This paper introduces Price-based Path-specified Transport Protocol (PPT) for NDN, using blockchain as a payment platform to support money transfers between autonomous nodes thus to incentivise packet forwarding. In PPTP, routers advertise their expected prices for packet forwarding and consumers estimate the costs and select the appropriate paths for content downloading. PPTP is still an on-going project therefore this paper will present the design principle and planed functions, and show how PPTP are related to other existing blockchain-based networking solutions.
Resilient Composition of Drone Services for Delivery
Venue: arXiv cs.NI 2020
Year: 2020

We propose a novel resilient drone service composition framework for delivery in dynamic weather conditions. We use a skyline approach to select an optimal set of candidate drone services at the source node in a skyway network. Drone services are initially composed using a novel constraint-aware deterministic lookahead algorithm using the multi-armed bandit tree exploration. We propose a heuristic-based resilient service composition approach that adapts to runtime changes and periodically updates the composition to meet delivery expectations. Experimental results prove the efficiency of the proposed approach.
Physical-Layer Security for Two-Hop Air-to-Underwater Communication Systems With Fixed-Gain Amplify-and-Forward Relaying
Venue: arXiv cs.IT 2020
Year: 2020

We analyze a secure two-hop mixed radio frequency (RF) and underwater wireless optical communication (UWOC) system using a fixed-gain amplify-and-forward (AF) relay. The UWOC channel is modeled using a unified mixture exponential-generalized Gamma distribution to consider the combined effects of air bubbles and temperature gradients on transmission characteristics. Both legitimate and eavesdropping RF channels are modeled using flexible $α-μ$ distributions. Specifically, we first derive both the probability density function (PDF) and cumulative distribution function (CDF) of the received signal-to-noise ratio (SNR) of the mixed RF and UWOC system. Based on the PDF and CDF expressions, we derive the closed-form expressions for the tight lower bound of the secrecy outage probability (SOP) and the probability of non-zero secrecy capacity (PNZ), which are both expressed in terms bivariate Fox's $H$-function. To utilize these analytical expressions, we derive asymptotic expressions of SOP and PNZ using only elementary functions. Also, we use asymptotic expressions to determine the optimal transmitting power to maximize energy efficiency. Further, we thoroughly investigate the effect of levels of air bubbles and temperature gradients in the UWOC channel, and study nonlinear characteristics of the transmission medium and the number of multipath clusters of the RF channel on the secrecy performance. Finally, all analyses are validated using Monte Carlo simulation.
Dynamic Horizon Value Estimation for Model-based Reinforcement Learning
Venue: arXiv cs.LG 2020
Year: 2020

Existing model-based value expansion methods typically leverage a world model for value estimation with a fixed rollout horizon to assist policy learning. However, the fixed rollout with an inaccurate model has a potential to harm the learning process. In this paper, we investigate the idea of using the model knowledge for value expansion adaptively. We propose a novel method called Dynamic-horizon Model-based Value Expansion (DMVE) to adjust the world model usage with different rollout horizons. Inspired by reconstruction-based techniques that can be applied for visual data novelty detection, we utilize a world model with a reconstruction module for image feature extraction, in order to acquire more precise value estimation. The raw and the reconstructed images are both used to determine the appropriate horizon for adaptive value expansion. On several benchmark visual control tasks, experimental results show that DMVE outperforms all baselines in sample efficiency and final performance, indicating that DMVE can achieve more effective and accurate value estimation than state-of-the-art model-based methods.
Interactive Steering of Hierarchical Clustering
Venue: arXiv cs.LG 2020
Year: 2020

Hierarchical clustering is an important technique to organize big data for exploratory data analysis. However, existing one-size-fits-all hierarchical clustering methods often fail to meet the diverse needs of different users. To address this challenge, we present an interactive steering method to visually supervise constrained hierarchical clustering by utilizing both public knowledge (e.g., Wikipedia) and private knowledge from users. The novelty of our approach includes 1) automatically constructing constraints for hierarchical clustering using knowledge (knowledge-driven) and intrinsic data distribution (data-driven), and 2) enabling the interactive steering of clustering through a visual interface (user-driven). Our method first maps each data item to the most relevant items in a knowledge base. An initial constraint tree is then extracted using the ant colony optimization algorithm. The algorithm balances the tree width and depth and covers the data items with high confidence. Given the constraint tree, the data items are hierarchically clustered using evolutionary Bayesian rose tree. To clearly convey the hierarchical clustering results, an uncertainty-aware tree visualization has been developed to enable users to quickly locate the most uncertain sub-hierarchies and interactively improve them. The quantitative evaluation and case study demonstrate that the proposed approach facilitates the building of customized clustering trees in an efficient and effective manner.
Economic Theoretic LEO Satellite Coverage Control: An Auction-based Framework
Venue: arXiv cs.NI 2020
Year: 2020

Recently, ultra-dense low earth orbit (LEO) satelliteconstellation over high-frequency bands has considered as one ofpromising solutions to supply coverage all over the world. Givensatellite constellations, efficient beam coverage schemes should beemployed at satellites to provide seamless services and full-viewcoverage. In LEO systems, hybrid wide and spot beam coverageschemes are generally used, where the LEO provides a widebeam for large area coverage and additional several steering spotbeams for high speed data access. In this given setting, schedulingmultiple spot beams is essentially required. In order to achievethis goal, Vickery-Clarke-Groves (VCG) auction-based trustfulalgorithm is proposed in this paper for scheduling multiple spotbeams for more efficient seamless services and full-view coverage.
Identifying synergies in private and public transportation
Venue: arXiv cs.CY 2020
Year: 2020

In this paper, we explore existing synergies between private and public transportation as provided by taxi and bus services on the level of individual trips. While these modes are typically separated for economic reasons, in a future with shared Autonomous Vehicles (AVs) providing cheap and efficient transportation services, such distinctions will blur. Consequently, optimization based on real-time data will allow exploiting parallels in demand in a dynamic way, such as the proposed approach of the current work. New operational and pricing strategies will then evolve, providing service in a more efficient way and utilizing a dynamic landscape of urban transportation. In the current work, we evaluate existing parallels between individual bus and taxi trips in two Asian cities and show how exploiting these synergies could lead to an increase in transportation service quality.