OpenRaiser/Intern-Atlas huggingface dataset:
Online Estimation and Adaptation for Random Access with Successive Interference Cancellation
Venue: arXiv cs.IT 2020
Year: 2020

This paper proposes an adaptive transmission algorithm for slotted random access systems supporting the successive interference cancellation (SIC) at the access point (AP). When multiple users transmit packets simultaneously in a slot, owing to the SIC technique, the AP is able to decode them through SIC resolve procedures (SRPs), which may occupy multiple consequent slots. While such an SRP could potentially improve the system throughput, how to fully exploit this capability in practical systems is still questionable. In particular, the number of active users contending for the channel varies over time which complicates the algorithm design. By fully exploiting the potential of SIC, the proposed algorithm is designed to maximize the system throughput and minimize the access delay. For this purpose, an online estimation is introduced to estimate the number of active users in real-time and controls their transmissions accordingly. It is shown that the throughput of the proposed algorithm can reach up to 0.693 packets/slot under practical assumptions, which is the first result achieving the throughput limit proved by Yu-Giannakis. It is further shown that the system throughput of 0.559 packets/slot (80.6$\%$ of the throughput limit) is still achievable when the SIC capability is restricted by two.
Adaptive Control of Embedding Strength in Image Watermarking using Neural Networks
Venue: arXiv cs.MM 2020
Year: 2020

Digital image watermarking has been widely used in different applications such as copyright protection of digital media, such as audio, image, and video files. Two opposing criteria of robustness and transparency are the goals of watermarking methods. In this paper, we propose a framework for determining the appropriate embedding strength factor. The framework can use most DWT and DCT based blind watermarking approaches. We use Mask R-CNN on the COCO dataset to find a good strength factor for each sub-block. Experiments show that this method is robust against different attacks and has good transparency.
Probabilistic K-means Clustering via Nonlinear Programming
Venue: arXiv cs.LG 2020
Year: 2020

K-means is a classical clustering algorithm with wide applications. However, soft K-means, or fuzzy c-means at m=1, remains unsolved since 1981. To address this challenging open problem, we propose a novel clustering model, i.e. Probabilistic K-Means (PKM), which is also a nonlinear programming model constrained on linear equalities and linear inequalities. In theory, we can solve the model by active gradient projection, while inefficiently. Thus, we further propose maximum-step active gradient projection and fast maximum-step active gradient projection to solve it more efficiently. By experiments, we evaluate the performance of PKM and how well the proposed methods solve it in five aspects: initialization robustness, clustering performance, descending stability, iteration number, and convergence speed.
Learning to Multi-Task Learn for Better Neural Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

Scarcity of parallel sentence pairs is a major challenge for training high quality neural machine translation (NMT) models in bilingually low-resource scenarios, as NMT is data-hungry. Multi-task learning is an elegant approach to inject linguistic-related inductive biases into NMT, using auxiliary syntactic and semantic tasks, to improve generalisation. The challenge, however, is to devise effective training schedules, prescribing when to make use of the auxiliary tasks during the training process to fill the knowledge gaps of the main translation task, a setting referred to as biased-MTL. Current approaches for the training schedule are based on hand-engineering heuristics, whose effectiveness vary in different MTL settings. We propose a novel framework for learning the training schedule, ie learning to multi-task learn, for the MTL setting of interest. We formulate the training schedule as a Markov decision process which paves the way to employ policy learning methods to learn the scheduling policy. We effectively and efficiently learn the training schedule policy within the imitation learning framework using an oracle policy algorithm that dynamically sets the importance weights of auxiliary tasks based on their contributions to the generalisability of the main NMT task. Experiments on low-resource NMT settings show the resulting automatically learned training schedulers are competitive with the best heuristics, and lead to up to +1.1 BLEU score improvements.
Linking Social Media Posts to News with Siamese Transformers
Venue: arXiv cs.IR 2020
Year: 2020

Many computational social science projects examine online discourse surrounding a specific trending topic. These works often involve the acquisition of large-scale corpora relevant to the event in question to analyze aspects of the response to the event. Keyword searches present a precision-recall trade-off and crowd-sourced annotations, while effective, are costly. This work aims to enable automatic and accurate ad-hoc retrieval of comments discussing a trending topic from a large corpus, using only a handful of seed news articles.
Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle
Venue: arXiv cs.AI 2020
Year: 2020

Autonomous underwater vehicle (AUV) plays an increasingly important role in ocean exploration. Existing AUVs are usually not fully autonomous and generally limited to pre-planning or pre-programming tasks. Reinforcement learning (RL) and deep reinforcement learning have been introduced into the AUV design and research to improve its autonomy. However, these methods are still difficult to apply directly to the actual AUV system because of the sparse rewards and low learning efficiency. In this paper, we proposed a deep interactive reinforcement learning method for path following of AUV by combining the advantages of deep reinforcement learning and interactive RL. In addition, since the human trainer cannot provide human rewards for AUV when it is running in the ocean and AUV needs to adapt to a changing environment, we further propose a deep reinforcement learning method that learns from both human rewards and environmental rewards at the same time. We test our methods in two path following tasks---straight line and sinusoids curve following of AUV by simulating in the Gazebo platform. Our experimental results show that with our proposed deep interactive RL method, AUV can converge faster than a DQN learner from only environmental reward. Moreover, AUV learning with our deep RL from both human and environmental rewards can also achieve a similar or even better performance than that with the deep interactive RL method and can adapt to the actual environment by further learning from environmental rewards.
A Complex Networks Approach to Find Latent Clusters of Terrorist Groups
Venue: arXiv cs.CY 2020
Year: 2020

Given the extreme heterogeneity of actors and groups participating in terrorist actions, investigating and assessing their characteristics can be important to extract relevant information and enhance the knowledge on their behaviors. The present work will seek to achieve this goal via a complex networks approach. This approach will allow finding latent clusters of similar terror groups using information on their operational characteristics. Specifically, using open access data of terrorist attacks occurred worldwide from 1997 to 2016, we build a multi-partite network that includes terrorist groups and related information on tactics, weapons, targets, active regions. We propose a novel algorithm for cluster formation that expands our earlier work that solely used Gower's coefficient of similarity via the application of Von Neumann entropy for mode-weighting. This novel approach is compared with our previous Gower-based method and a heuristic clustering technique that only focuses on groups' ideologies. The comparative analysis demonstrates that the entropy-based approach tends to reliably reflect the structure of the data that naturally emerges from the baseline Gower-based method. Additionally, it provides interesting results in terms of behavioral and ideological characteristics of terrorist groups. We furthermore show that the ideology-based procedure tends to distort or hide existing patterns. Among the main statistical results, our work reveals that groups belonging to opposite ideologies can share very common behaviors and that Islamist/jihadist groups hold peculiar behavioral characteristics with respect to the others. Limitations and potential work directions are also discussed, introducing the idea of a dynamic entropy-based framework.
Inductive Document Network Embedding with Topic-Word Attention
Venue: arXiv cs.LG 2020
Year: 2020

Document network embedding aims at learning representations for a structured text corpus i.e. when documents are linked to each other. Recent algorithms extend network embedding approaches by incorporating the text content associated with the nodes in their formulations. In most cases, it is hard to interpret the learned representations. Moreover, little importance is given to the generalization to new documents that are not observed within the network. In this paper, we propose an interpretable and inductive document network embedding method. We introduce a novel mechanism, the Topic-Word Attention (TWA), that generates document representations based on the interplay between word and topic representations. We train these word and topic vectors through our general model, Inductive Document Network Embedding (IDNE), by leveraging the connections in the document network. Quantitative evaluations show that our approach achieves state-of-the-art performance on various networks and we qualitatively show that our model produces meaningful and interpretable representations of the words, topics and documents.
Superbubbles as an Empirical Characteristic of Directed Networks
Venue: arXiv cs.SI 2020
Year: 2020

Superbubbles are acyclic induced subgraphs of a digraph with single entrance and exit that naturally arise in the context of genome assembly and the analysis of genome alignments in computational biology. These structures can be computed in linear time and are confined to non-symmetric digraphs. We demonstrate empirically that graph parameters derived from superbubbles provide a convenient means of distinguishing different classes of real-world graphical models, while being largely unrelated to simple, commonly used parameters.
A stable SPH with adaptive B-spline kernel
Venue: arXiv cs.CE 2020
Year: 2020

Tensile instability, often observed in smoothed particle hydrodynamics (SPH), is a numerical artifact that manifests itself by unphysical clustering or separation of particles. The instability originates in estimating the derivatives of the smoothing functions which, when interact with material constitution may result in negative stiffness in the discretized system. In the present study, a stable formulation of SPH is developed where the kernel function is continuously adapted at every material point depending on its state of stress. Bspline basis function with a variable intermediate knot is used as the kernel function. The shape of the kernel function is then modified by changing the intermediate knot position such that the condition associated with instability does not arise. While implementing the algorithm the simplicity and computational efficiency of SPH are not compromised. One-dimensional dispersion analysis is performed to understand the effect adaptive kernel on the stability. Finally, the efficacy of the algorithm is demonstrated through some benchmark elastic dynamics problems.
QSOR: Quantum-Safe Onion Routing
Venue: arXiv cs.CR 2020
Year: 2020

In this work, we propose a study on the use of post-quantum cryptographic primitives for the Tor network in order to make it safe in a quantum world. With this aim, the underlying keying material has first been analysed. We observe that breaking the security of the algorithms/protocols that use long- and medium-term keys (usually RSA keys) have the highest impact in security. Therefore, we investigate the cost of quantum-safe variants. These include key generation, key encapsulation and decapsulation. Six different post-quantum cryptographic algorithms that ensure level 1 NIST security are evaluated. We further target the Tor circuit creation operation and evaluate the overhead of the post-quantum variant. This comparative study is performed through a reference implementation based on SweetOnions that simulates Tor with slight simplifications. We show that a quantum-safe Tor circuit creation is possible and suggest two versions - one that can be used in a purely quantum-safe setting, and one that can be used in a hybrid setting.
Optimizing the Efficiency of Accelerated Reliability Testing for the Internet Router Motherboard
Venue: arXiv cs.SE 2020
Year: 2020

With the rapid development of internet Router, the complexity of its mainboard has been growing dramatically. The high reliability requirement renders the number of testing cases increasing exponentially, which becomes the bottleneck that prevents further elevation of the production efficiency. In this work, we develop a novel optimization method of two major steps to largely reduce the testing time and increase the testing efficiency. In the first step, it optimizes the selection of test cases given the required amount of testing time reduction while ensuring the coverage of failures. In the second step, selected test cases are optimally scheduled to maximize the efficiency of mainboard testing. A numerical experiment is investigated to illustrate the effectiveness of the proposed methods. The results show that the optimal subset of the test cases can be selected satisfying the 10% testing time reduction requirement, and the effectiveness index of the scheduled sequence can be improved by more than 75% with test case sequencing. Moreover, our method can self-adjust to the new failure data, which realizes the automatic configuration of board test cases.
Explaining the Explainer: A First Theoretical Analysis of LIME
Venue: arXiv cs.LG 2020
Year: 2020

Machine learning is used more and more often for sensitive applications, sometimes replacing humans in critical decision-making processes. As such, interpretability of these algorithms is a pressing need. One popular algorithm to provide interpretability is LIME (Local Interpretable Model-Agnostic Explanation). In this paper, we provide the first theoretical analysis of LIME. We derive closed-form expressions for the coefficients of the interpretable model when the function to explain is linear. The good news is that these coefficients are proportional to the gradient of the function to explain: LIME indeed discovers meaningful features. However, our analysis also reveals that poor choices of parameters can lead LIME to miss important features.
SVIRO: Synthetic Vehicle Interior Rear Seat Occupancy Dataset and Benchmark
Venue: arXiv cs.CV 2020
Year: 2020

We release SVIRO, a synthetic dataset for sceneries in the passenger compartment of ten different vehicles, in order to analyze machine learning-based approaches for their generalization capacities and reliability when trained on a limited number of variations (e.g. identical backgrounds and textures, few instances per class). This is in contrast to the intrinsically high variability of common benchmark datasets, which focus on improving the state-of-the-art of general tasks. Our dataset contains bounding boxes for object detection, instance segmentation masks, keypoints for pose estimation and depth images for each synthetic scenery as well as images for each individual seat for classification. The advantage of our use-case is twofold: The proximity to a realistic application to benchmark new approaches under novel circumstances while reducing the complexity to a more tractable environment, such that applications and theoretical questions can be tested on a more challenging dataset as toy problems. The data and evaluation server are available under https://sviro.kl.dfki.de.
Autoencoding Undirected Molecular Graphs With Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Discrete structure rules for validating molecular structures are usually limited to fulfillment of the octet rule or similar simple deterministic heuristics. We propose a model, inspired by language modeling from natural language processing, with the ability to learn from a collection of undirected molecular graphs, enabling fitting of any underlying structure rule present in the collection. We introduce an adaption to the popular Transformer model, which can learn relationships between atoms and bonds. To our knowledge, the Transformer adaption is the first model that is trained to solve the unsupervised task of recovering partially observed molecules. In this work, we assess how different degrees of information impact performance w.r.t. to fitting the QM9 dataset, which conforms to the octet rule, and to fitting the ZINC dataset, which contains hypervalent molecules and ions requiring the model to learn a more complex structure rule. More specifically, we test a full discrete graph with bond order information, a full discrete graph with only connectivity, a bag-of-neighbors, a bag-of-atoms, and a count-based unigram statistics. These results provide encouraging evidence that neural networks, even when only connectivity is available, can learn arbitrary molecular structure rules specific to a dataset, as the Transformer adaption surpasses a strong octet rule baseline on the ZINC dataset.
On the computational content of Zorn's lemma
Venue: arXiv cs.LO 2020
Year: 2020

We give a computational interpretation to an abstract instance of Zorn's lemma formulated as a wellfoundedness principle in the language of arithmetic in all finite types. This is achieved through Gödel's functional interpretation, and requires the introduction of a novel form of recursion over non-wellfounded partial orders whose existence in the model of total continuous functionals is proven using domain theoretic techniques. We show that a realizer for the functional interpretation of open induction over the lexicographic ordering on sequences follows as a simple application of our main results.
A Unified Conversational Assistant Framework for Business Process Automation
Venue: arXiv cs.AI 2020
Year: 2020

Business process automation is a booming multi-billion-dollar industry that promises to remove menial tasks from workers' plates -- through the introduction of autonomous agents -- and free up their time and brain power for more creative and engaging tasks. However, an essential component to the successful deployment of such autonomous agents is the ability of business users to monitor their performance and customize their execution. A simple and user-friendly interface with a low learning curve is necessary to increase the adoption of such agents in banking, insurance, retail and other domains. As a result, proactive chatbots will play a crucial role in the business automation space. Not only can they respond to users' queries and perform actions on their behalf but also initiate communication with the users to inform them of the system's behavior. This will provide business users a natural language interface to interact with, monitor and control autonomous agents. In this work, we present a multi-agent orchestration framework to develop such proactive chatbots by discussing the types of skills that can be composed into agents and how to orchestrate these agents. Two use cases on a travel preapproval business process and a loan application business process are adopted to qualitatively analyze the proposed framework based on four criteria: performance, coding overhead, scalability, and agent overlap.
Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks
Venue: arXiv cs.CL 2020
Year: 2020

Learners that are exposed to the same training data might generalize differently due to differing inductive biases. In neural network models, inductive biases could in theory arise from any aspect of the model architecture. We investigate which architectural factors affect the generalization behavior of neural sequence-to-sequence models trained on two syntactic tasks, English question formation and English tense reinflection. For both tasks, the training set is consistent with a generalization based on hierarchical structure and a generalization based on linear order. All architectural factors that we investigated qualitatively affected how models generalized, including factors with no clear connection to hierarchical structure. For example, LSTMs and GRUs displayed qualitatively different inductive biases. However, the only factor that consistently contributed a hierarchical bias across tasks was the use of a tree-structured model rather than a model with sequential recurrence, suggesting that human-like syntactic generalization requires architectural syntactic structure.
Unsupervised multi-modal Styled Content Generation
Venue: arXiv cs.GR 2020
Year: 2020

The emergence of deep generative models has recently enabled the automatic generation of massive amounts of graphical content, both in 2D and in 3D. Generative Adversarial Networks (GANs) and style control mechanisms, such as Adaptive Instance Normalization (AdaIN), have proved particularly effective in this context, culminating in the state-of-the-art StyleGAN architecture. While such models are able to learn diverse distributions, provided a sufficiently large training set, they are not well-suited for scenarios where the distribution of the training data exhibits a multi-modal behavior. In such cases, reshaping a uniform or normal distribution over the latent space into a complex multi-modal distribution in the data domain is challenging, and the generator might fail to sample the target distribution well. Furthermore, existing unsupervised generative models are not able to control the mode of the generated samples independently of the other visual attributes, despite the fact that they are typically disentangled in the training data. In this paper, we introduce UMMGAN, a novel architecture designed to better model multi-modal distributions, in an unsupervised fashion. Building upon the StyleGAN architecture, our network learns multiple modes, in a completely unsupervised manner, and combines them using a set of learned weights. We demonstrate that this approach is capable of effectively approximating a complex distribution as a superposition of multiple simple ones. We further show that UMMGAN effectively disentangles between modes and style, thereby providing an independent degree of control over the generated content.