OpenRaiser/Intern-Atlas huggingface dataset:
Analyzing the Structure of Mondrian's 1920-1940 Compositions
Venue: arXiv cs.CY 2020
Year: 2020

Mondrian was one of the most significant painters of the 20th century. He was a prominent member of DeStijl, the movement which revolutionized art by setting it free of the obligation to make images of existing objects, persons, or situations. DeStijl was one of the interrelated movements in early 20th century Europe including Cubism, Constructivism, the Futurists, and Dada. Their disruptive ideas changed the meaning of Western art. It was Mondrian, more than anyone else, who worked restlessly to find expression for the purest possible kind of beauty and truth, based on a theory called Neoplasticism. He was already famous during his lifetime and still now, his name is almost synonym for modern art. We analyze the structure of the system of black lines in his paintings and put the hypothesis to the test that the paintings could be obtained by recursive (binary) splitting. We used a novel tailor-made interactive analysis tool and apply it to as many Mondrian paintings as possible (in total 147). The results will be explained in a visual manner, but we also present statistical findings from the analysis of the 147 paintings. Our main conclusion is that the hypothesis of splitting decomposition is in general not true. It is possible to make Mondrian-like compositions by splitting, yet one misses out on a great deal of Neoplastic beauty if one would work by splitting only. It is possible to consider all crossings as pairs of Tees, but that is clumsy, and it leaves out essential information. Moreover there are other important design decisions of Mondrian, such as the keeping-distance to the canvas-edge which are not well-described by splitting.
An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution
Venue: arXiv cs.CL 2020
Year: 2020

One critical issue of zero anaphora resolution (ZAR) is the scarcity of labeled data. This study explores how effectively this problem can be alleviated by data augmentation. We adopt a state-of-the-art data augmentation method, called the contextual data augmentation (CDA), that generates labeled training instances using a pretrained language model. The CDA has been reported to work well for several other natural language processing tasks, including text classification and machine translation. This study addresses two underexplored issues on CDA, that is, how to reduce the computational cost of data augmentation and how to ensure the quality of the generated data. We also propose two methods to adapt CDA to ZAR: [MASK]-based augmentation and linguistically-controlled masking. Consequently, the experimental results on Japanese ZAR show that our methods contribute to both the accuracy gain and the computation cost reduction. Our closer analysis reveals that the proposed method can improve the quality of the augmented training data when compared to the conventional CDA.
A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English
Venue: arXiv cs.CL 2020
Year: 2020

Transformer-based language models achieve high performance on various tasks, but we still lack understanding of the kind of linguistic knowledge they learn and rely on. We evaluate three models (BERT, RoBERTa, and ALBERT), testing their grammatical and semantic knowledge by sentence-level probing, diagnostic cases, and masked prediction tasks. We focus on relative clauses (in American English) as a complex phenomenon needing contextual information and antecedent identification to be resolved. Based on a naturalistic dataset, probing shows that all three models indeed capture linguistic knowledge about grammaticality, achieving high performance. Evaluation on diagnostic cases and masked prediction tasks considering fine-grained linguistic knowledge, however, shows pronounced model-specific weaknesses especially on semantic knowledge, strongly impacting models' performance. Our results highlight the importance of (a)model comparison in evaluation task and (b) building up claims of model performance and the linguistic knowledge they capture beyond purely probing-based evaluations.
Using Vision Videos in a Virtual Focus Group: Experiences and Recommendations
Venue: arXiv cs.SE 2020
Year: 2020

Facilitated meetings are an established practice for the requirements engineering activities elicitation and validation. Focus groups are one well-known technique to implement this practice. Several researchers already reported the successful use of vision videos to stimulate active discussions among the participants of on-site focus groups, e.g., for validating scenarios and eliciting feedback. These vision videos show scenarios of a system vision. In this way, the videos serve all parties involved as a visual reference point to actively disclose, discuss, and align their mental models of the future system to achieve shared understanding. In the joint project TrUSD, we had planned to conduct such an on-site focus group using a vision video to validate a scenario of a future software tool, the so-called Privacy Dashboard. However, the COVID-19 pandemic and its associated measures led to an increase in home and remote working, which also affected us. Therefore, we had to replan and conduct the focus group virtually. In this paper, we report about our experiences and recommendations for the use of vision videos in virtual focus groups.
DNN-Based Semantic Model for Rescoring N-best Speech Recognition List
Venue: arXiv cs.CL 2020
Year: 2020

The word error rate (WER) of an automatic speech recognition (ASR) system increases when a mismatch occurs between the training and the testing conditions due to the noise, etc. In this case, the acoustic information can be less reliable. This work aims to improve ASR by modeling long-term semantic relations to compensate for distorted acoustic features. We propose to perform this through rescoring of the ASR N-best hypotheses list. To achieve this, we train a deep neural network (DNN). Our DNN rescoring model is aimed at selecting hypotheses that have better semantic consistency and therefore lower WER. We investigate two types of representations as part of input features to our DNN model: static word embeddings (from word2vec) and dynamic contextual embeddings (from BERT). Acoustic and linguistic features are also included. We perform experiments on the publicly available dataset TED-LIUM mixed with real noise. The proposed rescoring approaches give significant improvement of the WER over the ASR system without rescoring models in two noisy conditions and with n-gram and RNNLM.
Semi-regular sequences and other random systems of equations
Venue: arXiv cs.CR 2020
Year: 2020

The security of multivariate cryptosystems and digital signature schemes relies on the hardness of solving a system of polynomial equations over a finite field. Polynomial system solving is also currently a bottleneck of index-calculus algorithms to solve the elliptic and hyperelliptic curve discrete logarithm problem. The complexity of solving a system of polynomial equations is closely related to the cost of computing Groebner bases, since computing the solutions of a polynomial system can be reduced to finding a lexicographic Groebner basis for the ideal generated by the equations. Several algorithms for computing such bases exist: We consider those based on repeated Gaussian elimination of Macaulay matrices. In this paper, we analyze the case of random systems, where random systems means either semi-regular systems, or quadratic systems in n variables which contain a regular sequence of n polynomials. We provide explicit formulae for bounds on the solving degree of semi-regular systems with m > n equations in n variables, for equations of arbitrary degrees for m = n+1, and for any m for systems of quadratic or cubic polynomials. In the appendix, we provide a table of bounds for the solving degree of semi-regular systems of m = n + k quadratic equations in n variables for 2 <= k; n <= 100 and online we provide the values of the bounds for 2 <= k; n <= 500. For quadratic systems which contain a regular sequence of n polynomials, we argue that the Eisenbud-Green-Harris Conjecture, if true, provides a sharp bound for their solving degree, which we compute explicitly.
Evaluation of Siamese Networks for Semantic Code Search
Venue: arXiv cs.SE 2020
Year: 2020

With the increase in the number of open repositories and discussion forums, the use of natural language for semantic code search has become increasingly common. The accuracy of the results returned by such systems, however, can be low due to 1) limited shared vocabulary between code and user query and 2) inadequate semantic understanding of user query and its relation to code syntax. Siamese networks are well suited to learning such joint relations between data, but have not been explored in the context of code search. In this work, we evaluate Siamese networks for this task by exploring multiple extraction network architectures. These networks independently process code and text descriptions before passing them to a Siamese network to learn embeddings in a common space. We experiment on two different datasets and discover that Siamese networks can act as strong regularizers on networks that extract rich information from code and text, which in turn helps achieve impressive performance on code search beating previous baselines on $2$ programming languages. We also analyze the embedding space of these networks and provide directions to fully leverage the power of Siamese networks for semantic code search.
SIMDive: Approximate SIMD Soft Multiplier-Divider for FPGAs with Tunable Accuracy
Venue: arXiv cs.AR 2020
Year: 2020

The ever-increasing quest for data-level parallelism and variable precision in ubiquitous multimedia and Deep Neural Network (DNN) applications has motivated the use of Single Instruction, Multiple Data (SIMD) architectures. To alleviate energy as their main resource constraint, approximate computing has re-emerged,albeit mainly specialized for their Application-Specific Integrated Circuit (ASIC) implementations. This paper, presents for the first time, an SIMD architecture based on novel multiplier and divider with tunable accuracy, targeted for Field-Programmable Gate Arrays (FPGAs). The proposed hybrid architecture implements Mitchell's algorithms and supports precision variability from 8 to 32 bits. Experimental results obtained from Vivado, multimedia and DNN applications indicate superiority of proposed architecture (both SISD and SIMD) over accurate and state-of-the-art approximate counterparts. In particular, the proposed SISD divider outperforms the accurate Intellectual Property (IP) divider provided by Xilinx with 4x higher speed and 4.6x less energy and tolerating only < 0.8% error. Moreover, the proposed SIMD multiplier-divider supersede accurate SIMD multiplier by achieving up to 26%, 45%, 36%, and 56% improvement in area, throughput, power, and energy, respectively.
Data-Driven Adaptive Task Allocation for Heterogeneous Multi-Robot Teams Using Robust Control Barrier Functions
Venue: arXiv cs.RO 2020
Year: 2020

Multi-robot task allocation is a ubiquitous problem in robotics due to its applicability in a variety of scenarios. Adaptive task-allocation algorithms account for unknown disturbances and unpredicted phenomena in the environment where robots are deployed to execute tasks. However, this adaptivity typically comes at the cost of requiring precise knowledge of robot models in order to evaluate the allocation effectiveness and to adjust the task assignment online. As such, environmental disturbances can significantly degrade the accuracy of the models which in turn negatively affects the quality of the task allocation. In this paper, we leverage Gaussian processes, differential inclusions, and robust control barrier functions to learn environmental disturbances in order to guarantee robust task execution. We show the implementation and the effectiveness of the proposed framework on a real multi-robot system.
RandomForestMLP: An Ensemble-Based Multi-Layer Perceptron Against Curse of Dimensionality
Venue: arXiv cs.LG 2020
Year: 2020

We present a novel and practical deep learning pipeline termed RandomForestMLP. This core trainable classification engine consists of a convolutional neural network backbone followed by an ensemble-based multi-layer perceptrons core for the classification task. It is designed in the context of self and semi-supervised learning tasks to avoid overfitting while training on very small datasets. The paper details the architecture of the RandomForestMLP and present different strategies for neural network decision aggregation. Then, it assesses its robustness to overfitting when trained on realistic image datasets and compares its classification performance with existing regular classifiers.
A Lane-Changing Prediction Method Based on Temporal Convolution Network
Venue: arXiv cs.LG 2020
Year: 2020

Lane-changing is an important driving behavior and unreasonable lane changes can result in potentially dangerous traffic collisions. Advanced Driver Assistance System (ADAS) can assist drivers to change lanes safely and efficiently. To capture the stochastic time series of lane-changing behavior, this study proposes a temporal convolutional network (TCN) to predict the long-term lane-changing trajectory and behavior. In addition, the convolutional neural network (CNN) and recurrent neural network (RNN) methods are considered as the benchmark models to demonstrate the learning ability of the TCN. The lane-changing dataset was collected by the driving simulator. The prediction performance of TCN is demonstrated from three aspects: different input variables, different input dimensions and different driving scenarios. Prediction results show that the TCN can accurately predict the long-term lane-changing trajectory and driving behavior with shorter computational time compared with two benchmark models. The TCN can provide accurate lane-changing prediction, which is one key information for the development of accurate ADAS.
Useful Policy Invariant Shaping from Arbitrary Advice
Venue: arXiv cs.LG 2020
Year: 2020

Reinforcement learning is a powerful learning paradigm in which agents can learn to maximize sparse and delayed reward signals. Although RL has had many impressive successes in complex domains, learning can take hours, days, or even years of training data. A major challenge of contemporary RL research is to discover how to learn with less data. Previous work has shown that domain information can be successfully used to shape the reward; by adding additional reward information, the agent can learn with much less data. Furthermore, if the reward is constructed from a potential function, the optimal policy is guaranteed to be unaltered. While such potential-based reward shaping (PBRS) holds promise, it is limited by the need for a well-defined potential function. Ideally, we would like to be able to take arbitrary advice from a human or other agent and improve performance without affecting the optimal policy. The recently introduced dynamic potential based advice (DPBA) method tackles this challenge by admitting arbitrary advice from a human or other agent and improves performance without affecting the optimal policy. The main contribution of this paper is to expose, theoretically and empirically, a flaw in DPBA. Alternatively, to achieve the ideal goals, we present a simple method called policy invariant explicit shaping (PIES) and show theoretically and empirically that PIES succeeds where DPBA fails.
The Mathematical Foundations of Manifold Learning
Venue: arXiv cs.LG 2020
Year: 2020

Manifold learning is a popular and quickly-growing subfield of machine learning based on the assumption that one's observed data lie on a low-dimensional manifold embedded in a higher-dimensional space. This thesis presents a mathematical perspective on manifold learning, delving into the intersection of kernel learning, spectral graph theory, and differential geometry. Emphasis is placed on the remarkable interplay between graphs and manifolds, which forms the foundation for the widely-used technique of manifold regularization. This work is written to be accessible to a broad mathematical audience, including machine learning researchers and practitioners interested in understanding the theorems underlying popular manifold learning algorithms and dimensionality reduction techniques.
Blocking Adversarial Influence in Social Networks
Venue: arXiv cs.SI 2020
Year: 2020

While social networks are widely used as a media for information diffusion, attackers can also strategically employ analytical tools, such as influence maximization, to maximize the spread of adversarial content through the networks. We investigate the problem of limiting the diffusion of negative information by blocking nodes and edges in the network. We formulate the interaction between the defender and the attacker as a Stackelberg game where the defender first chooses a set of nodes to block and then the attacker selects a set of seeds to spread negative information from. This yields an extremely complex bi-level optimization problem, particularly since even the standard influence measures are difficult to compute. Our approach is to approximate the attacker's problem as the maximum node domination problem. To solve this problem, we first develop a method based on integer programming combined with constraint generation. Next, to improve scalability, we develop an approximate solution method that represents the attacker's problem as an integer program, and then combines relaxation with duality to yield an upper bound on the defender's objective that can be computed using mixed integer linear programming. Finally, we propose an even more scalable heuristic method that prunes nodes from the consideration set based on their degree. Extensive experiments demonstrate the efficacy of our approaches.
Error- and Tamper-Tolerant State Estimation for Discrete Event Systems under Cost Constraints
Venue: arXiv cs.IT 2020
Year: 2020

This paper deals with the state estimation problem in discrete-event systems modeled with nondeterministic finite automata, partially observed via a sensor measuring unit whose measurements (reported observations) may be vitiated by a malicious attacker. The attacks considered in this paper include arbitrary deletions, insertions, or substitutions of observed symbols by taking into account a bounded number of attacks or, more generally, a total cost constraint (assuming that each deletion, insertion, or substitution bears a positive cost to the attacker). An efficient approach is proposed to describe possible sequences of observations that match the one received by the measuring unit, as well as their corresponding state estimates and associated total costs. We develop an algorithm to obtain the least-cost matching sequences by reconstructing only a finite number of possible sequences, which we subsequently use to efficiently perform state estimation. We also develop a technique for verifying tamper-tolerant diagnosability under attacks that involve a bounded number of deletions, insertions, and substitutions (or, more generally, under attacks of bounded total cost) by using a novel structure obtained by attaching attacks and costs to the original plant. The overall construction and verification procedure have complexity that is of O(|X|^2 C^2),where |X| is the number of states of the given finite automaton and C is the maximum total cost that is allowed for all the deletions, insertions, and substitutions. We determine the minimum value of C such that the attacker can coordinate its tampering action to keep the observer indefinitely confused while utilizing a finite number of attacks. Several examples are presented to demonstrate the proposed methods.
Parameter Efficient Deep Neural Networks with Bilinear Projections
Venue: arXiv cs.CV 2020
Year: 2020

Recent research on deep neural networks (DNNs) has primarily focused on improving the model accuracy. Given a proper deep learning framework, it is generally possible to increase the depth or layer width to achieve a higher level of accuracy. However, the huge number of model parameters imposes more computational and memory usage overhead and leads to the parameter redundancy. In this paper, we address the parameter redundancy problem in DNNs by replacing conventional full projections with bilinear projections. For a fully-connected layer with $D$ input nodes and $D$ output nodes, applying bilinear projection can reduce the model space complexity from $\mathcal{O}(D^2)$ to $\mathcal{O}(2D)$, achieving a deep model with a sub-linear layer size. However, structured projection has a lower freedom of degree compared to the full projection, causing the under-fitting problem. So we simply scale up the mapping size by increasing the number of output channels, which can keep and even boosts the model accuracy. This makes it very parameter-efficient and handy to deploy such deep models on mobile systems with memory limitations. Experiments on four benchmark datasets show that applying the proposed bilinear projection to deep neural networks can achieve even higher accuracies than conventional full DNNs, while significantly reduces the model size.
GAIN: Graph Attention & Interaction Network for Inductive Semi-Supervised Learning over Large-scale Graphs
Venue: arXiv cs.LG 2020
Year: 2020

Graph Neural Networks (GNNs) have led to state-of-the-art performance on a variety of machine learning tasks such as recommendation, node classification and link prediction. Graph neural network models generate node embeddings by merging nodes features with the aggregated neighboring nodes information. Most existing GNN models exploit a single type of aggregator (e.g., mean-pooling) to aggregate neighboring nodes information, and then add or concatenate the output of aggregator to the current representation vector of the center node. However, using only a single type of aggregator is difficult to capture the different aspects of neighboring information and the simple addition or concatenation update methods limit the expressive capability of GNNs. Not only that, existing supervised or semi-supervised GNN models are trained based on the loss function of the node label, which leads to the neglect of graph structure information. In this paper, we propose a novel graph neural network architecture, Graph Attention \& Interaction Network (GAIN), for inductive learning on graphs. Unlike the previous GNN models that only utilize a single type of aggregation method, we use multiple types of aggregators to gather neighboring information in different aspects and integrate the outputs of these aggregators through the aggregator-level attention mechanism. Furthermore, we design a graph regularized loss to better capture the topological relationship of the nodes in the graph. Additionally, we first present the concept of graph feature interaction and propose a vector-wise explicit feature interaction mechanism to update the node embeddings. We conduct comprehensive experiments on two node-classification benchmarks and a real-world financial news dataset. The experiments demonstrate our GAIN model outperforms current state-of-the-art performances on all the tasks.