OpenRaiser/Intern-Atlas huggingface dataset:
The Regression Tsetlin Machine: A Tsetlin Machine for Continuous Output Problems
Venue: arXiv cs.LG 2019
Year: 2019

The recently introduced Tsetlin Machine (TM) has provided competitive pattern classification accuracy in several benchmarks, composing patterns with easy-to-interpret conjunctive clauses in propositional logic. In this paper, we go beyond pattern classification by introducing a new type of TMs, namely, the Regression Tsetlin Machine (RTM). In all brevity, we modify the inner inference mechanism of the TM so that input patterns are transformed into a single continuous output, rather than to distinct categories. We achieve this by: (1) using the conjunctive clauses of the TM to capture arbitrarily complex patterns; (2) mapping these patterns to a continuous output through a novel voting and normalization mechanism; and (3) employing a feedback scheme that updates the TM clauses to minimize the regression error. The feedback scheme uses a new activation probability function that stabilizes the updating of clauses, while the overall system converges towards an accurate input-output mapping. The performance of the RTM is evaluated using six different artificial datasets with and without noise, in comparison with the Classic Tsetlin Machine (CTM) and the Multiclass Tsetlin Machine (MTM). Our empirical results indicate that the RTM obtains the best training and testing results for both noisy and noise-free datasets, with a smaller number of clauses. This, in turn, translates to higher regression accuracy, using significantly less computational resources.
Variable elimination in binary CSPs
Venue: arXiv cs.DS 2019
Year: 2019

We investigate rules which allow variable elimination in binary CSP (constraint satisfaction problem) instances while conserving satisfiability. We study variable-elimination rules based on the language of forbidden patterns enriched with counting and quantification over variables and values. We propose new rules and compare them, both theoretically and experimentally. We give optimised algorithms to apply these rules and show that each define a novel tractable class. Using our variable-elimination rules in preprocessing allowed us to solve more benchmark problems than without.
Enabling Practical Processing in and near Memory for Data-Intensive Computing
Venue: arXiv cs.DC 2019
Year: 2019

Modern computing systems suffer from the dichotomy between computation on one side, which is performed only in the processor (and accelerators), and data storage/movement on the other, which all other parts of the system are dedicated to. Due to this dichotomy, data moves a lot in order for the system to perform computation on it. Unfortunately, data movement is extremely expensive in terms of energy and latency, much more so than computation. As a result, a large fraction of system energy is spent and performance is lost solely on moving data in a modern computing system. In this work, we re-examine the idea of reducing data movement by performing Processing in Memory (PIM). PIM places computation mechanisms in or near where the data is stored (i.e., inside the memory chips, in the logic layer of 3D-stacked logic and DRAM, or in the memory controllers), so that data movement between the computation units and memory is reduced or eliminated. While the idea of PIM is not new, we examine two new approaches to enabling PIM: 1) exploiting analog properties of DRAM to perform massively-parallel operations in memory, and 2) exploiting 3D-stacked memory technology design to provide high bandwidth to in-memory logic. We conclude by discussing work on solving key challenges to the practical adoption of PIM.
A New Encounter Between Leader-Follower Tracking and Observer-Based Control: Towards Enhancing Robustness against Disturbances
Venue: arXiv cs.SY 2019
Year: 2019

This paper studies robust tracking control for a leader-follower multi-agent system (MAS) subject to disturbances. A challenging problem is considered here, which differs from those in the literature in two aspects. First, we consider the case when all the leader and follower agents are affected by disturbances, while the existing studies assume only the followers to suffer disturbances. Second, we assume the disturbances to be bounded only in rates of change rather than magnitude as in the literature. To address this new problem, we propose a novel observer-based distributed tracking control design. As a distinguishing feature, the followers can cooperatively estimate the disturbance affecting the leader to adjust their maneuvers accordingly, which is enabled by the design of the first-of-its-kind distributed disturbance observers. We build specific tracking control approaches for both first- and second-order MASs and prove that they can lead to bounded-error tracking, despite the challenges due to the relaxed assumptions about disturbances. We further perform simulation to validate the proposed approaches.
Decentralized Trusted Computing Base for Blockchain Infrastructure Security
Venue: arXiv cs.CR 2019
Year: 2019

There is a growing interest today in blockchain technology as a possible foundation for the future global financial ecosystem. However, in order for this future financial ecosystem to be truly global, with a high degree of interoperability and stability, a number challenges need to be addressed related to infrastructure security. One key aspect concerns the security and robustness of the systems that participate in the blockchain peer-to-peer networks. In this paper we discuss the notion of the decentralized trusted computing base as an extension of the TCB concept in trusted computing. We explore how a decentralized TCB can be useful to (i) harden individual nodes and systems in the blockchain infrastructure, and (ii) be the basis for secure group-oriented computations making within the P2P network of nodes that make-up the blockchain system.
RDMA Performance Isolation With Justitia
Venue: arXiv cs.NI 2019
Year: 2019

Despite its increasing popularity, most of RDMA's benefits such as ultra-low latency can be achieved only when running an application in isolation. Using microbenchmarks and real open-source RDMA applications, we identify a series of performance anomalies when multiple applications coexist and show that such anomalies are pervasive across InfiniBand, RoCEv2, and iWARP. They arise due to a fundamental tradeoff between performance isolation and work conservation, which the state-of-the-art RDMA congestion control protocols such as DCQCN cannot resolve. We present Justitia to address these performance anomalies. Justitia is a software-only, host-based, and easy-to-deploy solution that maximizes RNIC utilization while guaranteeing performance isolation via shaping, rate limiting, and pacing at senders. Our evaluation of Justitia on multiple RDMA implementations show that Justitia effectively isolates different types of traffic and significantly improves latency (by up to 56.9x) and throughput (by up to 9.7x) of real-world RDMA-based applications without compromising low CPU usage or modifying the applications.
Sparse Optimization Problem with s-difference Regularization
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, a s-difference type regularization for sparse recovery problem is proposed, which is the difference of the normal penalty function R(x) and its corresponding struncated function R (xs). First, we show the equivalent conditions between the L0 constrained problem and the unconstrained s-difference penalty regularized problem. Next, we choose the forward-backward splitting (FBS) method to solve the nonconvex regularizes function and further derive some closed-form solutions for the proximal mapping of the s-difference regularization with some commonly used R(x), which makes the FBS easy and fast. We also show that any cluster point of the sequence generated by the proposed algorithm converges to a stationary point. Numerical experiments demonstrate the efficiency of the proposed s-difference regularization in comparison with some other existing penalty functions.
Unified Generator-Classifier for Efficient Zero-Shot Learning
Venue: arXiv cs.CV 2019
Year: 2019

Generative models have achieved state-of-the-art performance for the zero-shot learning problem, but they require re-training the classifier every time a new object category is encountered. The traditional semantic embedding approaches, though very elegant, usually do not perform at par with their generative counterparts. In this work, we propose an unified framework termed GenClass, which integrates the generator with the classifier for efficient zero-shot learning, thus combining the representative power of the generative approaches and the elegance of the embedding approaches. End-to-end training of the unified framework not only eliminates the requirement of additional classifier for new object categories as in the generative approaches, but also facilitates the generation of more discriminative and useful features. Extensive evaluation on three standard zero-shot object classification datasets, namely AWA, CUB and SUN shows the effectiveness of the proposed approach. The approach without any modification, also gives state-of-the-art performance for zero-shot action classification, thus showing its generalizability to other domains.
Multitask Deep Learning with Spectral Knowledge for Hyperspectral Image Classification
Venue: arXiv cs.CV 2019
Year: 2019

In this letter, we propose a multitask deep learning method for classification of multiple hyperspectral data in a single training. Deep learning models have achieved promising results on hyperspectral image classification, but their performance highly rely on sufficient labeled samples, which are scarce on hyperspectral images. However, samples from multiple data sets might be sufficient to train one deep learning model, thereby improving its performance. To do so, we trained an identical feature extractor for all data, and the extracted features were fed into corresponding Softmax classifiers. Spectral knowledge was introduced to ensure that the shared features were similar across domains. Four hyperspectral data sets were used in the experiments. We achieved higher classification accuracies on three data sets (Pavia University, Pavia Center, and Indian Pines) and competitive results on the Salinas Valley data compared with the baseline. Spectral knowledge was useful to prevent the deep network from overfitting when the data shared similar spectral response. The proposed method tested on two deep CNNs successfully shows its ability to utilize samples from multiple data sets and enhance networks' performance.
Semantic categories of artifacts and animals reflect efficient coding
Venue: arXiv cs.CL 2019
Year: 2019

It has been argued that semantic categories across languages reflect pressure for efficient communication. Recently, this idea has been cast in terms of a general information-theoretic principle of efficiency, the Information Bottleneck (IB) principle, and it has been shown that this principle accounts for the emergence and evolution of named color categories across languages, including soft structure and patterns of inconsistent naming. However, it is not yet clear to what extent this account generalizes to semantic domains other than color. Here we show that it generalizes to two qualitatively different semantic domains: names for containers, and for animals. First, we show that container naming in Dutch and French is near-optimal in the IB sense, and that IB broadly accounts for soft categories and inconsistent naming patterns in both languages. Second, we show that a hierarchy of animal categories derived from IB captures cross-linguistic tendencies in the growth of animal taxonomies. Taken together, these findings suggest that fundamental information-theoretic principles of efficient coding may shape semantic categories across languages and across domains.
Information search in a professional context - exploring a collection of professional search tasks
Venue: arXiv cs.IR 2019
Year: 2019

Search conducted in a work context is an everyday activity that has been around since long before the Web was invented, yet we still seem to understand little about its general characteristics. With this paper we aim to contribute to a better understanding of this large but rather multi-faceted area of `professional search'. Unlike task-based studies that aim at measuring the effectiveness of search methods, we chose to take a step back by conducting a survey among professional searchers to understand their typical search tasks. By doing so we offer complementary insights into the subject area. We asked our respondents to provide actual search tasks they have worked on, information about how these were conducted and details on how successful they eventually were. We then manually coded the collection of 56 search tasks with task characteristics and relevance criteria, and used the coded dataset for exploration purposes. Despite the relatively small scale of this study, our data provides enough evidence that professional search is indeed very different from Web search in many key respects and that this is a field that offers many avenues for future research.
Explainable AI for Trees: From Local Explanations to Global Understanding
Venue: arXiv cs.LG 2019
Year: 2019

Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are the most popular non-linear predictive models used in practice today, yet comparatively little attention has been paid to explaining their predictions. Here we significantly improve the interpretability of tree-based models through three main contributions: 1) The first polynomial time algorithm to compute optimal explanations based on game theory. 2) A new type of explanation that directly measures local feature interaction effects. 3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high-quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These tools enable us to i) identify high magnitude but low frequency non-linear mortality risk factors in the general US population, ii) highlight distinct population sub-groups with shared risk characteristics, iii) identify non-linear interaction effects among risk factors for chronic kidney disease, and iv) monitor a machine learning model deployed in a hospital by identifying which features are degrading the model's performance over time. Given the popularity of tree-based machine learning models, these improvements to their interpretability have implications across a broad set of domains.
The Language of Biometrics: Analysing Public Perceptions
Venue: arXiv cs.CR 2019
Year: 2019

There is an increasing shift in technology towards biometric solutions, but one of the biggest barriers to widespread use is the acceptance by the users. In this paper we investigate the understanding, awareness and acceptance of biometrics by the general public. The primary research method was a survey, which had 282 respondents, designed to gauge public opinion around biometrics. Additionally, qualitative data was captured in the form of the participants' definition of the term \textit{biometrics}. We applied thematic analysis as well as an automated Word Vector analysis to this data to provide a deeper insight into the perceptions and understanding of the term. Our results demonstrate that while there is generally a reasonable level of understanding of what biometrics are, this is typically limited to the techniques that are most familiar to participants (e.g., fingerprints or facial recognition). Most notably individuals' awareness overlooks emerging areas such as behavioural biometrics (e.g., gait). This was also apparent when we compared participants' views to definitions provided by official, published sources (e.g., ISO, NIST, OED, DHS). Overall, this article provides unique insight into the perceptions and understanding of biometrics as well as areas where users may lack knowledge on biometric applications.
Mega-Reward: Achieving Human-Level Play without Extrinsic Rewards
Venue: arXiv cs.AI 2019
Year: 2019

Intrinsic rewards were introduced to simulate how human intelligence works; they are usually evaluated by intrinsically-motivated play, i.e., playing games without extrinsic rewards but evaluated with extrinsic rewards. However, none of the existing intrinsic reward approaches can achieve human-level performance under this very challenging setting of intrinsically-motivated play. In this work, we propose a novel megalomania-driven intrinsic reward (called mega-reward), which, to our knowledge, is the first approach that achieves human-level performance in intrinsically-motivated play. Intuitively, mega-reward comes from the observation that infants' intelligence develops when they try to gain more control on entities in an environment; therefore, mega-reward aims to maximize the control capabilities of agents on given entities in a given environment. To formalize mega-reward, a relational transition model is proposed to bridge the gaps between direct and latent control. Experimental studies show that mega-reward (i) can greatly outperform all state-of-the-art intrinsic reward approaches, (ii) generally achieves the same level of performance as Ex-PPO and professional human-level scores, and (iii) has also a superior performance when it is incorporated with extrinsic rewards.
Predictive Ensemble Learning with Application to Scene Text Detection
Venue: arXiv cs.CV 2019
Year: 2019

Deep learning based approaches have achieved significant progresses in different tasks like classification, detection, segmentation, and so on. Ensemble learning is widely known to further improve performance by combining multiple complementary models. It is easy to apply ensemble learning for classification tasks, for example, based on averaging, voting, or other methods. However, for other tasks (like object detection) where the outputs are varying in quantity and unable to be simply compared, the ensemble of multiple models become difficult. In this paper, we propose a new method called Predictive Ensemble Learning (PEL), based on powerful predictive ability of deep neural networks, to directly predict the best performing model among a pool of base models for each test example, thus transforming ensemble learning to a traditional classification task. Taking scene text detection as the application, where no suitable ensemble learning strategy exists, PEL can significantly improve the performance, compared to either individual state-of-the-art models, or the fusion of multiple models by non-maximum suppression. Experimental results show the possibility and potential of PEL in predicting different models' performance based only on a query example, which can be extended for ensemble learning in many other complex tasks.
Passport: Enabling Accurate Country-Level Router Geolocation using Inaccurate Sources
Venue: arXiv cs.NI 2019
Year: 2019

When does Internet traffic cross international borders? This question has major geopolitical, legal and social implications and is surprisingly difficult to answer. A critical stumbling block is a dearth of tools that accurately map routers traversed by Internet traffic to the countries in which they are located. This paper presents Passport: a new approach for efficient, accurate country-level router geolocation and a system that implements it. Passport provides location predictions with limited active measurements, using machine learning to combine information from IP geolocation databases, router hostnames, whois records, and ping measurements. We show that Passport substantially outperforms existing techniques, and identify cases where paths traverse countries with implications for security, privacy, and performance.
Integrating Objects into Monocular SLAM: Line Based Category Specific Models
Venue: arXiv cs.RO 2019
Year: 2019

We propose a novel Line based parameterization for category specific CAD models. The proposed parameterization associates 3D category-specific CAD model and object under consideration using a dictionary based RANSAC method that uses object Viewpoints as prior and edges detected in the respective intensity image of the scene. The association problem is posed as a classical Geometry problem rather than being dataset driven, thus saving the time and labour that one invests in annotating dataset to train Keypoint Network for different category objects. Besides eliminating the need of dataset preparation, the approach also speeds up the entire process as this method processes the image only once for all objects, thus eliminating the need of invoking the network for every object in an image across all images. A 3D-2D edge association module followed by a resection algorithm for lines is used to recover object poses. The formulation optimizes for shape and pose of the object, thus aiding in recovering object 3D structure more accurately. Finally, a Factor Graph formulation is used to combine object poses with camera odometry to formulate a SLAM problem.
A New Look at an Old Problem: A Universal Learning Approach to Linear Regression
Venue: arXiv cs.LG 2019
Year: 2019

Linear regression is a classical paradigm in statistics. A new look at it is provided via the lens of universal learning. In applying universal learning to linear regression the hypotheses class represents the label $y\in {\cal R}$ as a linear combination of the feature vector $x^Tθ$ where $x\in {\cal R}^M$, within a Gaussian error. The Predictive Normalized Maximum Likelihood (pNML) solution for universal learning of individual data can be expressed analytically in this case, as well as its associated learnability measure. Interestingly, the situation where the number of parameters $M$ may even be larger than the number of training samples $N$ can be examined. As expected, in this case learnability cannot be attained in every situation; nevertheless, if the test vector resides mostly in a subspace spanned by the eigenvectors associated with the large eigenvalues of the empirical correlation matrix of the training data, linear regression can generalize despite the fact that it uses an ``over-parametrized'' model. We demonstrate the results with a simulation of fitting a polynomial to data with a possibly large polynomial degree.
A Comparison of Techniques for Sentiment Classification of Film Reviews
Venue: arXiv cs.CL 2019
Year: 2019

We undertake the task of comparing lexicon-based sentiment classification of film reviews with machine learning approaches. We look at existing methodologies and attempt to emulate and improve on them using a 'given' lexicon and a bag-of-words approach. We also utilise syntactical information such as part-of-speech and dependency relations. We will show that a simple lexicon-based classification achieves good results however machine learning techniques prove to be the superior tool. We also show that more features do not necessarily deliver better performance as well as elaborate on three further enhancements not tested in this article.