OpenRaiser/Intern-Atlas huggingface dataset:
Efficient Contextual Representation Learning Without Softmax Layer
Venue: arXiv cs.CL 2019
Year: 2019

Contextual representation models have achieved great success in improving various downstream tasks. However, these language-model-based encoders are difficult to train due to the large parameter sizes and high computational complexity. By carefully examining the training procedure, we find that the softmax layer (the output layer) causes significant inefficiency due to the large vocabulary size. Therefore, we redesign the learning objective and propose an efficient framework for training contextual representation models. Specifically, the proposed approach bypasses the softmax layer by performing language modeling with dimension reduction, and allows the models to leverage pre-trained word embeddings. Our framework reduces the time spent on the output layer to a negligible level, eliminates almost all the trainable parameters of the softmax layer and performs language modeling without truncating the vocabulary. When applied to ELMo, our method achieves a 4 times speedup and eliminates 80% trainable parameters while achieving competitive performance on downstream tasks.
FastFusionNet: New State-of-the-Art for DAWNBench SQuAD
Venue: arXiv cs.CL 2019
Year: 2019

In this technical report, we introduce FastFusionNet, an efficient variant of FusionNet [12]. FusionNet is a high performing reading comprehension architecture, which was designed primarily for maximum retrieval accuracy with less regard towards computational requirements. For FastFusionNets we remove the expensive CoVe layers [21] and substitute the BiLSTMs with far more efficient SRU layers [19]. The resulting architecture obtains state-of-the-art results on DAWNBench [5] while achieving the lowest training and inference time on SQuAD [25] to-date. The code is available at https://github.com/felixgwu/FastFusionNet.
Optimal Algorithms for Ski Rental with Soft Machine-Learned Predictions
Venue: arXiv cs.LG 2019
Year: 2019

We consider a variant of the classic Ski Rental online algorithm with applications to machine learning. In our variant, we allow the skier access to a black-box machine-learning algorithm that provides an estimate of the probability that there will be at most a threshold number of ski-days. We derive a class of optimal randomized algorithms to determine the strategy that minimizes the worst-case expected competitive ratio for the skier given a prediction from the machine learning algorithm,and analyze the performance and robustness of these algorithms.
An efficient numerical model for liquid water uptake in porous material and its parameter estimation
Venue: arXiv cs.CE 2019
Year: 2019

The goal of this study is to propose an efficient numerical model for the predictions of capillary adsorption phenomena in a porous material. The Scharfetter-Gummel numerical scheme is proposed to solve an advection-diffusion equation with gravity flux. Its advantages such as accuracy, relaxed stability condition, and reduced computational cost are discussed along with the study of linear and nonlinear cases. The reliability of the numerical model is evaluated by comparing the numerical predictions with experimental observations of liquid uptake in bricks. A parameter estimation problem is solved to adjust the uncertain coefficients of moisture diffusivity and hydraulic conductivity.
Video Summarization via Actionness Ranking
Venue: arXiv cs.CV 2019
Year: 2019

To automatically produce a brief yet expressive summary of a long video, an automatic algorithm should start by resembling the human process of summary generation. Prior work proposed supervised and unsupervised algorithms to train models for learning the underlying behavior of humans by increasing modeling complexity or craft-designing better heuristics to simulate human summary generation process. In this work, we take a different approach by analyzing a major cue that humans exploit for the summary generation; the nature and intensity of actions. We empirically observed that a frame is more likely to be included in human-generated summaries if it contains a substantial amount of deliberate motion performed by an agent, which is referred to as actionness. Therefore, we hypothesize that learning to automatically generate summaries involves an implicit knowledge of actionness estimation and ranking. We validate our hypothesis by running a user study that explores the correlation between human-generated summaries and actionness ranks. We also run a consensus and behavioral analysis between human subjects to ensure reliable and consistent results. The analysis exhibits a considerable degree of agreement among subjects within obtained data and verifying our initial hypothesis. Based on the study findings, we develop a method to incorporate actionness data to explicitly regulate a learning algorithm that is trained for summary generation. We assess the performance of our approach to four summarization benchmark datasets and demonstrate an evident advantage compared to state-of-the-art summarization methods.
On the Existence of Perfect Splitter Sets
Venue: arXiv cs.IT 2019
Year: 2019

Given integers $k_1, k_2$ with $0\le k_1<k_2$, the determinations of all positive integers $q$ for which there exists a perfect Splitter $B[-k_1, k_2](q)$ set is a wide open question in general. In this paper, we obtain new necessary and sufficient conditions for an odd prime $p$ such that there exists a nonsingular perfect $B[-1,3](p)$ set. We also give some necessary conditions for the existence of purely singular perfect splitter sets. In particular, we determine all perfect $B[-k_1, k_2](2^n)$ sets for any positive integers $k_1,k_2$ with $k_1+k_2\ge4$. We also prove that there are infinitely many prime $p$ such that there exists a perfect $B[-1,3](p)$ set.
Video Extrapolation with an Invertible Linear Embedding
Venue: arXiv cs.CV 2019
Year: 2019

We predict future video frames from complex dynamic scenes, using an invertible neural network as the encoder of a nonlinear dynamic system with latent linear state evolution. Our invertible linear embedding (ILE) demonstrates successful learning, prediction and latent state inference. In contrast to other approaches, ILE does not use any explicit reconstruction loss or simplistic pixel-space assumptions. Instead, it leverages invertibility to optimize the likelihood of image sequences exactly, albeit indirectly. Comparison with a state-of-the-art method demonstrates the viability of our approach.
Should All Temporal Difference Learning Use Emphasis?
Venue: arXiv cs.AI 2019
Year: 2019

Emphatic Temporal Difference (ETD) learning has recently been proposed as a convergent off-policy learning method. ETD was proposed mainly to address convergence issues of conventional Temporal Difference (TD) learning under off-policy training but it is different from conventional TD learning even under on-policy training. A simple counterexample provided back in 2017 pointed to a potential class of problems where ETD converges but TD diverges. In this paper, we empirically show that ETD converges on a few other well-known on-policy experiments whereas TD either diverges or performs poorly. We also show that ETD outperforms TD on the mountain car prediction problem. Our results, together with a similar pattern observed under off-policy training in prior works, suggest that ETD might be a good substitute over conventional TD.
Evaluation Mechanism of Collective Intelligence for Heterogeneous Agents Group
Venue: arXiv cs.AI 2019
Year: 2019

Collective intelligence is manifested when multiple agents coherently work in observation, interaction, decision-making and action. In this paper, we define and quantify the intelligence level of heterogeneous agents group with the improved Anytime Universal Intelligence Test(AUIT), based on an extension of the existing evaluation of homogeneous agents group. The relationship of intelligence level with agents composition, group size, spatial complexity and testing time is analyzed. The intelligence level of heterogeneous agents groups is compared with the homogeneous ones to analyze the effects of heterogeneity on collective intelligence. Our work will help to understand the essence of collective intelligence more deeply and reveal the effect of various key factors on group intelligence level.
Approximate Robust Control of Uncertain Dynamical Systems
Venue: arXiv cs.SY 2019
Year: 2019

This work studies the design of safe control policies for large-scale non-linear systems operating in uncertain environments. In such a case, the robust control framework is a principled approach to safety that aims to maximize the worst-case performance of a system. However, the resulting optimization problem is generally intractable for non-linear systems with continuous states. To overcome this issue, we introduce two tractable methods that are based either on sampling or on a conservative approximation of the robust objective. The proposed approaches are applied to the problem of autonomous driving.
Maximizing spreading influence via measuring influence overlap for social networks
Venue: arXiv cs.SI 2019
Year: 2019

Influence overlap is a universal phenomenon in influence spreading for social networks. In this paper, we argue that the redundant influence generated by influence overlap cause negative effect for maximizing spreading influence. Firstly, we present a theoretical method to calculate the influence overlap and record the redundant influence. Then in term of eliminating redundant influence, we present two algorithms, namely, Degree-Redundant-Influence (DRS) and Degree-Second-Neighborhood (DSN) for multiple spreaders identification. The experiments for four empirical social networks successfully verify the methods, and the spreaders selected by the DSN algorithm show smaller degree and k-core values.
Crowding in humans is unlike that in convolutional neural networks
Venue: arXiv cs.CV 2019
Year: 2019

Object recognition is a primary function of the human visual system. It has recently been claimed that the highly successful ability to recognise objects in a set of emergent computer vision systems---Deep Convolutional Neural Networks (DCNNs)---can form a useful guide to recognition in humans. To test this assertion, we systematically evaluated visual crowding, a dramatic breakdown of recognition in clutter, in DCNNs and compared their performance to extant research in humans. We examined crowding in three architectures of DCNNs with the same methodology as that used among humans. We manipulated multiple stimulus factors including inter-letter spacing, letter colour, size, and flanker location to assess the extent and shape of crowding in DCNNs. We found that crowding followed a predictable pattern across architectures that was different from that in humans. Some characteristic hallmarks of human crowding, such as invariance to size, the effect of target-flanker similarity, and confusions between target and flanker identities, were completely missing, minimised or even reversed. These data show that DCNNs, while proficient in object recognition, likely achieve this competence through a set of mechanisms that are distinct from those in humans. They are not necessarily equivalent models of human or primate object recognition and caution must be exercised when inferring mechanisms derived from their operation.
On the usage of the probability integral transform to reduce the complexity of multi-way fuzzy decision trees in Big Data classification problems
Venue: arXiv cs.LG 2019
Year: 2019

We present a new distributed fuzzy partitioning method to reduce the complexity of multi-way fuzzy decision trees in Big Data classification problems. The proposed algorithm builds a fixed number of fuzzy sets for all variables and adjusts their shape and position to the real distribution of training data. A two-step process is applied : 1) transformation of the original distribution into a standard uniform distribution by means of the probability integral transform. Since the original distribution is generally unknown, the cumulative distribution function is approximated by computing the q-quantiles of the training set; 2) construction of a Ruspini strong fuzzy partition in the transformed attribute space using a fixed number of equally distributed triangular membership functions. Despite the aforementioned transformation, the definition of every fuzzy set in the original space can be recovered by applying the inverse cumulative distribution function (also known as quantile function). The experimental results reveal that the proposed methodology allows the state-of-the-art multi-way fuzzy decision tree (FMDT) induction algorithm to maintain classification accuracy with up to 6 million fewer leaves.
Improved queue-size scaling for input-queued switches via graph factorization
Venue: arXiv cs.NI 2019
Year: 2019

This paper studies the scaling of the expected total queue size in an $n\times n$ input-queued switch, as a function of both the load $ρ$ and the system scale $n$. We provide a new class of scheduling policies under which the expected total queue size scales as $O\left( n(1-ρ)^{-4/3} \log \left(\max\{\frac{1}{1-ρ}, n\}\right)\right)$, over all $n$ and $ρ<1$, when the arrival rates are uniform. This improves over the previously best-known scalings in two regimes: $O\left(n^{1.5}(1-ρ)^{-1} \log \frac{1}{1-ρ}\right)$ when $Ω(n^{-1.5}) \le 1-ρ\le O(n^{-1})$ and $O\left(\frac{n\log n}{(1-ρ)^2}\right)$ when $1-ρ\geq Ω(n^{-1})$. A key ingredient in our method is a tight characterization of the largest $k$-factor of a random bipartite multigraph, which may be of independent interest.
Web-Based 3D and Haptic Interactive Environments for e-Learning, Simulation, and Training
Venue: arXiv cs.HC 2019
Year: 2019

Knowledge creation occurs in the process of social interaction. As our service-based society is evolving into a knowledge-based society there is an acute need for more effective collaboration and knowledge-sharing systems to be used by geographically scattered people. We present the use of Web3D components and standards, such as X3D, in combination with the haptic (tactile) paradigm, for the development of new communication channels for e-Learning and simulation.
TEEvil: Identity Lease via Trusted Execution Environments
Venue: arXiv cs.CR 2019
Year: 2019

We investigate identity lease, a new type of service in which users lease their identities to third parties by providing them with full or restricted access to their online accounts or credentials. We discuss how identity lease could be abused to subvert the digital society, facilitating the spread of fake news and subverting electronic voting by enabling the sale of votes. We show that the emergence of Trusted Execution Environments and anonymous cryptocurrencies, for the first time, allows the implementation of such a lease service while guaranteeing fairness, plausible deniability and anonymity, therefore shielding the users and account renters from prosecution. To show that such a service can be practically implemented, we build an example service that we call TEEvil leveraging Intel SGX and ZCash. Finally, we discuss defense mechanisms and challenges in the mitigation of identity lease services.
Parallel Index-based Stream Join on a Multicore CPU
Venue: arXiv cs.DB 2019
Year: 2019

There is increasing interest in using multicore processors to accelerate stream processing. For example, indexing sliding window content to enhance the performance of streaming queries is greatly improved by utilizing the computational capabilities of a multicore processor. However, designing an effective concurrency control mechanism that addresses the problem of concurrent indexing in highly dynamic settings remains a challenge. In this paper, we introduce an index data structure, called the Partitioned In-memory Merge-Tree, to address the challenges that arise when indexing highly dynamic data, which are common in streaming settings. To complement the index, we design an algorithm to realize a parallel index-based stream join that exploits the computational power of multicore processors. Our experiments using an octa-core processor show that our parallel stream join achieves up to 5.5 times higher throughput than a single-threaded approach.
Superseding traditional indexes by orchestrating learning and geometry
Venue: arXiv cs.DS 2019
Year: 2019

We design the first learned index that solves the dictionary problem with time and space complexity provably better than classic data structures for hierarchical memories, such as B-trees, and modern learned indexes. We call our solution the Piecewise Geometric Model index (PGM-index) because it turns the indexing of a sequence of keys into the coverage of a sequence of 2D-points via linear models (i.e. segments) suitably learned to trade query time vs space efficiency. This idea comes from some known heuristic results which we strengthen by showing that the minimal number of such segments can be computed via known and optimal streaming algorithms. Our index is then obtained by recursively applying this geometric idea that guarantees a smoothed adaptation to the "geometric complexity" of the input data. Finally, we propose a variant of the index that adapts not only to the distribution of the dictionary keys but also to their access frequencies, thus obtaining the first distribution-aware learned index. The second main contribution of this paper is the proposal and study of the concept of Multicriteria Data Structure, namely one that asks a data structure to adapt in an automatic way to the constraints imposed by the application of use. We show that our index is a multicriteria data structure because its significant flexibility in storage and query time can be exploited by a properly designed optimisation algorithm that efficiently finds its best design setting in order to match the input constraints. A thorough experimental analysis shows that our index and its multicriteria variant improve uniformly, over both time and space, classic and learned indexes up to several orders of magnitude.
A Nonlinear Model for Time Synchronization
Venue: arXiv cs.LG 2019
Year: 2019

The current algorithms are based on linear model, for example, Precision Time Protocol (PTP) which requires frequent synchronization in order to handle the effects of clock frequency drift. This paper introduces a nonlinear approach to clock time synchronize. This approach can accurately model the frequency shift. Therefore, the required time interval to synchronize clocks can be longer. Meanwhile, it also offers better performance and relaxes the synchronization process. The idea of the nonlinear algorithm and some numerical examples will be presented in this paper in detail.
Temporal Bibliographic Networks
Venue: arXiv cs.SI 2019
Year: 2019

We present two ways (instantaneous and cumulative) to transform bibliographic networks, using the works' publication year, into corresponding temporal networks based on temporal quantities. We also show how to use the addition of temporal quantities to define interesting temporal properties of nodes, links and their groups thus providing an insight into evolution of bibliographic networks. Using the multiplication of temporal networks we obtain different derived temporal networks providing us with new views on studied networks. The proposed approach is illustrated with examples from the collection of bibliographic networks on peer review.
Controlling Robots using Artificial Intelligence and a Consortium Blockchain
Venue: arXiv cs.RO 2019
Year: 2019

Blockchain is a disruptive technology that is normally used within financial applications, however it can be very beneficial also in certain robotic contexts, such as when an immutable register of events is required. Among the several properties of Blockchain that can be useful within robotic environments, we find not just immutability but also decentralization of the data, irreversibility, accessibility and non-repudiation. In this paper, we propose an architecture that uses blockchain as a ledger and smart-contract technology for robotic control by using external parties, Oracles, to process data. We show how to register events in a secure way, how it is possible to use smart-contracts to control robots and how to interface with external Artificial Intelligence algorithms for image analysis. The proposed architecture is modular and can be used in multiple contexts such as in manufacturing, network control, robot control, and others, since it is easy to integrate, adapt, maintain and extend to new domains.
Synthetic Knowing: The Politics of the Internet of Things
Venue: arXiv cs.CY 2019
Year: 2019

All knowing is material. The challenge for Information Systems (IS) research is to specify how knowing is material by drawing on theoretical characterizations of the digital. Synthetic knowing is knowing informed by theorizing digital materiality. We focus on two defining qualities: liquefaction (unhinging digital representations from physical objects, qualities, or processes) and open-endedness (extendable and generative). The Internet of Things (IoT) is crucial because sensors are vehicles of liquefaction. Their expanding scope for real-time seeing, hearing, tasting, smelling, and touching increasingly mimics phenomenologically perceived reality. Empirically, we present a longitudinal case study of IoT-rendered marine environmental monitoring by an oil and gas company operating in the politically contested Arctic. We characterize synthetic knowing into four concepts, the former three tied to liquefaction and the latter to open-endedness: (i) the objects of knowing are algorithmic phenomena; (ii) the sensors increasingly conjure up phenomenological reality; (iii) knowing is scoped (configurable); and (iv) open knowing/data is politically charged.
DimDraw -- A novel tool for drawing concept lattices
Venue: arXiv cs.CG 2019
Year: 2019

Concept lattice drawings are an important tool to visualize complex relations in data in a simple manner to human readers. Many attempts were made to transfer classical graph drawing approaches to order diagrams. Although those methods are satisfying for some lattices they unfortunately perform poorly in general. In this work we present a novel tool to draw concept lattices that is purely motivated by the order structure.
Selective Exposure shapes the Facebook News Diet
Venue: arXiv cs.SI 2019
Year: 2019

The social brain hypothesis fixes to 150 the number of social relationships we are able to maintain. Similar cognitive constraints emerge in several aspects of our daily life, from our mobility up to the way we communicate, and might even affect the way we consume information online. Indeed, despite the unprecedented amount of information we can access online, our attention span still remains limited. Furthermore, recent studies showed the tendency of users to ignore dissenting information but to interact with information adhering to their point of view. In this paper, we quantitatively analyze users' attention economy in news consumption on social media by analyzing 14M users interacting with 583 news outlets (pages) on Facebook over a time span of 6 years. In particular, we explore how users distribute their activity across news pages and topics. We find that, independently of their activity, users show the tendency to follow a very limited number of pages. On the other hand, users tend to interact with almost all the topics presented by their favored pages. Finally, we introduce a taxonomy accounting for users behavior to distinguish between patterns of selective exposure and interest. Our findings suggest that segregation of users in echo chambers might be an emerging effect of users' activity on social media and that selective exposure -- i.e. the tendency of users to consume information interest coherent with their preferences -- could be a major driver in their consumption patterns.