OpenRaiser/Intern-Atlas huggingface dataset:
Transformer++
Venue: arXiv cs.CL 2020
Year: 2020

Recent advancements in attention mechanisms have replaced recurrent neural networks and its variants for machine translation tasks. Transformer using attention mechanism solely achieved state-of-the-art results in sequence modeling. Neural machine translation based on the attention mechanism is parallelizable and addresses the problem of handling long-range dependencies among words in sentences more effectively than recurrent neural networks. One of the key concepts in attention is to learn three matrices, query, key, and value, where global dependencies among words are learned through linearly projecting word embeddings through these matrices. Multiple query, key, value matrices can be learned simultaneously focusing on a different subspace of the embedded dimension, which is called multi-head in Transformer. We argue that certain dependencies among words could be learned better through an intermediate context than directly modeling word-word dependencies. This could happen due to the nature of certain dependencies or lack of patterns that lend them difficult to be modeled globally using multi-head self-attention. In this work, we propose a new way of learning dependencies through a context in multi-head using convolution. This new form of multi-head attention along with the traditional form achieves better results than Transformer on the WMT 2014 English-to-German and English-to-French translation tasks. We also introduce a framework to learn POS tagging and NER information during the training of encoder which further improves results achieving a new state-of-the-art of 32.1 BLEU, better than existing best by 1.4 BLEU, on the WMT 2014 English-to-German and 44.6 BLEU, better than existing best by 1.1 BLEU, on the WMT 2014 English-to-French translation tasks. We call this Transformer++.
SAFE: Similarity-Aware Multi-Modal Fake News Detection
Venue: arXiv cs.CL 2020
Year: 2020

Effective detection of fake news has recently attracted significant attention. Current studies have made significant contributions to predicting fake news with less focus on exploiting the relationship (similarity) between the textual and visual information in news articles. Attaching importance to such similarity helps identify fake news stories that, for example, attempt to use irrelevant images to attract readers' attention. In this work, we propose a $\mathsf{S}$imilarity-$\mathsf{A}$ware $\mathsf{F}$ak$\mathsf{E}$ news detection method ($\mathsf{SAFE}$) which investigates multi-modal (textual and visual) information of news articles. First, neural networks are adopted to separately extract textual and visual features for news representation. We further investigate the relationship between the extracted features across modalities. Such representations of news textual and visual information along with their relationship are jointly learned and used to predict fake news. The proposed method facilitates recognizing the falsity of news articles based on their text, images, or their "mismatches." We conduct extensive experiments on large-scale real-world data, which demonstrate the effectiveness of the proposed method.
GASP! Generating Abstracts of Scientific Papers from Abstracts of Cited Papers
Venue: arXiv cs.CL 2020
Year: 2020

Creativity is one of the driving forces of human kind as it allows to break current understanding to envision new ideas, which may revolutionize entire fields of knowledge. Scientific research offers a challenging environment where to learn a model for the creative process. In fact, scientific research is a creative act in the formal settings of the scientific method and this creative act is described in articles. In this paper, we dare to introduce the novel, scientifically and philosophically challenging task of Generating Abstracts of Scientific Papers from abstracts of cited papers (GASP) as a text-to-text task to investigate scientific creativity, To foster research in this novel, challenging task, we prepared a dataset by using services where that solve the problem of copyright and, hence, the dataset is public available with its standard split. Finally, we experimented with two vanilla summarization systems to start the analysis of the complexity of the GASP task.
PAS: Prediction-based Adaptive Sleeping for Environment Monitoring in Sensor Networks
Venue: arXiv cs.DC 2020
Year: 2020

Energy efficiency has proven to be an important factor dominating the working period of WSN surveillance systems. Intensive studies have been done to provide energy efficient power management mechanisms. In this paper, we present PAS, a Prediction-based Adaptive Sleeping mechanism for environment monitoring sensor networks to conserve energy. PAS focuses on the diffusion stimulus (DS) scenario, which is very common and important in the application of environment monitoring. Different with most of previous works, PAS explores the features of DS spreading process to obtain higher energy efficiency. In PAS, sensors determine their sleeping schedules based on the observed emergency of DS spreading. While sensors near the DS boundary stay awake to accurately capture the possible stimulus arrival, the far away sensors turn into sleeping mode to conserve energy. Simulation experiment shows that PAS largely reduces the energy cost without decreasing system performance
PL${}_{1}$P -- Point-line Minimal Problems under Partial Visibility in Three Views
Venue: arXiv cs.CV 2020
Year: 2020

We present a complete classification of minimal problems for generic arrangements of points and lines in space observed partially by three calibrated perspective cameras when each line is incident to at most one point. This is a large class of interesting minimal problems that allows missing observations in images due to occlusions and missed detections. There is an infinite number of such minimal problems; however, we show that they can be reduced to 140616 equivalence classes by removing superfluous features and relabeling the cameras. We also introduce camera-minimal problems, which are practical for designing minimal solvers, and show how to pick a simplest camera-minimal problem for each minimal problem. This simplification results in 74575 equivalence classes. Only 76 of these were known; the rest are new. In order to identify problems that have potential for practical solving of image matching and 3D reconstruction, we present several smaller natural subfamilies of camera-minimal problems as well as compute solution counts for all camera-minimal problems which have less than 300 solutions for generic data.
Super-reflective Data: Speculative Imaginings of a World Where Data Works for People
Venue: arXiv cs.HC 2020
Year: 2020

It's the year 2020, and every space and place on- and off-line has been augmented with digital things that observe, record, transmit, and compute, for the purposes of recording endless data traces of what is happening in the world. Individually, these things (and the invisible services the power them) have reached considerable sophistication in their ability to analyse and dissect such observations, turning streams of audio and video into informative data fragments. Yet somehow, individuals as end-users of platforms and services have not seen the full potential of such data. In this speculative paper, we propose two hypothetical mini scenarios different from our current digital world. In the former, instead of hoarding it, data controllers turn captured data over to those who need it as quickly as possible, working together to combine, validate, and refine it for maximum usefulness. This simultaneously addresses the data fragmentation and privacy problem, by handing over long-term data governance to those that value it the most In the latter, we discuss ethical dilemmas using the long-term use of such rich data and its tendency to cause people to relentlessly optimise.
Self-Sovereign Identity for IoT environments: A Perspective
Venue: arXiv cs.CR 2020
Year: 2020

This paper analyses the concept of Self-Sovereign Identity (SSI), an emerging approach for establishing digital identity, in the context of the Internet of Things (IoT). We contrast existing approaches for identity on the Internet, such as cloud-based accounts and digital certificates, with SSI standards such as Decentralized Identifiers (DIDs) and Verifiable Credentials (VCs). To the best of our knowledge, this is the first thorough comparison of these approaches. The benefits and challenges of using DIDs and VCs to identify and authenticate IoT devices and their respective users are discussed. In the end, we establish that SSI, with its owner-centric, privacy-aware and decentrailized approach, provides a viable and attractive option for secure identification of IoT devices and users.
SIMBA: A Skyrmionic In-Memory Binary Neural Network Accelerator
Venue: arXiv cs.ET 2020
Year: 2020

Magnetic skyrmions are emerging as potential candidates for next generation non-volatile memories. In this paper, we propose an in-memory binary neural network (BNN) accelerator based on the non-volatile skyrmionic memory, which we call as SIMBA. SIMBA consumes 26.7 mJ of energy and 2.7 ms of latency when running an inference on a VGG-like BNN. Furthermore, we demonstrate improvements in the performance of SIMBA by optimizing material parameters such as saturation magnetization, anisotropic energy and damping ratio. Finally, we show that the inference accuracy of BNNs is robust against the possible stochastic behavior of SIMBA (88.5% +/- 1%).
Efficient Schedulability Test for Dynamic-Priority Scheduling of Mixed-Criticality Real-Time Systems
Venue: arXiv cs.OS 2020
Year: 2020

Systems in many safety-critical application domains are subject to certification requirements. In such a system, there are typically different applications providing functionalities that have varying degrees of criticality. Consequently, the certification requirements for functionalities at these different criticality levels are also varying, with very high levels of assurance required for a highly critical functionality, whereas relatively low levels of assurance required for a less critical functionality. Considering the timing assurance given to various applications in the form of guaranteed budgets within deadlines, a theory of real-time scheduling for such multi-criticality systems has been under development in the recent past. In particular, an algorithm called Earliest Deadline First with Virtual Deadlines (EDF-VD) has shown a lot of promise for systems with two criticality levels, especially in terms of practical performance demonstrated through experiment results. In this paper we design a new schedulability test for EDF-VD that extend these performance benefits to multi-criticality systems. We propose a new test based on demand bound functions and also present a novel virtual deadline assignment strategy. Through extensive experiments we show that the proposed technique significantly outperforms existing strategies for a variety of generic real-time systems.
Scan Correlation -- Revealing distributed scan campaigns
Venue: arXiv cs.CR 2020
Year: 2020

Public networks are exposed to port scans from the Internet. Attackers search for vulnerable services they can exploit. In large scan campaigns, attackers often utilize different machines to perform distributed scans, which impedes their detection and might also camouflage the actual goal of the scanning campaign. In this paper, we present a correlation algorithm to detect scans, identify potential relations among them, and reassemble them to larger campaigns. We evaluate our approach on real-world Internet traffic and our results indicate that it can summarize and characterize standalone and distributed scan campaigns based on their tools and intention.
The Sequent Calculus of Skew Monoidal Categories
Venue: arXiv cs.LO 2020
Year: 2020

Szlachányi's skew monoidal categories are a well-motivated variation of monoidal categories in which the unitors and associator are not required to be natural isomorphisms, but merely natural transformations in a particular direction. We present a sequent calculus for skew monoidal categories, building on the recent formulation by one of the authors of a sequent calculus for the Tamari order (skew semigroup categories). In this calculus, antecedents consist of a stoup (an optional formula) followed by a context, and the connectives behave like in the standard monoidal sequent calculus except that the left rules may only be applied in stoup position. We prove that this calculus is sound and complete with respect to existence of maps in the free skew monoidal category, and moreover that it captures equality of maps once a suitable equivalence relation is imposed on derivations. We then identify a subsystem of focused derivations and establish that it contains exactly one canonical representative from each equivalence class. This coherence theorem leads directly to simple procedures for deciding equality of maps in the free skew monoidal category and for enumerating any homset without duplicates. Finally, and in the spirit of Lambek's work, we describe the close connection between this proof-theoretic analysis and Bourke and Lack's recent characterization of skew monoidal categories as left representable skew multicategories. We have formalized this development in the dependently typed programming language Agda.
On the Reproduction of Real Wireless Channel Occupancy in ns-3
Venue: arXiv cs.NI 2020
Year: 2020

In wireless networking R&D we typically depend on simulation and experimentation to evaluate and validate new networking solutions. While simulations allow full control over the scenario conditions, real-world experiments are influenced by external random phenomena and may produce hardly repeatable and reproducible results, impacting the validation of the solution under evaluation. Previously, we have proposed the Trace-based Simulation (TS) approach to address the problem. TS uses traces of radio link quality and position of nodes to accurately reproduce past experiments in ns-3. Yet, in its current version, the TS approach is not compatible with scenarios where the radio spectrum is shared with concurrent networks, as it does not reproduce their channel occupancy. In this paper, we introduce the InterferencePropagationLossModel and a modified MacLow to allow reproducing the channel occupancy experienced in past experiments. To validate the proposed models, the network throughput was measured in different experiments performed in the w-iLab.t testbed, controlling the channel occupancy introduced by concurrent networks. The experimental results were then compared with the network throughput achieved using the improved TS approach, the legacy TS approach, and pure simulation, validating the new proposed models and confirming their relevance to reproduce experiments previously executed in real environments.
Capturing document context inside sentence-level neural machine translation models with self-training
Venue: arXiv cs.CL 2020
Year: 2020

Neural machine translation (NMT) has arguably achieved human level parity when trained and evaluated at the sentence-level. Document-level neural machine translation has received less attention and lags behind its sentence-level counterpart. The majority of the proposed document-level approaches investigate ways of conditioning the model on several source or target sentences to capture document context. These approaches require training a specialized NMT model from scratch on parallel document-level corpora. We propose an approach that doesn't require training a specialized model on parallel document-level corpora and is applied to a trained sentence-level NMT model at decoding time. We process the document from left to right multiple times and self-train the sentence-level model on pairs of source sentences and generated translations. Our approach reinforces the choices made by the model, thus making it more likely that the same choices will be made in other sentences in the document. We evaluate our approach on three document-level datasets: NIST Chinese-English, WMT'19 Chinese-English and OpenSubtitles English-Russian. We demonstrate that our approach has higher BLEU score and higher human preference than the baseline. Qualitative analysis of our approach shows that choices made by model are consistent across the document.
Hitting Long Directed Cycles is Fixed-Parameter Tractable
Venue: arXiv cs.DS 2020
Year: 2020

In the Directed Long Cycle Hitting Set} problem we are given a directed graph $G$, and the task is to find a set $S$ of at most $k$ vertices/arcs such that $G-S$ has no cycle of length longer than $\ell$. We show that the problem can be solved in time $2^{\mathcal O(\ell k^3\log k + k^5\log k\log\ell)}\cdot n^{\mathcal O(1)}$, that is, it is fixed-parameter tractable (FPT) parameterized by $k$ and $\ell$. This algorithm can be seen as a far-reaching generalization of the fixed-parameter tractability of {\sc Mixed Graph Feedback Vertex Set} [Bonsma and Lokshtanov WADS 2011], which is already a common generalization of the fixed-parameter tractability of (undirected) {\sc Feedback Vertex Set} and the {\sc Directed Feedback Vertex Set} problems, two classic results in parameterized algorithms. The algorithm requires significant insights into the structure of graphs without directed cycles length longer than $\ell$ and can be seen as an exact version of the approximation algorithm following from the Erd{ő}s-P{ó}sa property for long cycles in directed graphs proved by Kreutzer and Kawarabayashi [STOC 2015].
A Safety Framework for Critical Systems Utilising Deep Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

Increasingly sophisticated mathematical modelling processes from Machine Learning are being used to analyse complex data. However, the performance and explainability of these models within practical critical systems requires a rigorous and continuous verification of their safe utilisation. Working towards addressing this challenge, this paper presents a principled novel safety argument framework for critical systems that utilise deep neural networks. The approach allows various forms of predictions, e.g., future reliability of passing some demands, or confidence on a required reliability level. It is supported by a Bayesian analysis using operational data and the recent verification and validation techniques for deep learning. The prediction is conservative -- it starts with partial prior knowledge obtained from lifecycle activities and then determines the worst-case prediction. Open challenges are also identified.
On Tracking Java Methods with Git Mechanisms
Venue: arXiv cs.SE 2020
Year: 2020

Method-level historical information is useful in research on mining software repositories such as fault-prone module detection or evolutionary coupling identification. An existing technique named Historage converts a Git repository of a Java project to a finer-grained one. In a finer-grained repository, each Java method exists as a single file. Treating Java methods as files has an advantage, which is that Java methods can be tracked with Git mechanisms. The biggest benefit of tracking methods with Git mechanisms is that it can easily connect with any other tools and techniques build on Git infrastructure. However, Historage's tracking has an issue of accuracy, especially on small methods. More concretely, in the case that a small method is renamed or moved to another class, Historage has a limited capability to track the method. In this paper, we propose a new technique, FinerGit, to improve the trackability of Java methods with Git mechanisms. We implement FinerGit as a system and apply it to 182 open source software projects, which include 1,768K methods in total. The experimental results show that our tool has a higher capability of tracking methods in the case that methods are renamed or moved to other classes.
A Methodology for Automating Assurance Case Generation
Venue: arXiv cs.RO 2020
Year: 2020

Safety Case has become an integral component for safety-certification in various Cyber Physical System domains including automotive, aviation, medical devices, and military. The certification processes for these systems are stringent and require robust safety assurance arguments and substantial evidence backing. Despite the strict requirements, current practices still rely on manual methods that are brittle, do not have a systematic approach or thorough consideration of sound arguments. In addition, stringent certification requirements and ever-increasing system complexity make ad-hoc, manual assurance case generation (ACG) inefficient, time consuming, and expensive. To improve the current state of practice, we introduce a structured ACG tool which uses system design artifacts, accumulated evidence, and developer expertise to construct a safety case and evaluate it in an automated manner. We also illustrate the applicability of the ACG tool on a remote-control car testbed case study.
How Powerful Are Randomly Initialized Pointcloud Set Functions?
Venue: arXiv cs.CV 2020
Year: 2020

We study random embeddings produced by untrained neural set functions, and show that they are powerful representations which well capture the input features for downstream tasks such as classification, and are often linearly separable. We obtain surprising results that show that random set functions can often obtain close to or even better accuracy than fully trained models. We investigate factors that affect the representative power of such embeddings quantitatively and qualitatively.
Can Society Function Without Ethical Agents? An Informational Perspective
Venue: arXiv cs.GT 2020
Year: 2020

Many facts are learned through the intermediation of individuals with special access to information, such as law enforcement officers, officials with a security clearance, or experts with specific knowledge. This paper considers whether societies can learn about such facts when information is cheap to manipulate, produced sequentially, and these individuals are devoid of ethical motive. The answer depends on an "information attrition" condition pertaining to the amount of evidence available which distinguishes, for example, between reproducible scientific evidence and the evidence generated in a crime. Applications to institution enforcement, social cohesion, scientific progress, and historical revisionism are discussed.
Utilization Difference Based Partitioned Scheduling of Mixed-Criticality Systems
Venue: arXiv cs.OS 2020
Year: 2020

Mixed-Criticality (MC) systems consolidate multiple functionalities with different criticalities onto a single hardware platform. Such systems improve the overall resource utilization while guaranteeing resources to critical tasks. In this paper, we focus on the problem of partitioned multiprocessor MC scheduling, in particular the problem of designing efficient partitioning strategies. We develop two new partitioning strategies based on the principle of evenly distributing the difference between total high-critical utilization and total low-critical utilization for the critical tasks among all processors. By balancing this difference, we are able to reduce the pessimism in uniprocessor MC schedulability tests that are applied on each processor, thus improving overall schedulability. To evaluate the schedulability performance of the proposed strategies, we compare them against existing partitioned algorithms using extensive experiments. We show that the proposed strategies are effective with both dynamic-priority Earliest Deadline First with Virtual Deadlines (EDF-VD) and fixed-priority Adaptive Mixed-Criticality (AMC) algorithms. Specifically, our results show that the proposed strategies improve schedulability by as much as 28.1% and 36.2% for implicit and constrained-deadline task systems respectively.