OpenRaiser/Intern-Atlas huggingface dataset:
Sinusoidal Flow: A Fast Invertible Autoregressive Flow
Venue: arXiv cs.LG 2021
Year: 2021

Normalising flows offer a flexible way of modelling continuous probability distributions. We consider expressiveness, fast inversion and exact Jacobian determinant as three desirable properties a normalising flow should possess. However, few flow models have been able to strike a good balance among all these properties. Realising that the integral of a convex sum of sinusoidal functions squared leads to a bijective residual transformation, we propose Sinusoidal Flow, a new type of normalising flows that inherits the expressive power and triangular Jacobian from fully autoregressive flows while guaranteed by Banach fixed-point theorem to remain fast invertible and thereby obviate the need for sequential inversion typically required in fully autoregressive flows. Experiments show that our Sinusoidal Flow is not only able to model complex distributions, but can also be reliably inverted to generate realistic-looking samples even with many layers of transformations stacked.
Graph? Yes! Which one? Help!
Venue: arXiv cs.DB 2021
Year: 2021

Amazon Neptune is a graph database service that supports two graph (meta)models: W3C's Resource Description Framework (RDF) and Labeled Property Graphs (LPG). Customers opt in for one or the other model, and this choice determines which data modeling features can be used, and - perhaps more importantly - which query languages are available to query and manipulate the graph. The choice between the two technology stacks is difficult and requires consideration of data modeling aspects, query language features, their adequacy for current and future use cases, as well as many other factors (including developer preferences). Sometimes we see customers make the wrong choice with no easy way to reverse it later. It is therefore highly desirable that the choice of the query language can be made without consideration of what graph model is chosen, and can be easily revised or complemented at a later point. In this paper, we advocate and explore the idea of a single, unified graph data model that embraces both RDF and LPGs, and naturally supports different graph query languages on top. We investigate obstacles towards unifying the two graph data models, and propose an initial unifying model, dubbed "one graph" ("1G" for short), as the basis for moving forward.
Graph-based Heuristic Solution for Placing Distributed Video Processing Applications on Moving Vehicle Clusters
Venue: arXiv cs.NI 2021
Year: 2021

Vehicular fog computing (VFC) is envisioned as an extension of cloud and mobile edge computing to utilize the rich sensing and processing resources available in vehicles. We focus on slow-moving cars that spend a significant time in urban traffic congestion as a potential pool of on-board sensors, video cameras, and processing capacity. For leveraging the dynamic network and processing resources, we utilize a stochastic mobility model to select nodes with similar mobility patterns. We then design two distributed applications that are scaled in real-time and placed as multiple instances on selected vehicular fog nodes. We handle the unstable vehicular environment by a), Using real vehicle density data to build a realistic mobility model that helps in selecting nodes for service deployment b), Using community-detection algorithms for selecting a robust vehicular cluster using the predicted mobility behavior of vehicles. The stability of the chosen cluster is validated using a graph centrality measure, and c), Graph-based placement heuristics are developed to find the optimal placement of service graphs based on a multi-objective constrained optimization problem with the objective of efficient resource utilization. The heuristic solves an important problem of processing data generated from distributed devices by balancing the trade-off between increasing the number of service instances to have enough redundancy of processing instances to increase resilience in the service in case of node or link failure, versus reducing their number to minimise resource usage. We compare our heuristic to an integer linear program solution and a first-fit heuristic. Our approach performs better than these comparable schemes in terms of resource utilization and/or has a lesser service latency, which is a crucial requirement for safety-related applications.
Understanding the Role of Self-Supervised Learning in Out-of-Distribution Detection Task
Venue: arXiv cs.CV 2021
Year: 2021

Self-supervised learning (SSL) has achieved great success in a variety of computer vision tasks. However, the mechanism of how SSL works in these tasks remains a mystery. In this paper, we study how SSL can enhance the performance of the out-of-distribution (OOD) detection task. We first point out two general properties that a good OOD detector should have: 1) the overall feature space should be large and 2) the inlier feature space should be small. Then we demonstrate that SSL can indeed increase the intrinsic dimension of the overall feature space. In the meantime, SSL even has the potential to shrink the inlier feature space. As a result, there will be more space spared for the outliers, making OOD detection much easier. The conditions when SSL can shrink the inlier feature space is also discussed and validated. By understanding the role of SSL in the OOD detection task, our study can provide a guideline for designing better OOD detection algorithms. Moreover, this work can also shed light to other tasks where SSL can improve the performance.
A time-weighted metric for sets of trajectories to assess multi-object tracking algorithms
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes a metric for sets of trajectories to evaluate multi-object tracking algorithms that includes time-weighted costs for localisation errors of properly detected targets, for false targets, missed targets and track switches. The proposed metric extends the metric in [1] by including weights to the costs associated to different time steps. The time-weighted costs increase the flexibility of the metric [1] to fit more applications and user preferences. We first introduce a metric based on multi-dimensional assignments, and then its linear programming relaxation, which is computable in polynomial time and is also a metric. The metrics can also be extended to metrics on random finite sets of trajectories to evaluate and rank algorithms across different scenarios, each with a ground truth set of trajectories.
Estimation-Energy Tradeoff for Scalar Gauss-Markov Signals with Kalman Filtering
Venue: arXiv cs.IT 2021
Year: 2021

In this letter, we investigate a receiver architecture, which uses the received signal in order to simultaneously harvest energy and estimate a Gauss-Markov linear process. We study three communication scenarios: i) static channel, ii) Rayleigh block-fading channel, and iii) high power amplifier (HPA) nonlinearities at the transmitter side. Theoretical results for the minimum mean square error as well as the average harvested energy are given for all cases and the fundamental tradeoff between estimation quality and harvested energy is characterized. We show that channel fading improves the estimation performance while HPA requires an extended Kalman filter at the receiver and significantly affects both the estimation and the harvesting efficiency.
Enhanced User Grouping and Pairing Scheme for CoMP-NOMA-based Cellular Networks
Venue: arXiv cs.IT 2021
Year: 2021

Non-orthogonal multiple access (NOMA) has been identified as one of the promising technologies to enhance the spectral efficiency and throughput for the fifth generation (5G) and beyond 5G cellular networks. Alternatively, Coordinated multi-point transmission and reception (CoMP) improves the cell edge users' coverage. Thus, CoMP and NOMA can be used together to improve the overall coverage and throughput of the users. However, user grouping and pairing for CoMP-NOMA-based cellular networks have not been suitably studied in the existing literature. Motivated by this, we propose a user grouping and pairing scheme for a CoMP-NOMA-based system. Detailed numerical results are presented comparing the proposed scheme with the purely OMA-based benchmark system, NOMA only, and CoMP only systems. We show through simulation results that the proposed scheme offers a trade-off between throughput and coverage as compared to the existing NOMA or CoMP based system.
Simultaneous Neural Machine Translation with Constituent Label Prediction
Venue: arXiv cs.CL 2021
Year: 2021

Simultaneous translation is a task in which translation begins before the speaker has finished speaking, so it is important to decide when to start the translation process. However, deciding whether to read more input words or start to translate is difficult for language pairs with different word orders such as English and Japanese. Motivated by the concept of pre-reordering, we propose a couple of simple decision rules using the label of the next constituent predicted by incremental constituent label prediction. In experiments on English-to-Japanese simultaneous translation, the proposed method outperformed baselines in the quality-latency trade-off.
Exploring Content Moderation in the Decentralised Web: The Pleroma Case
Venue: arXiv cs.CY 2021
Year: 2021

Decentralising the Web is a desirable but challenging goal. One particular challenge is achieving decentralised content moderation in the face of various adversaries (e.g. trolls). To overcome this challenge, many Decentralised Web (DW) implementations rely on federation policies. Administrators use these policies to create rules that ban or modify content that matches specific rules. This, however, can have unintended consequences for many users. In this paper, we present the first study of federation policies on the DW, their in-the-wild usage, and their impact on users. We identify how these policies may negatively impact "innocent" users and outline possible solutions to avoid this problem in the future.
Part & Whole Extraction: Towards A Deep Understanding of Quantitative Facts for Percentages in Text
Venue: arXiv cs.CL 2021
Year: 2021

We study the problem of quantitative facts extraction for text with percentages. For example, given the sentence "30 percent of Americans like watching football, while 20% prefer to watch NBA.", our goal is to obtain a deep understanding of the percentage numbers ("30 percent" and "20%") by extracting their quantitative facts: part ("like watching football" and "prefer to watch NBA") and whole ("Americans). These quantitative facts can empower new applications like automated infographic generation. We formulate part and whole extraction as a sequence tagging problem. Due to the large gap between part/whole and its corresponding percentage, we introduce skip mechanism in sequence modeling, and achieved improved performance on both our task and the CoNLL-2003 named entity recognition task. Experimental results demonstrate that learning to skip in sequence tagging is promising.
An in-depth Analysis of Occasional and Recurring Collaborations in Online Music Co-creation
Venue: arXiv cs.HC 2021
Year: 2021

The success of online creative communities depends on the will of participants to create and derive content in a collaborative environment. Despite their growing popularity, the factors that lead to remixing existing content in online creative communities are not entirely understood. In this paper, we focus on overdubbing, that is, a dyadic collaboration where one author mixes one new track with an audio recording previously uploaded by another. We study musicians who collaborate regularly, that is, frequently overdub each other's songs. Building on frequent pattern mining techniques, we develop an approach to seek instances of such recurring collaborations in the Songtree community. We identify 43 instances involving two or three members with a similar reputation in the community. Our findings highlight common and different remix factors in occasional and recurring collaborations. Specifically, fresh and less mature songs are generally overdubbed more; instead, exchanging messages and invitations to collaborate are significant factors only for songs generated through recurring collaborations whereas author reputation (ranking) and applying metadata tags to songs have a positive effect only in occasional collaborations.
BuffetFS: Serve Yourself Permission Checks without Remote Procedure Calls
Venue: arXiv cs.DC 2021
Year: 2021

The remote procedure call (a.k.a. RPC) latency becomes increasingly significant in a distributed file system. We propose BuffetFS, a user-level file system that optimizes I/O performance by eliminating the RPCs caused by \texttt{open()} operation. By leveraging \texttt{open()} from file servers to clients, BuffetFS can restrain the procedure calls for permission checks locally, hence avoid RPCs during the initial stage to access a file. BuffetFS can further reduce response time when users are accessing a large number of small files. We implement a BuffetFS prototype and integrate it into a storage cluster. Our preliminary evaluation results show that BuffetFS can offer up to 70\% performance gain compared to the Lustre file system.
Learning Graph Representation of Person-specific Cognitive Processes from Audio-visual Behaviours for Automatic Personality Recognition
Venue: arXiv cs.CV 2021
Year: 2021

This approach builds on two following findings in cognitive science: (i) human cognition partially determines expressed behaviour and is directly linked to true personality traits; and (ii) in dyadic interactions individuals' nonverbal behaviours are influenced by their conversational partner behaviours. In this context, we hypothesise that during a dyadic interaction, a target subject's facial reactions are driven by two main factors, i.e. their internal (person-specific) cognitive process, and the externalised nonverbal behaviours of their conversational partner. Consequently, we propose to represent the target subjects (defined as the listener) person-specific cognition in the form of a person-specific CNN architecture that has unique architectural parameters and depth, which takes audio-visual non-verbal cues displayed by the conversational partner (defined as the speaker) as input, and is able to reproduce the target subject's facial reactions. Each person-specific CNN is explored by the Neural Architecture Search (NAS) and a novel adaptive loss function, which is then represented as a graph representation for recognising the target subject's true personality. Experimental results not only show that the produced graph representations are well associated with target subjects' personality traits in both human-human and human-machine interaction scenarios, and outperform the existing approaches with significant advantages, but also demonstrate that the proposed novel strategies such as adaptive loss, and the end-to-end vertices/edges feature learning, help the proposed approach in learning more reliable personality representations.
Automated Support for Unit Test Generation: A Tutorial Book Chapter
Venue: arXiv cs.SE 2021
Year: 2021

Unit testing is a stage of testing where the smallest segment of code that can be tested in isolation from the rest of the system - often a class - is tested. Unit tests are typically written as executable code, often in a format provided by a unit testing framework such as pytest for Python. Creating unit tests is a time and effort-intensive process with many repetitive, manual elements. To illustrate how AI can support unit testing, this chapter introduces the concept of search-based unit test generation. This technique frames the selection of test input as an optimization problem - we seek a set of test cases that meet some measurable goal of a tester - and unleashes powerful metaheuristic search algorithms to identify the best possible test cases within a restricted timeframe. This chapter introduces two algorithms that can generate pytest-formatted unit tests, tuned towards coverage of source code statements. The chapter concludes by discussing more advanced concepts and gives pointers to further reading for how artificial intelligence can support developers and testers when unit testing software.
Learning Robust Controllers Via Probabilistic Model-Based Policy Search
Venue: arXiv cs.LG 2021
Year: 2021

Model-based Reinforcement Learning estimates the true environment through a world model in order to approximate the optimal policy. This family of algorithms usually benefits from better sample efficiency than their model-free counterparts. We investigate whether controllers learned in such a way are robust and able to generalize under small perturbations of the environment. Our work is inspired by the PILCO algorithm, a method for probabilistic policy search. We show that enforcing a lower bound to the likelihood noise in the Gaussian Process dynamics model regularizes the policy updates and yields more robust controllers. We demonstrate the empirical benefits of our method in a simulation benchmark.
Open Rule Induction
Venue: arXiv cs.CL 2021
Year: 2021

Rules have a number of desirable properties. It is easy to understand, infer new knowledge, and communicate with other inference systems. One weakness of the previous rule induction systems is that they only find rules within a knowledge base (KB) and therefore cannot generalize to more open and complex real-world rules. Recently, the language model (LM)-based rule generation are proposed to enhance the expressive power of the rules. In this paper, we revisit the differences between KB-based rule induction and LM-based rule generation. We argue that, while KB-based methods inducted rules by discovering data commonalities, the current LM-based methods are "learning rules from rules". This limits these methods to only produce "canned" rules whose patterns are constrained by the annotated rules, while discarding the rich expressive power of LMs for free text. Therefore, in this paper, we propose the open rule induction problem, which aims to induce open rules utilizing the knowledge in LMs. Besides, we propose the Orion (\underline{o}pen \underline{r}ule \underline{i}nducti\underline{on}) system to automatically mine open rules from LMs without supervision of annotated rules. We conducted extensive experiments to verify the quality and quantity of the inducted open rules. Surprisingly, when applying the open rules in downstream tasks (i.e. relation extraction), these automatically inducted rules even outperformed the manually annotated rules.
Vaccine skepticism detection by network embedding
Venue: arXiv cs.SI 2021
Year: 2021

We demonstrate the applicability of network embedding to vaccine skepticism, a controversial topic of long-past history. With the Covid-19 pandemic outbreak at the end of 2019, the topic is more important than ever. Only a year after the first international cases were registered, multiple vaccines were developed and passed clinical testing. Besides the challenges of development, testing, and logistics, another factor that might play a significant role in the fight against the pandemic are people who are hesitant to get vaccinated, or even state that they will refuse any vaccine offered to them. Two groups of people commonly referred to as a) pro-vaxxer, those who support vaccinating people b) vax-skeptic, those who question vaccine efficacy or the need for general vaccination against Covid-19. It is very difficult to tell exactly how many people share each of these views. It is even more difficult to understand all the reasoning why vax-skeptic opinions are getting more popular. In this work, our intention was to develop techniques that are able to efficiently differentiate between pro-vaxxer and vax-skeptic content. After multiple data preprocessing steps, we analyzed the tweet text as well as the structure of user interactions on Twitter. We deployed several node embedding and community detection models that scale well for graphs with millions of edges.
A Pipeline for Graph-Based Monitoring of the Changes in the Information Space of Russian Social Media during the Lockdown
Venue: arXiv cs.SI 2021
Year: 2021

With the COVID-19 outbreak and the subsequent lockdown, social media became a vital communication tool. The sudden outburst of online activity influenced information spread and consumption patterns. It increases the relevance of studying the dynamics of social networks and developing data processing pipelines that allow a comprehensive analysis of social media data in the temporal dimension. This paper scopes the weekly dynamics of the information space represented by Russian social media (Twitter and Livejournal) during a critical period (massive COVID-19 outbreak and first governmental measures). The approach is twofold: a) build the time series of topic similarity indicators by identifying COVID-related topics in each week and measuring user contribution to the topic space, and b) cluster user activity and display user-topic relationships on graphs in a dashboard application. The paper describes the development of the pipeline, explains the choices made and provides a case study of the adaptation to virus control measures. The results confirm that social processes and behaviour in response to pandemic-triggered changes can be successfully traced in social media. Moreover, the adaptation trends revealed by psychological and sociological studies are reflected in our data and can be explored using the proposed method.
s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning
Venue: arXiv cs.CL 2021
Year: 2021

Pretrained bidirectional Transformers, such as BERT, have achieved significant improvements in a wide variety of language understanding tasks, while it is not straightforward to directly apply them for natural language generation. In this paper, we present a sequence-to-sequence fine-tuning toolkit s2s-ft, which adopts pretrained Transformers for conditional generation tasks. Inspired by UniLM, we implement three sequence-to-sequence fine-tuning algorithms, namely, causal fine-tuning, masked fine-tuning, and pseudo-masked fine-tuning. By leveraging the existing pretrained bidirectional Transformers, experimental results show that s2s-ft achieves strong performance on several benchmarks of abstractive summarization, and question generation. Moreover, we demonstrate that the package s2s-ft supports both monolingual and multilingual NLG tasks. The s2s-ft toolkit is available at https://github.com/microsoft/unilm/tree/master/s2s-ft.