OpenRaiser/Intern-Atlas huggingface dataset:
Natural Reweighted Wake-Sleep
Venue: arXiv cs.LG 2022
Year: 2022

Helmholtz Machines (HMs) are a class of generative models composed of two Sigmoid Belief Networks (SBNs), acting respectively as an encoder and a decoder. These models are commonly trained using a two-step optimization algorithm called Wake-Sleep (WS) and more recently by improved versions, such as Reweighted Wake-Sleep (RWS) and Bidirectional Helmholtz Machines (BiHM). The locality of the connections in an SBN induces sparsity in the Fisher Information Matrices associated to the probabilistic models, in the form of a finely-grained block-diagonal structure. In this paper we exploit this property to efficiently train SBNs and HMs using the natural gradient. We present a novel algorithm, called Natural Reweighted Wake-Sleep (NRWS), that corresponds to the geometric adaptation of its standard version. In a similar manner, we also introduce Natural Bidirectional Helmholtz Machine (NBiHM). Differently from previous work, we will show how for HMs the natural gradient can be efficiently computed without the need of introducing any approximation in the structure of the Fisher information matrix. The experiments performed on standard datasets from the literature show a consistent improvement of NRWS and NBiHM not only with respect to their non-geometric baselines but also with respect to state-of-the-art training algorithms for HMs. The improvement is quantified both in terms of speed of convergence as well as value of the log-likelihood reached after training.
A Review on Drivers Red Light Running Behavior Predictions and Technology Based Countermeasures
Venue: arXiv cs.AI 2022
Year: 2022

Red light running at signalised intersections is a growing road safety issue worldwide, leading to the rapid development of advanced intelligent transportation technologies and countermeasures. However, existing studies have yet to summarise and present the effect of these technology based innovations in improving safety. This paper represents a comprehensive review of red light running behaviour prediction methodologies and technology-based countermeasures. Specifically, the major focus of this study is to provide a comprehensive review on two streams of literature targeting red light running and stop and go behaviour at signalised intersection (1) studies focusing on modelling and predicting the red light running and stop and go related driver behaviour and (2) studies focusing on the effectiveness of different technology based countermeasures which combat such unsafe behaviour. The study provides a systematic guide to assist researchers and stakeholders in understanding how to best identify red light running and stop and go associated driving behaviour and subsequently implement countermeasures to combat such risky behaviour and improve the associated safety.
Superlinear Lower Bounds Based on ETH
Venue: arXiv cs.CC 2022
Year: 2022

We introduce techniques for proving superlinear conditional lower bounds for polynomial time problems. In particular, we show that CircuitSAT for circuits with m gates and log(m) inputs (denoted by log-CircuitSAT) is not decidable in essentially-linear time unless the exponential time hypothesis (ETH) is false and k-Clique is decidable in essentially-linear time in terms of the graph's size for all fixed k. Such conditional lower bounds have previously only been demonstrated relative to the strong exponential time hypothesis (SETH). Our results therefore offer significant progress towards proving unconditional superlinear time complexity lower bounds for natural problems in polynomial time.
Utility-based Resource Allocation and Pricing for Serverless Computing
Venue: arXiv cs.DC 2022
Year: 2022

Serverless computing platforms currently rely on basic pricing schemes that are static and do not reflect customer feedback. This leads to significant inefficiencies from a total utility perspective. As one of the fastest-growing cloud services, serverless computing provides an opportunity to better serve both users and providers through the incorporation of market-based strategies for pricing and resource allocation. With the help of utility functions to model the delay-sensitivity of customers, we propose a novel scheduler to allocate resources for serverless computing. The resulting resource allocation scheme is optimal in the sense that it maximizes the aggregate utility of all users across the system, thus maximizing social welfare. Our approach gives rise to a natural dynamic pricing scheme that is obtained by solving an optimization problem in its dual form. We further develop feedback mechanisms that allow the cloud provider to converge to optimal resource allocation, even when the users' utilities are private and unknown to the service provider. Simulations show that our approach can track market demand and achieve significantly higher social welfare (or, equivalently, cost savings for customers) compared to existing schemes.
Solving Problems on Generalized Convex Graphs via Mim-Width
Venue: arXiv cs.DS 2022
Year: 2022

A bipartite graph $G=(A,B,E)$ is ${\cal H}$-convex, for some family of graphs ${\cal H}$, if there exists a graph $H\in {\cal H}$ with $V(H)=A$ such that the set of neighbours in $A$ of each $b\in B$ induces a connected subgraph of $H$. Many $\mathsf{NP}$-complete problems, including problems such as Dominating Set, Feedback Vertex Set, Induced Matching and List $k$-Colouring, become polynomial-time solvable for ${\mathcal H}$-convex graphs when ${\mathcal H}$ is the set of paths. In this case, the class of ${\mathcal H}$-convex graphs is known as the class of convex graphs. The underlying reason is that the class of convex graphs has bounded mim-width. We extend the latter result to families of ${\mathcal H}$-convex graphs where (i) ${\mathcal H}$ is the set of cycles, or (ii) ${\mathcal H}$ is the set of trees with bounded maximum degree and a bounded number of vertices of degree at least $3$. As a consequence, we can re-prove and strengthen a large number of results on generalized convex graphs known in the literature. To complement result (ii), we show that the mim-width of ${\mathcal H}$-convex graphs is unbounded if ${\mathcal H}$ is the set of trees with arbitrarily large maximum degree or an arbitrarily large number of vertices of degree at least $3$. In this way we are able to determine complexity dichotomies for the aforementioned graph problems. Afterwards we perform a more refined width-parameter analysis, which shows even more clearly which width parameters are bounded for classes of ${\cal H}$-convex graphs.
DwarvesGraph: A High-Performance Graph Mining System with Pattern Decomposition
Venue: arXiv cs.DC 2022
Year: 2022

Graph pattern mining (GPM) is an important application that identifies structures from graphs. Despite the recent progress, the performance gap between the state-of-the-art GPM systems and an efficient algorithm--pattern decomposition--is still at least an order of magnitude. This paper clears the fundamental obstacles of adopting pattern decomposition to a GPM system. First, the performance of pattern decomposition algorithms depends on how to decompose the whole pattern into subpatterns. The original method performs complexity analysis of algorithms for different choices, and selects the one with the lowest complexity upper bound. Clearly, this approach is not feasible for average or even expert users. To solve this problem, we develop a GPM compiler with conventional and GPM-specific optimizations to generate algorithms for different decomposition choices, which are evaluated based on an accurate cost model. The executable of the GPM task is obtained from the algorithm with the best performance. Second, we propose a novel partial-embedding API that is sufficient to construct advanced GPM applications while preserving pattern decomposition algorithm advantages. Compared to state-of-the-art systems, our new GPM system, DwarvesGraph, developed based on the ideas, reduces the execution time of GPM on large graphs and patterns from days to a few hours with low programming effort.
A Survey on Evolutionary Neural Architecture Search
Venue: arXiv cs.NE 2022
Year: 2022

Deep Neural Networks (DNNs) have achieved great success in many applications. The architectures of DNNs play a crucial role in their performance, which is usually manually designed with rich expertise. However, such a design process is labour intensive because of the trial-and-error process, and also not easy to realize due to the rare expertise in practice. Neural Architecture Search (NAS) is a type of technology that can design the architectures automatically. Among different methods to realize NAS, Evolutionary Computation (EC) methods have recently gained much attention and success. Unfortunately, there has not yet been a comprehensive summary of the EC-based NAS algorithms. This paper reviews over 200 papers of most recent EC-based NAS methods in light of the core components, to systematically discuss their design principles as well as justifications on the design. Furthermore, current challenges and issues are also discussed to identify future research in this emerging field.
Robust Character Labeling in Movie Videos: Data Resources and Self-supervised Feature Adaptation
Venue: arXiv cs.CV 2022
Year: 2022

Robust face clustering is a vital step in enabling computational understanding of visual character portrayal in media. Face clustering for long-form content is challenging because of variations in appearance and lack of supporting large-scale labeled data. Our work in this paper focuses on two key aspects of this problem: the lack of domain-specific training or benchmark datasets, and adapting face embeddings learned on web images to long-form content, specifically movies. First, we present a dataset of over 169,000 face tracks curated from 240 Hollywood movies with weak labels on whether a pair of face tracks belong to the same or a different character. We propose an offline algorithm based on nearest-neighbor search in the embedding space to mine hard-examples from these tracks. We then investigate triplet-loss and multiview correlation-based methods for adapting face embeddings to hard-examples. Our experimental results highlight the usefulness of weakly labeled data for domain-specific feature adaptation. Overall, we find that multiview correlation-based adaptation yields more discriminative and robust face embeddings. Its performance on downstream face verification and clustering tasks is comparable to that of the state-of-the-art results in this domain. We also present the SAIL-Movie Character Benchmark corpus developed to augment existing benchmarks. It consists of racially diverse actors and provides face-quality labels for subsequent error analysis. We hope that the large-scale datasets developed in this work can further advance automatic character labeling in videos. All resources are available freely at https://sail.usc.edu/~ccmi/multiface.
Surrogate Model For Field Optimization Using Beta-VAE Based Regression
Venue: arXiv cs.LG 2022
Year: 2022

Oilfield development related decisions are made using reservoir simulation-based optimization study in which different production scenarios and well controls are compared. Such simulations are computationally expensive and so surrogate models are used to accelerate studies. Deep learning has been used in past to generate surrogates, but such models often fail to quantify prediction uncertainty and are not interpretable. In this work, beta-VAE based regression is proposed to generate simulation surrogates for use in optimization workflow. beta-VAE enables interpretable, factorized representation of decision variables in latent space, which is then further used for regression. Probabilistic dense layers are used to quantify prediction uncertainty and enable approximate Bayesian inference. Surrogate model developed using beta-VAE based regression finds interpretable and relevant latent representation. A reasonable value of beta ensures a good balance between factor disentanglement and reconstruction. Probabilistic dense layer helps in quantifying predicted uncertainty for objective function, which is then used to decide whether full-physics simulation is required for a case.
A Bidirectional Tree Tagging Scheme for Joint Medical Relation Extraction
Venue: arXiv cs.CL 2022
Year: 2022

Joint medical relation extraction refers to extracting triples, composed of entities and relations, from the medical text with a single model. One of the solutions is to convert this task into a sequential tagging task. However, in the existing works, the methods of representing and tagging the triples in a linear way failed to the overlapping triples, and the methods of organizing the triples as a graph faced the challenge of large computational effort. In this paper, inspired by the tree-like relation structures in the medical text, we propose a novel scheme called Bidirectional Tree Tagging (BiTT) to form the medical relation triples into two two binary trees and convert the trees into a word-level tags sequence. Based on BiTT scheme, we develop a joint relation extraction model to predict the BiTT tags and further extract medical triples efficiently. Our model outperforms the best baselines by 2.0\% and 2.5\% in F1 score on two medical datasets. What's more, the models with our BiTT scheme also obtain promising results in three public datasets of other domains.
Neural Topic Model via Optimal Transport
Venue: arXiv cs.IR 2022
Year: 2022

Recently, Neural Topic Models (NTMs) inspired by variational autoencoders have obtained increasingly research interest due to their promising results on text analysis. However, it is usually hard for existing NTMs to achieve good document representation and coherent/diverse topics at the same time. Moreover, they often degrade their performance severely on short documents. The requirement of reparameterisation could also comprise their training quality and model flexibility. To address these shortcomings, we present a new neural topic model via the theory of optimal transport (OT). Specifically, we propose to learn the topic distribution of a document by directly minimising its OT distance to the document's word distributions. Importantly, the cost matrix of the OT distance models the weights between topics and words, which is constructed by the distances between topics and words in an embedding space. Our proposed model can be trained efficiently with a differentiable loss. Extensive experiments show that our framework significantly outperforms the state-of-the-art NTMs on discovering more coherent and diverse topics and deriving better document representations for both regular and short texts.
Precoding and Scheduling for AoI Minimization in MIMO Broadcast Channels
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we consider a status updating system where updates are generated at a constant rate at $K$ sources and sent to the corresponding recipients through a noise-free broadcast channel. We assume that perfect channel state information (CSI) is available at the transmitter before each transmission, and the transmitter is able to utilize the CSI information to precode the updates. Our object is to design optimal precoding schemes to minimize the summed average \emph{age of information} (AoI) at the recipients. Under various assumptions on the size of each update $B$, the number of transmit antennas $M$, and the number of receive antennas $N$ at each user, this paper identifies the corresponding age-optimal precoding and transmission scheduling strategies. Specifically, for the case when $N=1$, a round-robin based updating scheme is shown to be optimal. For the two-user systems with $N>B$ or $M\notin[N:2N]$, framed updating schemes are proven to be optimal. For other cases in the two-user systems, a framed alternating updating scheme is proven to be $2$-optimal.
Twitter Corpus of the #BlackLivesMatter Movement And Counter Protests: 2013 to 2021
Venue: arXiv cs.SI 2022
Year: 2022

Black Lives Matter (BLM) is a decentralized social movement protesting violence against Black individuals and communities, with a focus on police brutality. The movement gained significant attention following the killings of Ahmaud Arbery, Breonna Taylor, and George Floyd in 2020. The #BlackLivesMatter social media hashtag has come to represent the grassroots movement, with similar hashtags counter protesting the BLM movement, such as #AllLivesMatter, and #BlueLivesMatter. We introduce a data set of 63.9 million tweets from 13.0 million users from over 100 countries which contain one of the following keywords: BlackLivesMatter, AllLivesMatter, and BlueLivesMatter. This data set contains all currently available tweets from the beginning of the BLM movement in 2013 to 2021. We summarize the data set and show temporal trends in use of both the BlackLivesMatter keyword and keywords associated with counter movements. Additionally, for each keyword, we create and release a set of Latent Dirichlet Allocation (LDA) topics (i.e., automatically clustered groups of semantically co-occuring words) to aid researchers in identifying linguistic patterns across the three keywords.
Learning to summarize from human feedback
Venue: arXiv cs.CL 2022
Year: 2022

As language models become more powerful, training and evaluation are increasingly bottlenecked by the data and metrics used for a particular task. For example, summarization models are often trained to predict human reference summaries and evaluated using ROUGE, but both of these metrics are rough proxies for what we really care about -- summary quality. In this work, we show that it is possible to significantly improve summary quality by training a model to optimize for human preferences. We collect a large, high-quality dataset of human comparisons between summaries, train a model to predict the human-preferred summary, and use that model as a reward function to fine-tune a summarization policy using reinforcement learning. We apply our method to a version of the TL;DR dataset of Reddit posts and find that our models significantly outperform both human reference summaries and much larger models fine-tuned with supervised learning alone. Our models also transfer to CNN/DM news articles, producing summaries nearly as good as the human reference without any news-specific fine-tuning. We conduct extensive analyses to understand our human feedback dataset and fine-tuned models We establish that our reward model generalizes to new datasets, and that optimizing our reward model results in better summaries than optimizing ROUGE according to humans. We hope the evidence from our paper motivates machine learning researchers to pay closer attention to how their training loss affects the model behavior they actually want.
Local Fast Rerouting with Low Congestion: A Randomized Approach
Venue: arXiv cs.NI 2022
Year: 2022

Most modern communication networks include fast rerouting mechanisms, implemented entirely in the data plane, to quickly recover connectivity after link failures. By relying on local failure information only, these data plane mechanisms provide very fast reaction times, but at the same time introduce an algorithmic challenge in case of multiple link failures: failover routes need to be robust to additional but locally unknown failures downstream. This paper presents local fast rerouting algorithms which not only provide a high degree of resilience against multiple link failures, but also ensure a low congestion on the resulting failover paths. We consider a randomized approach and focus on networks which are highly connected before the failures occur. Our main contribution are three simple algorithms which come with provable guarantees and provide interesting resilience-load tradeoffs, significantly outperforming any deterministic fast rerouting algorithm with high probability.
Controlling Fake News by Tagging: A Branching Process Analysis
Venue: arXiv cs.SI 2022
Year: 2022

The spread of fake news on online social networks (OSNs) has become a matter of concern. These platforms are also used for propagating important authentic information. Thus, there is a need for mitigating fake news without significantly influencing the spread of real news. We leverage users' inherent capabilities of identifying fake news and propose a warning-based control mechanism to curb this spread. Warnings are based on previous users' responses that indicate the authenticity of the news. We use population-size dependent continuous-time multi-type branching processes to describe the spreading under the warning mechanism. We also have new results towards these branching processes. The (time) asymptotic proportions of the individual populations are derived using stochastic approximation tools. Using these, relevant type 1, type 2 performances are derived and an appropriate optimization problem is solved. The proposed mechanism effectively controls fake news, with negligible influence on the propagation of authentic news. We validate performance measures using Monte Carlo simulations on network connections provided by Twitter data.
One-Clock Priced Timed Games with Negative Weights
Venue: arXiv cs.GT 2022
Year: 2022

Priced timed games are two-player zero-sum games played on priced timed automata (whose locations and transitions are labeled by weights modelling the cost of spending time in a state and executing an action, respectively). The goals of the players are to minimise and maximise the cost to reach a target location, respectively. We consider priced timed games with one clock and arbitrary integer weights and show that, for an important subclass of them (the so-called simple priced timed games), one can compute, in pseudo-polynomial time, the optimal values that the players can achieve, with their associated optimal strategies. As side results, we also show that one-clock priced timed games are determined and that we can use our result on simple priced timed games to solve the more general class of so-called negative-reset-acyclic priced timed games (with arbitrary integer weights and one clock). The decidability status of the full class of priced timed games with one-clock and arbitrary integer weights still remains open.
Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
Venue: arXiv cs.CR 2022
Year: 2022

Federated Learning (FL) allows multiple participants to train machine learning models collaboratively by keeping their datasets local while only exchanging model updates. Alas, this is not necessarily free from privacy and robustness vulnerabilities, e.g., via membership, property, and backdoor attacks. This paper investigates whether and to what extent one can use differential Privacy (DP) to protect both privacy and robustness in FL. To this end, we present a first-of-its-kind evaluation of Local and Central Differential Privacy (LDP/CDP) techniques in FL, assessing their feasibility and effectiveness. Our experiments show that both DP variants do d fend against backdoor attacks, albeit with varying levels of protection-utility trade-offs, but anyway more effectively than other robustness defenses. DP also mitigates white-box membership inference attacks in FL, and our work is the first to show it empirically. Neither LDP nor CDP, however, defend against property inference. Overall, our work provides a comprehensive, re-usable measurement methodology to quantify the trade-offs between robustness/privacy and utility in differentially private FL.