OpenRaiser/Intern-Atlas huggingface dataset:
RoMA: a Method for Neural Network Robustness Measurement and Assessment
Venue: arXiv cs.LG 2022
Year: 2022

Neural network models have become the leading solution for a large variety of tasks, such as classification, language processing, protein folding, and others. However, their reliability is heavily plagued by adversarial inputs: small input perturbations that cause the model to produce erroneous outputs. Adversarial inputs can occur naturally when the system's environment behaves randomly, even in the absence of a malicious adversary, and are a severe cause for concern when attempting to deploy neural networks within critical systems. In this paper, we present a new statistical method, called Robustness Measurement and Assessment (RoMA), which can measure the expected robustness of a neural network model. Specifically, RoMA determines the probability that a random input perturbation might cause misclassification. The method allows us to provide formal guarantees regarding the expected frequency of errors that a trained model will encounter after deployment. Our approach can be applied to large-scale, black-box neural networks, which is a significant advantage compared to recently proposed verification methods. We apply our approach in two ways: comparing the robustness of different models, and measuring how a model's robustness is affected by the magnitude of input perturbation. One interesting insight obtained through this work is that, in a classification network, different output labels can exhibit very different robustness levels. We term this phenomenon categorial robustness. Our ability to perform risk and robustness assessments on a categorial basis opens the door to risk mitigation, which may prove to be a significant step towards neural network certification in safety-critical applications.
Intelligent Reflecting Surface for Multi-Path Beam Routing with Active/Passive Beam Splitting and Combining
Venue: arXiv cs.IT 2022
Year: 2022

Intelligent reflecting surface (IRS) can be densely deployed in wireless networks to significantly enhance the communication channels. In this letter, we consider the downlink transmission from a multi-antenna base station (BS) to a single-antenna user, by exploiting the cooperative passive beamforming (CPB) and line-of-sight (LoS) path diversity gains of multi-IRS signal reflection. Unlike existing works where only one single multi-IRS reflection path from the BS to user is selected, we propose a new and more general multi-path beam routing scheme. Specifically, the BS sends the user's information signal via multiple orthogonal active beams (termed as active beam splitting), which point towards different IRSs. Then, these beamed signals are subsequently reflected by selected IRSs via their CPB in different paths, and finally coherently combined at the user's receiver (thus named {\it \textbf{passive beam combining}}). For this scheme, we formulate a new multi-path beam routing design problem to jointly optimize the number of IRS reflection paths, the selected IRSs for each of the reflection paths, the active/passive beamforming at the BS/each selected IRS, as well as the BS's power allocation over different active beams, so as to maximize the received signal power at the user. To solve this challenging problem, we first derive the optimal BS/IRS beamforming and BS power allocation for a given set of reflection paths. The clique-based approach in graph theory is then applied to solve the remaining multi-path selection problem efficiently. Simulation results show that our proposed multi-path beam routing scheme significantly outperforms its conventional single-path beam routing special case.
Detecting Anti-Vaccine Users on Twitter
Venue: arXiv cs.SI 2022
Year: 2022

Vaccine hesitancy, which has recently been driven by online narratives, significantly degrades the efficacy of vaccination strategies, such as those for COVID-19. Despite broad agreement in the medical community about the safety and efficacy of available vaccines, a large number of social media users continue to be inundated with false information about vaccines and are indecisive or unwilling to be vaccinated. The goal of this study is to better understand anti-vaccine sentiment by developing a system capable of automatically identifying the users responsible for spreading anti-vaccine narratives. We introduce a publicly available Python package capable of analyzing Twitter profiles to assess how likely that profile is to share anti-vaccine sentiment in the future. The software package is built using text embedding methods, neural networks, and automated dataset generation and is trained on several million tweets. We find this model can accurately detect anti-vaccine users up to a year before they tweet anti-vaccine hashtags or keywords. We also show examples of how text analysis helps us understand anti-vaccine discussions by detecting moral and emotional differences between anti-vaccine spreaders on Twitter and regular users. Our results will help researchers and policy-makers understand how users become anti-vaccine and what they discuss on Twitter. Policy-makers can utilize this information for better targeted campaigns that debunk harmful anti-vaccination myths.
SABMIS: Sparse approximation based blind multi-image steganography scheme
Venue: arXiv cs.CR 2022
Year: 2022

We hide grayscale secret images into a grayscale cover image, which is considered to be a challenging steganography problem. Our goal is to develop a steganography scheme with enhanced embedding capacity while preserving the visual quality of the stego-image as well as the extracted secret image, and ensuring that the stego-image is resistant to steganographic attacks. The novel embedding rule of our scheme helps to hide secret image sparse coefficients into the oversampled cover image sparse coefficients in a staggered manner. The stego-image is constructed by using ADMM to solve the LASSO formulation of the underlying minimization problem. Finally, the secret images are extracted from the constructed stego-image using the reverse of our embedding rule. Using these components together, to achieve the above mentioned competing goals, forms our most novel contribution. We term our scheme SABMIS (Sparse Approximation Blind Multi-Image Steganography). We perform extensive experiments on several standard images. By choosing the size of the secret images to be half of the of cover image, we obtain embedding capacities of 2 bpp (bits per pixel), 4 bpp, 6 bpp, and 8 bpp while embedding one, two, three, and four secret images, respectively. Our focus is on hiding multiple secret images. For the case of hiding two and three secret images, our embedding capacities are higher than all the embedding capacities obtained in the literature until now. For the case of hiding four secret images, although our capacity is slightly lower than one work, we do better on the other two goals; a) very little deterioration in the quality of the stego-images and extracted secret images, and b) inherently and designed-to-be resistant to steganographic attacks. Additionally, we demonstrate that SABMIS executes in few minutes, and show its application on two real-life problems.
Multiwave COVID-19 Prediction from Social Awareness using Web Search and Mobility Data
Venue: arXiv cs.SI 2022
Year: 2022

Recurring outbreaks of COVID-19 have posed enduring effects on global society, which calls for a predictor of pandemic waves using various data with early availability. Existing prediction models that forecast the first outbreak wave using mobility data may not be applicable to the multiwave prediction, because the evidence in the USA and Japan has shown that mobility patterns across different waves exhibit varying relationships with fluctuations in infection cases. Therefore, to predict the multiwave pandemic, we propose a Social Awareness-Based Graph Neural Network (SAB-GNN) that considers the decay of symptom-related web search frequency to capture the changes in public awareness across multiple waves. Our model combines GNN and LSTM to model the complex relationships among urban districts, inter-district mobility patterns, web search history, and future COVID-19 infections. We train our model to predict future pandemic outbreaks in the Tokyo area using its mobility and web search data from April 2020 to May 2021 across four pandemic waves collected by Yahoo Japan Corporation under strict privacy protection rules. Results demonstrate our model outperforms state-of-the-art baselines such as ST-GNN, MPNN, and GraphLSTM. Though our model is not computationally expensive (only 3 layers and 10 hidden neurons), the proposed model enables public agencies to anticipate and prepare for future pandemic outbreaks.
Grounding force-directed network layouts with latent space models
Venue: arXiv cs.SI 2022
Year: 2022

Force-directed layout algorithms are ubiquitously-used tools for network visualisation across a multitude of scientific disciplines. However, they lack theoretical grounding which allows to interpret their outcomes rigorously and can guide the choice of specific algorithms for certain data sets. We propose an approach building on latent space models, which assume that the probability of nodes forming a tie depends on their distance in an unobserved latent space. From such latent space models, we derive force equations for a force-directed layout algorithm. Since the forces infer positions which maximise the likelihood of the given network under the latent space model, the force-directed layout becomes interpretable. We implement these forces for unweighted and weighted networks and spatialise different real-world networks. Comparison to existing layout algorithms (not grounded in an interpretable model) reveals that node groups are placed in similar configurations, while said algorithms show a stronger intra-cluster separation of nodes, as well as a tendency to separate clusters more strongly in retweet networks. We also explore the possibility of visualising data traditionally not seen as network data, such as survey data.
Sinkformers: Transformers with Doubly Stochastic Attention
Venue: arXiv cs.LG 2022
Year: 2022

Attention based models such as Transformers involve pairwise interactions between data points, modeled with a learnable attention matrix. Importantly, this attention matrix is normalized with the SoftMax operator, which makes it row-wise stochastic. In this paper, we propose instead to use Sinkhorn's algorithm to make attention matrices doubly stochastic. We call the resulting model a Sinkformer. We show that the row-wise stochastic attention matrices in classical Transformers get close to doubly stochastic matrices as the number of epochs increases, justifying the use of Sinkhorn normalization as an informative prior. On the theoretical side, we show that, unlike the SoftMax operation, this normalization makes it possible to understand the iterations of self-attention modules as a discretized gradient-flow for the Wasserstein metric. We also show in the infinite number of samples limit that, when rescaling both attention matrices and depth, Sinkformers operate a heat diffusion. On the experimental side, we show that Sinkformers enhance model accuracy in vision and natural language processing tasks. In particular, on 3D shapes classification, Sinkformers lead to a significant improvement.
Voting algorithms for unique games on complete graphs
Venue: arXiv cs.DS 2022
Year: 2022

An approximation algorithm for a constraint satisfaction problem is called robust if it outputs an assignment satisfying a $(1 - f(ε))$-fraction of the constraints on any $(1-ε)$-satisfiable instance, where the loss function $f$ is such that $f(ε) \rightarrow 0$ as $ε\rightarrow 0$. Moreover, the runtime of a robust algorithm should not depend in any way on $ε$. In this paper, we present such an algorithm for Min-Unique-Games on complete graphs with $q$ labels. Specifically, the loss function is $f(ε) = (ε+ c_ε ε^2)$, where $c_ε$ is a constant depending on $ε$ such that $\lim_{ε\rightarrow 0} c_ε = 16$. The runtime of our algorithm is $O(qn^3)$ (with no dependence on $ε$) and can run in time $O(qn^2)$ using a randomized implementation with a slightly larger constant $c_ε$. Our algorithm is combinatorial and uses voting to find an assignment. It can furthermore be used to provide a PTAS for Min-Unique-Games on complete graphs, recovering a result of Karpinski and Schudy with a simpler algorithm and proof. We also prove NP-hardness for Min-Unique-Games on complete graphs and (using a randomized reduction) even in the case where the constraints form a cyclic permutation, which is also known as Min-Linear-Equations-mod-$q$ on complete graphs.
A Taxonomy for Inference in Causal Model Families
Venue: arXiv cs.LG 2022
Year: 2022

Neurally-parameterized Structural Causal Models in the Pearlian notion to causality, referred to as NCM, were recently introduced as a step towards next-generation learning systems. However, said NCM are only concerned with the learning aspect of causal inference but totally miss out on the architecture aspect. That is, actual causal inference within NCM is intractable in that the NCM won't return an answer to a query in polynomial time. This insight follows as corollary to the more general statement on the intractability of arbitrary SCM parameterizations, which we prove in this work through classical 3-SAT reduction. Since future learning algorithms will be required to deal with both high dimensional data and highly complex mechanisms governing the data, we ultimately believe work on tractable inference for causality to be decisive. We also show that not all ``causal'' models are created equal. More specifically, there are models capable of answering causal queries that are not SCM, which we refer to as \emph{partially causal models} (PCM). We provide a tabular taxonomy in terms of tractability properties for all of the different model families, namely correlation-based, PCM and SCM. To conclude our work, we also provide some initial ideas on how to overcome parts of the intractability of causal inference with SCM by showing an example of how parameterizing an SCM with SPN modules can at least allow for tractable mechanisms. We hope that our impossibility result alongside the taxonomy for tractability in causal models can raise awareness for this novel research direction since achieving success with causality in real world downstream tasks will not only depend on learning correct models as we also require having the practical ability to gain access to model inferences.
Bandits with Dynamic Arm-acquisition Costs
Venue: arXiv cs.LG 2022
Year: 2022

We consider a bandit problem where at any time, the decision maker can add new arms to her consideration set. A new arm is queried at a cost from an "arm-reservoir" containing finitely many "arm-types," each characterized by a distinct mean reward. The cost of query reflects in a diminishing probability of the returned arm being optimal, unbeknown to the decision maker; this feature encapsulates defining characteristics of a broad class of operations-inspired online learning problems, e.g., those arising in markets with churn, or those involving allocations subject to costly resource acquisition. The decision maker's goal is to maximize her cumulative expected payoffs over a sequence of n pulls, oblivious to the statistical properties as well as types of the queried arms. We study two natural modes of endogeneity in the reservoir distribution, and characterize a necessary condition for achievability of sub-linear regret in the problem. We also discuss a UCB-inspired adaptive algorithm that is long-run-average optimal whenever said condition is satisfied, thereby establishing its tightness.
Cybersecurity Misinformation Detection on Social Media: Case Studies on Phishing Reports and Zoom's Threats
Venue: arXiv cs.CY 2022
Year: 2022

Prior work has extensively studied misinformation related to news, politics, and health, however, misinformation can also be about technological topics. While less controversial, such misinformation can severely impact companies' reputations and revenues, and users' online experiences. Recently, social media has also been increasingly used as a novel source of knowledgebase for extracting timely and relevant security threats, which are fed to the threat intelligence systems for better performance. However, with possible campaigns spreading false security threats, these systems can become vulnerable to poisoning attacks. In this work, we proposed novel approaches for detecting misinformation about cybersecurity and privacy threats on social media, focusing on two topics with different types of misinformation: phishing websites and Zoom's security & privacy threats. We developed a framework for detecting inaccurate phishing claims on Twitter. Using this framework, we could label about 9% of URLs and 22% of phishing reports as misinformation. We also proposed another framework for detecting misinformation related to Zoom's security and privacy threats on multiple platforms. Our classifiers showed great performance with more than 98% accuracy. Employing these classifiers on the posts from Facebook, Instagram, Reddit, and Twitter, we found respectively that about 18%, 3%, 4%, and 3% of posts were misinformation. In addition, we studied the characteristics of misinformation posts, their authors, and their timelines, which helped us identify campaigns.
New Bounds for the Flock-of-Birds Problem
Venue: arXiv cs.DC 2022
Year: 2022

In this paper, we continue a line of work on obtaining succinct population protocols for Presburger-definable predicates. More specifically, we focus on threshold predicates. These are predicates of the form $n\ge d$, where $n$ is a free variable and $d$ is a constant. For every $d$, we establish a 1-aware population protocol for this predicate with $\log_2 d + \min\{e, z\} + O(1)$ states, where $e$ (resp., $z$) is the number of $1$'s (resp., $0$'s) in the binary representation of $d$ (resp., $d - 1$). This improves upon an upper bound $4\log_2 d + O(1)$ due to Blondin et al. We also show that any 1-aware protocol for our problem must have at least $\log_2(d)$ states. This improves upon a lower bound $\log_3 d$ due to Blondin et al.
A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysis
Venue: arXiv cs.SE 2022
Year: 2022

Logs are widely used to record runtime information of software systems, such as the timestamp and the importance of an event, the unique ID of the source of the log, and a part of the state of a task's execution. The rich information of logs enables system developers (and operators) to monitor the runtime behaviors of their systems and further track down system problems and perform analysis on log data in production settings. However, the prior research on utilizing logs is scattered and that limits the ability of new researchers in this field to quickly get to the speed and hampers currently active researchers to advance this field further. Therefore, this paper surveys and provides a systematic literature review and mapping of the contemporary logging practices and log statements' mining and monitoring techniques and their applications such as in system failure detection and diagnosis. We study a large number of conference and journal papers that appeared on top-level peer-reviewed venues. Additionally, we draw high-level trends of ongoing research and categorize publications into subdivisions. In the end, and based on our holistic observations during this survey, we provide a set of challenges and opportunities that will lead the researchers in academia and industry in moving the field forward.
Recommender Systems meet Mechanism Design
Venue: arXiv cs.GT 2022
Year: 2022

Machine learning has developed a variety of tools for learning and representing high-dimensional distributions with structure. Recent years have also seen big advances in designing multi-item mechanisms. Akin to overfitting, however, these mechanisms can be extremely sensitive to the Bayesian prior that they target, which becomes problematic when that prior is only approximately known. At the same time, even if access to the exact Bayesian prior is given, it is known that optimal or even approximately optimal multi-item mechanisms run into sample, computational, representation and communication intractability barriers. We consider a natural class of multi-item mechanism design problems with very large numbers of items, but where the bidders' value distributions can be well-approximated by a topic model akin to those used in recommendation systems with very large numbers of possible recommendations. We propose a mechanism design framework for this setting, building on a recent robustification framework by Brustle et al., which disentangles the statistical challenge of estimating a multi-dimensional prior from the task of designing a good mechanism for it, and robustifies the performance of the latter against the estimation error of the former. We provide an extension of this framework appropriate for our setting, which allows us to exploit the expressive power of topic models to reduce the effective dimensionality of the mechanism design problem and remove the dependence of its computational, communication and representation complexity on the number of items.
Fast Gradient Non-sign Methods
Venue: arXiv cs.CV 2022
Year: 2022

Adversarial attacks make their success in DNNs, and among them, gradient-based algorithms become one of the mainstreams. Based on the linearity hypothesis, under $\ell_\infty$ constraint, $sign$ operation applied to the gradients is a good choice for generating perturbations. However, side-effects from such operation exist since it leads to the bias of direction between real gradients and perturbations. In other words, current methods contain a gap between real gradients and actual noises, which leads to biased and inefficient attacks. Therefore in this paper, based on the Taylor expansion, the bias is analyzed theoretically, and the correction of $sign$, i.e., Fast Gradient Non-sign Method (FGNM), is further proposed. Notably, FGNM is a general routine that seamlessly replaces the conventional $sign$ operation in gradient-based attacks with negligible extra computational cost. Extensive experiments demonstrate the effectiveness of our methods. Specifically, for untargeted black-box attacks, ours outperform them by 27.5% at most and 9.5% on average. For targeted attacks against defense models, it is 15.1% and 12.7%. Our anonymous code is publicly available at https://github.com/yaya-cheng/FGNM
Planning for Risk-Aversion and Expected Value in MDPs
Venue: arXiv cs.AI 2022
Year: 2022

Planning in Markov decision processes (MDPs) typically optimises the expected cost. However, optimising the expectation does not consider the risk that for any given run of the MDP, the total cost received may be unacceptably high. An alternative approach is to find a policy which optimises a risk-averse objective such as conditional value at risk (CVaR). However, optimising the CVaR alone may result in poor performance in expectation. In this work, we begin by showing that there can be multiple policies which obtain the optimal CVaR. This motivates us to propose a lexicographic approach which minimises the expected cost subject to the constraint that the CVaR of the total cost is optimal. We present an algorithm for this problem and evaluate our approach on four domains. Our results demonstrate that our lexicographic approach improves the expected cost compared to the state of the art algorithm, while achieving the optimal CVaR.