OpenRaiser/Intern-Atlas huggingface dataset:
Time-based Repair for Asynchronous Wait Flaky Tests in Web Testing
Venue: arXiv cs.SE 2023
Year: 2023

Asynchronous waits are one of the most prevalent root causes of flaky tests and a major time-influential factor of web application testing. To investigate the characteristics of asynchronous wait flaky tests and their fixes in web testing, we build a dataset of 49 reproducible flaky tests, from 26 open-source projects, caused by asynchronous waits, along with their corresponding developer-written fixes. Our study of these flaky tests reveals that in approximately 63% of them (31 out of 49), developers addressed Asynchronous Wait flaky tests by adapting the wait time, even for cases where the root causes lie elsewhere. Based on this finding, we propose TRaf, an automated time-based repair method for asynchronous wait flaky tests in web applications. TRaf tackles the flakiness issues by suggesting a proper waiting time for each asynchronous call in a web application, using code similarity and past change history. The core insight is that as developers often make similar mistakes more than once, hints for the efficient wait time exist in the current or past codebase. Our analysis shows that TRaf can suggest a shorter wait time to resolve the test flakiness compared to developer-written fixes, reducing the test execution time by 11.1%. With additional dynamic tuning of the new wait time, TRaf further reduces the execution time by 20.2%.
Depth-bounded epistemic logic
Venue: arXiv cs.LO 2023
Year: 2023

Epistemic logics model how agents reason about their beliefs and the beliefs of other agents. Existing logics typically assume the ability of agents to reason perfectly about propositions of unbounded modal depth. We present DBEL, an extension of S5 that models agents that can reason about epistemic formulas only up to a specific modal depth. To support explicit reasoning about agent depths, DBEL includes depth atoms $E_a^d$ (agent $a$ has depth exactly $d$) and $P_a^d$ (agent $a$ has depth at least $d$). We provide a sound and complete axiomatization of DBEL. We extend DBEL to support public announcements for bounded depth agents and show how the resulting DPAL logic generalizes standard axioms from public announcement logic. We present two alternate extensions and identify two undesirable properties, amnesia and knowledge leakage, that these extensions have but DPAL does not. We provide axiomatizations of these logics as well as complexity results for satisfiability and model checking. Finally, we use these logics to illustrate how agents with bounded modal depth reason in the classical muddy children problem, including upper and lower bounds on the depth knowledge necessary for agents to successfully solve the problem.
Size-stochastic Knapsack Online Contention Resolution Schemes
Venue: arXiv cs.GT 2023
Year: 2023

Online contention resolution schemes (OCRSs) are effective rounding techniques for online stochastic combinatorial optimization problems. These schemes randomly and sequentially round a fractional solution to a relaxed problem that can be formulated in advance. In this study, we propose OCRSs for online stochastic generalized assignment problems. In the problem of our OCRSs, sequentially arriving items are packed into a single knapsack, and their sizes are revealed only after insertion. The goal of the problem is to maximize the acceptance probability, which is the smallest probability among the items being placed in the knapsack. Since the item sizes are unknown beforehand, a capacity overflow may occur. We consider two distinct settings: the hard constraint, where items that cause overflow are rejected, and the soft constraint setting, where such items are accepted. Under the hard constraint setting, we present an algorithm with an acceptance probability of $1/3$ and prove that no algorithm can achieve an acceptance probability greater than $3/7$. Under the soft constraint setting, we propose an algorithm with an acceptance probability of $1/2$ and demonstrate that this is best possible.
MADDM: Multi-Advisor Dynamic Binary Decision-Making by Maximizing the Utility
Venue: arXiv cs.AI 2023
Year: 2023

Being able to infer ground truth from the responses of multiple imperfect advisors is a problem of crucial importance in many decision-making applications, such as lending, trading, investment, and crowd-sourcing. In practice, however, gathering answers from a set of advisors has a cost. Therefore, finding an advisor selection strategy that retrieves a reliable answer and maximizes the overall utility is a challenging problem. To address this problem, we propose a novel strategy for optimally selecting a set of advisers in a sequential binary decision-making setting, where multiple decisions need to be made over time. Crucially, we assume no access to ground truth and no prior knowledge about the reliability of advisers. Specifically, our approach considers how to simultaneously (1) select advisors by balancing the advisors' costs and the value of making correct decisions, (2) learn the trustworthiness of advisers dynamically without prior information by asking multiple advisers, and (3) make optimal decisions without access to the ground truth, improving this over time. We evaluate our algorithm through several numerical experiments. The results show that our approach outperforms two other methods that combine state-of-the-art models.
SEAL: A Strategy-Proof and Privacy-Preserving UAV Computation Offloading Framework
Venue: arXiv cs.GT 2023
Year: 2023

Due to the limited battery and computing resource, offloading unmanned aerial vehicles (UAVs)' computation tasks to ground infrastructure, e.g., vehicles, is a fundamental framework. Under such an open and untrusted environment, vehicles are reluctant to share their computing resource unless provisioning strong incentives, privacy protection, and fairness guarantee. Precisely, without strategy-proofness guarantee, the strategic vehicles can overclaim participation costs so as to conduct market manipulation. Without the fairness provision, vehicles can deliberately abort the assigned tasks without any punishments, and UAVs can refuse to pay by the end, causing an exchange dilemma. Lastly, the strategy-proofness and fairness provision typically require transparent payment/task results exchange under public audit, which may disclose sensitive information of vehicles and make the privacy preservation a foremost issue. To achieve the three design goals, we propose SEAL, an integrated framework to address strategy-proof, fair, and privacy-preserving UAV computation offloading. SEAL deploys a strategy-proof reverse combinatorial auction mechanism to optimize UAVs' task offloading under practical constraints while ensuring economic-robustness and polynomial-time efficiency. Based on smart contracts and hashchain micropayment, SEAL implements a fair on-chain exchange protocol to realize the atomic completion of batch payments and computing results in multi-round auctions. In addition, a privacy-preserving off-chain auction protocol is devised with the assistance of the trusted processor to efficiently protect vehicles' bid privacy. Using rigorous theoretical analysis and extensive simulations, we validate that SEAL can effectively prevent vehicles from manipulating, ensure privacy protection and fairness, improve the offloading efficiency.
Exploring In-Context Learning Capabilities of Foundation Models for Generating Knowledge Graphs from Text
Venue: arXiv cs.CL 2023
Year: 2023

Knowledge graphs can represent information about the real-world using entities and their relations in a structured and semantically rich manner and they enable a variety of downstream applications such as question-answering, recommendation systems, semantic search, and advanced analytics. However, at the moment, building a knowledge graph involves a lot of manual effort and thus hinders their application in some situations and the automation of this process might benefit especially for small organizations. Automatically generating structured knowledge graphs from a large volume of natural language is still a challenging task and the research on sub-tasks such as named entity extraction, relation extraction, entity and relation linking, and knowledge graph construction aims to improve the state of the art of automatic construction and completion of knowledge graphs from text. The recent advancement of foundation models with billions of parameters trained in a self-supervised manner with large volumes of training data that can be adapted to a variety of downstream tasks has helped to demonstrate high performance on a large range of Natural Language Processing (NLP) tasks. In this context, one emerging paradigm is in-context learning where a language model is used as it is with a prompt that provides instructions and some examples to perform a task without changing the parameters of the model using traditional approaches such as fine-tuning. This way, no computing resources are needed for re-training/fine-tuning the models and the engineering effort is minimal. Thus, it would be beneficial to utilize such capabilities for generating knowledge graphs from text.
Covariate-distance Weighted Regression (CWR): A Case Study for Estimation of House Prices
Venue: arXiv cs.LG 2023
Year: 2023

Geographically weighted regression (GWR) is a popular tool for modeling spatial heterogeneity in a regression model. However, the current weighting function used in GWR only considers the geographical distance, while the attribute similarity is totally ignored. In this study, we proposed a covariate weighting function that combines the geographical distance and attribute distance. The covariate-distance weighted regression (CWR) is the extension of GWR including geographical distance and attribute distance. House prices are affected by numerous factors, such as house age, floor area, and land use. Prediction model is used to help understand the characteristics of regional house prices. The CWR was used to understand the relationship between the house price and controlling factors. The CWR can consider the geological and attribute distances, and produce accurate estimates of house price that preserve the weight matrix for geological and attribute distance functions. Results show that the house attributes/conditions and the characteristics of the house, such as floor area and house age, might affect the house price. After factor selection, in which only house age and floor area of a building are considered, the RMSE of the CWR model can be improved by 2.9%-26.3% for skyscrapers when compared to the GWR. CWR can effectively reduce estimation errors from traditional spatial regression models and provide novel and feasible models for spatial estimation.
Denoising Diffusion Models for Plug-and-Play Image Restoration
Venue: arXiv cs.CV 2023
Year: 2023

Plug-and-play Image Restoration (IR) has been widely recognized as a flexible and interpretable method for solving various inverse problems by utilizing any off-the-shelf denoiser as the implicit image prior. However, most existing methods focus on discriminative Gaussian denoisers. Although diffusion models have shown impressive performance for high-quality image synthesis, their potential to serve as a generative denoiser prior to the plug-and-play IR methods remains to be further explored. While several other attempts have been made to adopt diffusion models for image restoration, they either fail to achieve satisfactory results or typically require an unacceptable number of Neural Function Evaluations (NFEs) during inference. This paper proposes DiffPIR, which integrates the traditional plug-and-play method into the diffusion sampling framework. Compared to plug-and-play IR methods that rely on discriminative Gaussian denoisers, DiffPIR is expected to inherit the generative ability of diffusion models. Experimental results on three representative IR tasks, including super-resolution, image deblurring, and inpainting, demonstrate that DiffPIR achieves state-of-the-art performance on both the FFHQ and ImageNet datasets in terms of reconstruction faithfulness and perceptual quality with no more than 100 NFEs. The source code is available at {\url{https://github.com/yuanzhi-zhu/DiffPIR}}
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance
Venue: arXiv cs.CL 2023
Year: 2023

Progress in NLP is increasingly measured through benchmarks; hence, contextualizing progress requires understanding when and why practitioners may disagree about the validity of benchmarks. We develop a taxonomy of disagreement, drawing on tools from measurement modeling, and distinguish between two types of disagreement: 1) how tasks are conceptualized and 2) how measurements of model performance are operationalized. To provide evidence for our taxonomy, we conduct a meta-analysis of relevant literature to understand how NLP tasks are conceptualized, as well as a survey of practitioners about their impressions of different factors that affect benchmark validity. Our meta-analysis and survey across eight tasks, ranging from coreference resolution to question answering, uncover that tasks are generally not clearly and consistently conceptualized and benchmarks suffer from operationalization disagreements. These findings support our proposed taxonomy of disagreement. Finally, based on our taxonomy, we present a framework for constructing benchmarks and documenting their limitations.
Algorithmic Censoring in Dynamic Learning Systems
Venue: arXiv cs.LG 2023
Year: 2023

Dynamic learning systems subject to selective labeling exhibit censoring, i.e. persistent negative predictions assigned to one or more subgroups of points. In applications like consumer finance, this results in groups of applicants that are persistently denied and thus never enter into the training data. In this work, we formalize censoring, demonstrate how it can arise, and highlight difficulties in detection. We consider safeguards against censoring - recourse and randomized-exploration - both of which ensure we collect labels for points that would otherwise go unobserved. The resulting techniques allow examples from censored groups to enter into the training data and correct the model. Our results highlight the otherwise unmeasured harms of censoring and demonstrate the effectiveness of mitigation strategies across a range of data generating processes.
Adaptive Federated Pruning in Hierarchical Wireless Networks
Venue: arXiv cs.LG 2023
Year: 2023

Federated Learning (FL) is a promising privacy-preserving distributed learning framework where a server aggregates models updated by multiple devices without accessing their private datasets. Hierarchical FL (HFL), as a device-edge-cloud aggregation hierarchy, can enjoy both the cloud server's access to more datasets and the edge servers' efficient communications with devices. However, the learning latency increases with the HFL network scale due to the increasing number of edge servers and devices with limited local computation capability and communication bandwidth. To address this issue, in this paper, we introduce model pruning for HFL in wireless networks to reduce the neural network scale. We present the convergence analysis of an upper on the l2 norm of gradients for HFL with model pruning, analyze the computation and communication latency of the proposed model pruning scheme, and formulate an optimization problem to maximize the convergence rate under a given latency threshold by jointly optimizing the pruning ratio and wireless resource allocation. By decoupling the optimization problem and using Karush Kuhn Tucker (KKT) conditions, closed-form solutions of pruning ratio and wireless resource allocation are derived. Simulation results show that our proposed HFL with model pruning achieves similar learning accuracy compared with the HFL without model pruning and reduces about 50 percent communication cost.
Scalable and Robust Tensor Ring Decomposition for Large-scale Data
Venue: arXiv cs.LG 2023
Year: 2023

Tensor ring (TR) decomposition has recently received increased attention due to its superior expressive performance for high-order tensors. However, the applicability of traditional TR decomposition algorithms to real-world applications is hindered by prevalent large data sizes, missing entries, and corruption with outliers. In this work, we propose a scalable and robust TR decomposition algorithm capable of handling large-scale tensor data with missing entries and gross corruptions. We first develop a novel auto-weighted steepest descent method that can adaptively fill the missing entries and identify the outliers during the decomposition process. Further, taking advantage of the tensor ring model, we develop a novel fast Gram matrix computation (FGMC) approach and a randomized subtensor sketching (RStS) strategy which yield significant reduction in storage and computational complexity. Experimental results demonstrate that the proposed method outperforms existing TR decomposition methods in the presence of outliers, and runs significantly faster than existing robust tensor completion algorithms.
Geometric Hitting Set for Line-Constrained Disks and Related Problems
Venue: arXiv cs.CG 2023
Year: 2023

Given a set $P$ of $n$ weighted points and a set $S$ of $m$ disks in the plane, the hitting set problem is to compute a subset $P'$ of points of $P$ such that each disk contains at least one point of $P'$ and the total weight of all points of $P'$ is minimized. The problem is known to be NP-hard. In this paper, we consider a line-constrained version of the problem in which all disks are centered on a line $\ell$. We present an $O((m+n)\log(m+n)+κ\log m)$ time algorithm for the problem, where $κ$ is the number of pairs of disks that intersect. For the unit-disk case where all disks have the same radius, the running time can be reduced to $O((n + m)\log(m + n))$. In addition, we solve the problem in $O((m + n)\log(m + n))$ time in the $L_{\infty}$ and $L_1$ metrics, in which a disk is a square and a diamond, respectively. Our techniques can also be used to solve other geometric hitting set problems. For example, given in the plane a set $P$ of $n$ weighted points and a set $S$ of $n$ half-planes, we solve in $O(n^4\log n)$ time the problem of finding a minimum weight hitting set of $P$ for $S$. This improves the previous best algorithm of $O(n^6)$ time by nearly a quadratic factor.
Capturing Humans' Mental Models of AI: An Item Response Theory Approach
Venue: arXiv cs.LG 2023
Year: 2023

Improving our understanding of how humans perceive AI teammates is an important foundation for our general understanding of human-AI teams. Extending relevant work from cognitive science, we propose a framework based on item response theory for modeling these perceptions. We apply this framework to real-world experiments, in which each participant works alongside another person or an AI agent in a question-answering setting, repeatedly assessing their teammate's performance. Using this experimental data, we demonstrate the use of our framework for testing research questions about people's perceptions of both AI agents and other people. We contrast mental models of AI teammates with those of human teammates as we characterize the dimensionality of these mental models, their development over time, and the influence of the participants' own self-perception. Our results indicate that people expect AI agents' performance to be significantly better on average than the performance of other humans, with less variation across different types of problems. We conclude with a discussion of the implications of these findings for human-AI interaction.
SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting
Venue: arXiv cs.CL 2023
Year: 2023

Building end-to-end task bots and maintaining their integration with new functionalities using minimal human efforts is a long-standing challenge in dialog research. Recently large language models (LLMs) have demonstrated exceptional proficiency in conversational engagement and adherence to instructions across various downstream tasks. In this work, we introduce SGP-TOD, Schema-Guided Prompting for building Task-Oriented Dialog systems effortlessly based on LLMs. Utilizing the symbolic knowledge -- task schema, we instruct fixed LLMs to generate appropriate responses on novel tasks, circumventing the need for training data. Specifically, SGP-TOD comprises three components: a LLM for engaging with users, a DST Prompter to aid the LLM with dialog state tracking, which is then used to retrieve database items, and a Policy Prompter to elicit proper responses adhering to the provided dialog policy. Experimental results on Multiwoz, RADDLE and STAR datasets show that our training-free strategy SGP-TOD, without any task-specific data, yields state-of-the-art (SOTA) zero-shot performance, greatly surpasses the few-shot approaches. In a domain-extension setting, SGP-TOD aptly adapts to new functionalities by merely adding supplementary schema rules. We make our code and data publicly available.
FiMReSt: Finite Mixture of Multivariate Regulated Skew-t Kernels -- A Flexible Probabilistic Model for Multi-Clustered Data with Asymmetrically-Scattered Non-Gaussian Kernels
Venue: arXiv cs.LG 2023
Year: 2023

Recently skew-t mixture models have been introduced as a flexible probabilistic modeling technique taking into account both skewness in data clusters and the statistical degree of freedom (S-DoF) to improve modeling generalizability, and robustness to heavy tails and skewness. In this paper, we show that the state-of-the-art skew-t mixture models fundamentally suffer from a hidden phenomenon named here as "S-DoF explosion," which results in local minima in the shapes of normal kernels during the non-convex iterative process of expectation maximization. For the first time, this paper provides insights into the instability of the S-DoF, which can result in the divergence of the kernels from the mixture of t-distribution, losing generalizability and power for modeling the outliers. Thus, in this paper, we propose a regularized iterative optimization process to train the mixture model, enhancing the generalizability and resiliency of the technique. The resulting mixture model is named Finite Mixture of Multivariate Regulated Skew-t (FiMReSt) Kernels, which stabilizes the S-DoF profile during optimization process of learning. To validate the performance, we have conducted a comprehensive experiment on several real-world datasets and a synthetic dataset. The results highlight (a) superior performance of the FiMReSt, (b) generalizability in the presence of outliers, and (c) convergence of S-DoF.
Adaptive Network Embedding with Arbitrary Multiple Information Sources in Attributed Graphs
Venue: arXiv cs.SI 2023
Year: 2023

Graph representation learning (a.k.a. network embedding) is a significant topic of network analysis, due to its effectiveness to support various graph inference tasks. In this paper, we study the representation learning with multiple information sources in attributed graphs. Recent studies usually focus on several specific sources (e.g., high-order proximity and node attributes) but few of them can be extended to incorporate other available sources not specified. In addition, most existing methods assume that all the integrated sources share consistent latent features but may ignore the possible inconsistency among them, lacking the required robustness. To address these issues, we propose a novel adaptive hybrid graph representation (AHGR) method from a view of graph reweighting, where each information source is formulated as a corresponding auxiliary graph, enabling AHGR to integrate arbitrary available information sources. Moreover, a new transition relation among the reweighted graphs is then introduced to perceive and resist the possible inconsistency among multiple sources, enhancing the robustness of AHGR. We verify the effectiveness of AHGR on a series of synthetic and real attributed graphs, where it presents superior performance over other baselines.