OpenRaiser/Intern-Atlas huggingface dataset:
A Lightweight Approach of Human-Like Playtesting
Venue: arXiv cs.SE 2021
Year: 2021

A playtest is the process in which human testers are recruited to play video games and to reveal software bugs. Manual testing is expensive and time-consuming, especially when there are many mobile games to test and every software version requires for extensive testing before being released. Existing testing frameworks (e.g., Android Monkey) are limited because they adopt no domain knowledge to play games. Learning-based tools (e.g., Wuji) involve a huge amount of training data and computation before testing any game. This paper presents LIT -- our lightweight approach to generalize playtesting tactics from manual testing, and to adopt the generalized tactics to automate game testing. LIT consists of two phases. In Phase I, while a human plays an Android game app G for a short period of time (e.g., eight minutes), \tool records the user's actions (e.g., swipe) and the scene before each action. Based on the collected data, LIT generalizes a set of \emph{context-aware, abstract playtesting tactics} which describe under what circumstances, what actions can be taken to play the game. In Phase II, LIT tests G based on the generalized tactics. Namely, given a randomly generated game scene, LIT searches match for the abstract context of any inferred tactic; if there is a match, LIT customizes the tactic and generates a feasible event to play the game. Our evaluation with nine games shows LIT to outperform two state-of-the-art tools. This implies that by automating playtest, LIT will significantly reduce manual testing and boost the quality of game apps.
A Survey of RDF Stores & SPARQL Engines for Querying Knowledge Graphs
Venue: arXiv cs.DB 2021
Year: 2021

RDF has seen increased adoption in recent years, prompting the standardization of the SPARQL query language for RDF, and the development of local and distributed engines for processing SPARQL queries. This survey paper provides a comprehensive review of techniques and systems for querying RDF knowledge graphs. While other reviews on this topic tend to focus on the distributed setting, the main focus of the work is on providing a comprehensive survey of state-of-the-art storage, indexing and query processing techniques for efficiently evaluating SPARQL queries in a local setting (on one machine). To keep the survey self-contained, we also provide a short discussion on graph partitioning techniques used in the distributed setting. We conclude by discussing contemporary research challenges for further improving SPARQL query engines. This extended version also provides a survey of over one hundred SPARQL query engines and the techniques they use, along with twelve benchmarks and their features.
AutoPreview: A Framework for Autopilot Behavior Understanding
Venue: arXiv cs.AI 2021
Year: 2021

The behavior of self driving cars may differ from people expectations, (e.g. an autopilot may unexpectedly relinquish control). This expectation mismatch can cause potential and existing users to distrust self driving technology and can increase the likelihood of accidents. We propose a simple but effective framework, AutoPreview, to enable consumers to preview a target autopilot potential actions in the real world driving context before deployment. For a given target autopilot, we design a delegate policy that replicates the target autopilot behavior with explainable action representations, which can then be queried online for comparison and to build an accurate mental model. To demonstrate its practicality, we present a prototype of AutoPreview integrated with the CARLA simulator along with two potential use cases of the framework. We conduct a pilot study to investigate whether or not AutoPreview provides deeper understanding about autopilot behavior when experiencing a new autopilot policy for the first time. Our results suggest that the AutoPreview method helps users understand autopilot behavior in terms of driving style comprehension, deployment preference, and exact action timing prediction.
Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians
Venue: arXiv cs.RO 2021
Year: 2021

Robotic navigation in environments shared with other robots or humans remains challenging because the intentions of the surrounding agents are not directly observable and the environment conditions are continuously changing. Local trajectory optimization methods, such as model predictive control (MPC), can deal with those changes but require global guidance, which is not trivial to obtain in crowded scenarios. This paper proposes to learn, via deep Reinforcement Learning (RL), an interaction-aware policy that provides long-term guidance to the local planner. In particular, in simulations with cooperative and non-cooperative agents, we train a deep network to recommend a subgoal for the MPC planner. The recommended subgoal is expected to help the robot in making progress towards its goal and accounts for the expected interaction with other agents. Based on the recommended subgoal, the MPC planner then optimizes the inputs for the robot satisfying its kinodynamic and collision avoidance constraints. Our approach is shown to substantially improve the navigation performance in terms of number of collisions as compared to prior MPC frameworks, and in terms of both travel time and number of collisions compared to deep RL methods in cooperative, competitive and mixed multiagent scenarios.
Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency
Venue: arXiv cs.LG 2021
Year: 2021

Local SGD is a promising approach to overcome the communication overhead in distributed learning by reducing the synchronization frequency among worker nodes. Despite the recent theoretical advances of local SGD in empirical risk minimization, the efficiency of its counterpart in minimax optimization remains unexplored. Motivated by large scale minimax learning problems, such as adversarial robust learning and training generative adversarial networks (GANs), we propose local Stochastic Gradient Descent Ascent (local SGDA), where the primal and dual variables can be trained locally and averaged periodically to significantly reduce the number of communications. We show that local SGDA can provably optimize distributed minimax problems in both homogeneous and heterogeneous data with reduced number of communications and establish convergence rates under strongly-convex-strongly-concave and nonconvex-strongly-concave settings. In addition, we propose a novel variant local SGDA+, to solve nonconvex-nonconcave problems. We give corroborating empirical evidence on different distributed minimax problems.
Understanding Robustness in Teacher-Student Setting: A New Perspective
Venue: arXiv cs.LG 2021
Year: 2021

Adversarial examples have appeared as a ubiquitous property of machine learning models where bounded adversarial perturbation could mislead the models to make arbitrarily incorrect predictions. Such examples provide a way to assess the robustness of machine learning models as well as a proxy for understanding the model training process. Extensive studies try to explain the existence of adversarial examples and provide ways to improve model robustness (e.g. adversarial training). While they mostly focus on models trained on datasets with predefined labels, we leverage the teacher-student framework and assume a teacher model, or oracle, to provide the labels for given instances. We extend Tian (2019) in the case of low-rank input data and show that student specialization (trained student neuron is highly correlated with certain teacher neuron at the same layer) still happens within the input subspace, but the teacher and student nodes could differ wildly out of the data subspace, which we conjecture leads to adversarial examples. Extensive experiments show that student specialization correlates strongly with model robustness in different scenarios, including student trained via standard training, adversarial training, confidence-calibrated adversarial training, and training with robust feature dataset. Our studies could shed light on the future exploration about adversarial examples, and enhancing model robustness via principled data augmentation.
Perspectives and solutions towards intelligent ambient assisted living systems
Venue: arXiv cs.HC 2021
Year: 2021

The population of the elderly people has kept increasing rapidly over the world in the past decades. Solutions that are able to effectively support the elderly people to live independently at their home are thus urgently needed. Ambient assisted living (AAL) aims to provide products and services with ambient intelligence to build a safe environment around people in need. With the high prevalence of multiple chronic diseases, the elderly people often need different levels of care management to prolong independent living at home. An effective AAL system should provide the required clinical support as an extension to the services provided in hospitals. Following the rapid growth of available data, together with the wide application of machine learning technologies, we are now able to build intelligent ambient assisted systems to fulfil such a request. This paper discusses different levels of intelligence in AAL. We also introduce our solution for building an intelligent AAL system with the discussed technologies. Taking semantic web technology as its backbone, such an AAL system is able to aggregate information from different sources, solve the semantic gap between different data sources, and perform adaptive and personalized carepath management based on the ambient environment.
Towards a Unified Framework for Fair and Stable Graph Representation Learning
Venue: arXiv cs.LG 2021
Year: 2021

As the representations output by Graph Neural Networks (GNNs) are increasingly employed in real-world applications, it becomes important to ensure that these representations are fair and stable. In this work, we establish a key connection between counterfactual fairness and stability and leverage it to propose a novel framework, NIFTY (uNIfying Fairness and stabiliTY), which can be used with any GNN to learn fair and stable representations. We introduce a novel objective function that simultaneously accounts for fairness and stability and develop a layer-wise weight normalization using the Lipschitz constant to enhance neural message passing in GNNs. In doing so, we enforce fairness and stability both in the objective function as well as in the GNN architecture. Further, we show theoretically that our layer-wise weight normalization promotes counterfactual fairness and stability in the resulting representations. We introduce three new graph datasets comprising of high-stakes decisions in criminal justice and financial lending domains. Extensive experimentation with the above datasets demonstrates the efficacy of our framework.
Data-Driven Characterization and Detection of COVID-19 Themed Malicious Websites
Venue: arXiv cs.CR 2021
Year: 2021

COVID-19 has hit hard on the global community, and organizations are working diligently to cope with the new norm of "work from home". However, the volume of remote work is unprecedented and creates opportunities for cyber attackers to penetrate home computers. Attackers have been leveraging websites with COVID-19 related names, dubbed COVID-19 themed malicious websites. These websites mostly contain false information, fake forms, fraudulent payments, scams, or malicious payloads to steal sensitive information or infect victims' computers. In this paper, we present a data-driven study on characterizing and detecting COVID-19 themed malicious websites. Our characterization study shows that attackers are agile and are deceptively crafty in designing geolocation targeted websites, often leveraging popular domain registrars and top-level domains. Our detection study shows that the Random Forest classifier can detect COVID-19 themed malicious websites based on the lexical and WHOIS features defined in this paper, achieving a 98% accuracy and 2.7% false-positive rate.
Exploring the Effect of Resolution on the Usability of Locimetric Authentication
Venue: arXiv cs.CR 2021
Year: 2021

Locimetric authentication is a form of graphical authentication in which users validate their identity by selecting predetermined points on a predetermined image. Its primary advantage over the ubiquitous text-based approach stems from users' superior ability to remember visual information over textual information, coupled with the authentication process being transformed to one requiring recognition (instead of recall). Ideally, these differentiations enable users to create more complex passwords, which theoretically are more secure. Yet locimetric authentication has one significant weakness: hot-spots. This term refers to areas of an image that users gravitate towards, and which consequently have a higher probability of being selected. Although many strategies have been proposed to counter the hot-spot problem, one area that has received little attention is that of resolution. The hypothesis here is that high-resolution images would afford the user a larger password space, and consequently any hot-spots would dissipate. We employ an experimental approach, where users generate a series of locimetric passwords on either low- or high-resolution images. Our research reveals the presence of hot-spots even in high-resolution images, albeit at a lower level than that exhibited with low-resolution images. We conclude by reinforcing that other techniques - such as existing or new software controls or training - need to be utilized to mitigate the emergence of hot-spots with the locimetric scheme.
Improving Robustness of Learning-based Autonomous Steering Using Adversarial Images
Venue: arXiv cs.CV 2021
Year: 2021

For safety of autonomous driving, vehicles need to be able to drive under various lighting, weather, and visibility conditions in different environments. These external and environmental factors, along with internal factors associated with sensors, can pose significant challenges to perceptual data processing, hence affecting the decision-making and control of the vehicle. In this work, we address this critical issue by introducing a framework for analyzing robustness of the learning algorithm w.r.t varying quality in the image input for autonomous driving. Using the results of sensitivity analysis, we further propose an algorithm to improve the overall performance of the task of "learning to steer". The results show that our approach is able to enhance the learning outcomes up to 48%. A comparative study drawn between our approach and other related techniques, such as data augmentation and adversarial training, confirms the effectiveness of our algorithm as a way to improve the robustness and generalization of neural network training for autonomous driving.
Single-angle Radon samples based reconstruction of functions in refinable shift-invariant space
Venue: arXiv cs.IT 2021
Year: 2021

The traditional approaches to computerized tomography (CT) depend on the samples of Radon transform at multiple angles. In optics, the real time imaging requires the reconstruction of an object by the samples of Radon transform at a single angle (SA). Driven by this and motivated by the connection between Bin Han's construction of wavelet frames (e.g [13]) and Radon transform, in refinable shift-invariant spaces (SISs) we investigate the SA-Radon sample based reconstruction problem. We have two main theorems. The fist main theorem states that, any compactly supported function in a SIS generated by a general refinable function can be determined by its Radon samples at an appropriate angle. Motivated by the extensive application of positive definite (PD) functions to interpolation of scattered data, we also investigate the SA reconstruction problem in a class of (refinable) box-spline generated SISs. Thanks to the PD property of the Radon transform of such spline, our second main theorem states that, the reconstruction of compactly supported functions in these spline generated SISs can be achieved by the samples of Radon transform at almost every angle. Numerical simulation is conducted to check the result.
Optimal Pricing of Information
Venue: arXiv cs.GT 2021
Year: 2021

A decision maker is choosing between an active action (e.g., purchase a house, invest certain stock) and a passive action. The payoff of the active action depends on the buyer's private type and also an unknown state of nature. An information seller can design experiments to reveal information about the realized state to the decision maker, and would like to maximize profit from selling such information. We characterize, in closed-form, the revenue-optimal information selling mechanism for the seller. After eliciting the buyer's type, the optimal mechanism charges the buyer an upfront payment and then simply reveals whether the realized state exceeds a certain threshold or not. The optimal mechanism features both price discrimination and information discrimination. The special buyer type who is a-priori indifferent between the active and passive action benefits the most from participating in the mechanism.
DoA-LF: A Location Fingerprint Positioning Algorithm with Millimeter-Wave
Venue: arXiv cs.IT 2021
Year: 2021

Location fingerprint (LF) has been widely applied in indoor positioning. However, the existing studies on LF mostly focus on the fingerprint of WiFi below 6 GHz, bluetooth, ultra wideband (UWB), etc. The LF with millimeter-wave (mmWave) was rarely addressed. Since mmWave has the characteristics of narrow beam, fast signal attenuation and wide bandwidth, etc., the positioning error can be reduced. In this paper, an LF positioning method with mmWave is proposed, which is named as DoA-LF. Besides received signal strength indicator (RSSI) of access points (APs), the fingerprint database contains direction of arrival (DoA) information of APs, which is obtained via DoA estimation. Then the impact of the number of APs, the interval of reference points (RPs), the channel model of mmWave and the error of DoA estimation algorithm on positioning error is analyzed with Cramer-Rao lower bound (CRLB). Finally, the proposed DoA-LF algorithm with mmWave is verified through simulations. The simulation results have proved that mmWave can reduce the positioning error due to the fact that mmWave has larger path loss exponent and smaller variance of shadow fading compared with low frequency signals. Besides, accurate DoA estimation can reduce the positioning error.
Potential Impacts of Smart Homes on Human Behavior: A Reinforcement Learning Approach
Venue: arXiv cs.AI 2021
Year: 2021

We aim to investigate the potential impacts of smart homes on human behavior. To this end, we simulate a series of human models capable of performing various activities inside a reinforcement learning-based smart home. We then investigate the possibility of human behavior being altered as a result of the smart home and the human model adapting to one-another. We design a semi-Markov decision process human task interleaving model based on hierarchical reinforcement learning that learns to make decisions to either pursue or leave an activity. We then integrate our human model in the smart home which is based on Q-learning. We show that a smart home trained on a generic human model is able to anticipate and learn the thermal preferences of human models with intrinsic rewards similar to the generic model. The hierarchical human model learns to complete each activity and set optimal thermal settings for maximum comfort. With the smart home, the number of time steps required to change the thermal settings are reduced for the human models. Interestingly, we observe that small variations in the human model reward structures can lead to the opposite behavior in the form of unexpected switching between activities which signals changes in human behavior due to the presence of the smart home.
Dual-MTGAN: Stochastic and Deterministic Motion Transfer for Image-to-Video Synthesis
Venue: arXiv cs.CV 2021
Year: 2021

Generating videos with content and motion variations is a challenging task in computer vision. While the recent development of GAN allows video generation from latent representations, it is not easy to produce videos with particular content of motion patterns of interest. In this paper, we propose Dual Motion Transfer GAN (Dual-MTGAN), which takes image and video data as inputs while learning disentangled content and motion representations. Our Dual-MTGAN is able to perform deterministic motion transfer and stochastic motion generation. Based on a given image, the former preserves the input content and transfers motion patterns observed from another video sequence, and the latter directly produces videos with plausible yet diverse motion patterns based on the input image. The proposed model is trained in an end-to-end manner, without the need to utilize pre-defined motion features like pose or facial landmarks. Our quantitative and qualitative results would confirm the effectiveness and robustness of our model in addressing such conditioned image-to-video tasks.
Contact Tracing: Computational Bounds, Limitations and Implications
Venue: arXiv cs.SI 2021
Year: 2021

Contact tracing has been extensively studied from different perspectives in recent years. However, there is no clear indication of why this intervention has proven effective in some epidemics (SARS) and mostly ineffective in some others (COVID-19). Here, we perform an exhaustive evaluation of random testing and contact tracing on novel superspreading random networks to try to identify which epidemics are more containable with such measures. We also explore the suitability of positive rates as a proxy of the actual infection statuses of the population. Moreover, we propose novel ideal strategies to explore the potential limits of both testing and tracing strategies. Our study counsels caution, both at assuming epidemic containment and at inferring the actual epidemic progress, with current testing or tracing strategies. However, it also brings a ray of light for the future, with the promise of the potential of novel testing strategies that can achieve great effectiveness.
Predicting gender and age categories in English conversations using lexical, non-lexical, and turn-taking features
Venue: arXiv cs.CL 2021
Year: 2021

This paper examines gender and age salience and (stereo)typicality in British English talk with the aim to predict gender and age categories based on lexical, phrasal and turn-taking features. We examine the SpokenBNC, a corpus of around 11.4 million words of British English conversations and identify behavioural differences between speakers that are labelled for gender and age categories. We explore differences in language use and turn-taking dynamics and identify a range of characteristics that set the categories apart. We find that female speakers tend to produce more and slightly longer turns, while turns by male speakers feature a higher type-token ratio and a distinct range of minimal particles such as "eh", "uh" and "em". Across age groups, we observe, for instance, that swear words and laughter characterize young speakers' talk, while old speakers tend to produce more truncated words. We then use the observed characteristics to predict gender and age labels of speakers per conversation and per turn as a classification task, showing that non-lexical utterances such as minimal particles that are usually left out of dialog data can contribute to setting the categories apart.