OpenRaiser/Intern-Atlas huggingface dataset:
Towards Tackling MaxSAT by Combining Nested Monte Carlo with Local Search
Venue: arXiv cs.AI 2023
Year: 2023

Recent work proposed the UCTMAXSAT algorithm to address Maximum Satisfiability Problems (MaxSAT) and shown improved performance over pure Stochastic Local Search algorithms (SLS). UCTMAXSAT is based on Monte Carlo Tree Search but it uses SLS instead of purely random playouts. In this work, we introduce two algorithmic variations over UCTMAXSAT. We carry an empirical analysis on MaxSAT benchmarks from recent competitions and establish that both ideas lead to performance improvements. First, a nesting of the tree search inspired by the Nested Monte Carlo Search algorithm is effective on most instance types in the benchmark. Second, we observe that using a static flip limit in SLS, the ideal budget depends heavily on the instance size and we propose to set it dynamically. We show that it is a robust way to achieve comparable performance on a variety of instances without requiring additional tuning.
Data Structures for Deviation Payoffs
Venue: arXiv cs.GT 2023
Year: 2023

We present new data structures for representing symmetric normal-form games. These data structures are optimized for efficiently computing the expected utility of each unilateral pure-strategy deviation from a symmetric mixed-strategy profile. The cumulative effect of numerous incremental innovations is a dramatic speedup in the computation of symmetric mixed-strategy Nash equilibria, making it practical to represent and solve games with dozens to hundreds of players. These data structures naturally extend to role-symmetric and action-graph games with similar benefits.
Q-Cogni: An Integrated Causal Reinforcement Learning Framework
Venue: arXiv cs.LG 2023
Year: 2023

We present Q-Cogni, an algorithmically integrated causal reinforcement learning framework that redesigns Q-Learning with an autonomous causal structure discovery method to improve the learning process with causal inference. Q-Cogni achieves optimal learning with a pre-learned structural causal model of the environment that can be queried during the learning process to infer cause-and-effect relationships embedded in a state-action space. We leverage on the sample efficient techniques of reinforcement learning, enable reasoning about a broader set of policies and bring higher degrees of interpretability to decisions made by the reinforcement learning agent. We apply Q-Cogni on the Vehicle Routing Problem (VRP) and compare against state-of-the-art reinforcement learning algorithms. We report results that demonstrate better policies, improved learning efficiency and superior interpretability of the agent's decision making. We also compare this approach with traditional shortest-path search algorithms and demonstrate the benefits of our causal reinforcement learning framework to high dimensional problems. Finally, we apply Q-Cogni to derive optimal routing decisions for taxis in New York City using the Taxi & Limousine Commission trip record data and compare with shortest-path search, reporting results that show 85% of the cases with an equal or better policy derived from Q-Cogni in a real-world domain.
Bayesian Networks for Named Entity Prediction in Programming Community Question Answering
Venue: arXiv cs.LG 2023
Year: 2023

Within this study, we propose a new approach for natural language processing using Bayesian networks to predict and analyze the context and how this approach can be applied to the Community Question Answering domain. We discuss how Bayesian networks can detect semantic relationships and dependencies between entities, and this is connected to different score-based approaches of structure-learning. We compared the Bayesian networks with different score metrics, such as the BIC, BDeu, K2 and Chow-Liu trees. Our proposed approach out-performs the baseline model at the precision metric. We also discuss the influence of penalty terms on the structure of Bayesian networks and how they can be used to analyze the relationships between entities. In addition, we examine the visualization of directed acyclic graphs to analyze semantic relationships. The article further identifies issues with detecting certain semantic classes that are separated in the structure of directed acyclic graphs. Finally, we evaluate potential improvements for the Bayesian network approach.
On the Complexity of Recognizing Nerves of Convex Sets
Venue: arXiv cs.CG 2023
Year: 2023

We study the problem of recognizing whether a given abstract simplicial complex $K$ is the $k$-skeleton of the nerve of $j$-dimensional convex sets in $\mathbb{R}^d$. We denote this problem by $R(k,j,d)$. As a main contribution, we unify the results of many previous works under this framework and show that many of these works in fact imply stronger results than explicitly stated. This allows us to settle the complexity status of $R(1,j,d)$, which is equivalent to the problem of recognizing intersection graphs of $j$-dimensional convex sets in $\mathbb{R}^d$, for any $j$ and $d$. Furthermore, we point out some trivial cases of $R(k,j,d)$, and demonstrate that $R(k,j,d)$ is ER-complete for $j\in\{d-1,d\}$ and $k\geq d$.
Makeup Extraction of 3D Representation via Illumination-Aware Image Decomposition
Venue: arXiv cs.CV 2023
Year: 2023

Facial makeup enriches the beauty of not only real humans but also virtual characters; therefore, makeup for 3D facial models is highly in demand in productions. However, painting directly on 3D faces and capturing real-world makeup are costly, and extracting makeup from 2D images often struggles with shading effects and occlusions. This paper presents the first method for extracting makeup for 3D facial models from a single makeup portrait. Our method consists of the following three steps. First, we exploit the strong prior of 3D morphable models via regression-based inverse rendering to extract coarse materials such as geometry and diffuse/specular albedos that are represented in the UV space. Second, we refine the coarse materials, which may have missing pixels due to occlusions. We apply inpainting and optimization. Finally, we extract the bare skin, makeup, and an alpha matte from the diffuse albedo. Our method offers various applications for not only 3D facial models but also 2D portrait images. The extracted makeup is well-aligned in the UV space, from which we build a large-scale makeup dataset and a parametric makeup model for 3D faces. Our disentangled materials also yield robust makeup transfer and illumination-aware makeup interpolation/removal without a reference image.
Improving Representational Continuity via Continued Pretraining
Venue: arXiv cs.LG 2023
Year: 2023

We consider the continual representation learning setting: sequentially pretrain a model $M'$ on tasks $T_1, \ldots, T_T$, and then adapt $M'$ on a small amount of data from each task $T_i$ to check if it has forgotten information from old tasks. Under a kNN adaptation protocol, prior work shows that continual learning methods improve forgetting over naive training (SGD). In reality, practitioners do not use kNN classifiers -- they use the adaptation method that works best (e.g., fine-tuning) -- here, we find that strong continual learning baselines do worse than naive training. Interestingly, we find that a method from the transfer learning community (LP-FT) outperforms naive training and the other continual learning methods. Even with standard kNN evaluation protocols, LP-FT performs comparably with strong continual learning methods (while being simpler and requiring less memory) on three standard benchmarks: sequential CIFAR-10, CIFAR-100, and TinyImageNet. LP-FT also reduces forgetting in a real world satellite remote sensing dataset (FMoW), and a variant of LP-FT gets state-of-the-art accuracies on an NLP continual learning benchmark.
Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics
Venue: arXiv cs.SD 2023
Year: 2023

This paper aims to test whether a multi-modal approach for music emotion recognition (MER) performs better than a uni-modal one on high-level song features and lyrics. We use 11 song features retrieved from the Spotify API, combined lyrics features including sentiment, TF-IDF, and Anew to predict valence and arousal (Russell, 1980) scores on the Deezer Mood Detection Dataset (DMDD) (Delbouys et al., 2018) with 4 different regression models. We find that out of the 11 high-level song features, mainly 5 contribute to the performance, multi-modal features do better than audio alone when predicting valence. We made our code publically available.
Adaptive Control of IoT/M2M Devices in Smart Buildings using Heterogeneous Wireless Networks
Venue: arXiv cs.NI 2023
Year: 2023

With the rapid development of wireless communication technology, the Internet of Things (IoT) and Machine-to-Machine (M2M) are becoming essential for many applications. One of the most emblematic IoT/M2M applications is smart buildings. The current Building Automation Systems (BAS) are limited by many factors, including the lack of integration of IoT and M2M technologies, unfriendly user interfacing, and the lack of a convergent solution. Therefore, this paper proposes a better approach of using heterogeneous wireless networks consisting of Wireless Sensor Networks (WSNs) and Mobile Cellular Networks (MCNs) for IoT/M2M smart building systems. One of the most significant outcomes of this research is to provide accurate readings to the server, and very low latency, through which users can easily control and monitor remotely the proposed system that consists of several innovative services, namely smart parking, garden irrigation automation, intrusion alarm, smart door, fire and gas detection, smart lighting, smart medication reminder, and indoor air quality monitoring. All these services are designed and implemented to control and monitor from afar the building via our free mobile application named Raniso which is a local server that allows remote control of the building. This IoT/M2M smart building system is customizable to meet the needs of users, improving safety and quality of life while reducing energy consumption. Additionally, it helps prevent the loss of resources and human lives by detecting and managing risks.
Guiding the Sequential Experiments in Autonomous Experimentation Platforms through EI-based Bayesian Optimization and Bayesian Model Averaging
Venue: arXiv cs.LG 2023
Year: 2023

Autonomous Experimentation Platforms (AEPs) are advanced manufacturing platforms that, under intelligent control, can sequentially search the material design space (MDS) and identify parameters with the desired properties. At the heart of the intelligent control of these AEPs is the policy guiding the sequential experiments, which is to choose the location to carry out the next experiment. In such cases, a balance between exploitation and exploration must be achieved. A Bayesian Optimization (BO) framework with Expected Improvement based (EI-based) acquisition function can effectively search the MDS and guide where to conduct the next experiments so that the underlying relationship can be identified with a smaller number of experiments. The traditional BO framework tries to optimize a black box objective function in a sequential manner by relying on a single model. However, this single-model approach does not account for model uncertainty. Bayesian Model Averaging (BMA) addresses this issue by working with multiple models and thus considering the uncertainty in the models. In this work, we first apply the conventional BO algorithm with the most popular EI-based experiment policy in a real-life fatigue dataset for steel to predict the fatigue strength of steel. Afterward, we apply BMA to the same dataset by working with a set of predictive models and compare the performance of BMA with the traditional BO algorithm, which relies on a single model for approximation. We compare the results in terms of RMSE and find that BMA performs better than EI-based BO in the prediction task by considering the model uncertainty in its framework.
Autoencoders as Pattern Filters
Venue: arXiv cs.LG 2023
Year: 2023

We discuss a simple approach to transform autoencoders into "pattern filters". Besides filtering, we show how this simple approach can be used also to build robust classifiers, by learning to filter only patterns of a given class.
Asynchronous Persistence with ASAP
Venue: arXiv cs.AR 2023
Year: 2023

Supporting atomic durability of updates for persistent memories is typically achieved with Write-Ahead Logging (WAL). WAL flushes log entries to persistent memory before making the actual data persistent to ensure that a consistent state can be recovered if a crash occurs. Performing WAL in hardware is attractive because it makes most aspects of log management transparent to software, and it completes log persist operations (LPs) and data persist operations (DPs) in the background, overlapping them with the execution of other instructions. Prior hardware logging solutions commit atomic regions synchronously. Once the end of a region is reached, all outstanding persist operations required for the region to commit must be completed before instruction execution may proceed. For undo logging, LPs and DPs are both performed synchronously to ensure that the region commits synchronously. For redo logging, DPs can be performed asynchronously, but LPs are performed synchronously to ensure that the region commits synchronously. In both cases, waiting for synchronous persist operations (LP or DP) at the end of an atomic region causes atomic regions to incur high latency. To tackle this limitation, we propose ASAP, a hardware logging solution that allows atomic regions to commit asynchronously. That is, once the end of an atomic region is reached, instruction execution may proceed without waiting for outstanding persist operations to complete. As such, both LPs and DPs can be performed asynchronously. The challenge with allowing atomic regions to commit asynchronously is that it can lead to control and data dependence violations in the commit order of the atomic regions, leaving data in an unrecoverable state in case of a crash. To address this issue, ASAP tracks and enforces control and data dependencies between atomic regions in hardware to ensure that the regions commit in the proper order.
Cross-lingual Knowledge Transfer via Distillation for Multilingual Information Retrieval
Venue: arXiv cs.IR 2023
Year: 2023

In this paper, we introduce the approach behind our submission for the MIRACL challenge, a WSDM 2023 Cup competition that centers on ad-hoc retrieval across 18 diverse languages. Our solution contains two neural-based models. The first model is a bi-encoder re-ranker, on which we apply a cross-lingual distillation technique to transfer ranking knowledge from English to the target language space. The second model is a cross-encoder re-ranker trained on multilingual retrieval data generated using neural machine translation. We further fine-tune both models using MIRACL training data and ensemble multiple rank lists to obtain the final result. According to the MIRACL leaderboard, our approach ranks 8th for the Test-A set and 2nd for the Test-B set among the 16 known languages.
An accurate and efficient approach to probabilistic conflict prediction
Venue: arXiv cs.RO 2023
Year: 2023

Conflict prediction is a vital component of path planning for autonomous vehicles. Prediction methods must be accurate for reliable navigation, but also computationally efficient to enable online path planning. Efficient prediction methods are especially crucial when testing large sets of candidate trajectories. We present a prediction method that has the same accuracy as existing methods, but up to an order of magnitude faster. This is achieved by rewriting the conflict prediction problem in terms of the first-passage time distribution using a dimension-reduction transform. First-passage time distributions are analytically derived for a subset of Gaussian processes describing vehicle motion. The proposed method is applicable to 2-D stochastic processes where the mean can be approximated by line segments, and the conflict boundary can be approximated by piece-wise straight lines. The proposed method was tested in simulation and compared to two probability flow methods, as well as a recent instantaneous conflict probability method. The results demonstrate a significant decrease of computation time.
Navigating Multi-Stakeholder Incentives and Preferences: Co-Designing Alternatives for the Future of Gig Worker Well-Being
Venue: arXiv cs.HC 2023
Year: 2023

Gig workers, and the products and services they provide, play an increasingly ubiquitous role in our daily lives. But despite growing evidence suggesting that worker well-being in gig economy platforms have become significant societal problems, few studies have investigated possible solutions. We take a stride in this direction by engaging workers, platform employees, and local regulators in a series of speed dating workshops using storyboards based on real-life situations to rapidly elicit stakeholder preferences for addressing financial, physical, and social issues related to worker well-being. Our results reveal that existing public and platformic infrastructures fall short in providing workers with resources needed to perform gigs, surfacing a need for multi-platform collaborations, technological innovations, as well as changes in regulations, labor laws, and the public's perception of gig workers, among others. Drawing from multi-stakeholder findings, we discuss these implications for technology, policy, and service as well as avenues for collaboration.
A Self-Supervised Learning-based Approach to Clustering Multivariate Time-Series Data with Missing Values (SLAC-Time): An Application to TBI Phenotyping
Venue: arXiv cs.LG 2023
Year: 2023

Self-supervised learning approaches provide a promising direction for clustering multivariate time-series data. However, real-world time-series data often include missing values, and the existing approaches require imputing missing values before clustering, which may cause extensive computations and noise and result in invalid interpretations. To address these challenges, we present a Self-supervised Learning-based Approach to Clustering multivariate Time-series data with missing values (SLAC-Time). SLAC-Time is a Transformer-based clustering method that uses time-series forecasting as a proxy task for leveraging unlabeled data and learning more robust time-series representations. This method jointly learns the neural network parameters and the cluster assignments of the learned representations. It iteratively clusters the learned representations with the K-means method and then utilizes the subsequent cluster assignments as pseudo-labels to update the model parameters. To evaluate our proposed approach, we applied it to clustering and phenotyping Traumatic Brain Injury (TBI) patients in the Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACK-TBI) study. Our experiments demonstrate that SLAC-Time outperforms the baseline K-means clustering algorithm in terms of silhouette coefficient, Calinski Harabasz index, Dunn index, and Davies Bouldin index. We identified three TBI phenotypes that are distinct from one another in terms of clinically significant variables as well as clinical outcomes, including the Extended Glasgow Outcome Scale (GOSE) score, Intensive Care Unit (ICU) length of stay, and mortality rate. The experiments show that the TBI phenotypes identified by SLAC-Time can be potentially used for developing targeted clinical trials and therapeutic strategies.
Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators
Venue: arXiv cs.LG 2023
Year: 2023

It is becoming increasingly imperative to design robust ML defenses. However, recent work has found that many defenses that initially resist state-of-the-art attacks can be broken by an adaptive adversary. In this work we take steps to simplify the design of defenses and argue that white-box defenses should eschew randomness when possible. We begin by illustrating a new issue with the deployment of randomized defenses that reduces their security compared to their deterministic counterparts. We then provide evidence that making defenses deterministic simplifies robustness evaluation, without reducing the effectiveness of a truly robust defense. Finally, we introduce a new defense evaluation framework that leverages a defense's deterministic nature to better evaluate its adversarial robustness.
Age Minimization with Energy and Distortion Constraints
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we consider a status update system, where an access point collects measurements from multiple sensors that monitor a common physical process, fuses them, and transmits the aggregated sample to the destination over an erasure channel. Under a typical information fusion scheme, the distortion of the fused sample is inversely proportional to the number of measurements received. Our goal is to minimize the long-term average age while satisfying the average energy and general age-based distortion requirements. Specifically, we focus on the setting in which the distortion requirement is stricter when the age of the update is older. We show that the optimal policy is a mixture of two stationary, deterministic, threshold-based policies, each of which is optimal for a parameterized problem that aims to minimize the weighted sum of the age and energy under the distortion constraint. We then derive analytically the associated optimal average age-cost function and characterize its performance in the large threshold regime, the results of which shed critical insights on the tradeoff among age, energy, and the distortion of the samples. We have also developed a closed-form solution for the special case when the distortion requirement is independent of the age, arguably the most important setting for practical applications.
Pretraining De-Biased Language Model with Large-scale Click Logs for Document Ranking
Venue: arXiv cs.IR 2023
Year: 2023

Pre-trained language models have achieved great success in various large-scale information retrieval tasks. However, most of pretraining tasks are based on counterfeit retrieval data where the query produced by the tailored rule is assumed as the user's issued query on the given document or passage. Therefore, we explore to use large-scale click logs to pretrain a language model instead of replying on the simulated queries. Specifically, we propose to use user behavior features to pretrain a debiased language model for document ranking. Extensive experiments on Baidu desensitization click logs validate the effectiveness of our method. Our team on WSDM Cup 2023 Pre-training for Web Search won the 1st place with a Discounted Cumulative Gain @ 10 (DCG@10) score of 12.16525 on the final leaderboard.