OpenRaiser/Intern-Atlas huggingface dataset:
GenSF: Simultaneous Adaptation of Generative Pre-trained Models and Slot Filling
Venue: arXiv cs.CL 2021
Year: 2021

In transfer learning, it is imperative to achieve strong alignment between a pre-trained model and a downstream task. Prior work has done this by proposing task-specific pre-training objectives, which sacrifices the inherent scalability of the transfer learning paradigm. We instead achieve strong alignment by simultaneously modifying both the pre-trained model and the formulation of the downstream task, which is more efficient and preserves the scalability of transfer learning. We present GenSF (Generative Slot Filling), which leverages a generative pre-trained open-domain dialog model for slot filling. GenSF (1) adapts the pre-trained model by incorporating inductive biases about the task and (2) adapts the downstream task by reformulating slot filling to better leverage the pre-trained model's capabilities. GenSF achieves state-of-the-art results on two slot filling datasets with strong gains in few-shot and zero-shot settings. We achieve a 9 F1 score improvement in zero-shot slot filling. This highlights the value of strong alignment between the pre-trained model and the downstream task.
Multi-Resource List Scheduling of Moldable Parallel Jobs under Precedence Constraints
Venue: arXiv cs.DC 2021
Year: 2021

The scheduling literature has traditionally focused on a single type of resource (e.g., computing nodes). However, scientific applications in modern High-Performance Computing (HPC) systems process large amounts of data, hence have diverse requirements on different types of resources (e.g., cores, cache, memory, I/O). All of these resources could potentially be exploited by the runtime scheduler to improve the application performance. In this paper, we study multi-resource scheduling to minimize the makespan of computational workflows comprised of parallel jobs subject to precedence constraints. The jobs are assumed to be moldable, allowing the scheduler to flexibly select a variable set of resources before execution. We propose a multi-resource, list-based scheduling algorithm, and prove that, on a system with $d$ types of schedulable resources, our algorithm achieves an approximation ratio of $1.619d+2.545\sqrt{d}+1$ for any $d$, and a ratio of $d+O(\sqrt[3]{d^2})$ for large $d$. We also present improved results for independent jobs and for jobs with special precedence constraints (e.g., series-parallel graphs and trees). Finally, we prove a lower bound of $d$ on the approximation ratio of any list scheduling scheme with local priority considerations. To the best of our knowledge, these are the first approximation results for moldable workflows with multiple resource requirements.
User Acceptance of Gender Stereotypes in Automated Career Recommendations
Venue: arXiv cs.CY 2021
Year: 2021

Currently, there is a surge of interest in fair Artificial Intelligence (AI) and Machine Learning (ML) research which aims to mitigate discriminatory bias in AI algorithms, e.g. along lines of gender, age, and race. While most research in this domain focuses on developing fair AI algorithms, in this work, we show that a fair AI algorithm on its own may be insufficient to achieve its intended results in the real world. Using career recommendation as a case study, we build a fair AI career recommender by employing gender debiasing machine learning techniques. Our offline evaluation showed that the debiased recommender makes fairer career recommendations without sacrificing its accuracy. Nevertheless, an online user study of more than 200 college students revealed that participants on average prefer the original biased system over the debiased system. Specifically, we found that perceived gender disparity is a determining factor for the acceptance of a recommendation. In other words, our results demonstrate we cannot fully address the gender bias issue in AI recommendations without addressing the gender bias in humans.
The Power of Randomization: Efficient and Effective Algorithms for Constrained Submodular Maximization
Venue: arXiv cs.DS 2021
Year: 2021

Submodular optimization has numerous applications such as crowdsourcing and viral marketing. In this paper, we study the fundamental problem of non-negative submodular function maximization subject to a $k$-system constraint, which generalizes many other important constraints in submodular optimization such as cardinality constraint, matroid constraint, and $k$-extendible system constraint. The existing approaches for this problem achieve the best-known approximation ratio of $k+2\sqrt{k+2}+3$ (for a general submodular function) based on deterministic algorithmic frameworks. We propose several randomized algorithms that improve upon the state-of-the-art algorithms in terms of approximation ratio and time complexity, both under the non-adaptive setting and the adaptive setting. The empirical performance of our algorithms is extensively evaluated in several applications related to data mining and social computing, and the experimental results demonstrate the superiorities of our algorithms in terms of both utility and efficiency.
Self-orthogonal codes over a non-unital ring and combinatorial matrices
Venue: arXiv cs.IT 2021
Year: 2021

There is a local ring $E$ of order $4,$ without identity for the multiplication, defined by generators and relations as $E=\langle a,b \mid 2a=2b=0,\, a^2=a,\, b^2=b,\,ab=a,\, ba=b\rangle.$ We study a special construction of self-orthogonal codes over $E,$ based on combinatorial matrices related to two-class association schemes, Strongly Regular Graphs (SRG), and Doubly Regular Tournaments (DRT). We construct quasi self-dual codes over $E,$ and Type IV codes, that is, quasi self-dual codes whose all codewords have even Hamming weight. All these codes can be represented as formally self-dual additive codes over $\F_4.$ The classical invariant theory bound for the weight enumerators of this class of codesimproves the known bound on the minimum distance of Type IV codes over $E.$
Variational Policy Search using Sparse Gaussian Process Priors for Learning Multimodal Optimal Actions
Venue: arXiv cs.RO 2021
Year: 2021

Policy search reinforcement learning has been drawing much attention as a method of learning a robot control policy. In particular, policy search using such non-parametric policies as Gaussian process regression can learn optimal actions with high-dimensional and redundant sensors as input. However, previous methods implicitly assume that the optimal action becomes unique for each state. This assumption can severely limit such practical applications as robot manipulations since designing a reward function that appears in only one optimal action for complex tasks is difficult. The previous methods might have caused critical performance deterioration because the typical non-parametric policies cannot capture the optimal actions due to their unimodality. We propose novel approaches in non-parametric policy searches with multiple optimal actions and offer two different algorithms commonly based on a sparse Gaussian process prior and variational Bayesian inference. The following are the key ideas: 1) multimodality for capturing multiple optimal actions and 2) mode-seeking for capturing one optimal action by ignoring the others. First, we propose a multimodal sparse Gaussian process policy search that uses multiple overlapped GPs as a prior. Second, we propose a mode-seeking sparse Gaussian process policy search that uses the student-t distribution for a likelihood function. The effectiveness of those algorithms is demonstrated through applications to object manipulation tasks with multiple optimal actions in simulations.
Heterogeneous Edge-Enhanced Graph Attention Network For Multi-Agent Trajectory Prediction
Venue: arXiv cs.RO 2021
Year: 2021

Simultaneous trajectory prediction for multiple heterogeneous traffic participants is essential for the safe and efficient operation of connected automated vehicles under complex driving situations in the real world. The multi-agent prediction task is challenging, as the motions of traffic participants are affected by many factors, including their individual dynamics, their interactions with surrounding agents, the traffic infrastructures, and the number and modalities of the target agents. To further advance the trajectory prediction techniques, in this work we propose a three-channel framework together with a novel Heterogeneous Edge-enhanced graph ATtention network (HEAT), which is able to deal with the heterogeneity of the target agents and traffic participants involved. Specifically, the agent's dynamics are extracted from their historical states using type-specific encoders. The inter-agent interactions are represented with a directed edge-featured heterogeneous graph, and then interaction features are extracted using the proposed HEAT network. Besides, the map features are shared across all agents by introducing a selective gate mechanism. And finally, the trajectories of multi-agent are executed simultaneously. Validations using both urban and highway driving datasets show that the proposed model can realize simultaneous trajectory predictions for multiple agents under complex traffic situations, and achieve state-of-the-art performance with respect to prediction accuracy, demonstrating its feasibility and effectiveness.
Self-training Guided Adversarial Domain Adaptation For Thermal Imagery
Venue: arXiv cs.CV 2021
Year: 2021

Deep models trained on large-scale RGB image datasets have shown tremendous success. It is important to apply such deep models to real-world problems. However, these models suffer from a performance bottleneck under illumination changes. Thermal IR cameras are more robust against such changes, and thus can be very useful for the real-world problems. In order to investigate efficacy of combining feature-rich visible spectrum and thermal image modalities, we propose an unsupervised domain adaptation method which does not require RGB-to-thermal image pairs. We employ large-scale RGB dataset MS-COCO as source domain and thermal dataset FLIR ADAS as target domain to demonstrate results of our method. Although adversarial domain adaptation methods aim to align the distributions of source and target domains, simply aligning the distributions cannot guarantee perfect generalization to the target domain. To this end, we propose a self-training guided adversarial domain adaptation method to promote generalization capabilities of adversarial domain adaptation methods. To perform self-training, pseudo labels are assigned to the samples on the target thermal domain to learn more generalized representations for the target domain. Extensive experimental analyses show that our proposed method achieves better results than the state-of-the-art adversarial domain adaptation methods. The code and models are publicly available.
Capacity of Gaussian Arbitrarily-Varying Fading Channels
Venue: arXiv cs.IT 2021
Year: 2021

This paper considers an arbitrarily-varying fading channel consisting of one transmitter, one receiver and an arbitrarily varying adversary. The channel is assumed to have additive Gaussian noise and fast fading of the gain from the legitimate user to the receiver. We study four variants of the problem depending on whether the transmitter and/or adversary have access to the fading gains; we assume the receiver always knows the fading gains. In two variants the adversary does not have access to the gains, we show that the capacity corresponds to the capacity of a standard point-to-point fading channel with increased noise variance. The capacity of the other two cases, in which the adversary has knowledge of the channel gains, are determined by the worst-case noise variance as a function of the channel gain subject to the jammer's power constraint; if the jammer has enough power, then it can imitate the legitimate user's channel, causing the capacity to drop to zero. We also show that having the channel gains causally or non-causally at the encoder and/or the adversary does not change the capacity, except for the case where all parties know the channel gains. In this case, if the transmitter knows the gains non-causally, while the adversary knows the gains causally, then it is possible for the legitimate users to keep a secret from the adversary. We show that in this case the capacity is always positive.
Group-based Bi-Directional Recurrent Wavelet Neural Networks for Video Super-Resolution
Venue: arXiv cs.CV 2021
Year: 2021

Video super-resolution (VSR) aims to estimate a high-resolution (HR) frame from a low-resolution (LR) frames. The key challenge for VSR lies in the effective exploitation of spatial correlation in an intra-frame and temporal dependency between consecutive frames. However, most of the previous methods treat different types of the spatial features identically and extract spatial and temporal features from the separated modules. It leads to lack of obtaining meaningful information and enhancing the fine details. In VSR, there are three types of temporal modeling frameworks: 2D convolutional neural networks (CNN), 3D CNN, and recurrent neural networks (RNN). Among them, the RNN-based approach is suitable for sequential data. Thus the SR performance can be greatly improved by using the hidden states of adjacent frames. However, at each of time step in a recurrent structure, the RNN-based previous works utilize the neighboring features restrictively. Since the range of accessible motion per time step is narrow, there are still limitations to restore the missing details for dynamic or large motion. In this paper, we propose a group-based bi-directional recurrent wavelet neural networks (GBR-WNN) to exploit the sequential data and spatio-temporal information effectively for VSR. The proposed group-based bi-directional RNN (GBR) temporal modeling framework is built on the well-structured process with the group of pictures (GOP). We propose a temporal wavelet attention (TWA) module, in which attention is adopted for both spatial and temporal features. Experimental results demonstrate that the proposed method achieves superior performance compared with state-of-the-art methods in both of quantitative and qualitative evaluations.
Towards Continuous Safety Assessment in Context of DevOps
Venue: arXiv cs.SE 2021
Year: 2021

Traditionally, promoted by the internet companies, continuous delivery is more and more appealing to industries which develop systems with safety-critical functions. Since safety-critical systems must meet regulatory requirements and require specific safety assessment processes in addition to the normal development steps, enabling continuous delivery of software in safety-critical systems requires the automation of the safety assessment process in the delivery pipeline. In this paper, we outline a continuous delivery pipeline for realizing continuous safety assessment in software-intensive safety-critical systems based on model-based safety assessment methods.
Real-World Evaluation of the Impact of Automated Driving System Technology on Driver Gaze Behavior, Reaction Time and Trust
Venue: arXiv cs.HC 2021
Year: 2021

Recent developments in advanced driving assistance systems (ADAS) that rely on some level of autonomy have led the automobile industry and research community to investigate the impact they might have on driving performance. However, most of the research performed so far is based on simulated environments. In this study, we investigated the behavior of drivers in a vehicle with automated driving system (ADS) capabilities in a real-life driving scenario. We analyzed their response to a take over request (TOR) at two different driving speeds while being engaged in non-driving-related tasks (NDRT). Results from the performed experiments showed that driver reaction time to a TOR, gaze behavior and self-reported trust in automation were affected by the type of NDRT being concurrently performed and driver reaction time and gaze behavior additionally depended on the driving or vehicle speed at the time of TOR.
Cascaded Span Extraction and Response Generation for Document-Grounded Dialog
Venue: arXiv cs.CL 2021
Year: 2021

This paper summarizes our entries to both subtasks of the first DialDoc shared task which focuses on the agent response prediction task in goal-oriented document-grounded dialogs. The task is split into two subtasks: predicting a span in a document that grounds an agent turn and generating an agent response based on a dialog and grounding document. In the first subtask, we restrict the set of valid spans to the ones defined in the dataset, use a biaffine classifier to model spans, and finally use an ensemble of different models. For the second subtask, we use a cascaded model which grounds the response prediction on the predicted span instead of the full document. With these approaches, we obtain significant improvements in both subtasks compared to the baseline.
TimeLens: Event-based Video Frame Interpolation
Venue: arXiv cs.CV 2021
Year: 2021

State-of-the-art frame interpolation methods generate intermediate frames by inferring object motions in the image from consecutive key-frames. In the absence of additional information, first-order approximations, i.e. optical flow, must be used, but this choice restricts the types of motions that can be modeled, leading to errors in highly dynamic scenarios. Event cameras are novel sensors that address this limitation by providing auxiliary visual information in the blind-time between frames. They asynchronously measure per-pixel brightness changes and do this with high temporal resolution and low latency. Event-based frame interpolation methods typically adopt a synthesis-based approach, where predicted frame residuals are directly applied to the key-frames. However, while these approaches can capture non-linear motions they suffer from ghosting and perform poorly in low-texture regions with few events. Thus, synthesis-based and flow-based approaches are complementary. In this work, we introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both. We extensively evaluate our method on three synthetic and two real benchmarks where we show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods. Finally, we release a new large-scale dataset in highly dynamic scenarios, aimed at pushing the limits of existing methods.
Coresets for constrained k-median and k-means clustering in low dimensional Euclidean space
Venue: arXiv cs.DS 2021
Year: 2021

We study (Euclidean) $k$-median and $k$-means with constraints in the streaming model. There have been recent efforts to design unified algorithms to solve constrained $k$-means problems without using knowledge of the specific constraint at hand aside from mild assumptions like the polynomial computability of feasibility under the constraint (compute if a clustering satisfies the constraint) or the presence of an efficient assignment oracle (given a set of centers, produce an optimal assignment of points to the centers which satisfies the constraint). These algorithms have a running time exponential in $k$, but can be applied to a wide range of constraints. We demonstrate that a technique proposed in 2019 for solving a specific constrained streaming $k$-means problem, namely fair $k$-means clustering, actually implies streaming algorithms for all these constraints. These work for low dimensional Euclidean space. [Note that there are more algorithms for streaming fair $k$-means today, in particular they exist for high dimensional spaces now as well.]
Understanding Information Spreading Mechanisms During COVID-19 Pandemic by Analyzing the Impact of Tweet Text and User Features for Retweet Prediction
Venue: arXiv cs.SI 2021
Year: 2021

COVID-19 has affected the world economy and the daily life routine of almost everyone. It has been a hot topic on social media platforms such as Twitter, Facebook, etc. These social media platforms enable users to share information with other users who can reshare this information, thus causing this information to spread. Twitter's retweet functionality allows users to share the existing content with other users without altering the original content. Analysis of social media platforms can help in detecting emergencies during pandemics that lead to taking preventive measures. One such type of analysis is predicting the number of retweets for a given COVID-19 related tweet. Recently, CIKM organized a retweet prediction challenge for COVID-19 tweets focusing on using numeric features only. However, our hypothesis is, tweet text may play a vital role in an accurate retweet prediction. In this paper, we combine numeric and text features for COVID-19 related retweet predictions. For this purpose, we propose two CNN and RNN based models and evaluate the performance of these models on a publicly available TweetsCOV19 dataset using seven different evaluation metrics. Our evaluation results show that combining tweet text with numeric features improves the performance of retweet prediction significantly.
Self-Guided Contrastive Learning for BERT Sentence Representations
Venue: arXiv cs.CL 2021
Year: 2021

Although BERT and its variants have reshaped the NLP landscape, it still remains unclear how best to derive sentence embeddings from such pre-trained Transformers. In this work, we propose a contrastive learning method that utilizes self-guidance for improving the quality of BERT sentence representations. Our method fine-tunes BERT in a self-supervised fashion, does not rely on data augmentation, and enables the usual [CLS] token embeddings to function as sentence vectors. Moreover, we redesign the contrastive learning objective (NT-Xent) and apply it to sentence representation learning. We demonstrate with extensive experiments that our approach is more effective than competitive baselines on diverse sentence-related tasks. We also show it is efficient at inference and robust to domain shifts.
A2MM: Mitigating Frontrunning, Transaction Reordering and Consensus Instability in Decentralized Exchanges
Venue: arXiv cs.CR 2021
Year: 2021

The asset trading volume on blockchain-based exchanges (DEX) increased substantially since the advent of Automated Market Makers (AMM). Yet, AMMs and their forks compete on the same blockchain, incurring unnecessary network and block-space overhead, by attracting sandwich attackers and arbitrage competitions. Moreover, conceptually speaking, a blockchain is one database, and we find little reason to partition this database into multiple competing exchanges, which then necessarily require price synchronization through arbitrage. This paper shows that DEX arbitrage and trade routing among similar AMMs can be performed efficiently and atomically on-chain within smart contracts. These insights lead us to create a new AMM design, an Automated Arbitrage Market Maker, short A2MM DEX. A2MM aims to unite multiple AMMs to reduce overheads, costs and increase blockchain security. With respect to Miner Extractable Value (MEV), A2MM serves as a decentralized design for users to atomically collect MEV, mitigating the dangers of centralized MEV relay services. We show that A2MM offers essential security benefits. First, A2MM strengthens the blockchain consensus security by mitigating the competitive exploitation of MEV, therefore reducing the risks of consensus forks. A2MM reduces the network layer overhead of competitive transactions, improves network propagation, leading to less stale blocks and better blockchain security. Through trade routing, A2MM reduces the predatory risks of sandwich attacks by taking advantage of the minimum profitable victim input. A2MM also offers financial benefits to traders. Failed swap transactions from competitive trading occupy valuable block space, implying an upward pressure on transaction fees. Our evaluations shows that A2MM frees up 32.8% block-space of AMM-related transactions. In expectation, A2MM's revenue allows to reduce swap fees by 90%.