OpenRaiser/Intern-Atlas huggingface dataset:
You Cannot Easily Catch Me: A Low-Detectable Adversarial Patch for Object Detectors
Venue: arXiv cs.CV 2021
Year: 2021

Blind spots or outright deceit can bedevil and deceive machine learning models. Unidentified objects such as digital "stickers," also known as adversarial patches, can fool facial recognition systems, surveillance systems and self-driving cars. Fortunately, most existing adversarial patches can be outwitted, disabled and rejected by a simple classification network called an adversarial patch detector, which distinguishes adversarial patches from original images. An object detector classifies and predicts the types of objects within an image, such as by distinguishing a motorcyclist from the motorcycle, while also localizing each object's placement within the image by "drawing" so-called bounding boxes around each object, once again separating the motorcyclist from the motorcycle. To train detectors even better, however, we need to keep subjecting them to confusing or deceitful adversarial patches as we probe for the models' blind spots. For such probes, we came up with a novel approach, a Low-Detectable Adversarial Patch, which attacks an object detector with small and texture-consistent adversarial patches, making these adversaries less likely to be recognized. Concretely, we use several geometric primitives to model the shapes and positions of the patches. To enhance our attack performance, we also assign different weights to the bounding boxes in terms of loss function. Our experiments on the common detection dataset COCO as well as the driving-video dataset D2-City show that LDAP is an effective attack method, and can resist the adversarial patch detector.
Submodular Optimization Beyond Nonnegativity: Adaptive Seed Selection in Incentivized Social Advertising
Venue: arXiv cs.DS 2021
Year: 2021

The idea of social advertising (or social promotion) is to select a group of influential individuals (a.k.a \emph{seeds}) to help promote some products or ideas through an online social networks. There are two major players in the social advertising ecosystem: advertiser and platform. The platform sells viral engagements such as "like"s to advertisers by inserting their ads into the feed of seeds. These seeds receive monetary incentives from the platform in exchange for their participation in the social advertising campaign. Once an ad is engaged by a follower of some seed, the platform receives a fixed amount of payment, called cost per engagement, from the advertiser. The ad could potentially attract more engagements from followers' followers and trigger a viral contagion. At the beginning of a campaign, the advertiser submits a budget to the platform and this budget can be used for two purposes: recruiting seeds and paying for the viral engagements generated by the seeds. Note that the first part of payment goes to the seeds and the latter one is the actual revenue collected by the platform. In this setting, the problem for the platform is to recruit a group of seeds such that she can collect the largest possible amount of revenue subject to the budget constraint. We formulate this problem as a seed selection problem whose objective function is non-monotone and it might take on negative values, making existing results on submodular optimization and influence maximization not applicable to our setting. We study this problem under both non-adaptive and adaptive settings. Although we focus on social advertising in this paper, our results apply to any optimization problems whose objective function is the expectation of the minimum of a stochastic submodular function and a linear function.
Library of Congress Subject Heading (LCSH) Browsing and Natural Language Searching
Venue: arXiv cs.IR 2021
Year: 2021

Controlled topical vocabularies (CVs) are built into information systems to aid browsing and retrieval of items that may be unfamiliar, but it is unclear how this feature should be integrated with standard keyword searching. Few systems or scholarly prototypes have attempted this, and none have used the most widely used CV, the Library of Congress Subject Headings (LCSH), which organizes monograph collections in academic libraries throughout the world. This paper describes a working prototype of a Web application that concurrently allows topic exploration using an outline tree view of the LCSH hierarchy and natural language keyword searching of a real-world Science and Engineering bibliographic collection. Pilot testing shows the system is functional, and work to fit the complex LCSH structure into a usable hierarchy is ongoing. This study contributes to knowledge of the practical design decisions required when developing linked interactions between topical hierarchy browsing and natural language searching, which promise to facilitate information discovery and exploration.
MatSciBERT: A Materials Domain Language Model for Text Mining and Information Extraction
Venue: arXiv cs.CL 2021
Year: 2021

An overwhelmingly large amount of knowledge in the materials domain is generated and stored as text published in peer-reviewed scientific literature. Recent developments in natural language processing, such as bidirectional encoder representations from transformers (BERT) models, provide promising tools to extract information from these texts. However, direct application of these models in the materials domain may yield suboptimal results as the models themselves may not be trained on notations and jargon that are specific to the domain. Here, we present a materials-aware language model, namely, MatSciBERT, which is trained on a large corpus of scientific literature published in the materials domain. We further evaluate the performance of MatSciBERT on three downstream tasks, namely, abstract classification, named entity recognition, and relation extraction, on different materials datasets. We show that MatSciBERT outperforms SciBERT, a language model trained on science corpus, on all the tasks. Further, we discuss some of the applications of MatSciBERT in the materials domain for extracting information, which can, in turn, contribute to materials discovery or optimization. Finally, to make the work accessible to the larger materials community, we make the pretrained and finetuned weights and the models of MatSciBERT freely accessible.
NetSD: Remote Access to Integrated SD Cards of Embedded Devices
Venue: arXiv cs.OH 2021
Year: 2021

Digitalization continuously pervades all areas and the Internet of Things (IoT) is still on the rise. This leads to an increased need for efficiency in the development of embedded devices and systems composed thereof. Hybrid testbeds are common environments to representatively assess, e.g., hardware-software interaction, interoperability, and scalability. Although automation is inevitable to achieve efficiency, not all devices offer interfaces to be fully software-controlled. Most notably, block devices tend to be inaccessible for software outside a Device under Test (DuT), especially when the latter is in a dysfunctional state. This paper introduces the Networked SD card (NetSD) which enables remote access to removable block devices. The proposed system consists of a hardware part, which enables multiplexed access to a block device (e.g., an SD card) and a software part which enables remote access to the block device (e.g., via HTTP or network block device). NetSD thus adds testing and automation possibilities to DuTs without the need to modify their hard- or software. During the hardware design, we fund that different SD transfer modes and access profiles (read or write focus) benefit from different pull-up resistor configurations for the data lines.
Dataset: Analysis of IFTTT Recipes to Study How Humans Use Internet-of-Things (IoT) Devices
Venue: arXiv cs.HC 2021
Year: 2021

With the rapid development and usage of Internet-of-Things (IoT) and smart-home devices, researchers continue efforts to improve the "smartness" of those devices to address daily needs in people's lives. Such efforts usually begin with understanding evolving user behaviors on how humans utilize the devices and what they expect in terms of their behavior. However, while research efforts abound, there is a very limited number of datasets that researchers can use to both understand how people use IoT devices and to evaluate algorithms or systems for smart spaces. In this paper, we collect and characterize more than 50,000 recipes from the online If-This-Then-That (IFTTT) service to understand a seemingly straightforward but complicated question: "What kinds of behaviors do humans expect from their IoT devices?"
Near-Optimal Distance Oracles for Vertex-Labeled Planar Graphs
Venue: arXiv cs.DS 2021
Year: 2021

Given an undirected $n$-vertex planar graph $G=(V,E,ω)$ with non-negative edge weight function $ω:E\rightarrow \mathbb R$ and given an assigned label to each vertex, a vertex-labeled distance oracle is a data structure which for any query consisting of a vertex $u$ and a label $λ$ reports the shortest path distance from $u$ to the nearest vertex with label $λ$. We show that if there is a distance oracle for undirected $n$-vertex planar graphs with non-negative edge weights using $s(n)$ space and with query time $q(n)$, then there is a vertex-labeled distance oracle with $\tilde{O}(s(n))$ space and $\tilde{O}(q(n))$ query time. Using the state-of-the-art distance oracle of Long and Pettie, our construction produces a vertex-labeled distance oracle using $n^{1+o(1)}$ space and query time $\tilde O(1)$ at one extreme, $\tilde O(n)$ space and $n^{o(1)}$ query time at the other extreme, as well as such oracles for the full tradeoff between space and query time obtained in their paper. This is the first non-trivial exact vertex-labeled distance oracle for planar graphs and, to our knowledge, for any interesting graph class other than trees.
Determining Standard Occupational Classification Codes from Job Descriptions in Immigration Petitions
Venue: arXiv cs.LG 2021
Year: 2021

Accurate specification of standard occupational classification (SOC) code is critical to the success of many U.S. work visa applications. Determination of correct SOC code relies on careful study of job requirements and comparison to definitions given by the U.S. Bureau of Labor Statistics, which is often a tedious activity. In this paper, we apply methods from natural language processing (NLP) to computationally determine SOC code based on job description. We implement and empirically evaluate a broad variety of predictive models with respect to quality of prediction and training time, and identify models best suited for this task.
On the Trustworthiness of Tree Ensemble Explainability Methods
Venue: arXiv cs.LG 2021
Year: 2021

The recent increase in the deployment of machine learning models in critical domains such as healthcare, criminal justice, and finance has highlighted the need for trustworthy methods that can explain these models to stakeholders. Feature importance methods (e.g. gain and SHAP) are among the most popular explainability methods used to address this need. For any explainability technique to be trustworthy and meaningful, it has to provide an explanation that is accurate and stable. Although the stability of local feature importance methods (explaining individual predictions) has been studied before, there is yet a knowledge gap about the stability of global features importance methods (explanations for the whole model). Additionally, there is no study that evaluates and compares the accuracy of global feature importance methods with respect to feature ordering. In this paper, we evaluate the accuracy and stability of global feature importance methods through comprehensive experiments done on simulations as well as four real-world datasets. We focus on tree-based ensemble methods as they are used widely in industry and measure the accuracy and stability of explanations under two scenarios: 1) when inputs are perturbed 2) when models are perturbed. Our findings provide a comparison of these methods under a variety of settings and shed light on the limitations of global feature importance methods by indicating their lack of accuracy with and without noisy inputs, as well as their lack of stability with respect to: 1) increase in input dimension or noise in the data; 2) perturbations in models initialized by different random seeds or hyperparameter settings.
HUMBI: A Large Multiview Dataset of Human Body Expressions and Benchmark Challenge
Venue: arXiv cs.CV 2021
Year: 2021

This paper presents a new large multiview dataset called HUMBI for human body expressions with natural clothing. The goal of HUMBI is to facilitate modeling view-specific appearance and geometry of five primary body signals including gaze, face, hand, body, and garment from assorted people. 107 synchronized HD cameras are used to capture 772 distinctive subjects across gender, ethnicity, age, and style. With the multiview image streams, we reconstruct high fidelity body expressions using 3D mesh models, which allows representing view-specific appearance. We demonstrate that HUMBI is highly effective in learning and reconstructing a complete human model and is complementary to the existing datasets of human body expressions with limited views and subjects such as MPII-Gaze, Multi-PIE, Human3.6M, and Panoptic Studio datasets. Based on HUMBI, we formulate a new benchmark challenge of a pose-guided appearance rendering task that aims to substantially extend photorealism in modeling diverse human expressions in 3D, which is the key enabling factor of authentic social tele-presence. HUMBI is publicly available at http://humbi-data.net
Tree-Constrained Graph Neural Networks For Argument Mining
Venue: arXiv cs.CL 2021
Year: 2021

We propose a novel architecture for Graph Neural Networks that is inspired by the idea behind Tree Kernels of measuring similarity between trees by taking into account their common substructures, named fragments. By imposing a series of regularization constraints to the learning problem, we exploit a pooling mechanism that incorporates such notion of fragments within the node soft assignment function that produces the embeddings. We present an extensive experimental evaluation on a collection of sentence classification tasks conducted on several argument mining corpora, showing that the proposed approach performs well with respect to state-of-the-art techniques.
Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning
Venue: arXiv cs.CL 2021
Year: 2021

Establishing retrieval-based dialogue systems that can select appropriate responses from the pre-built index has gained increasing attention from researchers. For this task, the adoption of pre-trained language models (such as BERT) has led to remarkable progress in a number of benchmarks. There exist two common approaches, including cross-encoders which perform full attention over the inputs, and bi-encoders that encode the context and response separately. The former gives considerable improvements in accuracy but is often inapplicable in practice for large-scale retrieval given the cost of the full attention required for each sample at test time. The latter is efficient for billions of indexes but suffers from sub-optimal performance. In this work, we propose to combine the best of both worlds to build a retrieval system. Specifically, we employ a fast bi-encoder to replace the traditional feature-based pre-retrieval model (such as BM25) and set the response re-ranking model as a more complicated architecture (such as cross-encoder). To further improve the effectiveness of our framework, we train the pre-retrieval model and the re-ranking model at the same time via mutual learning, which enables two models to learn from each other throughout the training process. We conduct experiments on two benchmarks and evaluation results demonstrate the efficiency and effectiveness of our proposed framework.
Batched Thompson Sampling
Venue: arXiv cs.LG 2021
Year: 2021

We introduce a novel anytime Batched Thompson sampling policy for multi-armed bandits where the agent observes the rewards of her actions and adjusts her policy only at the end of a small number of batches. We show that this policy simultaneously achieves a problem dependent regret of order $O(\log(T))$ and a minimax regret of order $O(\sqrt{T\log(T)})$ while the number of batches can be bounded by $O(\log(T))$ independent of the problem instance over a time horizon $T$. We also show that in expectation the number of batches used by our policy can be bounded by an instance dependent bound of order $O(\log\log(T))$. These results indicate that Thompson sampling maintains the same performance in this batched setting as in the case when instantaneous feedback is available after each action, while requiring minimal feedback. These results also indicate that Thompson sampling performs competitively with recently proposed algorithms tailored for the batched setting. These algorithms optimize the batch structure for a given time horizon $T$ and prioritize exploration in the beginning of the experiment to eliminate suboptimal actions. We show that Thompson sampling combined with an adaptive batching strategy can achieve a similar performance without knowing the time horizon $T$ of the problem and without having to carefully optimize the batch structure to achieve a target regret bound (i.e. problem dependent vs minimax regret) for a given $T$.
Inverse airfoil design method for generating varieties of smooth airfoils using conditional WGAN-gp
Venue: arXiv cs.LG 2021
Year: 2021

Machine learning models are recently utilized for airfoil shape generation methods. It is desired to obtain airfoil shapes that satisfies required lift coefficient. Generative adversarial networks (GAN) output reasonable airfoil shapes. However, shapes obtained from ordinal GAN models are not smooth, and they need smoothing before flow analysis. Therefore, the models need to be coupled with Bezier curves or other smoothing methods to obtain smooth shapes. Generating shapes without any smoothing methods is challenging. In this study, we employed conditional Wasserstein GAN with gradient penalty (CWGAN-GP) to generate airfoil shapes, and the obtained shapes are as smooth as those obtained using smoothing methods. With the proposed method, no additional smoothing method is needed to generate airfoils. Moreover, the proposed model outputs shapes that satisfy the lift coefficient requirements.
Real-Time Risk-Bounded Tube-Based Trajectory Safety Verification
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we address the real-time risk-bounded safety verification problem of continuous-time state trajectories of autonomous systems in the presence of uncertain time-varying nonlinear safety constraints. Risk is defined as the probability of not satisfying the uncertain safety constraints. Existing approaches to address the safety verification problems under uncertainties either are limited to particular classes of uncertainties and safety constraints, e.g., Gaussian uncertainties and linear constraints, or rely on sampling based methods. In this paper, we provide a fast convex algorithm to efficiently evaluate the probabilistic nonlinear safety constraints in the presence of arbitrary probability distributions and long planning horizons in real-time, without the need for uncertainty samples and time discretization. The provided approach verifies the safety of the given state trajectory and its neighborhood (tube) to account for the execution uncertainties and risk. In the provided approach, we first use the moments of the probability distributions of the uncertainties to transform the probabilistic safety constraints into a set of deterministic safety constraints. We then use convex methods based on sum-of-squares polynomials to verify the obtained deterministic safety constraints over the entire planning time horizon without time discretization. To illustrate the performance of the proposed method, we apply the provided method to the safety verification problem of self-driving vehicles and autonomous aerial vehicles.
Learning Reward Functions from Scale Feedback
Venue: arXiv cs.RO 2021
Year: 2021

Today's robots are increasingly interacting with people and need to efficiently learn inexperienced user's preferences. A common framework is to iteratively query the user about which of two presented robot trajectories they prefer. While this minimizes the users effort, a strict choice does not yield any information on how much one trajectory is preferred. We propose scale feedback, where the user utilizes a slider to give more nuanced information. We introduce a probabilistic model on how users would provide feedback and derive a learning framework for the robot. We demonstrate the performance benefit of slider feedback in simulations, and validate our approach in two user studies suggesting that scale feedback enables more effective learning in practice.
Neural Dependency Coding inspired Multimodal Fusion
Venue: arXiv cs.NE 2021
Year: 2021

Information integration from different modalities is an active area of research. Human beings and, in general, biological neural systems are quite adept at using a multitude of signals from different sensory perceptive fields to interact with the environment and each other. Recent work in deep fusion models via neural networks has led to substantial improvements over unimodal approaches in areas like speech recognition, emotion recognition and analysis, captioning and image description. However, such research has mostly focused on architectural changes allowing for fusion of different modalities while keeping the model complexity manageable. Inspired by recent neuroscience ideas about multisensory integration and processing, we investigate the effect of synergy maximizing loss functions. Experiments on multimodal sentiment analysis tasks: CMU-MOSI and CMU-MOSEI with different models show that our approach provides a consistent performance boost.
Online Primal-Dual Algorithms with Predictions for Packing Problems
Venue: arXiv cs.DS 2021
Year: 2021

The domain of online algorithms with predictions has been extensively studied for different applications such as scheduling, caching (paging), clustering, ski rental, etc. Recently, Bamas et al., aiming for an unified method, have provided a primal-dual framework for linear covering problems. They extended the online primal-dual method by incorporating predictions in order to achieve a performance beyond the worst-case case analysis. In this paper, we consider this research line and present a framework to design algorithms with predictions for non-linear packing problems. We illustrate the applicability of our framework in submodular maximization and in particular ad-auction maximization in which the optimal bound is given and supporting experiments are provided.
External knowledge transfer deployment inside a simple double agent Viterbi algorithm
Venue: arXiv cs.CL 2021
Year: 2021

We consider in this paper deploying external knowledge transfer inside a simple double agent Viterbi algorithm which is an algorithm firstly introduced by the author in his preprint "Hidden Markov Based Mathematical Model dedicated to Extract Ingredients from Recipe Text". The key challenge of this work lies in discovering the reason why our old model does have bad performances when it is confronted with estimating ingredient state for unknown words and see if deploying external knowledge transfer directly on calculating state matrix could be the solution instead of deploying it only on back propagating step.