OpenRaiser/Intern-Atlas huggingface dataset:
Download Cost of Private Updating
Venue: arXiv cs.IT 2021
Year: 2021

We consider the problem of privately updating a message out of $K$ messages from $N$ replicated and non-colluding databases. In this problem, a user has an outdated version of the message $\hat{W}_θ$ of length $L$ bits that differ from the current version $W_θ$ in at most $f$ bits. The user needs to retrieve $W_θ$ correctly using a private information retrieval (PIR) scheme with the least number of downloads without leaking any information about the message index $θ$ to any individual database. To that end, we propose a novel achievable scheme based on \emph{syndrome decoding}. Specifically, the user downloads the syndrome corresponding to $W_θ$, according to a linear block code with carefully designed parameters, using the optimal PIR scheme for messages with a length constraint. We derive lower and upper bounds for the optimal download cost that match if the term $\log_2\left(\sum_{i=0}^f \binom{L}{i}\right)$ is an integer. Our results imply that there is a significant reduction in the download cost if $f < \frac{L}{2}$ compared with downloading $W_θ$ directly using classical PIR approaches without taking the correlation between $W_θ$ and $\hat{W}_θ$ into consideration.
Automated essay scoring using efficient transformer-based language models
Venue: arXiv cs.CL 2021
Year: 2021

Automated Essay Scoring (AES) is a cross-disciplinary effort involving Education, Linguistics, and Natural Language Processing (NLP). The efficacy of an NLP model in AES tests it ability to evaluate long-term dependencies and extrapolate meaning even when text is poorly written. Large pretrained transformer-based language models have dominated the current state-of-the-art in many NLP tasks, however, the computational requirements of these models make them expensive to deploy in practice. The goal of this paper is to challenge the paradigm in NLP that bigger is better when it comes to AES. To do this, we evaluate the performance of several fine-tuned pretrained NLP models with a modest number of parameters on an AES dataset. By ensembling our models, we achieve excellent results with fewer parameters than most pretrained transformer-based models.
Multi-Domain Learning by Meta-Learning: Taking Optimal Steps in Multi-Domain Loss Landscapes by Inner-Loop Learning
Venue: arXiv cs.CV 2021
Year: 2021

We consider a model-agnostic solution to the problem of Multi-Domain Learning (MDL) for multi-modal applications. Many existing MDL techniques are model-dependent solutions which explicitly require nontrivial architectural changes to construct domain-specific modules. Thus, properly applying these MDL techniques for new problems with well-established models, e.g. U-Net for semantic segmentation, may demand various low-level implementation efforts. In this paper, given emerging multi-modal data (e.g., various structural neuroimaging modalities), we aim to enable MDL purely algorithmically so that widely used neural networks can trivially achieve MDL in a model-independent manner. To this end, we consider a weighted loss function and extend it to an effective procedure by employing techniques from the recently active area of learning-to-learn (meta-learning). Specifically, we take inner-loop gradient steps to dynamically estimate posterior distributions over the hyperparameters of our loss function. Thus, our method is model-agnostic, requiring no additional model parameters and no network architecture changes; instead, only a few efficient algorithmic modifications are needed to improve performance in MDL. We demonstrate our solution to a fitting problem in medical imaging, specifically, in the automatic segmentation of white matter hyperintensity (WMH). We look at two neuroimaging modalities (T1-MR and FLAIR) with complementary information fitting for our problem.
Unfounded Sets for Disjunctive Hybrid MKNF Knowledge Bases
Venue: arXiv cs.AI 2021
Year: 2021

Combining the closed-world reasoning of answer set programming (ASP) with the open-world reasoning of ontologies broadens the space of applications of reasoners. Disjunctive hybrid MKNF knowledge bases succinctly extend ASP and in some cases without increasing the complexity of reasoning tasks. However, in many cases, solver development is lagging behind. As the result, the only known method of solving disjunctive hybrid MKNF knowledge bases is based on guess-and-verify, as formulated by Motik and Rosati in their original work. A main obstacle is understanding how constraint propagation may be performed by a solver, which, in the context of ASP, centers around the computation of \textit{unfounded atoms}, the atoms that are false given a partial interpretation. In this work, we build towards improving solvers for hybrid MKNF knowledge bases with disjunctive rules: We formalize a notion of unfounded sets for these knowledge bases, identify lower complexity bounds, and demonstrate how we might integrate these developments into a solver. We discuss challenges introduced by ontologies that are not present in the development of solvers for disjunctive logic programs, which warrant some deviations from traditional definitions of unfounded sets. We compare our work with prior definitions of unfounded sets.
RbSyn: Type- and Effect-Guided Program Synthesis
Venue: arXiv cs.PL 2021
Year: 2021

In recent years, researchers have explored component-based synthesis, which aims to automatically construct programs that operate by composing calls to existing APIs. However, prior work has not considered efficient synthesis of methods with side effects, e.g., web app methods that update a database. In this paper, we introduce RbSyn, a novel type- and effect-guided synthesis tool for Ruby. An RbSyn synthesis goal is specified as the type for the target method and a series of test cases it must pass. RbSyn works by recursively generating well-typed candidate method bodies whose write effects match the read effects of the test case assertions. After finding a set of candidates that separately satisfy each test, RbSyn synthesizes a solution that branches to execute the correct candidate code under the appropriate conditions. We formalize RbSyn on a core, object-oriented language $λ_{syn}$ and describe how the key ideas of the model are scaled-up in our implementation for Ruby. We evaluated RbSyn on 19 benchmarks, 12 of which come from popular, open-source Ruby apps. We found that RbSyn synthesizes correct solutions for all benchmarks, with 15 benchmarks synthesizing in under 9 seconds, while the slowest benchmark takes 83 seconds. Using observed reads to guide synthesize is effective: using type-guidance alone times out on 10 of 12 app benchmarks. We also found that using less precise effect annotations leads to worse synthesis performance. In summary, we believe type- and effect-guided synthesis is an important step forward in synthesis of effectful methods from test cases.
Rip van Winkle's Razor: A Simple Estimate of Overfit to Test Data
Venue: arXiv cs.LG 2021
Year: 2021

Traditional statistics forbids use of test data (a.k.a. holdout data) during training. Dwork et al. 2015 pointed out that current practices in machine learning, whereby researchers build upon each other's models, copying hyperparameters and even computer code -- amounts to implicitly training on the test set. Thus error rate on test data may not reflect the true population error. This observation initiated {\em adaptive data analysis}, which provides evaluation mechanisms with guaranteed upper bounds on this difference. With statistical query (i.e. test accuracy) feedbacks, the best upper bound is fairly pessimistic: the deviation can hit a practically vacuous value if the number of models tested is quadratic in the size of the test set. In this work, we present a simple new estimate, {\em Rip van Winkle's Razor}. It relies upon a new notion of \textquotedblleft information content\textquotedblright\ of a model: the amount of information that would have to be provided to an expert referee who is intimately familiar with the field and relevant science/math, and who has been just been woken up after falling asleep at the moment of the creation of the test data (like \textquotedblleft Rip van Winkle\textquotedblright\ of the famous fairy tale). This notion of information content is used to provide an estimate of the above deviation which is shown to be non-vacuous in many modern settings.
Machine Biometrics -- Towards Identifying Machines in a Smart City Environment
Venue: arXiv cs.LG 2021
Year: 2021

This paper deals with the identification of machines in a smart city environment. The concept of machine biometrics is proposed in this work for the first time, as a way to authenticate machine identities interacting with humans in everyday life. This definition is imposed in modern years where autonomous vehicles, social robots, etc. are considered active members of contemporary societies. In this context, the case of car identification from the engine behavioral biometrics is examined. For this purpose, 22 sound features were extracted and their discrimination capabilities were tested in combination with 9 different machine learning classifiers, towards identifying 5 car manufacturers. The experimental results revealed the ability of the proposed biometrics to identify cars with high accuracy up to 98% for the case of the Multilayer Perceptron (MLP) neural network model.
Reinforcement Learning of Implicit and Explicit Control Flow in Instructions
Venue: arXiv cs.LG 2021
Year: 2021

Learning to flexibly follow task instructions in dynamic environments poses interesting challenges for reinforcement learning agents. We focus here on the problem of learning control flow that deviates from a strict step-by-step execution of instructions -- that is, control flow that may skip forward over parts of the instructions or return backward to previously completed or skipped steps. Demand for such flexible control arises in two fundamental ways: explicitly when control is specified in the instructions themselves (such as conditional branching and looping) and implicitly when stochastic environment dynamics require re-completion of instructions whose effects have been perturbed, or opportunistic skipping of instructions whose effects are already present. We formulate an attention-based architecture that meets these challenges by learning, from task reward only, to flexibly attend to and condition behavior on an internal encoding of the instructions. We test the architecture's ability to learn both explicit and implicit control in two illustrative domains -- one inspired by Minecraft and the other by StarCraft -- and show that the architecture exhibits zero-shot generalization to novel instructions of length greater than those in a training set, at a performance level unmatched by two baseline recurrent architectures and one ablation architecture.
Characterizing the Landscape of COVID-19 Themed Cyberattacks and Defenses
Venue: arXiv cs.CR 2021
Year: 2021

COVID-19 (Coronavirus) hit the global society and economy with a big surprise. In particular, work-from-home has become a new norm for employees. Despite the fact that COVID-19 can equally attack innocent people and cybercriminals, it is ironic to see surges in cyberattacks leveraging COVID-19 as a theme, dubbed COVID-19 themed cyberattacks or COVID-19 attacks for short, which represent a new phenomenon that has yet to be systematically understood. In this paper, we make the first step towards fully characterizing the landscape of these attacks, including their sophistication via the Cyber Kill Chain model. We also explore the solution space of defenses against these attacks.
Cybersecurity Threats in Connected and Automated Vehicles based Federated Learning Systems
Venue: arXiv cs.CR 2021
Year: 2021

Federated learning (FL) is a machine learning technique that aims at training an algorithm across decentralized entities holding their local data private. Wireless mobile networks allow users to communicate with other fixed or mobile users. The road traffic network represents an infrastructure-based configuration of a wireless mobile network where the Connected and Automated Vehicles (CAV) represent the communicating entities. Applying FL in a wireless mobile network setting gives rise to a new threat in the mobile environment that is very different from the traditional fixed networks. The threat is due to the intrinsic characteristics of the wireless medium and is caused by the characteristics of the vehicular networks such as high node-mobility and rapidly changing topology. Most cyber defense techniques depend on highly reliable and connected networks. This paper explores falsified information attacks, which target the FL process that is ongoing at the RSU. We identified a number of attack strategies conducted by the malicious CAVs to disrupt the training of the global model in vehicular networks. We show that the attacks were able to increase the convergence time and decrease the accuracy the model. We demonstrate that our attacks bypass FL defense strategies in their primary form and highlight the need for novel poisoning resilience defense mechanisms in the wireless mobile setting of the future road networks.
LazyTensor: combining eager execution with domain-specific compilers
Venue: arXiv cs.PL 2021
Year: 2021

Domain-specific optimizing compilers have demonstrated significant performance and portability benefits, but require programs to be represented in their specialized IRs. Existing frontends to these compilers suffer from the "language subset problem" where some host language features are unsupported in the subset of the user's program that interacts with the domain-specific compiler. By contrast, define-by-run ML frameworks-colloquially called "eager" mode-are popular due to their ease of use and expressivity, where the full power of the host programming language can be used. LazyTensor is a technique to target domain specific compilers without sacrificing define-by-run ergonomics. Initially developed to support PyTorch on Cloud TPUs, the technique, along with a substantially shared implementation, has been used by Swift for TensorFlow across CPUs, GPUs, and TPUs, demonstrating the generality of the approach across (1) Tensor implementations, (2) hardware accelerators, and (3) programming languages.
MixSearch: Searching for Domain Generalized Medical Image Segmentation Architectures
Venue: arXiv cs.CV 2021
Year: 2021

Considering the scarcity of medical data, most datasets in medical image analysis are an order of magnitude smaller than those of natural images. However, most Network Architecture Search (NAS) approaches in medical images focused on specific datasets and did not take into account the generalization ability of the learned architectures on unseen datasets as well as different domains. In this paper, we address this point by proposing to search for generalizable U-shape architectures on a composited dataset that mixes medical images from multiple segmentation tasks and domains creatively, which is named MixSearch. Specifically, we propose a novel approach to mix multiple small-scale datasets from multiple domains and segmentation tasks to produce a large-scale dataset. Then, a novel weaved encoder-decoder structure is designed to search for a generalized segmentation network in both cell-level and network-level. The network produced by the proposed MixSearch framework achieves state-of-the-art results compared with advanced encoder-decoder networks across various datasets.
Large-scale Quantum Approximate Optimization via Divide-and-Conquer
Venue: arXiv cs.ET 2021
Year: 2021

Quantum Approximate Optimization Algorithm (QAOA) is a promising hybrid quantum-classical algorithm for solving combinatorial optimization problems. However, it cannot overcome qubit limitation for large-scale problems. Furthermore, the execution time of QAOA scales exponentially with the problem size. We propose a Divide-and-Conquer QAOA (DC-QAOA) to address the above challenges for graph maximum cut (MaxCut) problem. The algorithm works by recursively partitioning a larger graph into smaller ones whose MaxCut solutions are obtained with small-size NISQ computers. The overall solution is retrieved from the sub-solutions by applying the combination policy of quantum state reconstruction. Multiple partitioning and reconstruction methods are proposed/ compared. DC-QAOA achieves 97.14% approximation ratio (20.32% higher than classical counterpart), and 94.79% expectation value (15.80% higher than quantum annealing). DC-QAOA also reduces the time complexity of conventional QAOA from exponential to quadratic.
Algorithmic Correspondence for Hybrid Logic with Binder
Venue: arXiv cs.LO 2021
Year: 2021

In the present paper, we develop the algorithmic correspondence theory for hybrid logic with binder. We define the class of Sahlqvist inequalities, each inequality of which is shown to have a first-order frame correspondent effectively computable by an algorithm ALBA.
Graph-based Semi-supervised Learning: A Comprehensive Review
Venue: arXiv cs.LG 2021
Year: 2021

Semi-supervised learning (SSL) has tremendous value in practice due to its ability to utilize both labeled data and unlabelled data. An important class of SSL methods is to naturally represent data as graphs such that the label information of unlabelled samples can be inferred from the graphs, which corresponds to graph-based semi-supervised learning (GSSL) methods. GSSL methods have demonstrated their advantages in various domains due to their uniqueness of structure, the universality of applications, and their scalability to large scale data. Focusing on this class of methods, this work aims to provide both researchers and practitioners with a solid and systematic understanding of relevant advances as well as the underlying connections among them. This makes our paper distinct from recent surveys that cover an overall picture of SSL methods while neglecting fundamental understanding of GSSL methods. In particular, a major contribution of this paper lies in a new generalized taxonomy for GSSL, including graph regularization and graph embedding methods, with the most up-to-date references and useful resources such as codes, datasets, and applications. Furthermore, we present several potential research directions as future work with insights into this rapidly growing field.
Mitigating Domain Mismatch in Face Recognition Using Style Matching
Venue: arXiv cs.CV 2021
Year: 2021

Despite outstanding performance on public benchmarks, face recognition still suffers due to domain mismatch between training (source) and testing (target) data. Furthermore, these domains are not shared classes, which complicates domain adaptation. Since this is also a fine-grained classification problem which does not strictly follow the low-density separation principle, conventional domain adaptation approaches do not resolve these problems. In this paper, we formulate domain mismatch in face recognition as a style mismatch problem for which we propose two methods. First, we design a domain discriminator with human-level judgment to mine target-like images in the training data to mitigate the domain gap. Second, we extract style representations in low-level feature maps of the backbone model, and match the style distributions of the two domains to find a common style representation. Evaluations on verification and open-set and closed-set identification protocols show that both methods yield good improvements, and that performance is more robust if they are combined. Our approach is competitive with related work, and its effectiveness is verified in a practical application.
Lightweight Key-Dependent Dynamic S-Boxes based on Hyperelliptic Curve for IoT Devices
Venue: arXiv cs.CR 2021
Year: 2021

Security is one of the main issues in Internet of Things (IoT). Encryption plays a curtail role in making these systems secure. Substitution Box (S-Box) has an effective impact in block encryption methods. Due to the restricted resource capacities of IoT nodes, providing a lightweight S-Box is a challenging problem. This paper presents a key-dependent S-Box using Hyperelliptic curve. The proposed S-Box is analytically evaluated using performance criteria including bijection, nonlinearity, strict avalanche effect, and algebraic degree. The evaluation results endorse that the offered S-Box production algorithm is considerably an effective way to generate cryptographic strong S-Box.
Engineering Swarms of Cyber-Physical Systems with the CPSwarm Workbench
Venue: arXiv cs.MA 2021
Year: 2021

Engineering swarms of cyber-physical systems (CPSs) is a complex process. We present the CPSwarm workbench that creates an automated design workflow to ease this process. This formalized workflow guides the user from modeling, to code generation, to deployment, both in simulation and on CPS hardware platforms. The workbench combines existing and emerging tools to solve real-world CPS swarm problems. As a proof-of-concept, we use the workbench to design a swarm of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) for a search and rescue (SAR) use case. We evaluate the resulting swarm behaviors on three levels. First, abstract simulations for rapid prototyping. Second, detailed simulation to test the correctness of the results. Third, deployment on hardware to demonstrate the applicability. We measure the swarm performance in terms of area covered and victims rescued. The results show that the performance of the swarm is proportional to its size. Despite some manual steps, the proposed workbench shows to be well suited to ease the complicated task of deploying a swarm of CPSs.
Can Romeo and Juliet Meet? Or Rendezvous Games with Adversaries on Graphs
Venue: arXiv cs.DM 2021
Year: 2021

We introduce the rendezvous game with adversaries. In this game, two players, {\sl Facilitator} and {\sl Disruptor}, play against each other on a graph. Facilitator has two agents, and Disruptor has a team of $k$ agents located in some vertices of the graph. They take turns in moving their agents to adjacent vertices (or staying). Facilitator wins if his agents meet in some vertex of the graph. The goal of Disruptor is to prevent the rendezvous of Facilitator's agents. Our interest is to decide whether Facilitator can win. It appears that, in general, the problem is PSPACE-hard and, when parameterized by $k$, co-W[2]-hard. Moreover, even the game's variant where we ask whether Facilitator can ensure the meeting of his agents within $τ$ steps is co-NP-complete already for $τ=2$. On the other hand, for chordal and $P_5$-free graphs, we prove that the problem is solvable in polynomial time. These algorithms exploit an interesting relation of the game and minimum vertex cuts in certain graph classes. Finally, we show that the problem is fixed-parameter tractable parameterized by both the graph's neighborhood diversity and $τ$.